Backends

AWS.jl supports two "backends" which serve as HTTP clients to reach the AWS REST API.

The backend can be specified in two ways: by setting the global AWS.DEFAULT_BACKEND, or by setting the backend on a per-request basis by setting the "backend" key in params dictionaries:

using AWS
@service S3 use_response_type = true
result = S3.get_object(bucket, key, Dict("backend" => AWS.DownloadsBackend()))

Note: use_response_type=true is not needed here for the backend selection to work; it is just a recommended option in general. See @service for more.

AWS.AbstractBackend — Type

AWS.AbstractBackend

An abstract type representing a "backend" to use as an HTTP client to connect to the AWS REST API.

source

AWS.DEFAULT_BACKEND — Constant

AWS.DEFAULT_BACKEND = Ref{AbstractBackend}()

This specifies the default backend to use. This can be modified to change the default backend used by AWS.jl:

using AWS
AWS.DEFAULT_BACKEND[] = AWS.DownloadsBackend()

As an alternative, the backend can be specified on a per-request basis, by adding a pair "backend" => AWS.DownloadsBackend() to the params argument of AWS.jl functions.

Warning

Setting the AWS.DEFAULT_BACKEND is a global change that affects all packages in your Julia session using AWS.jl. Therefore, it is not recommended for library code to change the default backend, and instead set the backend on a per-request basis if needed (or ask the user to set a default backend). If you do wish to change the default backend inside package code which is precompiled, then it must be changed from within the __init__ method. See the Julia manual for more on module initialization.

source

AWS.HTTPBackend — Type

AWS.HTTPBackend <: AWS.AbstractBackend

This backend uses HTTP.jl as an HTTP client to connect to the AWS REST API, and has one field:

http_options::AbstractDict{Symbol,<:Any}

which defaults to an empty dictionary. This field provides default options to use, which can be any of the keyword arguments to HTTP.request. These options are overwritten by any per-request options.

This is the default backend, and the only option until AWS.jl v1.57.0. Therefore, it has been used more often in practice, and may be more mature. Note, however, HTTP.jl currently (March 2022) has issues with concurrency (see HTTP.jl#517). Therefore, it may be advisable to switch to the DownloadsBackend if you are using concurrency.

source

AWS.DownloadsBackend — Type

DownloadsBackend <: AWS.AbstractBackend

This backend uses the Downloads.jl stdlib to use libcurl as an HTTP client to connect to the AWS REST API.

It has one field,

downloader::Union{Nothing,Downloads.Downloader}

which is the Downloads.Downloader to use. If set to nothing, the default, then a global downloader object will be used.

Downloads.jl tends to perform better under concurrent operation than HTTP.jl, particularly with @async / asyncmap. Note that threading (e.g. @spawn or @threads) with Downloads.jl is broken on Julia releases prior to 1.8 (https://github.com/JuliaLang/Downloads.jl/issues/182#issuecomment-1069269944).

source