Emr Containers

This page documents function available when using the Emr_Containers module, created with @service Emr_Containers.

Index

Documentation

Main.Emr_Containers.cancel_job_runMethod
cancel_job_run(job_run_id, virtual_cluster_id)
cancel_job_run(job_run_id, virtual_cluster_id, params::Dict{String,<:Any})

Cancels a job run. A job run is a unit of work, such as a Spark jar, PySpark script, or SparkSQL query, that you submit to Amazon EMR on EKS.

Arguments

  • job_run_id: The ID of the job run to cancel.
  • virtual_cluster_id: The ID of the virtual cluster for which the job run will be canceled.
source
Main.Emr_Containers.create_job_templateMethod
create_job_template(client_token, job_template_data, name)
create_job_template(client_token, job_template_data, name, params::Dict{String,<:Any})

Creates a job template. Job template stores values of StartJobRun API request in a template and can be used to start a job run. Job template allows two use cases: avoid repeating recurring StartJobRun API request values, enforcing certain values in StartJobRun API request.

Arguments

  • client_token: The client token of the job template.
  • job_template_data: The job template data which holds values of StartJobRun API request.
  • name: The specified name of the job template.

Optional Parameters

Optional parameters can be passed as a params::Dict{String,<:Any}. Valid keys are:

  • "kmsKeyArn": The KMS key ARN used to encrypt the job template.
  • "tags": The tags that are associated with the job template.
source
Main.Emr_Containers.create_managed_endpointMethod
create_managed_endpoint(client_token, execution_role_arn, name, release_label, type, virtual_cluster_id)
create_managed_endpoint(client_token, execution_role_arn, name, release_label, type, virtual_cluster_id, params::Dict{String,<:Any})

Creates a managed endpoint. A managed endpoint is a gateway that connects Amazon EMR Studio to Amazon EMR on EKS so that Amazon EMR Studio can communicate with your virtual cluster.

Arguments

  • client_token: The client idempotency token for this create call.
  • execution_role_arn: The ARN of the execution role.
  • name: The name of the managed endpoint.
  • release_label: The Amazon EMR release version.
  • type: The type of the managed endpoint.
  • virtual_cluster_id: The ID of the virtual cluster for which a managed endpoint is created.

Optional Parameters

Optional parameters can be passed as a params::Dict{String,<:Any}. Valid keys are:

  • "certificateArn": The certificate ARN provided by users for the managed endpoint. This field is under deprecation and will be removed in future releases.
  • "configurationOverrides": The configuration settings that will be used to override existing configurations.
  • "tags": The tags of the managed endpoint.
source
Main.Emr_Containers.create_virtual_clusterMethod
create_virtual_cluster(client_token, container_provider, name)
create_virtual_cluster(client_token, container_provider, name, params::Dict{String,<:Any})

Creates a virtual cluster. Virtual cluster is a managed entity on Amazon EMR on EKS. You can create, describe, list and delete virtual clusters. They do not consume any additional resource in your system. A single virtual cluster maps to a single Kubernetes namespace. Given this relationship, you can model virtual clusters the same way you model Kubernetes namespaces to meet your requirements.

Arguments

  • client_token: The client token of the virtual cluster.
  • container_provider: The container provider of the virtual cluster.
  • name: The specified name of the virtual cluster.

Optional Parameters

Optional parameters can be passed as a params::Dict{String,<:Any}. Valid keys are:

  • "tags": The tags assigned to the virtual cluster.
source
Main.Emr_Containers.delete_job_templateMethod
delete_job_template(template_id)
delete_job_template(template_id, params::Dict{String,<:Any})

Deletes a job template. Job template stores values of StartJobRun API request in a template and can be used to start a job run. Job template allows two use cases: avoid repeating recurring StartJobRun API request values, enforcing certain values in StartJobRun API request.

Arguments

  • template_id: The ID of the job template that will be deleted.
source
Main.Emr_Containers.delete_managed_endpointMethod
delete_managed_endpoint(endpoint_id, virtual_cluster_id)
delete_managed_endpoint(endpoint_id, virtual_cluster_id, params::Dict{String,<:Any})

Deletes a managed endpoint. A managed endpoint is a gateway that connects Amazon EMR Studio to Amazon EMR on EKS so that Amazon EMR Studio can communicate with your virtual cluster.

Arguments

  • endpoint_id: The ID of the managed endpoint.
  • virtual_cluster_id: The ID of the endpoint's virtual cluster.
source
Main.Emr_Containers.delete_virtual_clusterMethod
delete_virtual_cluster(virtual_cluster_id)
delete_virtual_cluster(virtual_cluster_id, params::Dict{String,<:Any})

Deletes a virtual cluster. Virtual cluster is a managed entity on Amazon EMR on EKS. You can create, describe, list and delete virtual clusters. They do not consume any additional resource in your system. A single virtual cluster maps to a single Kubernetes namespace. Given this relationship, you can model virtual clusters the same way you model Kubernetes namespaces to meet your requirements.

Arguments

  • virtual_cluster_id: The ID of the virtual cluster that will be deleted.
source
Main.Emr_Containers.describe_job_runMethod
describe_job_run(job_run_id, virtual_cluster_id)
describe_job_run(job_run_id, virtual_cluster_id, params::Dict{String,<:Any})

Displays detailed information about a job run. A job run is a unit of work, such as a Spark jar, PySpark script, or SparkSQL query, that you submit to Amazon EMR on EKS.

Arguments

  • job_run_id: The ID of the job run request.
  • virtual_cluster_id: The ID of the virtual cluster for which the job run is submitted.
source
Main.Emr_Containers.describe_job_templateMethod
describe_job_template(template_id)
describe_job_template(template_id, params::Dict{String,<:Any})

Displays detailed information about a specified job template. Job template stores values of StartJobRun API request in a template and can be used to start a job run. Job template allows two use cases: avoid repeating recurring StartJobRun API request values, enforcing certain values in StartJobRun API request.

Arguments

  • template_id: The ID of the job template that will be described.
source
Main.Emr_Containers.describe_managed_endpointMethod
describe_managed_endpoint(endpoint_id, virtual_cluster_id)
describe_managed_endpoint(endpoint_id, virtual_cluster_id, params::Dict{String,<:Any})

Displays detailed information about a managed endpoint. A managed endpoint is a gateway that connects Amazon EMR Studio to Amazon EMR on EKS so that Amazon EMR Studio can communicate with your virtual cluster.

Arguments

  • endpoint_id: This output displays ID of the managed endpoint.
  • virtual_cluster_id: The ID of the endpoint's virtual cluster.
source
Main.Emr_Containers.describe_virtual_clusterMethod
describe_virtual_cluster(virtual_cluster_id)
describe_virtual_cluster(virtual_cluster_id, params::Dict{String,<:Any})

Displays detailed information about a specified virtual cluster. Virtual cluster is a managed entity on Amazon EMR on EKS. You can create, describe, list and delete virtual clusters. They do not consume any additional resource in your system. A single virtual cluster maps to a single Kubernetes namespace. Given this relationship, you can model virtual clusters the same way you model Kubernetes namespaces to meet your requirements.

Arguments

  • virtual_cluster_id: The ID of the virtual cluster that will be described.
source
Main.Emr_Containers.get_managed_endpoint_session_credentialsMethod
get_managed_endpoint_session_credentials(credential_type, endpoint_id, execution_role_arn, virtual_cluster_id)
get_managed_endpoint_session_credentials(credential_type, endpoint_id, execution_role_arn, virtual_cluster_id, params::Dict{String,<:Any})

Generate a session token to connect to a managed endpoint.

Arguments

  • credential_type: Type of the token requested. Currently supported and default value of this field is “TOKEN.”
  • endpoint_id: The ARN of the managed endpoint for which the request is submitted.
  • execution_role_arn: The IAM Execution Role ARN that will be used by the job run.
  • virtual_cluster_id: The ARN of the Virtual Cluster which the Managed Endpoint belongs to.

Optional Parameters

Optional parameters can be passed as a params::Dict{String,<:Any}. Valid keys are:

  • "clientToken": The client idempotency token of the job run request.
  • "durationInSeconds": Duration in seconds for which the session token is valid. The default duration is 15 minutes and the maximum is 12 hours.
  • "logContext": String identifier used to separate sections of the execution logs uploaded to S3.
source
Main.Emr_Containers.list_job_runsMethod
list_job_runs(virtual_cluster_id)
list_job_runs(virtual_cluster_id, params::Dict{String,<:Any})

Lists job runs based on a set of parameters. A job run is a unit of work, such as a Spark jar, PySpark script, or SparkSQL query, that you submit to Amazon EMR on EKS.

Arguments

  • virtual_cluster_id: The ID of the virtual cluster for which to list the job run.

Optional Parameters

Optional parameters can be passed as a params::Dict{String,<:Any}. Valid keys are:

  • "createdAfter": The date and time after which the job runs were submitted.
  • "createdBefore": The date and time before which the job runs were submitted.
  • "maxResults": The maximum number of job runs that can be listed.
  • "name": The name of the job run.
  • "nextToken": The token for the next set of job runs to return.
  • "states": The states of the job run.
source
Main.Emr_Containers.list_job_templatesMethod
list_job_templates()
list_job_templates(params::Dict{String,<:Any})

Lists job templates based on a set of parameters. Job template stores values of StartJobRun API request in a template and can be used to start a job run. Job template allows two use cases: avoid repeating recurring StartJobRun API request values, enforcing certain values in StartJobRun API request.

Optional Parameters

Optional parameters can be passed as a params::Dict{String,<:Any}. Valid keys are:

  • "createdAfter": The date and time after which the job templates were created.
  • "createdBefore": The date and time before which the job templates were created.
  • "maxResults": The maximum number of job templates that can be listed.
  • "nextToken": The token for the next set of job templates to return.
source
Main.Emr_Containers.list_managed_endpointsMethod
list_managed_endpoints(virtual_cluster_id)
list_managed_endpoints(virtual_cluster_id, params::Dict{String,<:Any})

Lists managed endpoints based on a set of parameters. A managed endpoint is a gateway that connects Amazon EMR Studio to Amazon EMR on EKS so that Amazon EMR Studio can communicate with your virtual cluster.

Arguments

  • virtual_cluster_id: The ID of the virtual cluster.

Optional Parameters

Optional parameters can be passed as a params::Dict{String,<:Any}. Valid keys are:

  • "createdAfter": The date and time after which the endpoints are created.
  • "createdBefore": The date and time before which the endpoints are created.
  • "maxResults": The maximum number of managed endpoints that can be listed.
  • "nextToken": The token for the next set of managed endpoints to return.
  • "states": The states of the managed endpoints.
  • "types": The types of the managed endpoints.
source
Main.Emr_Containers.list_tags_for_resourceMethod
list_tags_for_resource(resource_arn)
list_tags_for_resource(resource_arn, params::Dict{String,<:Any})

Lists the tags assigned to the resources.

Arguments

  • resource_arn: The ARN of tagged resources.
source
Main.Emr_Containers.list_virtual_clustersMethod
list_virtual_clusters()
list_virtual_clusters(params::Dict{String,<:Any})

Lists information about the specified virtual cluster. Virtual cluster is a managed entity on Amazon EMR on EKS. You can create, describe, list and delete virtual clusters. They do not consume any additional resource in your system. A single virtual cluster maps to a single Kubernetes namespace. Given this relationship, you can model virtual clusters the same way you model Kubernetes namespaces to meet your requirements.

Optional Parameters

Optional parameters can be passed as a params::Dict{String,<:Any}. Valid keys are:

  • "containerProviderId": The container provider ID of the virtual cluster.
  • "containerProviderType": The container provider type of the virtual cluster. Amazon EKS is the only supported type as of now.
  • "createdAfter": The date and time after which the virtual clusters are created.
  • "createdBefore": The date and time before which the virtual clusters are created.
  • "maxResults": The maximum number of virtual clusters that can be listed.
  • "nextToken": The token for the next set of virtual clusters to return.
  • "states": The states of the requested virtual clusters.
source
Main.Emr_Containers.start_job_runMethod
start_job_run(client_token, virtual_cluster_id)
start_job_run(client_token, virtual_cluster_id, params::Dict{String,<:Any})

Starts a job run. A job run is a unit of work, such as a Spark jar, PySpark script, or SparkSQL query, that you submit to Amazon EMR on EKS.

Arguments

  • client_token: The client idempotency token of the job run request.
  • virtual_cluster_id: The virtual cluster ID for which the job run request is submitted.

Optional Parameters

Optional parameters can be passed as a params::Dict{String,<:Any}. Valid keys are:

  • "configurationOverrides": The configuration overrides for the job run.
  • "executionRoleArn": The execution role ARN for the job run.
  • "jobDriver": The job driver for the job run.
  • "jobTemplateId": The job template ID to be used to start the job run.
  • "jobTemplateParameters": The values of job template parameters to start a job run.
  • "name": The name of the job run.
  • "releaseLabel": The Amazon EMR release version to use for the job run.
  • "retryPolicyConfiguration": The retry policy configuration for the job run.
  • "tags": The tags assigned to job runs.
source
Main.Emr_Containers.tag_resourceMethod
tag_resource(resource_arn, tags)
tag_resource(resource_arn, tags, params::Dict{String,<:Any})

Assigns tags to resources. A tag is a label that you assign to an Amazon Web Services resource. Each tag consists of a key and an optional value, both of which you define. Tags enable you to categorize your Amazon Web Services resources by attributes such as purpose, owner, or environment. When you have many resources of the same type, you can quickly identify a specific resource based on the tags you've assigned to it. For example, you can define a set of tags for your Amazon EMR on EKS clusters to help you track each cluster's owner and stack level. We recommend that you devise a consistent set of tag keys for each resource type. You can then search and filter the resources based on the tags that you add.

Arguments

  • resource_arn: The ARN of resources.
  • tags: The tags assigned to resources.
source
Main.Emr_Containers.untag_resourceMethod
untag_resource(resource_arn, tag_keys)
untag_resource(resource_arn, tag_keys, params::Dict{String,<:Any})

Removes tags from resources.

Arguments

  • resource_arn: The ARN of resources.
  • tag_keys: The tag keys of the resources.
source