Create and Manage an Amazon S3 Data Source

This page describes the native-s3 (Amazon S3 data sources) endpoint.

Private preview: The Amazon S3 integration is available to select accounts. Reach out to your Immuta representative for details.

Additional fields may be included in some responses you receive; however, these attributes are for internal purposes and are therefore undocumented.

Amazon S3 workflow

1. Create an Amazon S3 data source.

2. Search Amazon S3 data sources.

3. Delete Amazon S3 data sources.

Create a data source

POST /native-s3/handler

Save the provided connection information as a data source.

Payload parameters

The table below outlines the parameters for creating an S3 data source.

Parameter	Description	Required or optional	Accepted values
type string	The type of integration.	Required	Native S3
integrationId number	The unique identifier of the S3 integration.	Required	-
dataSources.dataSourceName string	The name of the S3 data source you want to create.	Required	-
dataSources.prefix string	The S3 prefix that creates a data source for the prefix, bucket, or object provided in the path.	Required	-

Response schema

The table below outlines the response schema for successful requests.

Property	Description
dataSourceId integer	The unique identifier of the data source.
prefix string	The S3 path of the prefix, bucket, or object used to create the data source.
dataSourceName string	The name of the data source.

Request example

The following request saves the provided connection information as a data source.

curl -X 'POST' \
    'https://<your-immuta-url.com>/native-s3/handler' \
    -H 'accept: application/json' \
    -H 'Content-Type: application/json' \
    -H 'Authorization: <your-api-key>' \
    -d '{
    "type": "Native S3",
    "integrationId": <id-of-your-s3-integration>,
    "dataSources": [
      {
        "dataSourceName": "<name-of-data-source>",
        "prefix": "</data-source-prefix>"
      }
    ]
    }'

Response example

The response returns the ID, name, and prefix of the data source. See the response schema above for details about the response schema. An unsuccessful response returns the status code and an error message.

{
  "dataSourceId": 1,
  "prefix": "research-data-source",
  "dataSourceName": "s3://research-data-source (64*****50499/us-east-2)"
}

Search Amazon S3 data sources

GET /native-s3/handler/{handlerId}

Get the handler metadata associated with the provided handler ID.

Query parameter

Attribute	Description	Required or optional
handlerId integer	The specific handler ID.	Required

Response schema

Attribute	Description
id integer	The data source ID.
metadata.prefix string	The S3 path of the prefix, bucket, or object used to create the data source.
metadata.integrationId integer	The unique identifier of the integration.
metadata.dataSourceName string	The name of the data source.
type string	The handler type (nativeS3Handler).
awsLocationPath string	The base S3 location prefix that Immuta used to register the S3 data source.

Request example

The following request returns the handler metadata associated with the provided handler ID.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/native-s3/handler/1

Response example

{
  "id": 1,
  "metadata": {
    "prefix": "research-data-source",
    "integrationId": 2,
    "dataSourceName": "s3://research-data-source (64*****50499/us-east-2)"
  },
  "type": "nativeS3Handler",
  "awsLocationPath": "s3://"
}

Delete an Amazon S3 data source

DELETE /dataSource/{dataSourceId}

Delete a data source. This will perform a soft delete on the first call and a hard delete the second time.

Query parameters

Attribute	Description	Required
dataSourceId	integer The data source ID.	Yes

Response schema

Attribute	Description
success boolean	If true, the request to disable or delete the data source was successful.
id integer	The data source ID.
schemaEvolutionId integer	The schema evolution ID.
name string	The data source name.
disabled boolean	If true, the data source is disabled.
handlerDeleteErrorMessage string	The delete error message.

Request example

The following request disables the data source 1.

curl \
    --request DELETE \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/dataSource/1

Response example

{
  "success": true,
  "id": 1,
  "schemaEvolutionId": 1,
  "name": "s3://research-data-source (64*****50499/us-east-2)",
  "disabled": true,
  "handlerDeleteErrorMessage": null
}