Search S3 Buckets

These endpoints allow you to communicate with Immuta the same way you would with S3, making Immuta easy to integrate with tools you may already be using to work with S3. In this integration, Immuta implements a single bucket (with data sources broken up as sub-directories under that bucket), since some S3 tools only support the new virtual-hosted style requests.

The endpoints outlined below support basic AWS functionality; the requests and responses for each are identical to those in S3.

Additional fields may be included in some responses you receive; however, these attributes are for internal purposes and are therefore undocumented.

S3 integration workflow

1. List the bucket all data sources are grouped under.

2. List the contents of a specific bucket.

3. List the contents of a specific blob.

List the bucket all data sources are grouped under

GET /s3p

Return constant bucket that all data sources are grouped under.

Response Parameters

Attribute	Description
body	string Lists the bucket, organization, and owner of the bucket.

Request example

The following request returns the bucket all data sources are grouped under for the organization.

curl \
    --request GET \
    --header "Authorization: AWS <API KEY>:immuta" \
    https://demo.immuta.com/s3p

Response example

The following response lists the bucket immuta.

<?xml version="1.0" encoding="UTF-8"?>
<ListAllMyBucketsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
    <Owner>
        <ID>Immuta</ID>
        <DisplayName>Immuta Inc</DisplayName>
    </Owner>
    <Buckets>
        <Bucket>
            <Name>immuta</Name>
            <CreationDate>2006-03-19T00:00:00.000Z</CreationDate>
        </Bucket>
    </Buckets>
</ListAllMyBucketsResult>

List the contents of a specific bucket

GET /s3p/{bucket}

Return the contents of a given bucket (data sources or data source blobs).

Query Parameters

Attribute	Description	Required
bucket	string Name of the bucket (this should match the bucket name in configuration).	Yes
prefix	string The prefix to start a search under.	No
max-keys	number Max number of content objects to return.	No
delimiter	string The terminating character for a search.	No
marker	string When provided, only prefixes after this marker will be returned.	No
location	string If provided, will instead return the bucket location.	No
encoding-type	string Encoding type for content keys.	No
list-type	integer If set to 2, will use v2 of the API response.	No
fetch-owner	boolean When true, will return owner field in v2 API response.	No
start-after	string Only prefixes after this marker will be returned. (This is the v2 API version of marker).	No

Response Parameters**

Attribute	Description
body	string Lists the data sources and/or data source blobs in the bucket.

Request example

The following request returns a list of data sources in the bucket immuta.

curl \
    --request GET \
    --header "Authorization: AWS <API KEY>:immuta" \
    https://demo.immuta.com/s3p/immuta

Response example

The following response lists the data source blob immuta.

<?xml version="1.0" encoding="UTF-8"?>
<ListBucketResult xmlns="http://doc.s3.amazonaws.com/2006-03-01/">
    <IsTruncated>false</IsTruncated>
    <Marker></Marker>
    <Name>immuta</Name>
    <Prefix>/</Prefix>
    <MaxKeys>1000</MaxKeys>
    <Delimiter></Delimiter>
</ListBucketResult>

List the contents of a specific blob

GET /s3p/{bucket}/{dataSource}/{key}

Return a specific blob.

Query Parameters

Attribute	Description	Required
bucket	string Name of the bucket (this should match the bucket name in configuration).	Yes
dataSource	string Immuta S3 Folder to search in.	Yes
key	string Key path to search for the specified file.	Yes

Response Parameters

Attribute	Description
body	string Lists the data sources and/or data source blobs in the bucket.

Request example

The following request returns the contents of the single file that exists in the requested directory in immuta.

curl \
    --request GET \
    --header "Authorization: AWS <API KEY>:immuta" \
    https://demo.immuta.com/s3p/immuta/my_data_source/path/to/file/myfile.json

Response example

The following response lists the size, storage class, owner, and the last date of modification of the single file.

<?xml version="1.0" encoding="UTF-8"?>
<ListBucketResult xmlns="http://doc.s3.amazonaws.com/2006-03-01/">
    <IsTruncated>false</IsTruncated>
    <Marker></Marker>
    <Name>immuta</Name>
    <Prefix>my_data_source/path/to/file</Prefix>
    <MaxKeys>1000</MaxKeys>
    <Delimiter>/</Delimiter>
    <Contents>
        <Key>my_data_source/path/to/file/myfile.json</Key>
        <LastModified>2018-11-05T21:25:04.000Z</LastModified>
        <ETag>5b0810c82a69a70e552cece19b20585fc94b67fe4eaa8b</ETag>
        <Size>389</Size>
        <StorageClass>STANDARD</StorageClass>
        <Owner>
            <ID>Immuta</ID>
            <DisplayName>Immuta</DisplayName>
        </Owner>
    </Contents>
</ListBucketResult>