1 of 11

Connect Data

Audience: Data Owners
Content Summary: This section of API documentation is specific to connecting your data to Immuta and managing Data Dictionaries.

Data sources can also be created and managed using the V2 API.

Section Contents

Azure blob storage API reference guide: Create an Azure blob storage data source.
Azure Synapse Analytics API reference guide: Create an Azure Synapse Analytics data source.
Databricks API reference guide: Create a Databricks data source.
Redshift API reference guide: Create a Redshift data source.
Snowflake API reference guide: Create a Snowflake data source.
Data dictionary API reference guide: Manage the data dictionary.

Create Data Sources

Azure Blob Storage API Reference Guide

The azureblob endpoint allows you to connect and manage Azure Blob Storage data sources in Immuta.

Additional fields may be included in some responses you receive; however, these attributes are for internal purposes and are therefore undocumented.

Azure Blob workflow

Create a data source

POST /azureblob/handler

Save the provided connection for an Azure Blob Storage data source.

Payload parameters

Attribute

Description

Required

Response parameters

Request example

The following request saves the provided connection information (in example-payload.json) as a data source.

Request payload example

Response example

Get information about a data source

GET /azureblob/handler/{handlerId}

Return the handler metadata associated with the provided handler ID.

Query parameters

Response parameters

Request example

The following request returns the handler metadata associated with the provided handler ID.

Response example

Manage data sources

Update a specific data source

PUT /azureblob/handler/{handlerId}

Update the provided information for an Azure Blob Storage data source.

Query parameters

Response parameters

Request example

The following request with the payload below updates the metadata for the data source with the handler ID 18.

Payload example

Response example

Update multiple data sources

PUT /azureblob/bulk

Update the data source metadata associated with the provided connection string.

Payload parameters

Response parameters

Request example

The following request updates the autoIngest value to true for data sources with the connection string specified in the payload below.

Payload example

Response example

Re-crawl the data source

PUT /azureblob/handler/{handlerId}/crawl

Re-crawls the data source and updates the metadata.

Query parameters

Response parameters

The response returns a string of characters that identify the job run.

Request example

The following request re-crawls the data source.

Response example

Azure Data Lake Storage Gen2 API Reference Guide

**Deprecation notice**: Support for this database has been deprecated.

The adls-gen2 endpoint allows you to connect and manage Azure Data Lake Storage Gen2 data sources in Immuta.

Additional fields may be included in some responses you receive; however, these attributes are for internal purposes and are therefore undocumented.

Create a data source

**Duplicate data sources**: In order to avoid two data sources referencing the same table, users can not create duplicate data sources. If you attempt to create a duplicate data source using the API, you will encounter a warning stating "duplicate tables are specified in the payload."

POST /adls-gen2/handler

Save the provided connection information as a data source.

Payload Parameters

Attribute

Description

Required

Response Parameters

Request example

This request creates two Azure Data Lake Storage Gen2 data sources.

curl \
    --request POST \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://demo.immuta.com/adls-gen2/handler

Payload example

{
  "handler": {
    "metadata": {
      "tagAttributes": [],
      "eventTimeAttribute": "",
      "useDirectoryForTags": false,
      "sasToken": "your-sas-token",
      "sasTokenUrl": "your-sas-token-url",
      "authenticationMethod": "sasToken",
      "fileSystem": "bens-test-data"
    }
  },
  "dataSource": {
    "blobHandler": {
      "scheme": "https",
      "url": ""
    },
    "expiration": "2021-11-19T04:59:59.999Z",
    "blobHandlerType": "Azure Data Lake Storage Gen2",
    "recordFormat": "",
    "type": "ingested",
    "name": "Test-Data",
    "sqlTableName": "test_data"
  }
}

Response example

{
  "id": 17,
  "dataSourceId": 17
}

Get information about a data source

GET /adls-gen2/handler/{handlerId}

Get the handler metadata associated with the provided handler ID.

Query Parameters

Response Parameters

Request example

This request returns metadata for the handler with the ID 17.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://demo.immuta.com/adls-gen2/handler/17

Response example

{
  "dataSourceId": 17,
  "metadata": {
    "fileSystem": "bens-test-data",
    "accountName": "your-account-name",
    "sasTokenUrl": "https://your.blob.core.windows.net/",
    "ingestUserId": "azure data lake storage gen2_indexer_17",
    "tagAttributes": [],
    "dataSourceName": "Test-Data",
    "refreshInterval": 0,
    "eventTimeAttribute": "",
    "useDirectoryForTags": false,
    "authenticationMethod": "sasToken"
  },
  "type": "azureDataLakeStorageGen2Handler",
  "connectionString": "your-connection-string",
  "remoteTableDescription": null,
  "id": 17,
  "createdAt": "2021-10-19T16:41:16.161Z",
  "updatedAt": "2021-10-19T16:41:16.390Z"
}

Update a specific data source

PUT /adls-gen2/handler/{handlerId}

Update the data source metadata associated with the provided handler ID. This endpoint does not perform partial updates, but will allow the dictionary to be omitted. In this case, it uses the current dictionary.

Query Parameters

Payload Parameters

Response Parameters

Request example

This request updates the metadata for the data source with the handler ID 17.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://demo.immuta.com/adls-gen2/handler/17

Payload example

The payload below updates the dataSourceName to Customer Details.

{
  "dataSourceId": 17,
  "metadata": {
    "fileSystem": "bens-test-data",
    "accountName": "your-account-name",
    "sasTokenUrl": "https://your.blob.core.windows.net/",
    "ingestUserId": "azure data lake storage gen2_indexer_17",
    "tagAttributes": [],
    "dataSourceName": "Customer Details",
    "refreshInterval": 0,
    "eventTimeAttribute": "",
    "useDirectoryForTags": false,
    "authenticationMethod": "sasToken"
  },
  "type": "azureDataLakeStorageGen2Handler",
  "connectionString": "your-connection-string",
  "remoteTableDescription": null,
  "id": 17,
  "createdAt": "2021-10-19T16:41:16.161Z",
  "updatedAt": "2021-10-19T16:41:16.390Z"
}

Response example

{
  "id": 17,
  "ca": ["-----BEGIN CERTIFICATE-----\ncertificatedata\n-----END CERTIFICATE-----"]
}

Azure Synapse Analytics API Reference Guide

This page describes the asa (Azure Synapse Analytics data sources) endpoint.

Additional fields may be included in some responses you receive; however, these attributes are for internal purposes and are therefore undocumented.

ASA workflow

Create a data source

POST /asa/handler

Save the provided connection information as a data source.

Payload parameters

Attribute

Description

Required

Response parameters

Request example

The following request saves the provided connection information as a data source.

Request example payload

Response example

Search ASA data sources

Search for handler metadata

GET /asa/handler/{handlerId}

Get the handler metadata associated with the provided handler ID.

Query parameters

Response parameters

Request example

The following request returns the handler metadata associated with the provided handler ID.

Response example

Update ASA data sources

Update handler metadata

PUT /asa/handler/{handlerId}

Updates the handler metadata associated with the provided handler ID. This endpoint does not perform partial updates, but will allow the dictionary to be omitted. In this case it uses the current dictionary.

Query parameters

Response parameters

Request example

The following request updates the handler metadata (saved in example_payload.json) associated with the provided handler ID.

Request payload example

Response example

Update multiple data sources

PUT /asa/bulk

Updates the data source metadata associated with the provided connection string.

Query parameters

Response parameters

Request example

The following request updates the handler metadata for the handler ID specified in example_payload.json.

Request payload example

Response example

Recalculate high cardinality columns

PUT /asa/handler/{handlerId}/triggerHighCardinalityJob

Recalculates the high cardinality column for the provided handler ID.

Query parameters

Response parameters

The response returns a string of characters that identify the high cardinality job run.

Request example

The following request recalculates the high cardinality column for the provided handler ID.

Response example

Refresh a native view

PUT /asa/handler/{handlerId}/refreshNativeViewJob

Refresh the native view of a data source.

Query parameters

Response parameters

The response returns a string of characters that identifies the refresh view job run.

Request example

This request refreshes the view for the data source with the handler ID 7.

Response example

Databricks API Reference Guide

The databricks endpoint allows you to connect and manage Databricks data sources in Immuta.

Additional fields may be included in some responses you receive; however, these attributes are for internal purposes and are therefore undocumented.

Requirements

Databricks Spark integration

When exposing a table or view from an Immuta-enabled Databricks cluster, be sure that at least one of these traits is true:

The user exposing the tables has READ_METADATA and SELECT permissions on the target views/tables (specifically if Table ACLs are enabled).
The user exposing the tables is listed in the immuta.spark.acl.whitelist configuration on the target cluster.
The user exposing the tables is a Databricks workspace administrator.

Databricks Unity Catalog integration

When exposing a table from Databricks Unity Catalog, be sure the credentials used to register the data sources have the Databricks privileges listed below.

The following privileges on the parent catalogs and schemas of those tables:
- SELECT
- USE CATALOG
- USE SCHEMA
USE SCHEMA on system.information_schema

Azure Databricks Unity Catalog limitation

Set all table-level ownership on your Unity Catalog data sources to an individual user or service principal instead of a Databricks group before proceeding. Otherwise, Immuta cannot apply data policies to the table in Unity Catalog. See the for details.

Databricks workflow

Create a data source

POST /databricks/handler

Save the provided connection information as a data source.

Payload parameters

Response parameters

Request example

This request creates two Databricks data sources.

Payload example

Response example

Get information about a data source

GET /databricks/handler/{handlerId}

Get the handler metadata associated with the provided handler ID.

Query parameters

Response parameters

Request example

This request returns metadata for the handler with the ID 48.

Response example

Manage data sources

Update a specific data source

PUT /databricks/handler/{handlerId}

Query parameters

Payload parameters

Response parameters

Request example

This request updates the metadata for the data source with the handler ID 48.

Payload example

The payload below updates the dataSourceName to Cities.

Response example

Update multiple data sources

PUT /databricks/bulk

Update the data source metadata associated with the provided connection string.

Payload parameters

Response parameters

Request example

This request updates the metadata for all data sources with the connection string specified in example-payload.json.

Payload example

The payload below adds a certificate (certificate.json) to connect to the data sources with the provided connection.

Response example

Recalculate the high cardinality column for a data source

PUT /databricks/handler/{handlerId}/triggerHighCardinalityJob

Recalculate the high cardinality column for the specified data source.

Query parameters

Response parameters

The response returns a string of characters that identify the high cardinality job run.

Request example

This request re-runs the job that calculates the high cardinality column for the data source with the handler ID 47.

Response example

Presto API Reference Guide

This page describes the presto (Presto data sources) endpoint.

Additional fields may be included in some responses you receive; however, these attributes are for internal purposes and are therefore undocumented.

Presto workflow

Create a data source

POST /presto/handler

Save the provided connection information as a data source.

Payload parameters

Attribute

Description

Required

Response parameters

Request example

This request creates a Presto data source.

Payload example

Response example

Search Presto data sources

Search for handler metadata

GET /presto/handler/{handlerId}

Get the handler metadata associated with the provided handler ID.

Query parameters

Response parameters

Request example

The following request returns the handler metadata associated with the provided handler ID.

Response example

Update Presto data sources

Update handler metadata

PUT /presto/handler/{handlerId}

Query parameters

Response parameters

Request example

This request updates the data source name to Marketing Data for the data source with the handler ID 67.

Request payload example

Response example

Update multiple data sources

PUT /presto/bulk

Updates the data source metadata associated with the provided connection string.

Query parameters

Response parameters

Request example

This request updates the metadata for all data sources with the connection string specified in example-payload.json.

Payload example

The payload below adds a certificate (certificate.json) to the data sources with the provided connection.

Response example

Recalculate high cardinality columns

PUT /presto/handler/{handlerId}/triggerHighCardinalityJob

Recalculates the high cardinality column for the provided handler ID.

Query parameters

Response parameters

The response returns a string of characters that identify the high cardinality job run.

Request example

The following request recalculates the high cardinality column for the provided handler ID.

Response example

Snowflake API Reference Guide

The snowflake endpoint allows you to connect and manage Snowflake data sources in Immuta.

Additional fields may be included in some responses you receive; however, these attributes are for internal purposes and are therefore undocumented.

Snowflake workflow

Snowflake imported databases

Immuta does not support Snowflake tables from imported databases. Instead, create a view of the table and register that view as a data source.

Create a data source.
Get information about a data source.
Manage data sources.

Create a data source

POST /snowflake/handler

Save the provided connection information as a data source.

Payload parameters

Attribute

Description

Required

Response parameters

Request example

This request creates a Snowflake data source.

curl \
    --request POST \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://demo.immuta.com/snowflake/handler

Payload example

{
  "handler": {
    "metadata": {
      "ssl": true,
      "userFiles": [],
      "authenticationMethod": "userPassword",
      "username": "user",
      "password": "yourpassword",
      "port": 443,
      "hostname": "demo.us-east-1.snowflakecomputing.com",
      "warehouse": "YOUR_WH",
      "database": "ANALYTICS",
      "schema": "TEST",
      "table": "CUSTOMERS",
      "nativeViewName": "customers_immuta",
      "nativeSchemaName": "test_immuta",
      "nativeWorkspaceName": "immuta_analytics",
      "schemaProjectName": "Test",
      "bodataSchemaName": "test",
      "columns": [{
        "name": "CUSTOMER_ID",
        "dataType": "numeric(38,0)",
        "remoteType": "number(38,0)",
        "isPrimaryKey": false,
        "nullable": true
      }, {
        "name": "FIRST_NAME",
        "dataType": "text",
        "remoteType": "varchar(16777216)",
        "isPrimaryKey": false,
        "nullable": true
      }, {
        "name": "LAST_NAME",
        "dataType": "text",
        "remoteType": "varchar(16777216)",
        "isPrimaryKey": false,
        "nullable": true
      }, {
        "name": "FIRST_ORDER",
        "dataType": "date",
        "remoteType": "date",
        "isPrimaryKey": false,
        "nullable": true
      }, {
        "name": "MOST_RECENT_ORDER",
        "dataType": "date",
        "remoteType": "date",
        "isPrimaryKey": false,
        "nullable": true
      }, {
        "name": "NUMBER_OF_ORDERS",
        "dataType": "numeric(18,0)",
        "remoteType": "number(18,0)",
        "isPrimaryKey": false,
        "nullable": true
      }, {
        "name": "TOTAL_ORDER_AMOUNT",
        "dataType": "numeric",
        "remoteType": "number(38,6)",
        "isPrimaryKey": false,
        "nullable": true
      }],
      "eventTime": "FIRST_ORDER",
      "staleDataTolerance": 86400,
      "bodataTableName": "customers",
      "dataSourceName": "Customers"
    }
  },
  "dataSource": {
    "blobHandler": {
      "scheme": "https",
      "url": ""
    },
    "blobHandlerType": "Snowflake",
    "recordFormat": "",
    "type": "queryable",
    "schemaEvolutionId": null,
    "columnEvolutionEnabled": true,
    "name": "Customers",
    "sqlTableName": "customers"
  },
  "schemaEvolution": {
    "ownerProfileId": 2,
    "config": {
      "nameTemplate": {
        "nameFormat": "<Schema> <Tablename>",
        "tableFormat": "<tablename>",
        "sqlSchemaNameFormat": "<schema>",
        "schemaProjectNameFormat": "<Schema>"
      }
    },
    "schemas": []
  }
}

Response example

{
  "id": 30,
  "dataSourceId": 30,
  "warnings": []
}

Get information about a data source

GET /snowflake/handler/{handlerId}

Get the handler metadata associated with the provided handler ID.

Query parameters

Response parameters

Request example

This request returns metadata for the handler with the ID 30.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://demo.immuta.com/snowflake/handler/30

Response example

{
  "dataSourceId": 30,
  "metadata": {
    "ssl": true,
    "port": 443,
    "query": null,
    "table": "CUSTOMERS",
    "schema": "TEST",
    "database": "ANALYTICS",
    "hostname": "demo.us-east-1.snowflakecomputing.com",
    "username": "user",
    "eventTime": "FIRST_ORDER",
    "userFiles": [],
    "warehouse": "YOUR_WH",
    "dataSourceName": "Customers",
    "bodataTableName": "customers",
    "highCardinality": "CUSTOMER_ID",
    "bodataSchemaName": "test",
    "columnsNormalized": true,
    "schemaProjectName": "Test",
    "staleDataTolerance": 86400,
    "authenticationMethod": "userPassword"
  },
  "type": "odbcHandler",
  "connectionString": "user@demo.us-east-1.snowflakecomputing.com:443/ANALYTICS",
  "id": 30,
  "createdAt": "2021-10-01T16:19:23.562Z",
  "updatedAt": "2021-10-01T16:19:28.556Z",
  "dbms": {
    "name": "snowflake"
  }
}

Manage data sources

Update a specific data source

PUT /snowflake/handler/{handlerId}

Query parameters

Payload parameters

Response parameters

Request example

This request updates the metadata for the data source with the handler ID 30.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://demo.immuta.com/snowflake/handler/30

Payload example

The payload below updates the eventTime to MOST_RECENT_ORDER.

{
  "handler": {
    "policyHandler": null,
    "dataSourceId": 30,
    "metadata": {
      "ssl": true,
      "port": 443,
      "table": "CUSTOMERS",
      "schema": "TEST",
      "database": "ANALYTICS",
      "hostname": "demo.us-east-1.snowflakecomputing.com",
      "username": "user",
      "eventTime": "MOST_RECENT_ORDER",
      "userFiles": [],
      "warehouse": "YOUR_WH",
      "dataSourceName": "Customers",
      "bodataTableName": "customers",
      "highCardinality": "CUSTOMER_ID",
      "bodataSchemaName": "test",
      "columnsNormalized": true,
      "schemaProjectName": "Test",
      "staleDataTolerance": 86400,
      "authenticationMethod": "userPassword",
      "columns": [{
        "name": "CUSTOMER_ID",
        "dataType": "numeric(38,0)",
        "nullable": true,
        "remoteType": "number(38,0)",
        "isPrimaryKey": false,
        "remoteColumn": "CUSTOMER_ID"
      }, {
        "name": "FIRST_NAME",
        "dataType": "text",
        "nullable": true,
        "remoteType": "varchar(16777216)",
        "isPrimaryKey": false,
        "remoteColumn": "FIRST_NAME"
      }, {
        "name": "LAST_NAME",
        "dataType": "text",
        "nullable": true,
        "remoteType": "varchar(16777216)",
        "isPrimaryKey": false,
        "remoteColumn": "LAST_NAME"
      }, {
        "name": "FIRST_ORDER",
        "dataType": "date",
        "nullable": true,
        "remoteType": "date",
        "isPrimaryKey": false,
        "remoteColumn": "FIRST_ORDER"
      }, {
        "name": "MOST_RECENT_ORDER",
        "dataType": "date",
        "nullable": true,
        "remoteType": "date",
        "isPrimaryKey": false,
        "remoteColumn": "MOST_RECENT_ORDER"
      }, {
        "name": "NUMBER_OF_ORDERS",
        "dataType": "numeric(18,0)",
        "nullable": true,
        "remoteType": "number(18,0)",
        "isPrimaryKey": false,
        "remoteColumn": "NUMBER_OF_ORDERS"
      }, {
        "name": "TOTAL_ORDER_AMOUNT",
        "dataType": "numeric",
        "nullable": true,
        "remoteType": "number(38,6)",
        "isPrimaryKey": false,
        "remoteColumn": "TOTAL_ORDER_AMOUNT"
      }]
    },
    "type": "odbcHandler",
    "connectionString": "user@demo.us-east-1.snowflakecomputing.com:443/ANALYTICS",
    "id": 30,
    "createdAt": "2021-10-01T16:19:23.562Z",
    "updatedAt": "2021-10-01T17:19:35.853Z",
    "dbms": {
      "name": "snowflake"
    }
  }
}

Response example

{
  "id": 30,
  "ca": ["-----BEGIN CERTIFICATE-----\ncertificatedata\n-----END CERTIFICATE-----"],
  "metadata": {
    "columns": [{
      "name": "customer_id",
      "dataType": "numeric(38,0)",
      "remoteType": "number(38,0)",
      "isPrimaryKey": false,
      "nullable": true,
      "remoteColumn": "CUSTOMER_ID"
    }, {
      "name": "first_name",
      "dataType": "text",
      "remoteType": "varchar(16777216)",
      "isPrimaryKey": false,
      "nullable": true,
      "remoteColumn": "FIRST_NAME"
    }, {
      "name": "last_name",
      "dataType": "text",
      "remoteType": "varchar(16777216)",
      "isPrimaryKey": false,
      "nullable": true,
      "remoteColumn": "LAST_NAME"
    }, {
      "name": "first_order",
      "dataType": "date",
      "remoteType": "date",
      "isPrimaryKey": false,
      "nullable": true,
      "remoteColumn": "FIRST_ORDER"
    }, {
      "name": "most_recent_order",
      "dataType": "date",
      "remoteType": "date",
      "isPrimaryKey": false,
      "nullable": true,
      "remoteColumn": "MOST_RECENT_ORDER"
    }, {
      "name": "number_of_orders",
      "dataType": "numeric(18,0)",
      "remoteType": "number(18,0)",
      "isPrimaryKey": false,
      "nullable": true,
      "remoteColumn": "NUMBER_OF_ORDERS"
    }, {
      "name": "total_order_amount",
      "dataType": "numeric",
      "remoteType": "number(38,6)",
      "isPrimaryKey": false,
      "nullable": true,
      "remoteColumn": "TOTAL_ORDER_AMOUNT"
    }]
  }
}

Update multiple data sources

PUT /snowflake/bulk

Update the data source metadata associated with the provided connection string.

Payload parameters

Response parameters

Request example

This request updates the metadata for all data sources with the connection string specified in example-payload.json.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://demo.immuta.com/snowflake/bulk

Payload example

The payload below updates the database to ANALYST_DEMO for the provided connection string.

{
  "handler": {
    "metadata": {
      "ssl": true,
      "port": 443,
      "database": "ANALYST_DEMO",
      "hostname": "demo.us-east-1.snowflakecomputing.com",
      "username": "yourusername",
      "userFiles": [],
      "warehouse": "YOUR_WH",
      "authenticationMethod": "userPassword",
      "password": "yourpassword"
    }
  },
  "connectionString": "demo.us-east-1.snowflakecomputing.com:443/ANALYST_DEMO"
}

Response example

{
  "bulkId": "bulk_ds_update_54ada6bhashedvaluea0c80cecd9d62",
  "jobsCreated": 5
}

Recalculate the high cardinality column for a data source

PUT /snowflake/handler/{handlerId}/triggerHighCardinalityJob

Recalculate the high cardinality column for the specified data source.

Query parameters

Response parameters

The response returns a string of characters that identify the high cardinality job run.

Request example

This request re-runs the job that calculates the high cardinality column for the data source with the handler ID 30.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://demo.immuta.com/snowflake/handler/30/triggerHighCardinalityJob

Response example

c12fd320-22d8-11ec-b2b8-874838eeef05

Refresh a native view

PUT /snowflake/handler/{handlerId}/refreshNativeViewJob

Refresh the native view of a data source.

Query parameters

Response parameters

The response returns a string of characters that identifies the refresh view job run.

Request example

This request refreshes the view for the data source with the handler ID 7.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://demo.immuta.com/snowflake/handler/7/refreshNativeViewJob

Response example

53c256d0-eb57-11ec-b275-d95a8e998142

Data Dictionary API Reference Guide

The data dictionary API allows you to manage the data dictionary for your data sources.

Additional fields may be included in some responses you receive; however, these attributes are for internal purposes and are therefore undocumented.

Data dictionary workflow

Manage the dictionary for a data source.
Search dictionaries.
Delete a dictionary for a data source.

Manage data dictionaries

Method

Path

Purpose

Create a data dictionary

POST /dictionary/{dataSourceId}

Create the dictionary for the specified data source.

Query parameters

Attribute

Description

Required

Payload parameters

Response parameters

Request example

The following request creates a data dictionary (saved in example-payload.json) for the data source with ID 1.

curl \
    --request POST \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --header "Content-Type: application/json" \
    --data @example-payload.json \
    https://demo.immuta.com/dictionary/1

Payload example

{
  "metadata": [
    {
      "name": "notificationType",
      "dataType": "text",
      "remoteType": "text"
    },
    {
      "name": "actionBy",
      "dataType": "text",
      "remoteType": "integer"
    },
    {
      "name": "targetUser",
      "dataType": "integer",
      "remoteType": "integer"
    },
    {
      "name": "metadata",
      "dataType": "json",
      "remoteType": "json"
    },
    {
      "name": "id",
      "dataType": "integer",
      "remoteType": "integer"
    },
    {
      "name": "notifyInitiator",
      "dataType": "text",
      "remoteType": "boolean"
    },
    {
      "name": "eventTime",
      "dataType": "timestamp with time zone",
      "remoteType": "timestamp with time zone"
    }
  ]
}

Response example

{
  "createdAt": "2018-03-21T10:52:30.535Z",
  "dataSource": 1,
  "id": 1,
  "metadata": [
    {
      "name": "notificationType",
      "dataType": "text",
      "remoteType": "text"
    },
    {
      "name": "actionBy",
      "dataType": "text",
      "remoteType": "integer"
    },
    {
      "name": "targetUser",
      "dataType": "integer",
      "remoteType": "integer"
    },
    {
      "name": "metadata",
      "dataType": "json",
      "remoteType": "json"
    },
    {
      "name": "id",
      "dataType": "integer",
      "remoteType": "integer"
    },
    {
      "name": "notifyInitiator",
      "dataType": "text",
      "remoteType": "boolean"
    },
    {
      "name": "eventTime",
      "dataType": "timestamp with time zone",
      "remoteType": "timestamp with time zone"
    }
  ],
  "types": [
    "text",
    "integer",
    "json",
    "timestamp with time zone"
  ],
  "updatedAt": "2018-03-21T12:18:25.531Z"
}

Other status codes returned include:

Update a data dictionary

PUT /dictionary/{dataSourceId}

Update the dictionary for the specified data source.

Query parameters

Payload parameters

Response parameters

Request example

The request below updates the data dictionary for the data source with the ID 1.

curl \
    --request PUT \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --header "Content-Type: application/json" \
    --data @example-payload.json \
    https://demo.immuta.com/dictionary/1

Payload example

{
  "metadata": [
    {
      "name": "notificationType",
      "dataType": "text",
      "remoteType": "text"
    },
    {
      "name": "actionBy",
      "dataType": "text",
      "remoteType": "integer"
    },
    {
      "name": "targetUser",
      "dataType": "integer",
      "remoteType": "integer"
    },
    {
      "name": "metadata",
      "dataType": "json",
      "remoteType": "json"
    },
    {
      "name": "id",
      "dataType": "integer",
      "remoteType": "integer"
    },
    {
      "name": "notifyInitiator",
      "dataType": "text",
      "remoteType": "boolean"
    },
    {
      "name": "eventTime",
      "dataType": "timestamp with time zone",
      "remoteType": "timestamp with time zone"
    }
  ]
}

Response example

{
  "createdAt": "2018-03-21T10:52:30.535Z",
  "dataSource": 1,
  "id": 1,
  "metadata": [
    {
      "name": "notificationType",
      "dataType": "text",
      "remoteType": "text"
    },
    {
      "name": "actionBy",
      "dataType": "text",
      "remoteType": "integer"
    },
    {
      "name": "targetUser",
      "dataType": "integer",
      "remoteType": "integer"
    },
    {
      "name": "metadata",
      "dataType": "json",
      "remoteType": "json"
    },
    {
      "name": "id",
      "dataType": "integer",
      "remoteType": "integer"
    },
    {
      "name": "notifyInitiator",
      "dataType": "text",
      "remoteType": "boolean"
    },
    {
      "name": "eventTime",
      "dataType": "timestamp with time zone",
      "remoteType": "timestamp with time zone"
    }
  ],
  "types": [
    "text",
    "integer",
    "json",
    "timestamp with time zone"
  ],
  "updatedAt": "2018-03-21T12:18:25.531Z"
}

Other status codes returned include

Search data dictionaries

Get the dictionary for a specified data source

GET /dictionary/{dataSourceId}

Get the dictionary for the specified data source.

Query parameters

Response parameters

Request example

The request below gets the data dictionary for the data source with the ID 1.

curl \
    --request GET \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --header "Content-Type: application/json" \
    https://demo.immuta.com/dictionary/1

Response example

{
  "createdAt": "2018-03-21T10:52:30.535Z",
  "dataSource": 1,
  "id": 1,
  "metadata": [
    {
      "name": "notificationType",
      "dataType": "text",
      "remoteType": "text"
    },
    {
      "name": "actionBy",
      "dataType": "text",
      "remoteType": "integer"
    },
    {
      "name": "targetUser",
      "dataType": "integer",
      "remoteType": "integer"
    },
    {
      "name": "metadata",
      "dataType": "json",
      "remoteType": "json"
    },
    {
      "name": "id",
      "dataType": "integer",
      "remoteType": "integer"
    },
    {
      "name": "notifyInitiator",
      "dataType": "text",
      "remoteType": "boolean"
    },
    {
      "name": "eventTime",
      "dataType": "timestamp with time zone",
      "remoteType": "timestamp with time zone"
    }
  ],
  "types": [
    "text",
    "integer",
    "json",
    "timestamp with time zone"
  ],
  "updatedAt": "2018-03-21T12:18:25.531Z"
}

Search across all dictionary columns

GET /dictionary/columns

Search across all dictionary columns.

Query parameters

Response parameters

Request example

The following request searches for columns in all dictionaries that contain the text address in their name, with a limit of 10 results.

curl \
    --request GET \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --header "Content-Type: application/json" \
    https://demo.immuta.com/dictionary/columns?searchText=address&limit=10

Response example

[
  "address_city",
  "address_state",
  "address_street"
]

Delete a data dictionary

DELETE /dictionary/{dataSourceId}

Delete the data dictionary for the specified data source.

Query parameters

Request example

The request below deletes the data dictionary for the data source with ID 1.

curl \
    --request DELETE \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --header "Content-Type: application/json" \
    https://demo.immuta.com/dictionary/1

Response example

This endpoint returns {} on success.

Other status codes returned include

Azure Synapse Analytics API Reference Guide

This page describes the asa (Azure Synapse Analytics data sources) endpoint.

Additional fields may be included in some responses you receive; however, these attributes are for internal purposes and are therefore undocumented.

ASA workflow

Create a data source

POST /asa/handler

Save the provided connection information as a data source.

Payload parameters

Attribute

Description

Required

Response parameters

Attribute

Description

Request example

The following request saves the provided connection information as a data source.

curl \
    --request POST \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example_payload.json
    https://your-immuta-url.com/asa/handler

Request example payload

{
    "hits": [{
        "name": "Public Credit Accounts",
        "id": 1,
        "recordFormat": "Not Provided",
        "deleted": false,
        "description": null,
        "createdAt": "2021-09-09T14:12:09.511Z",
        "subscriptionPolicy": {
            "type": "subscription",
            "approvals": [{
                "requiredPermission": "OWNER",
                "specificApproverRequired": false
            }]
        },
        "schemaEvolutionId": 1,
        "recordCount": 0,
        "blobHandlerType": "Synapse",
        "subscriptionType": "approval",
        "sqlSchemaName": "public",
        "status": "failed",
        "subscriptionStatus": "owner",
        "connectionString": "your-username@your-dev-workspace.sql.azuresynapse.net:1433/public",
        "policy": "No Conflict",
        "policyHandlerType": "Builder",
        "native": null,
        "workspace": null
    }, {
        "name": "Public Credit Transactions",
        "id": 2,
        "recordFormat": "Not Provided",
        "deleted": false,
        "description": null,
        "createdAt": "2021-09-09T14:12:09.522Z",
        "subscriptionPolicy": {
            "type": "subscription",
            "approvals": [{
                "requiredPermission": "OWNER",
                "specificApproverRequired": false
            }]
        },
        "schemaEvolutionId": 1,
        "recordCount": 0,
        "blobHandlerType": "Synapse",
        "subscriptionType": "approval",
        "sqlSchemaName": "public",
        "status": "passed",
        "subscriptionStatus": "owner",
        "connectionString": "your-username@your-dev-workspace.sql.azuresynapse.net:1433/public",
        "policy": "No Conflict",
        "policyHandlerType": "Builder",
        "native": null,
        "workspace": null
    }, {
        "name": "Public Fake Medical Claims 2017",
        "id": 3,
        "recordFormat": "Not Provided",
        "deleted": false,
        "description": null,
        "createdAt": "2021-09-09T14:12:09.894Z",
        "subscriptionPolicy": null,
        "schemaEvolutionId": 1,
        "recordCount": 0,
        "blobHandlerType": "Synapse",
        "subscriptionType": "manual",
        "sqlSchemaName": "public",
        "status": "passed",
        "subscriptionStatus": "owner",
        "connectionString": "your-username@your-dev-workspace.sql.azuresynapse.net:1433/public",
        "policy": "No Conflict",
        "policyHandlerType": "Builder",
        "native": null,
        "workspace": null
    }, {
        "name": "Public Uciml Census Income Enriched",
        "id": 4,
        "recordFormat": "Not Provided",
        "deleted": false,
        "description": null,
        "createdAt": "2021-09-09T14:12:09.916Z",
        "subscriptionPolicy": null,
        "schemaEvolutionId": 1,
        "recordCount": 0,
        "blobHandlerType": "Synapse",
        "subscriptionType": "manual",
        "sqlSchemaName": "public",
        "status": "passed",
        "subscriptionStatus": "owner",
        "connectionString": "your-username@your-dev-workspace.sql.azuresynapse.net:1433/public",
        "policy": "None",
        "policyHandlerType": "None",
        "native": null,
        "workspace": null
    }],
    "facets": {
        "tags": [],
        "statuses": [{
            "name": "failed",
            "count": 0
        }, {
            "name": "passed",
            "count": 0
        }],
        "blobHandlerTypes": [{
            "name": "Synapse",
            "count": 0
        }],
        "editable": [{
            "name": "owned",
            "count": 4
        }, {
            "name": "expert",
            "count": 0
        }, {
            "name": "notSubscribed",
            "count": 0
        }, {
            "name": "disabled",
            "count": 0
        }]
    },
    "count": 4
}

Response example

{
  "connectionString": "your-username@your-dev-workspace.sql.azuresynapse.net:1433/public"
}

Search ASA data sources

Search for handler metadata

GET /asa/handler/{handlerId}

Get the handler metadata associated with the provided handler ID.

Query parameters

Attribute

Description

Required

Response parameters

Attribute

Description

Request example

The following request returns the handler metadata associated with the provided handler ID.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/asa/handler/67

Response example

{
  "dataSourceId": 67,
  "metadata": {
    "ssl": true,
    "port": 1433,
    "query": null,
    "table": "table_108",
    "schema": "elliott_dev",
    "database": "your_database",
    "hostname": "your-workspace.sql.azuresynapse.net",
    "username": "your-username",
    "eventTime": null,
    "dataSourceName": "Dev Table 108",
    "bodataTableName": "table_108",
    "disableClassify": false,
    "highCardinality": "col_1",
    "bodataSchemaName": "dev",
    "columnsNormalized": false,
    "schemaProjectName": "Dev",
    "staleDataTolerance": 2592000
  },
  "type": "odbcHandler",
  "connectionString": "your-username@your-workspace.sql.azuresynapse.net:1433/public",
  "id": 67,
  "createdAt": "2021-09-17T14:15:10.256Z",
  "updatedAt": "2021-09-17T14:15:19.547Z",
  "dbms": {
    "name": "asa"
  }
}

Update ASA data sources

Endpoint

Purpose

Update handler metadata

PUT /asa/handler/{handlerId}

Query parameters

Attribute

Description

Required

Response parameters

Attribute

Description

Request example

The following request updates the handler metadata (saved in example_payload.json) associated with the provided handler ID.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example_payload.json
    https://your-immuta-url.com/asa/handler/67

Request payload example

{
    "schemaEvolution": {
        "schemas": [
            "string"
        ],
        "disabled": false,
        "ownerProfileId": 0,
        "config": {
            "nameTemplate": {
                "tableFormat": "string",
                "nameFormat": "string",
                "sqlSchemaNameFormat": "string",
                "schemaProjectNameFormat": "string"
            }
        },
        "dataSourceConfig": {},
        "handlerMetadata": {},
        "connectionString": "string"
    },
    "handler": {
        "id": 0,
        "type": "string",
        "dataSourceId": 0,
        "metadata": {
            "blobId": "string",
            "eventTime": "string",
            "highCardinality": "string",
            "bodataSchemaName": "immuta",
            "bodataTableName": "string",
            "format": "string",
            "disableClassify": false,
            "staleDataTolerance": 2592000,
            "dataSourceName": "string",
            "schemaProjectName": "string",
            "username": "string",
            "password": "string",
            "ssl": false,
            "database": "string",
            "schema": "string",
            "table": "string",
            "query": "string",
            "columns": [{
                "name": "string",
                "dataType": "string",
                "remoteColumn": "string",
                "remoteType": "string",
                "srid": 0,
                "statistics": {},
                "nullable": false,
                "isPrimaryKey": false,
                "displayName": "string",
                "description": "string",
                "tags": [
                    "string"
                ],
                "catalogMetadata": {},
                "children": [
                    "string"
                ]
            }],
            "columnsNormalized": false,
            "nativeWorkspaceName": "string",
            "nativeSchemaName": "string",
            "nativeViewName": "string",
            "hostname": "string",
            "port": 0,
            "userFiles": [{
                "keyName": "string",
                "filename": "string",
                "content": "string",
                "userFilename": "string"
            }],
            "scheme": "string",
            "warehouse": "string",
            "connectionStringOptions": "string",
            "native": {
                "type": "string",
                "projectId": 0
            },
            "secureNativeView": false
        },
        "dbms": {
            "name": "string"
        },
        "connectionString": "string",
        "createdAt": "2021-09-20",
        "updatedAt": "2021-09-20",
        "policyHandler": {
            "visibilitySchema": {
                "fields": [
                    "string"
                ],
                "version": "2021-09-20"
            }
        }
    }
}

Response example

{
  "dataSourceId": 67,
  "metadata": {
    "ssl": true,
    "port": 1433,
    "query": null,
    "table": "table_108",
    "schema": "dev",
    "database": "your_database",
    "hostname": "your-workspace.sql.azuresynapse.net",
    "username": "your-username",
    "eventTime": null,
    "dataSourceName": "Dev Table 108",
    "bodataTableName": "table_108",
    "disableClassify": false,
    "highCardinality": "col_1",
    "bodataSchemaName": "dev",
    "columnsNormalized": false,
    "schemaProjectName": "Dev",
    "staleDataTolerance": 2592000
  },
  "type": "odbcHandler",
  "connectionString": "your-username@your-workspace.sql.azuresynapse.net:1433/public",
  "id": 67,
  "createdAt": "2021-09-17T14:15:10.256Z",
  "updatedAt": "2021-09-17T14:15:19.547Z",
  "dbms": {
    "name": "asa"
  }
}

Update multiple data sources

PUT /asa/bulk

Updates the data source metadata associated with the provided connection string.

Query parameters

Attribute

Description

Required

Response parameters

Attribute

Description

Request example

The following request updates the handler metadata for the handler ID specified in example_payload.json.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example_payload.json
    https://your-immuta-url.com/asa/bulk

Request payload example

{
    "ids": [
        0
    ],
    "connectionString": "string",
    "handler": {
        "metadata": {
            "blobId": "string",
            "eventTime": "string",
            "highCardinality": "string",
            "bodataTableName": "string",
            "format": "string",
            "disableClassify": false,
            "staleDataTolerance": 2592000,
            "dataSourceName": "string",
            "schemaProjectName": "string",
            "username": "string",
            "password": "string",
            "ssl": false,
            "database": "string",
            "schema": "string",
            "table": "string",
            "query": "string",
            "columns": [{
                "name": "string",
                "dataType": "string",
                "remoteColumn": "string",
                "remoteType": "string",
                "srid": 0,
                "statistics": {},
                "nullable": false,
                "isPrimaryKey": false,
                "displayName": "string",
                "description": "string",
                "tags": [
                    "string"
                ],
                "catalogMetadata": {},
                "children": [
                    "string"
                ]
            }],
            "columnsNormalized": false,
            "nativeWorkspaceName": "string",
            "nativeSchemaName": "string",
            "nativeViewName": "string",
            "authenticationMethod": "string",
            "hostname": "string",
            "port": 0,
            "sid": "string",
            "connectionStringOptions": "string",
            "paths": [
                "string"
            ],
            "clusterName": "string",
            "pathUris": [
                "string"
            ],
            "metastoreTables": [
                "string"
            ],
            "scheme": "string",
            "providers": "string",
            "ephemeral": false,
            "httpPath": "string",
            "userFiles": [{
                "keyName": "string",
                "filename": "string",
                "content": "string",
                "userFilename": "string"
            }],
            "warehouse": "string",
            "workspaceId": 0,
            "authDB": "string",
            "directory": "string",
            "secureNativeView": false,
            "bodataSchemaName": "string"
        }
    },
    "schemaEvolution": {
        "schemas": [
            "string"
        ],
        "disabled": false,
        "ownerProfileId": 0,
        "config": {
            "nameTemplate": {
                "tableFormat": "string",
                "nameFormat": "string",
                "sqlSchemaNameFormat": "string",
                "schemaProjectNameFormat": "string"
            }
        },
        "dataSourceConfig": {},
        "handlerMetadata": {},
        "connectionString": "string"
    }
}

Response example

{
  "bulkId": "bulk_ds_update_fe48d7fd4c594f96a89438cdb84ec0ba",
  "connectionString": "string@string:0/string",
  "jobsCreated": 1
}

Recalculate high cardinality columns

PUT /asa/handler/{handlerId}/triggerHighCardinalityJob

Recalculates the high cardinality column for the provided handler ID.

Query parameters

Attribute

Description

Required

Response parameters

The response returns a string of characters that identify the high cardinality job run.

Request example

The following request recalculates the high cardinality column for the provided handler ID.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/asa/handler/67/triggerHighCardinalityJob

Response example

25424a50-17df-11ec-b388-0fe1d33b5af1

Refresh a native view

PUT /asa/handler/{handlerId}/refreshNativeViewJob

Refresh the native view of a data source.

Query parameters

Attribute

Description

Required

Response parameters

The response returns a string of characters that identifies the refresh view job run.

Request example

This request refreshes the view for the data source with the handler ID 7.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://demo.immuta.com/asa/handler/7/refreshNativeViewJob

Response example

53c256d0-eb57-11ec-b275-d95a8e998142

Databricks API Reference Guide

The databricks endpoint allows you to connect and manage Databricks data sources in Immuta.

Additional fields may be included in some responses you receive; however, these attributes are for internal purposes and are therefore undocumented.

Requirements

Databricks Spark integration

When exposing a table or view from an Immuta-enabled Databricks cluster, be sure that at least one of these traits is true:

The user exposing the tables has READ_METADATA and SELECT permissions on the target views/tables (specifically if Table ACLs are enabled).
The user exposing the tables is listed in the immuta.spark.acl.whitelist configuration on the target cluster.
The user exposing the tables is a Databricks workspace administrator.

Databricks Unity Catalog integration

When exposing a table from Databricks Unity Catalog, be sure the credentials used to register the data sources have the Databricks privileges listed below.

The following privileges on the parent catalogs and schemas of those tables:
- SELECT
- USE CATALOG
- USE SCHEMA
USE SCHEMA on system.information_schema

Azure Databricks Unity Catalog limitation

Databricks workflow

Create a data source

POST /databricks/handler

Save the provided connection information as a data source.

Payload parameters

Attribute

Description

Required

Response parameters

Attribute

Description

Request example

This request creates two Databricks data sources.

curl \
    --request POST \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://demo.immuta.com/databricks/handler

Payload example

{
  "handler": [{
    "metadata": {
      "ssl": true,
      "userFiles": [],
      "authenticationMethod": "Access Token",
      "password": "your-password",
      "port": 443,
      "hostname": "your-hostname.cloud.databricks.com",
      "database": "default",
      "httpPath": "sql/your/http/0/path",
      "schemaProjectName": "Default",
      "staleDataTolerance": 86400,
      "bodataSchemaName": "default",
      "bodataTableName": "applicant_data",
      "dataSourceName": "Default Applicant Data",
      "table": "applicant_data",
      "schema": "default"
    }
  }, {
    "metadata": {
      "ssl": true,
      "userFiles": [],
      "authenticationMethod": "Access Token",
      "password": "your-password",
      "port": 443,
      "hostname": "your-hostname.cloud.databricks.com",
      "database": "default",
      "httpPath": "sql/your/http/0/path",
      "schemaProjectName": "Default",
      "staleDataTolerance": 86400,
      "bodataSchemaName": "default",
      "bodataTableName": "cities",
      "dataSourceName": "Default Cities",
      "table": "cities",
      "schema": "default"
    }
  }],
  "dataSource": {
    "blobHandler": {
      "scheme": "https",
      "url": ""
    },
    "blobHandlerType": "Databricks",
    "recordFormat": "",
    "type": "queryable",
    "schemaEvolutionId": null,
    "columnEvolutionEnabled": true
  },
  "schemaEvolution": {
    "ownerProfileId": 2,
    "config": {
      "nameTemplate": {
        "nameFormat": "<Schema> <Tablename>",
        "tableFormat": "<tablename>",
        "sqlSchemaNameFormat": "<schema>",
        "schemaProjectNameFormat": "<Schema>"
      }
    },
    "schemas": []
  }
}

Response example

{
  "connectionString": "your-hostname.cloud.databricks.com:443/default"
}

Get information about a data source

GET /databricks/handler/{handlerId}

Get the handler metadata associated with the provided handler ID.

Query parameters

Attribute

Description

Required

Response parameters

Attribute

Description

Request example

This request returns metadata for the handler with the ID 48.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://demo.immuta.com/databricks/handler/48

Response example

{
  "dataSourceId": 49,
  "metadata": {
    "ssl": true,
    "port": 443,
    "paths": ["/user/hive/warehouse/cities"],
    "query": null,
    "table": "cities",
    "schema": "default",
    "scheme": "dbfs",
    "database": "default",
    "hostname": "your-hostname.cloud.databricks.com",
    "httpPath": "sql/your/http/0/path",
    "pathUris": ["dbfs:/user/hive/warehouse/cities"],
    "ephemeral": true,
    "eventTime": null,
    "userFiles": [],
    "clusterName": null,
    "dataSourceName": "Default Cities",
    "bodataTableName": "cities",
    "metastoreTables": ["default.cities"],
    "bodataSchemaName": "default",
    "columnsNormalized": false,
    "schemaProjectName": "Default",
    "staleDataTolerance": 86400,
    "authenticationMethod": "Access Token"
  },
  "type": "queryable",
  "connectionString": "your-hostname.cloud.databricks.com:443/default",
  "id": 48,
  "createdAt": "2021-10-06T17:53:09.640Z",
  "updatedAt": "2021-10-06T17:53:09.882Z",
  "dbms": {
    "name": "databricks"
  }
}

Manage data sources

Method

Path

Purpose

Update a specific data source

PUT /databricks/handler/{handlerId}

Query parameters

Attribute

Description

Required

Payload parameters

Attribute

Description

Required

Response parameters

Attribute

Description

Request example

This request updates the metadata for the data source with the handler ID 48.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://demo.immuta.com/databricks/handler/48

Payload example

The payload below updates the dataSourceName to Cities.

{
  "handler": {
    "policyHandler": null,
    "dataSourceId": 49,
    "metadata": {
      "ssl": true,
      "port": 443,
      "paths": ["/user/hive/warehouse/cities"],
      "table": "cities",
      "schema": "default",
      "scheme": "dbfs",
      "database": "default",
      "hostname": "your-hostname.cloud.databricks.com",
      "httpPath": "sql/your/http/0/path",
      "pathUris": ["dbfs:/user/hive/warehouse/cities"],
      "ephemeral": true,
      "eventTime": null,
      "userFiles": [],
      "clusterName": null,
      "dataSourceName": "Cities",
      "bodataTableName": "cities",
      "metastoreTables": ["default.cities"],
      "bodataSchemaName": "default",
      "columnsNormalized": false,
      "schemaProjectName": "Default",
      "staleDataTolerance": 86400,
      "authenticationMethod": "Access Token",
      "columns": [{
        "name": "OBJECTID",
        "dataType": "bigint",
        "remoteType": "bigint",
        "nullable": true
      }, {
        "name": "URBID",
        "dataType": "bigint",
        "remoteType": "bigint",
        "nullable": true
      }, {
        "name": "LIGHTDCW",
        "dataType": "bigint",
        "remoteType": "bigint",
        "nullable": true
      }, {
        "name": "ES90POP",
        "dataType": "double precision",
        "remoteType": "double",
        "nullable": true
      }, {
        "name": "ES95POP",
        "dataType": "double precision",
        "remoteType": "double",
        "nullable": true
      }, {
        "name": "ES00POP",
        "dataType": "double precision",
        "remoteType": "double",
        "nullable": true
      }, {
        "name": "PCOUNT",
        "dataType": "bigint",
        "remoteType": "bigint",
        "nullable": true
      }, {
        "name": "SCHNM",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "NAME",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "SQKM_FINAL",
        "dataType": "double precision",
        "remoteType": "double",
        "nullable": true
      }, {
        "name": "ISO3",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "ISOURBID",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "REMOVED_PO",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "ADDED_POIN",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "YEAR_V1_01",
        "dataType": "double precision",
        "remoteType": "double",
        "nullable": true
      }, {
        "name": "POP_V1_01",
        "dataType": "double precision",
        "remoteType": "double",
        "nullable": true
      }, {
        "name": "Unsdcode",
        "dataType": "bigint",
        "remoteType": "bigint",
        "nullable": true
      }, {
        "name": "Countryeng",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "Continent",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "geometry",
        "dataType": "struct",
        "remoteType": "struct<__geom__:bigint,_is_empty:boolean,_ndim:bigint>",
        "nullable": true,
        "children": [{
          "name": "__geom__",
          "dataType": "bigint"
        }, {
          "name": "_is_empty",
          "dataType": "boolean"
        }, {
          "name": "_ndim",
          "dataType": "bigint"
        }]
      }, {
        "name": "wkt",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }],
      "password": "your-password"
    },
    "type": "queryable",
    "connectionString": "dbc-d3fe40ca-b4fb.cloud.databricks.com:443/default",
    "id": 48,
    "createdAt": "2021-10-06T17:53:09.640Z",
    "updatedAt": "2021-10-06T17:53:09.882Z",
    "dbms": {
      "name": "databricks"
    }
  }
}

Response example

{
  "id": 48,
  "ca": ["-----BEGIN CERTIFICATE-----\ncertificatedata\n-----END CERTIFICATE-----"],
  "metadata": {
    "columns": [{
      "name": "OBJECTID",
      "dataType": "bigint",
      "remoteType": "bigint",
      "nullable": true
    }, {
      "name": "URBID",
      "dataType": "bigint",
      "remoteType": "bigint",
      "nullable": true
    }, {
      "name": "LIGHTDCW",
      "dataType": "bigint",
      "remoteType": "bigint",
      "nullable": true
    }, {
      "name": "ES90POP",
      "dataType": "double precision",
      "remoteType": "double",
      "nullable": true
    }, {
      "name": "ES95POP",
      "dataType": "double precision",
      "remoteType": "double",
      "nullable": true
    }, {
      "name": "ES00POP",
      "dataType": "double precision",
      "remoteType": "double",
      "nullable": true
    }, {
      "name": "PCOUNT",
      "dataType": "bigint",
      "remoteType": "bigint",
      "nullable": true
    }, {
      "name": "SCHNM",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "NAME",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "SQKM_FINAL",
      "dataType": "double precision",
      "remoteType": "double",
      "nullable": true
    }, {
      "name": "ISO3",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "ISOURBID",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "REMOVED_PO",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "ADDED_POIN",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "YEAR_V1_01",
      "dataType": "double precision",
      "remoteType": "double",
      "nullable": true
    }, {
      "name": "POP_V1_01",
      "dataType": "double precision",
      "remoteType": "double",
      "nullable": true
    }, {
      "name": "Unsdcode",
      "dataType": "bigint",
      "remoteType": "bigint",
      "nullable": true
    }, {
      "name": "Countryeng",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "Continent",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "geometry",
      "dataType": "struct",
      "remoteType": "struct<__geom__:bigint,_is_empty:boolean,_ndim:bigint>",
      "nullable": true,
      "children": [{
        "name": "__geom__",
        "dataType": "bigint"
      }, {
        "name": "_is_empty",
        "dataType": "boolean"
      }, {
        "name": "_ndim",
        "dataType": "bigint"
      }]
    }, {
      "name": "wkt",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }]
  }
}

Update multiple data sources

PUT /databricks/bulk

Update the data source metadata associated with the provided connection string.

Payload parameters

Attribute

Description

Required

Response parameters

Attribute

Description

Request example

This request updates the metadata for all data sources with the connection string specified in example-payload.json.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://demo.immuta.com/databricks/bulk

Payload example

The payload below adds a certificate (certificate.json) to connect to the data sources with the provided connection.

{
  "handler": {
    "metadata": {
      "ssl": true,
      "port": 443,
      "database": "default",
      "hostname": "your-hostname.cloud.databricks.com",
      "userFiles": [{
        "keyName": "test",
        "filename": "6dc06a3310b9ba33c543e483d1e745b3ac9bc648.json",
        "userFilename": "certificate.json"
      }],
      "authenticationMethod": "Access Token",
      "password": "your-password",
      "httpPath": "sql/your/http/0/path"
    }
  },
  "connectionString": "your-hostname.cloud.databricks.com:443/default/default"
}

Response example

{
  "bulkId": "bulk_ds_update_9ae5bfd85a3a47a8b454c618043e2aa3",
  "connectionString": "your-hostname.cloud.databricks.com:443/default",
  "jobsCreated": 2
}

Recalculate the high cardinality column for a data source

PUT /databricks/handler/{handlerId}/triggerHighCardinalityJob

Recalculate the high cardinality column for the specified data source.

Query parameters

Attribute

Description

Required

Response parameters

The response returns a string of characters that identify the high cardinality job run.

Request example

This request re-runs the job that calculates the high cardinality column for the data source with the handler ID 47.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://demo.immuta.com/databricks/handler/47/triggerHighCardinalityJob

Response example

f6ac1ad0-26d0-11ec-8078-d36bbf5b90fb

Presto API Reference Guide

This page describes the presto (Presto data sources) endpoint.

Additional fields may be included in some responses you receive; however, these attributes are for internal purposes and are therefore undocumented.

Presto workflow

Create a data source

POST /presto/handler

Save the provided connection information as a data source.

Payload parameters

Attribute

Description

Required

Response parameters

Attribute

Description

Request example

This request creates a Presto data source.

curl \
    --request POST \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://demo.immuta.com/presto/handler

Payload example

{
  "handler": {
    "metadata": {
      "staleDataTolerance": 86400,
      "schemaProjectName": "Public",
      "bodataSchemaName": "public",
      "columns": [{
        "name": "c_customer_sk",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_customer_id",
        "dataType": "text",
        "remoteType": "char(16)",
        "nullable": true
      }, {
        "name": "c_current_cdemo_sk",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_current_hdemo_sk",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_current_addr_sk",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_first_shipto_date_sk",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_first_sales_date_sk",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_salutation",
        "dataType": "text",
        "remoteType": "varchar(10)",
        "nullable": true
      }, {
        "name": "c_first_name",
        "dataType": "text",
        "remoteType": "varchar(20)",
        "nullable": true
      }, {
        "name": "c_last_name",
        "dataType": "text",
        "remoteType": "varchar(30)",
        "nullable": true
      }, {
        "name": "c_preferred_cust_flag",
        "dataType": "text",
        "remoteType": "char(1)",
        "nullable": true
      }, {
        "name": "c_birth_day",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_birth_month",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_birth_year",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_birth_country",
        "dataType": "text",
        "remoteType": "varchar(20)",
        "nullable": true
      }, {
        "name": "c_login",
        "dataType": "text",
        "remoteType": "char(13)",
        "nullable": true
      }, {
        "name": "c_email_address",
        "dataType": "text",
        "remoteType": "varchar(50)",
        "nullable": true
      }, {
        "name": "c_last_review_date",
        "dataType": "text",
        "remoteType": "varchar(10)",
        "nullable": true
      }],
      "hostname": "example-presto.host.io",
      "port": 8080,
      "ssl": false,
      "authenticationMethod": "No Authentication",
      "connectionStringOptions": "",
      "userFiles": [],
      "database": "your_database",
      "sid": "postgres",
      "table": "customer",
      "schema": "public",
      "bodataTableName": "customer",
      "dataSourceName": "Customer"
    }
  },
  "dataSource": {
    "blobHandler": {
      "scheme": "https",
      "url": ""
    },
    "blobHandlerType": "Presto",
    "recordFormat": "",
    "type": "queryable",
    "schemaEvolutionId": null,
    "columnEvolutionEnabled": true,
    "name": "Customer",
    "sqlTableName": "customer"
  },
  "schemaEvolution": {
    "ownerProfileId": 1,
    "config": {
      "nameTemplate": {
        "nameFormat": "<Tablename>",
        "tableFormat": "<tablename>",
        "sqlSchemaNameFormat": "<schema>",
        "schemaProjectNameFormat": "<Schema>"
      }
    },
    "schemas": []
  }
}

Response example

{
  "connectionString": "your-username@us-east-1/tpc"
  "id": 1,
  "dataSourceId": 1,
  "dataSource": {
    "blobHandler": {
      "accessKey": "REDACTED",
      "url": "https://REDACTED/presto/handler/1",
      "ca": ["-----BEGIN CERTIFICATE-----\nyourdXRhIENcertificate\n-----END CERTIFICATE-----"],
      "manualDictionary": false
    },
    "blobHandlerType": "Presto",
    "recordFormat": "Not Provided",
    "type": "queryable",
    "schemaEvolutionId": 1,
    "columnEvolutionEnabled": true,
    "name": "Customer",
    "sqlTableName": "customer",
    "sqlSchemaName": "public",
    "workspace": null,
    "rowCount": 0,
    "seeded": false,
    "owner": {},
    "documentation": "# Customer",
    "statsExpiration": "2022-07-19T18:55:21.300Z",
    "recordCount": 0,
    "createdBy": 1,
    "policyHandler": null,
    "subscriptionType": "manual",
    "id": 1,
    "description": null,
    "deleted": false,
    "policyHandlerType": "None",
    "subscriptionPolicy": null,
    "globalPolicies": null,
    "status": null,
    "statusInfo": null,
    "expiration": null,
    "catalogMetadata": null,
    "sddTemplate": null,
    "createdAt": "2022-07-19T18:55:21.302Z",
    "updatedAt": "2022-07-19T18:55:21.302Z",
    "tags": []
  },
  "warnings": []
}

Search Presto data sources

Search for handler metadata

GET /presto/handler/{handlerId}

Get the handler metadata associated with the provided handler ID.

Query parameters

Attribute

Description

Required

Response parameters

Attribute

Description

Request example

The following request returns the handler metadata associated with the provided handler ID.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/presto/handler/67

Response example

{
  "dataSourceId": 67,
  "metadata": {
    "ssl": true,
    "port": 1433,
    "query": null,
    "table": "table_108",
    "schema": "elliott_dev",
    "database": "your_database",
    "hostname": "example-presto.host.io",
    "username": "your-username",
    "eventTime": null,
    "dataSourceName": "Dev Table 108",
    "bodataTableName": "table_108",
    "disableClassify": false,
    "highCardinality": "col_1",
    "bodataSchemaName": "dev",
    "columnsNormalized": false,
    "schemaProjectName": "Dev",
    "staleDataTolerance": 2592000
  },
  "type": "odbcHandler",
  "connectionString": "your-username@example-presto.host.io:1433/your_database",
  "id": 67,
  "createdAt": "2021-09-17T14:15:10.256Z",
  "updatedAt": "2021-09-17T14:15:19.547Z",
  "dbms": {
    "name": "presto"
  }
}

Update Presto data sources

Endpoint

Purpose

Update handler metadata

PUT /presto/handler/{handlerId}

Query parameters

Attribute

Description

Required

Response parameters

Attribute

Description

Request example

This request updates the data source name to Marketing Data for the data source with the handler ID 67.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example_payload.json
    https://your-immuta-url.com/presto/handler/67

Request payload example

{
  "handler": {
    "policyHandler": null,
    "dataSourceId": 67,
    "metadata": {
      "sid": "postgres",
      "ssl": false,
      "port": 8080,
      "query": null,
      "table": "customer",
      "schema": "public",
      "database": "your_database",
      "hostname": "presto-example.io",
      "ephemeral": true,
      "eventTime": null,
      "userFiles": [],
      "dataSourceName": "Marketing Data",
      "bodataTableName": "customer",
      "highCardinality": "c_customer_sk",
      "bodataSchemaName": "public",
      "columnsNormalized": false,
      "schemaProjectName": "Public",
      "staleDataTolerance": 0,
      "authenticationMethod": "No Authentication",
      "connectionStringOptions": "",
      "columns": [{
        "name": "c_customer_sk",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_customer_id",
        "dataType": "text",
        "remoteType": "char(16)",
        "nullable": true
      }, {
        "name": "c_current_cdemo_sk",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_current_hdemo_sk",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_current_addr_sk",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_first_shipto_date_sk",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_first_sales_date_sk",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_salutation",
        "dataType": "text",
        "remoteType": "varchar(10)",
        "nullable": true
      }, {
        "name": "c_first_name",
        "dataType": "text",
        "remoteType": "varchar(20)",
        "nullable": true
      }, {
        "name": "c_last_name",
        "dataType": "text",
        "remoteType": "varchar(30)",
        "nullable": true
      }, {
        "name": "c_preferred_cust_flag",
        "dataType": "text",
        "remoteType": "char(1)",
        "nullable": true
      }, {
        "name": "c_birth_day",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_birth_month",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_birth_year",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_birth_country",
        "dataType": "text",
        "remoteType": "varchar(20)",
        "nullable": true
      }, {
        "name": "c_login",
        "dataType": "text",
        "remoteType": "char(13)",
        "nullable": true
      }, {
        "name": "c_email_address",
        "dataType": "text",
        "remoteType": "varchar(50)",
        "nullable": true
      }, {
        "name": "c_last_review_date",
        "dataType": "text",
        "remoteType": "varchar(10)",
        "nullable": true
      }]
    },
    "type": "odbcHandler",
    "connectionString": "presto-example.io:8080/postgres/public",
    "id": 1,
    "createdAt": "2022-07-19T18:55:21.220Z",
    "updatedAt": "2022-07-19T18:55:23.466Z",
    "dbms": {
      "name": "presto"
    }
  }
}

Response example

{
  "id": 67,
  "ca": ["-----BEGIN CERTIFICATE-----\nMIIuyourcertificate\n-----END CERTIFICATE-----"],
  "metadata": {
    "columns": [{
      "name": "c_customer_sk",
      "dataType": "integer",
      "remoteType": "integer",
      "nullable": true
    }, {
      "name": "c_customer_id",
      "dataType": "text",
      "remoteType": "char(16)",
      "nullable": true
    }, {
      "name": "c_current_cdemo_sk",
      "dataType": "integer",
      "remoteType": "integer",
      "nullable": true
    }, {
      "name": "c_current_hdemo_sk",
      "dataType": "integer",
      "remoteType": "integer",
      "nullable": true
    }, {
      "name": "c_current_addr_sk",
      "dataType": "integer",
      "remoteType": "integer",
      "nullable": true
    }, {
      "name": "c_first_shipto_date_sk",
      "dataType": "integer",
      "remoteType": "integer",
      "nullable": true
    }, {
      "name": "c_first_sales_date_sk",
      "dataType": "integer",
      "remoteType": "integer",
      "nullable": true
    }, {
      "name": "c_salutation",
      "dataType": "text",
      "remoteType": "varchar(10)",
      "nullable": true
    }, {
      "name": "c_first_name",
      "dataType": "text",
      "remoteType": "varchar(20)",
      "nullable": true
    }, {
      "name": "c_last_name",
      "dataType": "text",
      "remoteType": "varchar(30)",
      "nullable": true
    }, {
      "name": "c_preferred_cust_flag",
      "dataType": "text",
      "remoteType": "char(1)",
      "nullable": true
    }, {
      "name": "c_birth_day",
      "dataType": "integer",
      "remoteType": "integer",
      "nullable": true
    }, {
      "name": "c_birth_month",
      "dataType": "integer",
      "remoteType": "integer",
      "nullable": true
    }, {
      "name": "c_birth_year",
      "dataType": "integer",
      "remoteType": "integer",
      "nullable": true
    }, {
      "name": "c_birth_country",
      "dataType": "text",
      "remoteType": "varchar(20)",
      "nullable": true
    }, {
      "name": "c_login",
      "dataType": "text",
      "remoteType": "char(13)",
      "nullable": true
    }, {
      "name": "c_email_address",
      "dataType": "text",
      "remoteType": "varchar(50)",
      "nullable": true
    }, {
      "name": "c_last_review_date",
      "dataType": "text",
      "remoteType": "varchar(10)",
      "nullable": true
    }]
  }
}

Update multiple data sources

PUT /presto/bulk

Updates the data source metadata associated with the provided connection string.

Query parameters

Attribute

Description

Required

Response parameters

Attribute

Description

Request example

This request updates the metadata for all data sources with the connection string specified in example-payload.json.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://demo.immuta.com/presto/bulk

Payload example

The payload below adds a certificate (certificate.json) to the data sources with the provided connection.

{
  "handler": {
    "metadata": {
      "sid": "postgres",
      "ssl": false,
      "port": 8080,
      "database": "your_database",
      "hostname": "presto-example.io",
      "userFiles": [{
        "keyName": "certificate",
        "filename": "576d8b38e9e8bc3749599489408a0b9f605b2a8d.md",
        "userFilename": "certificate.json"
      }],
      "authenticationMethod": "No Authentication",
      "connectionStringOptions": ""
    }
  },
  "connectionString": "presto-example.io:8080/postgres/your_database"
}

Response example

{
  "bulkId": "bulk_ds_update_657dd563e6e746069bf040de5e6909a9",
  "connectionString": "presto-example.io:8080/postgres/your_database",
  "jobsCreated": 4
}

Recalculate high cardinality columns

PUT /presto/handler/{handlerId}/triggerHighCardinalityJob

Recalculates the high cardinality column for the provided handler ID.

Query parameters

Attribute

Description

Required

Response parameters

The response returns a string of characters that identify the high cardinality job run.

Request example

The following request recalculates the high cardinality column for the provided handler ID.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/presto/handler/67/triggerHighCardinalityJob

Response example

25424a50-17df-11ec-b388-0fe1d33b5af1

Snowflake API Reference Guide

The snowflake endpoint allows you to connect and manage Snowflake data sources in Immuta.

Additional fields may be included in some responses you receive; however, these attributes are for internal purposes and are therefore undocumented.

Snowflake workflow

Snowflake imported databases

Immuta does not support Snowflake tables from imported databases. Instead, create a view of the table and register that view as a data source.

Create a data source.
Get information about a data source.
Manage data sources.

Create a data source

POST /snowflake/handler

Save the provided connection information as a data source.

Payload parameters

Attribute

Description

Required

Response parameters

Attribute

Description

Request example

This request creates a Snowflake data source.

curl \
    --request POST \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://demo.immuta.com/snowflake/handler

Payload example

{
  "handler": {
    "metadata": {
      "ssl": true,
      "userFiles": [],
      "authenticationMethod": "userPassword",
      "username": "user",
      "password": "yourpassword",
      "port": 443,
      "hostname": "demo.us-east-1.snowflakecomputing.com",
      "warehouse": "YOUR_WH",
      "database": "ANALYTICS",
      "schema": "TEST",
      "table": "CUSTOMERS",
      "nativeViewName": "customers_immuta",
      "nativeSchemaName": "test_immuta",
      "nativeWorkspaceName": "immuta_analytics",
      "schemaProjectName": "Test",
      "bodataSchemaName": "test",
      "columns": [{
        "name": "CUSTOMER_ID",
        "dataType": "numeric(38,0)",
        "remoteType": "number(38,0)",
        "isPrimaryKey": false,
        "nullable": true
      }, {
        "name": "FIRST_NAME",
        "dataType": "text",
        "remoteType": "varchar(16777216)",
        "isPrimaryKey": false,
        "nullable": true
      }, {
        "name": "LAST_NAME",
        "dataType": "text",
        "remoteType": "varchar(16777216)",
        "isPrimaryKey": false,
        "nullable": true
      }, {
        "name": "FIRST_ORDER",
        "dataType": "date",
        "remoteType": "date",
        "isPrimaryKey": false,
        "nullable": true
      }, {
        "name": "MOST_RECENT_ORDER",
        "dataType": "date",
        "remoteType": "date",
        "isPrimaryKey": false,
        "nullable": true
      }, {
        "name": "NUMBER_OF_ORDERS",
        "dataType": "numeric(18,0)",
        "remoteType": "number(18,0)",
        "isPrimaryKey": false,
        "nullable": true
      }, {
        "name": "TOTAL_ORDER_AMOUNT",
        "dataType": "numeric",
        "remoteType": "number(38,6)",
        "isPrimaryKey": false,
        "nullable": true
      }],
      "eventTime": "FIRST_ORDER",
      "staleDataTolerance": 86400,
      "bodataTableName": "customers",
      "dataSourceName": "Customers"
    }
  },
  "dataSource": {
    "blobHandler": {
      "scheme": "https",
      "url": ""
    },
    "blobHandlerType": "Snowflake",
    "recordFormat": "",
    "type": "queryable",
    "schemaEvolutionId": null,
    "columnEvolutionEnabled": true,
    "name": "Customers",
    "sqlTableName": "customers"
  },
  "schemaEvolution": {
    "ownerProfileId": 2,
    "config": {
      "nameTemplate": {
        "nameFormat": "<Schema> <Tablename>",
        "tableFormat": "<tablename>",
        "sqlSchemaNameFormat": "<schema>",
        "schemaProjectNameFormat": "<Schema>"
      }
    },
    "schemas": []
  }
}

Response example

{
  "id": 30,
  "dataSourceId": 30,
  "warnings": []
}

Get information about a data source

GET /snowflake/handler/{handlerId}

Get the handler metadata associated with the provided handler ID.

Query parameters

Attribute

Description

Required

Response parameters

Attribute

Description

Request example

This request returns metadata for the handler with the ID 30.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://demo.immuta.com/snowflake/handler/30

Response example

{
  "dataSourceId": 30,
  "metadata": {
    "ssl": true,
    "port": 443,
    "query": null,
    "table": "CUSTOMERS",
    "schema": "TEST",
    "database": "ANALYTICS",
    "hostname": "demo.us-east-1.snowflakecomputing.com",
    "username": "user",
    "eventTime": "FIRST_ORDER",
    "userFiles": [],
    "warehouse": "YOUR_WH",
    "dataSourceName": "Customers",
    "bodataTableName": "customers",
    "highCardinality": "CUSTOMER_ID",
    "bodataSchemaName": "test",
    "columnsNormalized": true,
    "schemaProjectName": "Test",
    "staleDataTolerance": 86400,
    "authenticationMethod": "userPassword"
  },
  "type": "odbcHandler",
  "connectionString": "user@demo.us-east-1.snowflakecomputing.com:443/ANALYTICS",
  "id": 30,
  "createdAt": "2021-10-01T16:19:23.562Z",
  "updatedAt": "2021-10-01T16:19:28.556Z",
  "dbms": {
    "name": "snowflake"
  }
}

Manage data sources

Method

Path

Purpose

Update a specific data source

PUT /snowflake/handler/{handlerId}

Query parameters

Attribute

Description

Required

Payload parameters

Attribute

Description

Required

Response parameters

Attribute

Description

Request example

This request updates the metadata for the data source with the handler ID 30.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://demo.immuta.com/snowflake/handler/30

Payload example

The payload below updates the eventTime to MOST_RECENT_ORDER.

{
  "handler": {
    "policyHandler": null,
    "dataSourceId": 30,
    "metadata": {
      "ssl": true,
      "port": 443,
      "table": "CUSTOMERS",
      "schema": "TEST",
      "database": "ANALYTICS",
      "hostname": "demo.us-east-1.snowflakecomputing.com",
      "username": "user",
      "eventTime": "MOST_RECENT_ORDER",
      "userFiles": [],
      "warehouse": "YOUR_WH",
      "dataSourceName": "Customers",
      "bodataTableName": "customers",
      "highCardinality": "CUSTOMER_ID",
      "bodataSchemaName": "test",
      "columnsNormalized": true,
      "schemaProjectName": "Test",
      "staleDataTolerance": 86400,
      "authenticationMethod": "userPassword",
      "columns": [{
        "name": "CUSTOMER_ID",
        "dataType": "numeric(38,0)",
        "nullable": true,
        "remoteType": "number(38,0)",
        "isPrimaryKey": false,
        "remoteColumn": "CUSTOMER_ID"
      }, {
        "name": "FIRST_NAME",
        "dataType": "text",
        "nullable": true,
        "remoteType": "varchar(16777216)",
        "isPrimaryKey": false,
        "remoteColumn": "FIRST_NAME"
      }, {
        "name": "LAST_NAME",
        "dataType": "text",
        "nullable": true,
        "remoteType": "varchar(16777216)",
        "isPrimaryKey": false,
        "remoteColumn": "LAST_NAME"
      }, {
        "name": "FIRST_ORDER",
        "dataType": "date",
        "nullable": true,
        "remoteType": "date",
        "isPrimaryKey": false,
        "remoteColumn": "FIRST_ORDER"
      }, {
        "name": "MOST_RECENT_ORDER",
        "dataType": "date",
        "nullable": true,
        "remoteType": "date",
        "isPrimaryKey": false,
        "remoteColumn": "MOST_RECENT_ORDER"
      }, {
        "name": "NUMBER_OF_ORDERS",
        "dataType": "numeric(18,0)",
        "nullable": true,
        "remoteType": "number(18,0)",
        "isPrimaryKey": false,
        "remoteColumn": "NUMBER_OF_ORDERS"
      }, {
        "name": "TOTAL_ORDER_AMOUNT",
        "dataType": "numeric",
        "nullable": true,
        "remoteType": "number(38,6)",
        "isPrimaryKey": false,
        "remoteColumn": "TOTAL_ORDER_AMOUNT"
      }]
    },
    "type": "odbcHandler",
    "connectionString": "user@demo.us-east-1.snowflakecomputing.com:443/ANALYTICS",
    "id": 30,
    "createdAt": "2021-10-01T16:19:23.562Z",
    "updatedAt": "2021-10-01T17:19:35.853Z",
    "dbms": {
      "name": "snowflake"
    }
  }
}

Response example

{
  "id": 30,
  "ca": ["-----BEGIN CERTIFICATE-----\ncertificatedata\n-----END CERTIFICATE-----"],
  "metadata": {
    "columns": [{
      "name": "customer_id",
      "dataType": "numeric(38,0)",
      "remoteType": "number(38,0)",
      "isPrimaryKey": false,
      "nullable": true,
      "remoteColumn": "CUSTOMER_ID"
    }, {
      "name": "first_name",
      "dataType": "text",
      "remoteType": "varchar(16777216)",
      "isPrimaryKey": false,
      "nullable": true,
      "remoteColumn": "FIRST_NAME"
    }, {
      "name": "last_name",
      "dataType": "text",
      "remoteType": "varchar(16777216)",
      "isPrimaryKey": false,
      "nullable": true,
      "remoteColumn": "LAST_NAME"
    }, {
      "name": "first_order",
      "dataType": "date",
      "remoteType": "date",
      "isPrimaryKey": false,
      "nullable": true,
      "remoteColumn": "FIRST_ORDER"
    }, {
      "name": "most_recent_order",
      "dataType": "date",
      "remoteType": "date",
      "isPrimaryKey": false,
      "nullable": true,
      "remoteColumn": "MOST_RECENT_ORDER"
    }, {
      "name": "number_of_orders",
      "dataType": "numeric(18,0)",
      "remoteType": "number(18,0)",
      "isPrimaryKey": false,
      "nullable": true,
      "remoteColumn": "NUMBER_OF_ORDERS"
    }, {
      "name": "total_order_amount",
      "dataType": "numeric",
      "remoteType": "number(38,6)",
      "isPrimaryKey": false,
      "nullable": true,
      "remoteColumn": "TOTAL_ORDER_AMOUNT"
    }]
  }
}

Update multiple data sources

PUT /snowflake/bulk

Update the data source metadata associated with the provided connection string.

Payload parameters

Attribute

Description

Required

Response parameters

Attribute

Description

Request example

This request updates the metadata for all data sources with the connection string specified in example-payload.json.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://demo.immuta.com/snowflake/bulk

Payload example

The payload below updates the database to ANALYST_DEMO for the provided connection string.

{
  "handler": {
    "metadata": {
      "ssl": true,
      "port": 443,
      "database": "ANALYST_DEMO",
      "hostname": "demo.us-east-1.snowflakecomputing.com",
      "username": "yourusername",
      "userFiles": [],
      "warehouse": "YOUR_WH",
      "authenticationMethod": "userPassword",
      "password": "yourpassword"
    }
  },
  "connectionString": "demo.us-east-1.snowflakecomputing.com:443/ANALYST_DEMO"
}

Response example

{
  "bulkId": "bulk_ds_update_54ada6bhashedvaluea0c80cecd9d62",
  "jobsCreated": 5
}

Recalculate the high cardinality column for a data source

PUT /snowflake/handler/{handlerId}/triggerHighCardinalityJob

Recalculate the high cardinality column for the specified data source.

Query parameters

Attribute

Description

Required

Response parameters

The response returns a string of characters that identify the high cardinality job run.

Request example

This request re-runs the job that calculates the high cardinality column for the data source with the handler ID 30.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://demo.immuta.com/snowflake/handler/30/triggerHighCardinalityJob

Response example

c12fd320-22d8-11ec-b2b8-874838eeef05

Refresh a native view

PUT /snowflake/handler/{handlerId}/refreshNativeViewJob

Refresh the native view of a data source.

Query parameters

Attribute

Description

Required

Response parameters

The response returns a string of characters that identifies the refresh view job run.

Request example

This request refreshes the view for the data source with the handler ID 7.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://demo.immuta.com/snowflake/handler/7/refreshNativeViewJob

Response example

53c256d0-eb57-11ec-b275-d95a8e998142

Trino API Reference Guide

The trino endpoint allows you to connect and manage Trino data sources in Immuta.

Additional fields may be included in some responses you receive; however, these attributes are for internal purposes and are therefore undocumented.

Trino workflow

Create a data source.
Get information about a data source.
Manage data sources.

Create a data source

POST /trino/handler

Save the provided connection information as a data source.

Payload parameters

Attribute

Description

Required

Response parameters

Attribute

Description

Request example

This request creates a Trino data source.

curl \
    --request POST \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://demo.immuta.com/trino/handler

Payload example

{
  "handler": {
    "metadata": {
      "staleDataTolerance": 86400,
      "schemaProjectName": "Public",
      "bodataSchemaName": "public",
      "columns": [{
        "name": "c_customer_sk",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_customer_id",
        "dataType": "text",
        "remoteType": "char(16)",
        "nullable": true
      }, {
        "name": "c_current_cdemo_sk",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_current_hdemo_sk",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_current_addr_sk",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_first_shipto_date_sk",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_first_sales_date_sk",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_salutation",
        "dataType": "text",
        "remoteType": "varchar(10)",
        "nullable": true
      }, {
        "name": "c_first_name",
        "dataType": "text",
        "remoteType": "varchar(20)",
        "nullable": true
      }, {
        "name": "c_last_name",
        "dataType": "text",
        "remoteType": "varchar(30)",
        "nullable": true
      }, {
        "name": "c_preferred_cust_flag",
        "dataType": "text",
        "remoteType": "char(1)",
        "nullable": true
      }, {
        "name": "c_birth_day",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_birth_month",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_birth_year",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_birth_country",
        "dataType": "text",
        "remoteType": "varchar(20)",
        "nullable": true
      }, {
        "name": "c_login",
        "dataType": "text",
        "remoteType": "char(13)",
        "nullable": true
      }, {
        "name": "c_email_address",
        "dataType": "text",
        "remoteType": "varchar(50)",
        "nullable": true
      }, {
        "name": "c_last_review_date",
        "dataType": "text",
        "remoteType": "varchar(10)",
        "nullable": true
      }],
      "hostname": "example-trino.host.io",
      "port": 8080,
      "ssl": false,
      "authenticationMethod": "No Authentication",
      "connectionStringOptions": "",
      "userFiles": [],
      "database": "public",
      "sid": "postgres",
      "table": "customer",
      "schema": "public",
      "bodataTableName": "customer",
      "dataSourceName": "Customer"
    }
  },
  "dataSource": {
    "blobHandler": {
      "scheme": "https",
      "url": ""
    },
    "blobHandlerType": "Trino",
    "recordFormat": "",
    "type": "queryable",
    "schemaEvolutionId": null,
    "columnEvolutionEnabled": true,
    "name": "Customer",
    "sqlTableName": "customer"
  },
  "schemaEvolution": {
    "ownerProfileId": 1,
    "config": {
      "nameTemplate": {
        "nameFormat": "<Tablename>",
        "tableFormat": "<tablename>",
        "sqlSchemaNameFormat": "<schema>",
        "schemaProjectNameFormat": "<Schema>"
      }
    },
    "schemas": []
  }
}

Response example

{
  "id": 1,
  "dataSourceId": 1,
  "dataSource": {
    "blobHandler": {
      "accessKey": "REDACTED",
      "url": "https://REDACTED/trino/handler/1",
      "ca": ["-----BEGIN CERTIFICATE-----\nyourdXRhIENcertificate\n-----END CERTIFICATE-----"],
      "manualDictionary": false
    },
    "blobHandlerType": "Trino",
    "recordFormat": "Not Provided",
    "type": "queryable",
    "schemaEvolutionId": 1,
    "columnEvolutionEnabled": true,
    "name": "Customer",
    "sqlTableName": "customer",
    "sqlSchemaName": "public",
    "workspace": null,
    "rowCount": 0,
    "seeded": false,
    "owner": {},
    "documentation": "# Customer",
    "statsExpiration": "2022-07-19T18:55:21.300Z",
    "recordCount": 0,
    "createdBy": 1,
    "policyHandler": null,
    "subscriptionType": "manual",
    "id": 1,
    "description": null,
    "deleted": false,
    "policyHandlerType": "None",
    "subscriptionPolicy": null,
    "globalPolicies": null,
    "status": null,
    "statusInfo": null,
    "catalogMetadata": null,
    "createdAt": "2022-07-19T18:55:21.302Z",
    "updatedAt": "2022-07-19T18:55:21.302Z",
    "tags": []
  },
  "warnings": []
}

Get information about a data source

GET /trino/handler/{handlerId}

Get the handler metadata associated with the provided handler ID.

Query parameters

Attribute

Description

Required

Response parameters

Attribute

Description

Request example

This request returns metadata for the handler with the ID 1.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://demo.immuta.com/trino/handler/1

Response example

{
  "dataSourceId": 1,
  "metadata": {
    "sid": "postgres",
    "ssl": false,
    "port": 8080,
    "query": null,
    "table": "customer",
    "schema": "public",
    "database": "public",
    "hostname": "trino-example-database.io",
    "ephemeral": true,
    "eventTime": null,
    "userFiles": [],
    "dataSourceName": "Customer",
    "bodataTableName": "customer",
    "highCardinality": "c_customer_sk",
    "bodataSchemaName": "public",
    "columnsNormalized": false,
    "schemaProjectName": "Public",
    "staleDataTolerance": 0,
    "authenticationMethod": "No Authentication",
    "connectionStringOptions": null
  },
  "type": "queryable",
  "connectionString": "trino-example-database.io:8080/postgres/public",
  "id": 1,
  "createdAt": "2022-07-19T18:55:21.220Z",
  "updatedAt": "2022-07-19T18:55:23.466Z",
  "dbms": {
    "name": "trino"
  }
}

Manage data sources

Method

Path

Purpose

Update a specific data source

PUT /trino/handler/{handlerId}

Query parameters

Attribute

Description

Required

Payload parameters

Attribute

Description

Required

Response parameters

Attribute

Description

Request example

This request updates the data source name to Marketing Data for the data source with the handler ID 1.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://demo.immuta.com/trino/handler/1

Payload example

{
  "handler": {
    "policyHandler": null,
    "dataSourceId": 1,
    "metadata": {
      "sid": "postgres",
      "ssl": false,
      "port": 8080,
      "query": null,
      "table": "customer",
      "schema": "public",
      "database": "public",
      "hostname": "trino-example.io",
      "ephemeral": true,
      "eventTime": null,
      "userFiles": [],
      "dataSourceName": "Marketing Data",
      "bodataTableName": "customer",
      "highCardinality": "c_customer_sk",
      "bodataSchemaName": "public",
      "columnsNormalized": false,
      "schemaProjectName": "Public",
      "staleDataTolerance": 0,
      "authenticationMethod": "No Authentication",
      "connectionStringOptions": "",
      "columns": [{
        "name": "c_customer_sk",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_customer_id",
        "dataType": "text",
        "remoteType": "char(16)",
        "nullable": true
      }, {
        "name": "c_current_cdemo_sk",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_current_hdemo_sk",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_current_addr_sk",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_first_shipto_date_sk",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_first_sales_date_sk",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_salutation",
        "dataType": "text",
        "remoteType": "varchar(10)",
        "nullable": true
      }, {
        "name": "c_first_name",
        "dataType": "text",
        "remoteType": "varchar(20)",
        "nullable": true
      }, {
        "name": "c_last_name",
        "dataType": "text",
        "remoteType": "varchar(30)",
        "nullable": true
      }, {
        "name": "c_preferred_cust_flag",
        "dataType": "text",
        "remoteType": "char(1)",
        "nullable": true
      }, {
        "name": "c_birth_day",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_birth_month",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_birth_year",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "c_birth_country",
        "dataType": "text",
        "remoteType": "varchar(20)",
        "nullable": true
      }, {
        "name": "c_login",
        "dataType": "text",
        "remoteType": "char(13)",
        "nullable": true
      }, {
        "name": "c_email_address",
        "dataType": "text",
        "remoteType": "varchar(50)",
        "nullable": true
      }, {
        "name": "c_last_review_date",
        "dataType": "text",
        "remoteType": "varchar(10)",
        "nullable": true
      }]
    },
    "type": "queryable",
    "connectionString": "trino-example.io:8080/postgres/public",
    "id": 1,
    "createdAt": "2022-07-19T18:55:21.220Z",
    "updatedAt": "2022-07-19T18:55:23.466Z",
    "dbms": {
      "name": "trino"
    }
  }
}

Response example

{
  "id": 1,
  "ca": ["-----BEGIN CERTIFICATE-----\nMIIuyourcertificate\n-----END CERTIFICATE-----"],
  "metadata": {
    "columns": [{
      "name": "c_customer_sk",
      "dataType": "integer",
      "remoteType": "integer",
      "nullable": true
    }, {
      "name": "c_customer_id",
      "dataType": "text",
      "remoteType": "char(16)",
      "nullable": true
    }, {
      "name": "c_current_cdemo_sk",
      "dataType": "integer",
      "remoteType": "integer",
      "nullable": true
    }, {
      "name": "c_current_hdemo_sk",
      "dataType": "integer",
      "remoteType": "integer",
      "nullable": true
    }, {
      "name": "c_current_addr_sk",
      "dataType": "integer",
      "remoteType": "integer",
      "nullable": true
    }, {
      "name": "c_first_shipto_date_sk",
      "dataType": "integer",
      "remoteType": "integer",
      "nullable": true
    }, {
      "name": "c_first_sales_date_sk",
      "dataType": "integer",
      "remoteType": "integer",
      "nullable": true
    }, {
      "name": "c_salutation",
      "dataType": "text",
      "remoteType": "varchar(10)",
      "nullable": true
    }, {
      "name": "c_first_name",
      "dataType": "text",
      "remoteType": "varchar(20)",
      "nullable": true
    }, {
      "name": "c_last_name",
      "dataType": "text",
      "remoteType": "varchar(30)",
      "nullable": true
    }, {
      "name": "c_preferred_cust_flag",
      "dataType": "text",
      "remoteType": "char(1)",
      "nullable": true
    }, {
      "name": "c_birth_day",
      "dataType": "integer",
      "remoteType": "integer",
      "nullable": true
    }, {
      "name": "c_birth_month",
      "dataType": "integer",
      "remoteType": "integer",
      "nullable": true
    }, {
      "name": "c_birth_year",
      "dataType": "integer",
      "remoteType": "integer",
      "nullable": true
    }, {
      "name": "c_birth_country",
      "dataType": "text",
      "remoteType": "varchar(20)",
      "nullable": true
    }, {
      "name": "c_login",
      "dataType": "text",
      "remoteType": "char(13)",
      "nullable": true
    }, {
      "name": "c_email_address",
      "dataType": "text",
      "remoteType": "varchar(50)",
      "nullable": true
    }, {
      "name": "c_last_review_date",
      "dataType": "text",
      "remoteType": "varchar(10)",
      "nullable": true
    }]
  }
}

Update multiple data sources

PUT /trino/bulk

Update the data source metadata associated with the provided connection string.

Payload parameters

Attribute

Description

Required

Response parameters

Attribute

Description

Request example

This request updates the metadata for all data sources with the connection string specified in example-payload.json.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://demo.immuta.com/trino/bulk

Payload example

The payload below adds a certificate (certificate.json) to the data sources with the provided connection.

{
  "handler": {
    "metadata": {
      "sid": "postgres",
      "ssl": false,
      "port": 8080,
      "database": "public",
      "hostname": "trino-example.io",
      "userFiles": [{
        "keyName": "certificate",
        "filename": "576d8b38e9e8bc3749599489408a0b9f605b2a8d.md",
        "userFilename": "certificate.json"
      }],
      "authenticationMethod": "No Authentication",
      "connectionStringOptions": ""
    }
  },
  "connectionString": "trino-example.io:8080/postgres/public"
}

Response example

{
  "bulkId": "bulk_ds_update_657dd563e6e746069bf040de5e6909a9",
  "connectionString": "trino-example.io:8080/postgres/public",
  "jobsCreated": 4
}

Recalculate the high cardinality column for a data source

PUT /trino/handler/{handlerId}/triggerHighCardinalityJob

Recalculate the high cardinality column for the specified data source.

Query parameters

Attribute

Description

Required

Response parameters

The response returns a string of characters that identify the high cardinality job run.

Request example

This request re-runs the job that calculates the high cardinality column for the data source with the handler ID 30.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://demo.immuta.com/trino/handler/30/triggerHighCardinalityJob

Response example

f6ac1ad0-26d0-11ec-8078-d36bbf5b90fb

Refresh a native view

PUT /trino/handler/{handlerId}/refreshNativeViewJob

Refresh the native view of a data source.

Query parameters

Attribute

Description

Required

Response parameters

The response returns a string of characters that identifies the refresh view job run.

Request example

This request refreshes the view for the data source with the handler ID 7.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://demo.immuta.com/trino/handler/7/refreshNativeViewJob

Response example

53c256d0-eb57-11ec-b275-d95a8e998142

Redshift API Reference Guide

The redshift endpoint allows you to connect and manage Redshift data sources in Immuta.

Additional fields may be included in some responses you receive; however, these attributes are for internal purposes and are therefore undocumented.

Redshift workflow

Create a data source.
Get information about a data source.
Manage data sources.

Create a data source

POST /redshift/handler

Save the provided connection information as a data source.

Payload parameters

Attribute

Description

Required

Response parameters

Attribute

Description

Request example

This request creates two Redshift data sources, which are specified in example-payload.json.

curl \
    --request POST \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://demo.immuta.com/redshift/handler

Payload example

{
  "handler": [{
    "metadata": {
      "ssl": true,
      "userFiles": [],
      "port": 5439,
      "hostname": "redshift.database.example.io",
      "database": "test",
      "username": "user",
      "password": "yourpassword",
      "schemaProjectName": "Army",
      "staleDataTolerance": 86400,
      "bodataSchemaName": "army",
      "bodataTableName": "army_records",
      "dataSourceName": "Army Army Records",
      "table": "army_records",
      "schema": "army",
      "nativeViewName": "army_records_immuta",
      "nativeSchemaName": "army_immuta",
      "nativeWorkspaceName": "immuta_test"
    }
  }, {
    "metadata": {
      "ssl": true,
      "userFiles": [],
      "port": 5439,
      "hostname": "redshift.database.example.io",
      "database": "test",
      "username": "user",
      "password": "your-password",
      "schemaProjectName": "Army",
      "staleDataTolerance": 86400,
      "bodataSchemaName": "army",
      "bodataTableName": "data",
      "dataSourceName": "Army Data",
      "table": "data",
      "schema": "army",
      "nativeViewName": "data_immuta",
      "nativeSchemaName": "army_immuta",
      "nativeWorkspaceName": "immuta_test"
    }
  }],
  "dataSource": {
    "blobHandler": {
      "scheme": "https",
      "url": ""
    },
    "blobHandlerType": "Redshift",
    "recordFormat": "",
    "type": "queryable",
    "schemaEvolutionId": null,
    "columnEvolutionEnabled": true
  },
  "schemaEvolution": {
    "ownerProfileId": 2,
    "config": {
      "nameTemplate": {
        "nameFormat": "<Schema> <Tablename>",
        "tableFormat": "<tablename>",
        "sqlSchemaNameFormat": "<schema>",
        "schemaProjectNameFormat": "<Schema>"
      }
    },
    "schemas": []
  }
}

Response example

{
  "connectionString":"user@redshift.database.example.io:5439/test"
}

Get information about a data source

GET /redshift/handler/{handlerId}

Get the handler metadata associated with the provided handler ID.

Query parameters

Attribute

Description

Required

Response parameters

Attribute

Description

Request example

This request returns metadata for the handler with the ID 41.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://demo.immuta.com/redshift/handler/41

Response Example

{
  "dataSourceId": 41,
  "metadata": {
    "ssl": true,
    "port": 5439,
    "query": null,
    "table": "data",
    "schema": "army",
    "database": "test",
    "hostname": "redshift.database.example.io",
    "username": "user",
    "eventTime": "fileinfo.creationtime",
    "userFiles": [],
    "dataSourceName": "Army Data",
    "bodataTableName": "data",
    "bodataSchemaName": "army",
    "columnsNormalized": false,
    "schemaProjectName": "Army",
    "staleDataTolerance": 86400
  },
  "type": "odbcHandler",
  "connectionString": "user@redshift.database.example.io:5439/test",
  "id": 41,
  "createdAt": "2021-10-01T18:15:09.139Z",
  "updatedAt": "2021-10-01T18:15:09.274Z",
  "dbms": {
    "name": "redshift"
  }
}

Manage data sources

Method

Path

Purpose

Update a specific data source

PUT /redshift/handler/{handlerId}

Query parameters

Attribute

Description

Required

Payload parameters

Attribute

Description

Required

Response parameters

Attribute

Description

Request example

This request updates the metadata for the data source with the handler ID 41.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://demo.immuta.com/redshift/handler/41

Payload example

The payload below removes the paragraph_count column from the data source.

{
  "handler": {
    "policyHandler": null,
    "dataSourceId": 41,
    "metadata": {
      "ssl": true,
      "port": 5439,
      "table": "data",
      "schema": "army",
      "database": "test",
      "hostname": "redshift.database.example.io",
      "username": "user",
      "eventTime": "fileinfo.creationtime",
      "userFiles": [],
      "dataSourceName": "Army Data",
      "bodataTableName": "data",
      "bodataSchemaName": "army",
      "columnsNormalized": false,
      "schemaProjectName": "Army",
      "staleDataTolerance": 86400,
      "columns": [{
        "name": "text",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "date",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "hashed_id",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "filename",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "storage_location",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "size",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "owner",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "dc:publisher",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "meta:page-count",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "publisher",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "author",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "extended-properties:template",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "application-version",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "revision-number",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "page-count",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "last-author",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "character-count-with-spaces",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "xmptpg:npages",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "creation-date",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "meta:last-author",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "extended-properties:application",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "meta:creation-date",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "meta:author",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "custom:contenttypeid",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "creator",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "x-parsed-by",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "content-type",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "modified",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "extended-properties:totaltime",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "application-name",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "meta:character-count-with-spaces",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "meta:save-date",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "line-count",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "template",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "meta:character-count",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "last-save-date",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "last-modified",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "dcterms:modified",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "meta:line-count",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "dcterms:created",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "word-count",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "extended-properties:company",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "dc:creator",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "meta:word-count",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "meta:paragraph-count",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "extended-properties:appversion",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "total-time",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "cp:revision",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "fileinfo.group",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "fileinfo.isdirectory",
        "dataType": "boolean",
        "remoteType": "boolean",
        "nullable": true
      }, {
        "name": "fileinfo.isother",
        "dataType": "boolean",
        "remoteType": "boolean",
        "nullable": true
      }, {
        "name": "fileinfo.permissions",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "fileinfo.issymboliclink",
        "dataType": "boolean",
        "remoteType": "boolean",
        "nullable": true
      }, {
        "name": "fileinfo.size",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "fileinfo.file",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "fileinfo.filename",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "fileinfo.filekey",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "fileinfo.isregularfile",
        "dataType": "boolean",
        "remoteType": "boolean",
        "nullable": true
      }, {
        "name": "fileinfo.creationtime",
        "dataType": "timestamp",
        "remoteType": "timestamp",
        "nullable": true
      }, {
        "name": "fileinfo.lastmodifiedtime",
        "dataType": "timestamp",
        "remoteType": "timestamp",
        "nullable": true
      }, {
        "name": "fileinfo.lastaccesstime",
        "dataType": "timestamp",
        "remoteType": "timestamp",
        "nullable": true
      }, {
        "name": "fileinfo.owner",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "topsecretboolean",
        "dataType": "double precision",
        "remoteType": "double precision",
        "nullable": true
      }, {
        "name": "secretboolean",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "confidentialsecretboolean",
        "dataType": "double precision",
        "remoteType": "double precision",
        "nullable": true
      }, {
        "name": "classificationlevel",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "releasability",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "compartment",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }],
      "password": "yourpassword"
    },
    "type": "odbcHandler",
    "connectionString": "user@redshift.database.example.io:5439/test",
    "id": 41,
    "createdAt": "2021-10-01T18:15:09.139Z",
    "updatedAt": "2021-10-01T18:15:09.274Z",
    "dbms": {
      "name": "redshift"
    }
  }
}

Response example

{
  "id": 41,
  "ca": ["-----BEGIN CERTIFICATE-----\ncertificatedata\n-----END CERTIFICATE-----"],
  "metadata": {
    "columns": [{
      "name": "text",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "date",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "hashed_id",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "filename",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "storage_location",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "size",
      "dataType": "integer",
      "remoteType": "integer",
      "nullable": true
    }, {
      "name": "owner",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "dc:publisher",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "meta:page-count",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "publisher",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "author",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "extended-properties:template",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "application-version",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "revision-number",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "page-count",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "last-author",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "character-count-with-spaces",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "xmptpg:npages",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "creation-date",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "meta:last-author",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "extended-properties:application",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "meta:creation-date",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "meta:author",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "custom:contenttypeid",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "creator",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "x-parsed-by",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "content-type",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "modified",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "extended-properties:totaltime",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "application-name",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "meta:character-count-with-spaces",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "meta:save-date",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "line-count",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "template",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "meta:character-count",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "last-save-date",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "last-modified",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "dcterms:modified",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "meta:line-count",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "dcterms:created",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "word-count",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "extended-properties:company",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "dc:creator",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "meta:word-count",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "meta:paragraph-count",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "extended-properties:appversion",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "total-time",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "cp:revision",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "fileinfo.group",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "fileinfo.isdirectory",
      "dataType": "boolean",
      "remoteType": "boolean",
      "nullable": true
    }, {
      "name": "fileinfo.isother",
      "dataType": "boolean",
      "remoteType": "boolean",
      "nullable": true
    }, {
      "name": "fileinfo.permissions",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "fileinfo.issymboliclink",
      "dataType": "boolean",
      "remoteType": "boolean",
      "nullable": true
    }, {
      "name": "fileinfo.size",
      "dataType": "integer",
      "remoteType": "integer",
      "nullable": true
    }, {
      "name": "fileinfo.file",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "fileinfo.filename",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "fileinfo.filekey",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "fileinfo.isregularfile",
      "dataType": "boolean",
      "remoteType": "boolean",
      "nullable": true
    }, {
      "name": "fileinfo.creationtime",
      "dataType": "timestamp",
      "remoteType": "timestamp",
      "nullable": true
    }, {
      "name": "fileinfo.lastmodifiedtime",
      "dataType": "timestamp",
      "remoteType": "timestamp",
      "nullable": true
    }, {
      "name": "fileinfo.lastaccesstime",
      "dataType": "timestamp",
      "remoteType": "timestamp",
      "nullable": true
    }, {
      "name": "fileinfo.owner",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "topsecretboolean",
      "dataType": "double precision",
      "remoteType": "double precision",
      "nullable": true
    }, {
      "name": "secretboolean",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "confidentialsecretboolean",
      "dataType": "double precision",
      "remoteType": "double precision",
      "nullable": true
    }, {
      "name": "classificationlevel",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "releasability",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "compartment",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }]
  }
}

Update multiple data sources

PUT /redshift/bulk

Update the data source metadata associated with the provided connection string.

Payload parameters

Attribute

Description

Required

Response parameters

Attribute

Description

Request example

This request updates the metadata for all data sources with the connection string specified in example-payload.json.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://demo.immuta.com/redshift/bulk

Payload example

The payload below adds a certificate (certificate.json) to connect to the data sources with the provided connection string.

{
  "handler": {
    "metadata": {
      "ssl": true,
      "port": 5439,
      "database": "test",
      "hostname": "redshift.database.example.io",
      "username": "user",
      "userFiles": [{
        "keyName": "testcert",
        "filename": "bf349cbbfd3bc4eaf4ace7824a599fbc7cb5f7d5.json",
        "userFilename": "certificate.json"
      }],
      "password": "yourpassword"
    }
  },
  "connectionString": "user@sredshift.database.example.io:5000/test"
}

Response example

{
  "bulkId": "bulk_ds_update_f50a8010dca03c07bcd0dc55eaf13c07",
  "connectionString": "yourusername@redshift.database.example.io:5000/tpc",
  "jobsCreated": 2
}

Recalculate the high cardinality column for a data source

PUT /redshift/handler/{handlerId}/triggerHighCardinalityJob

Recalculate the high cardinality column for the specified data source.

Query parameters

Attribute

Description

Required

Response parameters

The response returns a string of characters that identify the high cardinality job run.

Request example

This request re-runs the job that calculates the high cardinality column for the data source with the handler ID 41.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://demo.immuta.com/redshift/handler/41/triggerHighCardinalityJob

Response example

0f5f08d0-22ee-11ec-b2b8-874838eeef05

Refresh a native view

PUT /redshift/handler/{handlerId}/refreshNativeViewJob

Refresh the native view of a data source.

Query parameters

Attribute

Description

Required

Response parameters

The response returns a string of characters that identifies the refresh view job run.

Request example

This request refreshes the view for the data source with the handler ID 7.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://demo.immuta.com/redshift/handler/7/refreshNativeViewJob

Response example

53c256d0-eb57-11ec-b275-d95a8e998142