Create an Azure Synapse Analytics Data Source

Azure Synapse Analytics API reference guide

This page describes the asa (Azure Synapse Analytics data sources) endpoint.

Additional fields may be included in some responses you receive; however, these attributes are for internal purposes and are therefore undocumented.

ASA workflow

1. Create an Azure Synapse Analytics data source.

2. Search Azure Synapse Analytics data sources.

3. Update Azure Synapse Analytics data sources.

Create a data source

POST /asa/handler

Save the provided connection information as a data source.

Required Immuta permission: CREATE_DATA_SOURCE

Payload parameters

Attribute	Description	Required
private	boolean When false, the data source will be publicly available in the Immuta UI.	Yes
blobHandler	array[object] A list of full URLs providing the locations of all blob store handlers to use with this data source.	Yes
blobHandlerType	string Describes the type of underlying blob handler that will be used with this data source (e.g., MS SQL).	Yes
recordFormat	string The data format of blobs in the data source, such as json, xml, html, or jpeg.	Yes
type	string The type of data source: queryable (metadata is dynamically queried).	Yes
name	string The name of the data source. It must be unique within the Immuta tenant.	Yes
sqlTableName	string A string that represents this data source's table in Immuta.	Yes
organization	string The organization that owns the data source.	Yes
category	string The category of the data source.	No
description	string The description of the data source.	No
hasExamples	boolean When true, the data source contains examples.	No

Response parameters

Attribute	Description
id	integer The handler ID.
dataSourceId	integer The ID of the data source.
warnings	string This message describes issues with the created data source, such as the data source being unhealthy.
connectionString	string The connection string used to connect the data source to Immuta.

Request example

The following request saves the provided connection information as a data source.

curl \
    --request POST \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example_payload.json
    https://your-immuta-url.com/asa/handler

Request example payload

{
    "hits": [{
        "name": "Public Credit Accounts",
        "id": 1,
        "recordFormat": "Not Provided",
        "deleted": false,
        "description": null,
        "createdAt": "2021-09-09T14:12:09.511Z",
        "subscriptionPolicy": {
            "type": "subscription",
            "approvals": [{
                "requiredPermission": "OWNER",
                "specificApproverRequired": false
            }]
        },
        "schemaEvolutionId": 1,
        "recordCount": 0,
        "blobHandlerType": "Synapse",
        "subscriptionType": "approval",
        "sqlSchemaName": "public",
        "status": "failed",
        "subscriptionStatus": "owner",
        "connectionString": "[e-mail]:1433/public",
        "policy": "No Conflict",
        "policyHandlerType": "Builder",
        "native": null,
        "workspace": null
    }, {
        "name": "Public Credit Transactions",
        "id": 2,
        "recordFormat": "Not Provided",
        "deleted": false,
        "description": null,
        "createdAt": "2021-09-09T14:12:09.522Z",
        "subscriptionPolicy": {
            "type": "subscription",
            "approvals": [{
                "requiredPermission": "OWNER",
                "specificApproverRequired": false
            }]
        },
        "schemaEvolutionId": 1,
        "recordCount": 0,
        "blobHandlerType": "Synapse",
        "subscriptionType": "approval",
        "sqlSchemaName": "public",
        "status": "passed",
        "subscriptionStatus": "owner",
        "connectionString": "[e-mail]:1433/public",
        "policy": "No Conflict",
        "policyHandlerType": "Builder",
        "native": null,
        "workspace": null
    }, {
        "name": "Public Fake Medical Claims 2017",
        "id": 3,
        "recordFormat": "Not Provided",
        "deleted": false,
        "description": null,
        "createdAt": "2021-09-09T14:12:09.894Z",
        "subscriptionPolicy": null,
        "schemaEvolutionId": 1,
        "recordCount": 0,
        "blobHandlerType": "Synapse",
        "subscriptionType": "manual",
        "sqlSchemaName": "public",
        "status": "passed",
        "subscriptionStatus": "owner",
        "connectionString": "[e-mail]:1433/public",
        "policy": "No Conflict",
        "policyHandlerType": "Builder",
        "native": null,
        "workspace": null
    }, {
        "name": "Public Uciml Census Income Enriched",
        "id": 4,
        "recordFormat": "Not Provided",
        "deleted": false,
        "description": null,
        "createdAt": "2021-09-09T14:12:09.916Z",
        "subscriptionPolicy": null,
        "schemaEvolutionId": 1,
        "recordCount": 0,
        "blobHandlerType": "Synapse",
        "subscriptionType": "manual",
        "sqlSchemaName": "public",
        "status": "passed",
        "subscriptionStatus": "owner",
        "connectionString": "[e-mail]:1433/public",
        "policy": "None",
        "policyHandlerType": "None",
        "native": null,
        "workspace": null
    }],
    "facets": {
        "tags": [],
        "statuses": [{
            "name": "failed",
            "count": 0
        }, {
            "name": "passed",
            "count": 0
        }],
        "blobHandlerTypes": [{
            "name": "Synapse",
            "count": 0
        }],
        "editable": [{
            "name": "owned",
            "count": 4
        }, {
            "name": "expert",
            "count": 0
        }, {
            "name": "notSubscribed",
            "count": 0
        }, {
            "name": "disabled",
            "count": 0
        }]
    },
    "count": 4
}

Response example

{
  "connectionString": "[e-mail]:1433/public"
}

Search ASA data sources

Search for handler metadata

GET /asa/handler/{handlerId}

Get the handler metadata associated with the provided handler ID.

Query parameters

Attribute	Description	Required
handlerId	integer The specific handler ID.	Yes
skipCache	boolean If true the handler cache will be skipped when retrieving the handler data.	No

Response parameters

Attribute	Description
dataSourceId	integer The data source ID.
value	metadata Details regarding the handler.

Request example

The following request returns the handler metadata associated with the provided handler ID.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/asa/handler/67

Response example

{
  "dataSourceId": 67,
  "metadata": {
    "ssl": true,
    "port": 1433,
    "query": null,
    "table": "table_108",
    "schema": "elliott_dev",
    "database": "your_database",
    "hostname": "your-workspace.sql.azuresynapse.net",
    "username": "your-username",
    "eventTime": null,
    "dataSourceName": "Dev Table 108",
    "bodataTableName": "table_108",
    "disableClassify": false,
    "highCardinality": "col_1",
    "bodataSchemaName": "dev",
    "columnsNormalized": false,
    "schemaProjectName": "Dev",
    "staleDataTolerance": 2592000
  },
  "type": "queryable",
  "connectionString": "[e-mail]:1433/public",
  "id": 67,
  "createdAt": "2021-09-17T14:15:10.256Z",
  "updatedAt": "2021-09-17T14:15:19.547Z",
  "dbms": {
    "name": "asa"
  }
}

Update ASA data sources

Endpoint	Purpose
/asa/handler/{handlerId}	Updates the handler metadata associated with the provided handler ID.
/asa/bulk	Updates the data source metadata associated with the provided connection string.
/asa/handler/{handlerId}/triggerHighCardinalityJob	Recalculates the high cardinality column for the provided handler ID.
/asa/handler/{handlerId}/refreshNativeViewJob	Refresh the view of a data source.

Update handler metadata

PUT /asa/handler/{handlerId}

Updates the handler metadata associated with the provided handler ID. This endpoint does not perform partial updates, but will allow the dictionary to be omitted. In this case it uses the current dictionary.

Required: The global GOVERNANCE permission or be the data source owner

Query parameters

Attribute	Description	Required
handlerId	integer The specific handler ID.	Yes
skipCache	boolean If true the handler cache will be skipped when retrieving the handler data.	No

Response parameters

Attribute	Description
dataSourceId	integer The data source ID.
body	array[object] Details regarding the handler, including schema, name format, and data source metadata.

Request example

The following request updates the handler metadata (saved in example_payload.json) associated with the provided handler ID.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example_payload.json
    https://your-immuta-url.com/asa/handler/67

Request payload example

{
    "schemaEvolution": {
        "schemas": [
            "string"
        ],
        "disabled": false,
        "ownerProfileId": 0,
        "config": {
            "nameTemplate": {
                "tableFormat": "string",
                "nameFormat": "string",
                "sqlSchemaNameFormat": "string",
                "schemaProjectNameFormat": "string"
            }
        },
        "dataSourceConfig": {},
        "handlerMetadata": {},
        "connectionString": "string"
    },
    "handler": {
        "id": 0,
        "type": "string",
        "dataSourceId": 0,
        "metadata": {
            "blobId": "string",
            "eventTime": "string",
            "highCardinality": "string",
            "bodataSchemaName": "immuta",
            "bodataTableName": "string",
            "format": "string",
            "disableClassify": false,
            "staleDataTolerance": 2592000,
            "dataSourceName": "string",
            "schemaProjectName": "string",
            "username": "string",
            "password": "string",
            "ssl": false,
            "database": "string",
            "schema": "string",
            "table": "string",
            "query": "string",
            "columns": [{
                "name": "string",
                "dataType": "string",
                "remoteColumn": "string",
                "remoteType": "string",
                "srid": 0,
                "statistics": {},
                "nullable": false,
                "isPrimaryKey": false,
                "displayName": "string",
                "description": "string",
                "tags": [
                    "string"
                ],
                "catalogMetadata": {},
                "children": [
                    "string"
                ]
            }],
            "columnsNormalized": false,
            "nativeWorkspaceName": "string",
            "nativeSchemaName": "string",
            "nativeViewName": "string",
            "hostname": "string",
            "port": 0,
            "userFiles": [{
                "keyName": "string",
                "filename": "string",
                "content": "string",
                "userFilename": "string"
            }],
            "scheme": "string",
            "warehouse": "string",
            "connectionStringOptions": "string",
            "native": {
                "type": "string",
                "projectId": 0
            },
            "secureNativeView": false
        },
        "dbms": {
            "name": "string"
        },
        "connectionString": "string",
        "createdAt": "2021-09-20",
        "updatedAt": "2021-09-20",
        "policyHandler": {
            "visibilitySchema": {
                "fields": [
                    "string"
                ],
                "version": "2021-09-20"
            }
        }
    }
}

Response example

{
  "dataSourceId": 67,
  "metadata": {
    "ssl": true,
    "port": 1433,
    "query": null,
    "table": "table_108",
    "schema": "dev",
    "database": "your_database",
    "hostname": "your-workspace.sql.azuresynapse.net",
    "username": "your-username",
    "eventTime": null,
    "dataSourceName": "Dev Table 108",
    "bodataTableName": "table_108",
    "disableClassify": false,
    "highCardinality": "col_1",
    "bodataSchemaName": "dev",
    "columnsNormalized": false,
    "schemaProjectName": "Dev",
    "staleDataTolerance": 2592000
  },
  "type": "queryable",
  "connectionString": "[e-mail]:1433/public",
  "id": 67,
  "createdAt": "2021-09-17T14:15:10.256Z",
  "updatedAt": "2021-09-17T14:15:19.547Z",
  "dbms": {
    "name": "asa"
  }
}

Update multiple data sources

PUT /asa/bulk

Updates the data source metadata associated with the provided connection string.

Required: The global GOVERNANCE permission or be the data source owner

Query parameters

Attribute	Description	Required
body	array[object] This payload includes data source metadata and specifies the connection string.	Yes

Response parameters

Attribute	Description
bulkId	integer The bulk handler metadata ID.
connectionString	string The specified connection string.
jobsCreated	integer Number of jobs created.

Request example

The following request updates the handler metadata for the handler ID specified in example_payload.json.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example_payload.json
    https://your-immuta-url.com/asa/bulk

Request payload example

{
    "ids": [
        0
    ],
    "connectionString": "string",
    "handler": {
        "metadata": {
            "blobId": "string",
            "eventTime": "string",
            "highCardinality": "string",
            "bodataTableName": "string",
            "format": "string",
            "disableClassify": false,
            "staleDataTolerance": 2592000,
            "dataSourceName": "string",
            "schemaProjectName": "string",
            "username": "string",
            "password": "string",
            "ssl": false,
            "database": "string",
            "schema": "string",
            "table": "string",
            "query": "string",
            "columns": [{
                "name": "string",
                "dataType": "string",
                "remoteColumn": "string",
                "remoteType": "string",
                "srid": 0,
                "statistics": {},
                "nullable": false,
                "isPrimaryKey": false,
                "displayName": "string",
                "description": "string",
                "tags": [
                    "string"
                ],
                "catalogMetadata": {},
                "children": [
                    "string"
                ]
            }],
            "columnsNormalized": false,
            "nativeWorkspaceName": "string",
            "nativeSchemaName": "string",
            "nativeViewName": "string",
            "authenticationMethod": "string",
            "hostname": "string",
            "port": 0,
            "sid": "string",
            "connectionStringOptions": "string",
            "paths": [
                "string"
            ],
            "clusterName": "string",
            "pathUris": [
                "string"
            ],
            "metastoreTables": [
                "string"
            ],
            "scheme": "string",
            "providers": "string",
            "ephemeral": false,
            "httpPath": "string",
            "userFiles": [{
                "keyName": "string",
                "filename": "string",
                "content": "string",
                "userFilename": "string"
            }],
            "warehouse": "string",
            "workspaceId": 0,
            "authDB": "string",
            "directory": "string",
            "secureNativeView": false,
            "bodataSchemaName": "string"
        }
    },
    "schemaEvolution": {
        "schemas": [
            "string"
        ],
        "disabled": false,
        "ownerProfileId": 0,
        "config": {
            "nameTemplate": {
                "tableFormat": "string",
                "nameFormat": "string",
                "sqlSchemaNameFormat": "string",
                "schemaProjectNameFormat": "string"
            }
        },
        "dataSourceConfig": {},
        "handlerMetadata": {},
        "connectionString": "string"
    }
}

Response example

{
  "bulkId": "bulk_ds_update_fe48d7fd4c594f96a89438cdb84ec0ba",
  "connectionString": "string@string:0/string",
  "jobsCreated": 1
}

Recalculate high cardinality columns

PUT /asa/handler/{handlerId}/triggerHighCardinalityJob

Recalculates the high cardinality column for the provided handler ID.

Required: The global GOVERNANCE permission or be the data source owner

Query parameters

Attribute	Description	Required
handlerId	integer The specific handler ID.	Yes

Response parameters

The response returns a string of characters that identify the high cardinality job run.

Request example

The following request recalculates the high cardinality column for the provided handler ID.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://your-immuta-url.com/asa/handler/67/triggerHighCardinalityJob

Response example

25424a50-17df-11ec-b388-0fe1d33b5af1

Refresh a view

PUT /asa/handler/{handlerId}/refreshNativeViewJob

Refresh the view of a data source.

Required: The global GOVERNANCE permission or be the data source owner

Query parameters

Attribute	Description	Required
handlerId	integer The ID of the handler.	Yes

Response parameters

The response returns a string of characters that identifies the refresh view job run.

Request example

This request refreshes the view for the data source with the handler ID 7.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://demo.immuta.com/asa/handler/7/refreshNativeViewJob

Response example

53c256d0-eb57-11ec-b275-d95a8e998142