Create an Azure Blob Storage Data Source
Azure Blob Storage data source API reference guide
The azureblob
endpoint allows you to connect and manage Azure Blob Storage data sources in Immuta.
Additional fields may be included in some responses you receive; however, these attributes are for internal purposes and are therefore undocumented.
Azure Blob workflow
Create a data source
POST
/azureblob/handler
Save the provided connection for an Azure Blob Storage data source.
Payload parameters
Response parameters
Request example
The following request saves the provided connection information (in example-payload.json
) as a data source.
curl \
--request POST \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
--data @example-payload.json \
https://your-immuta-url.com/azureblob/handler
Request payload example
{
"handler": {
"metadata": {
"tagAttributes": [],
"eventTimeAttribute": "",
"useDirectoryForTags": false,
"sasToken": "?sv=your=sas?token",
"sasTokenUrl": "https://your.blob.example.windows.net/sastoken-url",
"container": "demodata"
}
},
"dataSource": {
"blobHandler": {
"scheme": "https",
"url": ""
},
"blobHandlerType": "Azure Blob Storage",
"recordFormat": "",
"type": "ingested",
"name": "dev",
"sqlTableName": "dev"
}
}
Response example
{
"id": 18,
"dataSourceId": 18
}
Get information about a data source
GET
/azureblob/handler/{handlerId}
Return the handler metadata associated with the provided handler ID.
Query parameters
Response parameters
Request example
The following request returns the handler metadata associated with the provided handler ID.
curl \
--request GET \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
https://your-immuta-url.com/azureblob/handler/67
Response example
{
"dataSourceId": 427,
"metadata": {
"container": "integration",
"accountName": "integration-tests",
"sasTokenUrl": "https://your.blob.example.windows.net/",
"ingestUserId": "azure blob storage_indexer_example",
"tagAttributes": [],
"dataSourceName": "Test",
"refreshInterval": 0,
"eventTimeAttribute": "",
"useDirectoryForTags": false
},
"type": "azureBlobStorageHandler",
"connectionString": "integration-tests/integration",
"id": 427,
"createdAt": "2021-09-22T18:45:47.744Z",
"updatedAt": "2021-09-22T18:45:47.969Z"
}
Manage data sources
Update a specific data source
PUT
/azureblob/handler/{handlerId}
Update the provided information for an Azure Blob Storage data source.
Query parameters
Response parameters
Request example
The following request with the payload below updates the metadata for the data source with the handler ID 18
.
curl \
--request PUT \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
--data @example-payload.json \
https://your-immuta-url.com/azureblob/handler/18
Payload example
{
"dataSourceId": 18,
"metadata": {
"container": "testdata",
"accountName": "integration-tests",
"sasTokenUrl": "https://your.blob.example.windows.net/",
"ingestUserId": "azure blob storage_indexer_example",
"tagAttributes": [],
"dataSourceName": "dev",
"refreshInterval": 0,
"eventTimeAttribute": "",
"useDirectoryForTags": false
},
"type": "azureBlobStorageHandler",
"connectionString": "your/testdata",
"id": 18,
"createdAt": "2021-09-23T18:47:52.976Z",
"updatedAt": "2021-09-23T18:47:53.194Z"
}
Response example
{
"id": 18,
"dataSourceId": 18,
"metadata": {
"sasToken": "2:your?sastoken==",
"container": "testdata",
"accountName": "your-account-name",
"sasTokenUrl": "2:your?sastokenurlTS",
"ingestAPIKey": "996samplee89c1apia7ckey9",
"ingestUserId": "azure blob storage_indexer_example",
"tagAttributes": [],
"dataSourceName": "dev",
"refreshInterval": 0,
"eventTimeAttribute": "",
"useDirectoryForTags": false
}
}
Update multiple data sources
PUT
/azureblob/bulk
Update the data source metadata associated with the provided connection string.
Payload parameters
Response parameters
Request example
The following request updates the autoIngest
value to true
for data sources with the connection string specified in the payload below.
curl \
--request PUT \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
--data @example-payload.json \
https://your-immuta-url.com/azureblob/bulk
Payload example
{
"ids": [
5, 6
],
"connectionString": "integration-tests/integration",
"handler": {
"metadata": {
"autoIngest": true
}
}
}
Response example
{
"bulkId": "bulk_ds_update_dd2600809bf8418dbea2706d6f456636",
"connectionString": "integration-tests/integration",
"jobsCreated": 0
}
Re-crawl the data source
PUT
/azureblob/handler/{handlerId}/crawl
Re-crawls the data source and updates the metadata.
Query parameters
Response parameters
The response returns a string of characters that identify the job run.
Request example
The following request re-crawls the data source.
curl \
--request PUT \
--header "Content-Type: application/json" \
--header "Authorization: Bearer dea464c07bd07300095caa8" \
https://your-immuta-url.com/azureblob/hanfler/427/crawl
Response example
a4de5af0-1be1-11ec-8131-6fe77107bfa9