Create a Redshift Data Source

Redshift data source API reference guide

The redshift endpoint allows you to connect and manage Redshift data sources in Immuta.

Additional fields may be included in some responses you receive; however, these attributes are for internal purposes and are therefore undocumented.

Redshift workflow

Create a data source

POST /redshift/handler

Save the provided connection information as a data source.

Payload parameters

Response parameters

Request example

This request creates two Redshift data sources, which are specified in example-payload.json.

curl \
    --request POST \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://demo.immuta.com/redshift/handler

Payload example

{
  "handler": [{
    "metadata": {
      "ssl": true,
      "userFiles": [],
      "port": 5439,
      "hostname": "redshift.database.example.io",
      "database": "test",
      "username": "user",
      "password": "yourpassword",
      "schemaProjectName": "Army",
      "staleDataTolerance": 86400,
      "bodataSchemaName": "army",
      "bodataTableName": "army_records",
      "dataSourceName": "Army Army Records",
      "table": "army_records",
      "schema": "army",
      "nativeViewName": "army_records_immuta",
      "nativeSchemaName": "army_immuta",
      "nativeWorkspaceName": "immuta_test"
    }
  }, {
    "metadata": {
      "ssl": true,
      "userFiles": [],
      "port": 5439,
      "hostname": "redshift.database.example.io",
      "database": "test",
      "username": "user",
      "password": "your-password",
      "schemaProjectName": "Army",
      "staleDataTolerance": 86400,
      "bodataSchemaName": "army",
      "bodataTableName": "data",
      "dataSourceName": "Army Data",
      "table": "data",
      "schema": "army",
      "nativeViewName": "data_immuta",
      "nativeSchemaName": "army_immuta",
      "nativeWorkspaceName": "immuta_test"
    }
  }],
  "dataSource": {
    "blobHandler": {
      "scheme": "https",
      "url": ""
    },
    "blobHandlerType": "Redshift",
    "recordFormat": "",
    "type": "queryable",
    "schemaEvolutionId": null,
    "columnEvolutionEnabled": true
  },
  "schemaEvolution": {
    "ownerProfileId": 2,
    "config": {
      "nameTemplate": {
        "nameFormat": "<Schema> <Tablename>",
        "tableFormat": "<tablename>",
        "sqlSchemaNameFormat": "<schema>",
        "schemaProjectNameFormat": "<Schema>"
      }
    },
    "schemas": []
  }
}

Response example

{
  "connectionString":"user@redshift.database.example.io:5439/test"
}

Get information about a data source

GET /redshift/handler/{handlerId}

Get the handler metadata associated with the provided handler ID.

Query parameters

Response parameters

Request example

This request returns metadata for the handler with the ID 41.

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    https://demo.immuta.com/redshift/handler/41

Response Example

{
  "dataSourceId": 41,
  "metadata": {
    "ssl": true,
    "port": 5439,
    "query": null,
    "table": "data",
    "schema": "army",
    "database": "test",
    "hostname": "redshift.database.example.io",
    "username": "user",
    "eventTime": "fileinfo.creationtime",
    "userFiles": [],
    "dataSourceName": "Army Data",
    "bodataTableName": "data",
    "bodataSchemaName": "army",
    "columnsNormalized": false,
    "schemaProjectName": "Army",
    "staleDataTolerance": 86400
  },
  "type": "odbcHandler",
  "connectionString": "user@redshift.database.example.io:5439/test",
  "id": 41,
  "createdAt": "2021-10-01T18:15:09.139Z",
  "updatedAt": "2021-10-01T18:15:09.274Z",
  "dbms": {
    "name": "redshift"
  }
}

Manage data sources

Update a specific data source

PUT /redshift/handler/{handlerId}

Update the data source metadata associated with the provided handler ID. This endpoint does not perform partial updates, but will allow the dictionary to be omitted. In this case, it uses the current dictionary.

Query parameters

Payload parameters

Response parameters

Request example

This request updates the metadata for the data source with the handler ID 41.

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://demo.immuta.com/redshift/handler/41

Payload example

The payload below removes the paragraph_count column from the data source.

{
  "handler": {
    "policyHandler": null,
    "dataSourceId": 41,
    "metadata": {
      "ssl": true,
      "port": 5439,
      "table": "data",
      "schema": "army",
      "database": "test",
      "hostname": "redshift.database.example.io",
      "username": "user",
      "eventTime": "fileinfo.creationtime",
      "userFiles": [],
      "dataSourceName": "Army Data",
      "bodataTableName": "data",
      "bodataSchemaName": "army",
      "columnsNormalized": false,
      "schemaProjectName": "Army",
      "staleDataTolerance": 86400,
      "columns": [{
        "name": "text",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "date",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "hashed_id",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "filename",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "storage_location",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "size",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "owner",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "dc:publisher",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "meta:page-count",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "publisher",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "author",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "extended-properties:template",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "application-version",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "revision-number",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "page-count",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "last-author",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "character-count-with-spaces",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "xmptpg:npages",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "creation-date",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "meta:last-author",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "extended-properties:application",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "meta:creation-date",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "meta:author",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "custom:contenttypeid",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "creator",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "x-parsed-by",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "content-type",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "modified",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "extended-properties:totaltime",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "application-name",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "meta:character-count-with-spaces",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "meta:save-date",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "line-count",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "template",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "meta:character-count",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "last-save-date",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "last-modified",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "dcterms:modified",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "meta:line-count",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "dcterms:created",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "word-count",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "extended-properties:company",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "dc:creator",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "meta:word-count",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "meta:paragraph-count",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "extended-properties:appversion",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "total-time",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "cp:revision",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "fileinfo.group",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "fileinfo.isdirectory",
        "dataType": "boolean",
        "remoteType": "boolean",
        "nullable": true
      }, {
        "name": "fileinfo.isother",
        "dataType": "boolean",
        "remoteType": "boolean",
        "nullable": true
      }, {
        "name": "fileinfo.permissions",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "fileinfo.issymboliclink",
        "dataType": "boolean",
        "remoteType": "boolean",
        "nullable": true
      }, {
        "name": "fileinfo.size",
        "dataType": "integer",
        "remoteType": "integer",
        "nullable": true
      }, {
        "name": "fileinfo.file",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "fileinfo.filename",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "fileinfo.filekey",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "fileinfo.isregularfile",
        "dataType": "boolean",
        "remoteType": "boolean",
        "nullable": true
      }, {
        "name": "fileinfo.creationtime",
        "dataType": "timestamp",
        "remoteType": "timestamp",
        "nullable": true
      }, {
        "name": "fileinfo.lastmodifiedtime",
        "dataType": "timestamp",
        "remoteType": "timestamp",
        "nullable": true
      }, {
        "name": "fileinfo.lastaccesstime",
        "dataType": "timestamp",
        "remoteType": "timestamp",
        "nullable": true
      }, {
        "name": "fileinfo.owner",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "topsecretboolean",
        "dataType": "double precision",
        "remoteType": "double precision",
        "nullable": true
      }, {
        "name": "secretboolean",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "confidentialsecretboolean",
        "dataType": "double precision",
        "remoteType": "double precision",
        "nullable": true
      }, {
        "name": "classificationlevel",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "releasability",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }, {
        "name": "compartment",
        "dataType": "text",
        "remoteType": "string",
        "nullable": true
      }],
      "password": "yourpassword"
    },
    "type": "odbcHandler",
    "connectionString": "user@redshift.database.example.io:5439/test",
    "id": 41,
    "createdAt": "2021-10-01T18:15:09.139Z",
    "updatedAt": "2021-10-01T18:15:09.274Z",
    "dbms": {
      "name": "redshift"
    }
  }
}

Response example

{
  "id": 41,
  "ca": ["-----BEGIN CERTIFICATE-----\ncertificatedata\n-----END CERTIFICATE-----"],
  "metadata": {
    "columns": [{
      "name": "text",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "date",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "hashed_id",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "filename",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "storage_location",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "size",
      "dataType": "integer",
      "remoteType": "integer",
      "nullable": true
    }, {
      "name": "owner",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "dc:publisher",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "meta:page-count",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "publisher",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "author",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "extended-properties:template",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "application-version",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "revision-number",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "page-count",
      "dataType": "text",
      "remoteType": "string",
      "nullable": true
    }, {
      "name": "last-author",
      "dataType": "text",
      "remoteType": "string",