Data Source Payload Attribute Details

connectionKey

The connectionKey is a unique identifier for the collection of data sources being created. If an existing connectionKey is used with new connection information, it will delete the old data sources and create new ones from the new information in the payload.

connection

Special Cases

• Athena: Also requires region and queryResultLocationBucket. queryResultLocationDirectory is optional. authenticationMethod can be none, accessKey (default: username = access key, password = secret key), or instanceRole.

• BigQuery: Does not require hostname and password. Requires sid, which is the GCP project ID, and userFiles with the keyName of KeyFilePath and the base64-encoded keyfile.json.

• Trino: authenticationMethod can be No Authentication, LDAP Authentication, or Kerberos Authentication.

nameTemplate

Available templates include

• <tablename>

• <schema>

• <database>

All cases of the name in Immuta should be lowercase.

For example, consider a table TPC.CUSTOMER that is given the following nameTemplate:

dataSourceFormat: <schema> <tablename>
tableFormat: <tablename>
schemaFormat: <schema>
schemaProjectNameFormat: <schema>

This nameTemplate will produce a data source named tpc.customer in a schema project named tpc.

options

owners

sources

This attribute configures which sources are created. If sources is not provided, all sources from the given connection will be created.

There are 3 types of sources than can be specified:

• all tables

• query

• table

Recommended: Specify All Tables

If you specify any sources (either tables or queries), but you still want to create data sources for the rest of the tables in the schema or database, you can specify all as a source:

sources:
  - all: true

Specify a Query

Immuta recommends creating a view in your native database instead of using this option, but if that is not possible, you can create data sources based on SQL statements:

sources:
  - query: “select * from table”
    naming:
      datasource: “My Source”,
      table: “my_source”,
      schema: “queries”

Specify a Table

If you want to select specific tables to be created as data sources, or if you want to tag individual data sources or columns within a data source, you need to leverage this parameter:

sources:
  - table: name_of_table
    schema: name_of_schema

Additional Options

When specifying a table or query there are other options that can be specified:

Columns

• If any columns are specified, those are the only columns that will be available in the data source.

• If no columns are specified, Immuta will look for new or removed columns on a schedule (by default, once a day) and add or remove columns from the data sources automatically as needed.

• New columns will be tagged New, so you can build a policy to automatically mask new columns until they are approved.

• Data Owners will be notified when columns are added or removed.

columns is an array of objects for each column:

Column Descriptions

You can add descriptions to columns without having to specify all the columns in the data source. columnDescriptions is an array of objects with the following schema:

columnDescriptions:
  - columnName: acct_num
    description: The account number

Tags

You can add tags to columns or data sources. tags is an object with the following schema:

tags:
  table:
    - Sensitive
    - Marketing
  columns:
    - columnName: acct_num
      tags:
        - unique_id