Skip to content

Connect Data Sources Using dbt Cloud Integration (Preview Feature)

Audience: Data Owners

Content Summary: This tutorial outlines the process of connecting a dbt Cloud integration. This integration allows Immuta to be connected to your dbt Cloud jobs so that updates run through dbt populate within Immuta as new data sources, column descriptions, data source descriptions, and tags.

This dbt Cloud integration is a Preview Feature that must be enabled by an Application Admin on the App Settings page.

1 - Connect a New dbt Integration

  1. Click the plus button in the top left of the Immuta console.
  2. Select New dbt Cloud Integration.

2 - Enter dbt Connection Information

  1. Input the API Key and the Account ID.
  2. Click Get Projects.

    dbt Get Projects

3 - Enter dbt Job Information

  1. Use the dropdown menu to select the dbt Project and dbt Job.

    dbt Job Information

  2. Add the webhook URL to any dbt job in your selected project with the same environment as the selected job to update Immuta data sources after each run.

4 - Enter Credentials

To connect the dbt Cloud project in Immuta, enter the credentials used to authenticate to the underlying database in dbt Cloud.

  1. Enter the Databricks API Token used to authenticate to the database in dbt Cloud.
  2. Opt to enter any Additional Connection String Options.
  3. Click Test Connection.
  1. Enter the Username and Password used to authenticate to the database in dbt Cloud.
  2. Opt to enter any Additional Connection String Options or Upload Certificates.
  3. Click Test Connection.

Select the Authentication Method from the dropdown menu.

  1. Enter the Snowflake Username and Password used to authenticate to the database in dbt Cloud.
  2. Opt to enter any Additional Connection String Options or Upload Certificates.
  3. Click Test Connection.
  1. Enter the Snowflake Username used to authenticate to the database in dbt Cloud.
  2. Click Select a File.
  3. Use the pop-up to select and upload a key pair file.
  4. Click Test Connection.

5 - Enter Basic Information

Provide information about your source to make it discoverable to users.

  1. Enter the SQL Schema Name Format to be the SQL name that the data sources pulled from the dbt integration will exist under in the Immuta Query Engine. It must include a schema macro but you may personalize the format using lowercase letters, numbers, and underscores. It may have up to 255 characters.
  2. Enter the Schema Project Name Format to be the name of the schema project in the Immuta UI. This field is disabled if the schema project already exists within Immuta.

  3. Select the Data Source Name Format, which will be the format of the name of the data sources pulled from the dbt integration in the Immuta UI.

    <Tablename>

    The data source name will be the name of the remote table, and the case of the data source name will match the case of the macro.

    <Schema><Tablename>

    The data source name will be the name of the remote schema followed by the name of the remote table, and the case of the data source name will match the cases of the macros.

    Custom

    Enter a custom template for the Data Source Name. You may personalize this field as you wish, but it must include a tablename macro. The case of the macro will apply to the data source name (i.e., <Tablename> will result in "Data Source Name," <tablename> will result in "data source name," and <TABLENAME> will result in "DATA SOURCE NAME").

    Custom Schema Name

  4. Enter the SQL Table Name Format, which will be the format of the name of the tables in the Immuta Query Engine. It must include a table name macro, but you may personalize the format using lowercase letters, numbers, and underscores. It may have up to 255 characters.

    Data Source Creation Basic Information

What's Next

Now that you've created a date source, you can choose to continue to the next page or to one of these tutorials:

Manage Data Sources Write a Local Policy