Connections are an improvement from the existing process for not only onboarding your data sources but also managing the integration. However, there are some differences between the two processes that should be noted and understood before you start with the upgrade.
API changes: See the API changes pages for a complete breakdown of the APIs that will not work once you begin the upgrade. These changes will mostly affect users with automated API calls around schema monitoring and data source registration.
Automated data source names: Previously, you could name data sources manually. However, data sources from connections are automatically named using the information (database, schema, table) from your data platform.
Schema projects phased out: With integrations, many settings and the connection info for data sources were controlled in the schema project. This functionality is no longer needed with connections and now you can control connection details in a central spot.
New hierarchy display: With integrations, tables were brought in as data sources and presented as a flat list on the data source list page. With connections, databases and schemas are displayed as objects too.
Change from schema monitoring to object sync: Object metadata synchronization between Immuta and your data platform is no longer optional but always required:
If schema monitoring is off before the upgrade: Once the connection is registered, everything the system user can see will be pulled into Immuta and, if it didn't already exist in Immuta, it will be an inactive object. These inactive objects exist so you can see them, but policy is not protecting the objects, and they will not appear as data sources.
If schema monitoring is on before the upgrade: Once the connection is registered, everything the system user can see will be pulled into Immuta. If it already existed in Immuta, it will be an active object and continue to appear as data source.
Public preview
This feature is public preview and available to select accounts. Reach out to your Immuta support professional to enable it on your tenant.
Connections allow you to register your data objects in a technology through a single connection, making data registration more scalable for your organization. Instead of registering schema and databases individually, you can register them all at once and allow Immuta to monitor your data platform for changes so that data sources are added and removed automatically to reflect the state of data on your data platform.
Once you register your connection, Immuta presents a hierarchical view of your data that reflects the hierarchy of objects in your data platform:
Account (Snowflake) or Metastore (Databricks Unity Catalog)
Database
Schema
Tables: These represent the individual objects in your data platform, and when active, become data sources
Beyond making the registration of your data more intuitive, connections provides more control. Instead of performing operations on individual schemas or tables, you can perform operations (such as object sync) at the connection level.
See the Snowflake or Databricks Unity Catalog connection registration how-to guides for a list of requirements.
With connections, you configure the integration and register data sources simultaneously. Once you save your connection, Immuta manages and applies Snowflake or Unity Catalog governance features to data objects registered in Immuta.
Then, Immuta crawls your connection to register all tables within every schema and database that the Snowflake role or Databricks account credentials you provided during the registration has access to. The object metadata, user metadata, and policy definitions are stored in the Immuta metadata database, and this metadata is used to enforce policies for users accessing this data.
After initial registration, your connection can be crawled in two ways:
Periodic crawl: This crawl happens once every 24 hours. Currently, updating this schedule is not configurable.
Manual crawl: You can manually trigger a crawl of your connection.
During these subsequent crawls of your connection, Immuta identifies tables, schemas, or databases that have been added or removed. If tables are added, new data sources are created in Immuta. If remote tables are deleted, the corresponding data sources and data objects will be removed from Immuta.
For more information about the Snowflake or Databricks Unity Catalog integration and and how policies are enforced, see the Snowflake integration reference guide or Databricks Unity Catalog integration reference guide.
When there is an active policy that targets the New tag, Immuta sends validation requests to data owners for the following changes made in the remote data platform:
Column added: Immuta applies the New tag on the column that has been added and sends a request to the data owner to validate if the new column contains sensitive data. Once the data owner confirms they have validated the content of the column, Immuta removes the New tag from it and as a result any policy that targets the New column tag no longer applies.
Column data type changed: Immuta applies the New tag on the column where the data type has been changed and sends a request to the data owner to validate if the column contains sensitive data. Once the data owner confirms they have validated the content of the column, Immuta removes the New tag from it and as a result any policy that targets the New column tag no longer applies.
Column deleted: Immuta deletes the column from the data source's data dictionary in Immuta. Then, Immuta sends a request to the data owner to validate the deleted column.
Data source created: Immuta applies the New tag on the data source that has been newly created and sends a request to the data owner to validate if the new data source contains sensitive data. Once the data owner confirms they have validated the content of the data source, Immuta removes the New tag from it and as a result any policy that targets the New data source tag no longer applies.
For instructions on how to view and manage your tasks and requests in the Immuta UI, see the Manage access requests guide. To view and manage your tasks and requests via the Immuta API, see the Manage data source requests section of the API documentation.
When registering a connection, Immuta sets the connection to the recommended default settings to protect your . The recommended settings are described below:
Object sync: This setting allows Immuta to monitor the connection for changes. When Immuta identifies a new table, a data source will automatically be created. Similarly, if remote tables are deleted, the corresponding data sources and data objects will be deleted in Immuta. This setting is enabled by default and cannot be disabled.
Default run schedule: This sets the time interval for Immuta to check for new objects. By default, this schedule is set to 24 hours.
Sensitive data discovery: This setting enables sensitive data discovery and allows you to select the sensitive data discovery framework that Immuta will apply to your data objects. This setting is enabled by default to use the preconfigured or global framework.
Impersonation: This setting enable and defines the role for user impersonation in Snowflake. User impersonation is not supported in the Databricks Unity Catalog integration. This setting is disabled by default.
Project workspaces: This setting enables Snowflake project workspaces. If you use Snowflake secure data sharing with Immuta, enable this setting, as project workspaces are required. If you use Snowflake table grants, disable this setting; project workspaces cannot be used when Snowflake table grants are enabled. Project workspaces are not supported in the Databricks Unity Catalog integration. This setting is disabled by default.
Deregistering a connection automatically deletes all of its child objects in Immuta. However, Immuta will not remove the objects in your Snowflake or Databricks account.
Snowflake and Databricks Unity Catalog are currently the only integrations that support connections
Databricks Unity Catalog: Delta shares are unsupported.
Public preview
This feature is public preview and available to select accounts. Reach out to your Immuta support professional to enable it on your tenant.
Connections allow you to register your data objects in a technology through a single connection, making data registration more scalable for your organization. Instead of registering schema and databases individually, you can register them all at once and allow Immuta to monitor your data platform for changes so that data sources are added and removed automatically to reflect the state of data on your data platform.
Native integrations are now connections. Once the upgrade is complete, you will control most integration settings at the connection level via the Infrastructure tab in Immuta.
| Integrations (existing) | Connections (new) |
|---|---|
Snowflake OAuth
Username and password
Key pair
Token
*M2M OAuth is not yet supported.
Unsupported technologies
The following technologies are not yet supported with connections:
Azure Synapse Analytics
Databricks Spark
Google BigQuery
Redshift
S3
Starburst (Trino)
The tables below outline Immuta features, their availability with integrations, and their availability with connections.
There will be no policy downtime on your data sources while performing the upgrade.
The supported object types are the same for both data sources with integrations and data sources with connections.
Table
View
Materialized view
External table
Event table
Iceberg table
Dynamic table
Table
View
Materialized view
Streaming table
External table
Foreign table
With connections, your data sources are ingested and presented to reflect the infrastructure hierarchy of your connected data platform. For example, this is what the new hierarchy will look like for a Snowflake connection:
Connections will not change any tags currently applied on your data sources.
If you previously ingested data sources using the V2 /data endpoint this limitation applies to you.
The V2 /data endpoint allows users to register data sources and attach a tag automatically when the data sources are registered in Immuta.
The V2 /data endpoint is not supported with a connection, and there is no substitution for this behavior at this time. If you require default tags for newly onboarded data sources, please reach out to your Immuta support professional before upgrading.
Schema monitoring is renamed to object sync with connections, as it can also monitor for changes at database and connection level.
During object sync, Immuta crawls your connection to ingest metadata for every database, schema, and table that the Snowflake role or Databricks account credentials you provided during the configuration has access to. Upon completion of the upgrade, the tables' states depend on your previous schema monitoring settings:
If you had schema monitoring enabled on a schema: All tables from that schema will be registered in Immuta as active data sources.
If you had schema monitoring disabled on a schema: All tables from that schema (that were not already registered in Immuta) will be registered as inactive data sources. They are visible from the infrastructure view, but are not listed as data sources until they are activated.
After the initial upgrade, your connection is periodically crawled every 24 hours to keep your tables in Immuta in sync. Additionally, users can also manually crawl metadata via the UI or API.
Object sync provides additional controls compared to schema monitoring:
Object status: Connections, databases, schemas and tables can be marked active, which for tables make them appear as data sources, or inactive. These statuses are inherited to all lower objects by default, but that can be overridden. For example, if you make a database inactive, all schemas and tables within that database will also be inactive. However, if you want one of those tables to be a data source, you can manually activate it.
Activate new data objects: This setting controls what state new objects are registered as in Immuta when found by object sync.
Active: New data objects found by object sync will automatically be activated and tables will be registered as data sources.
Inactive: This is the default. New data objects found by object sync will be inactive.
Data sources with integrations, required users to manually create the schema monitoring job in Databricks. However, this job has been fully automated on data sources with connections, and this step is no longer necessary.
Consolidating integration setup and data source registration into a single connection significantly simplifies programmatic interaction with the Immuta APIs. Actions that used to be managed through multiple different endpoints can now be achieved through one simple and standardized one. As a result, multiple API endpoints are blocked once a user has upgraded their connection.
All blocked APIs will send an error indicating "400 Bad Request - [...]. Use the /data endpoint." This error indicates that you will need to update your processes that are calling the Immuta APIs to leverage the new /data endpoint instead. For details, see the API changes page.
| Action | Deprecated endpoint | Use this with connections instead |
|---|
| Feature | Integrations (existing) | Connections (new) |
|---|---|---|
| Feature | Integrations (existing) | Connections (new) |
|---|---|---|
| Integrations (existing) | Connections (new) |
|---|---|
| Permission | Action | Object |
|---|---|---|
| Permission | Action | Object |
|---|---|---|
| Integrations (existing) | Connections (new) | |
|---|---|---|
Integrations are set up from the Immuta app settings page or via the API. These integrations establish a relationship between Immuta and your data platform for policy orchestration. Then tables are registered as data sources through an additional step with separate credentials. Schemas and databases are not reflected in the UI.
Integrations and data sources are set up together with a single connection per account between Immuta and your data platform. Based on the privileges granted to the Immuta system user, metadata from databases, schemas, and tables is automatically pulled into Immuta and continuously monitored for any changes.
User impersonation
Project workspaces
Snowflake lineage
Supported
Supported
Query audit
Supported
Supported
Tag ingestion
Supported
Supported
User impersonation
Not supported
Not supported
Project workspaces
Not supported
Not supported
Query audit
Supported
Supported
Tag ingestion
Supported
Supported
Catalog isolation support
Supported
Not supported
Integration
Connection
-
Database
-
Schema
Data source
Data source (once activated, becomes available for policy enforcement)
APPLICATION_ADMIN
Configure integration
Integration
CREATE_DATA_SOURCE
Register tables
Data source
Data owner
Manage data sources
Data source
CREATE_DATA_SOURCE
Register the connection
Connection, database, schema, data source
GOVERNANCE
Manage all connection
Connection, database, schema, data source
Infrastructure admin
Manage a connection
Connection, database, schema, data source
Data owner
Manage data objects
Connection, database, schema, data source
Name
Schema monitoring and column detection
Object sync
Where to turn on?
Enable (optionally) when configuring a data source
Enabled by default
Where to update the feature?
Enable or disable from the schema project
Object sync cannot be disabled
Default schedule
Every 24 hours
Every 24 hours
Can you adjust the default schedule?
No
No
Create a single data source | Step 1: Ensure your system user has been granted access to the relevant object in the data platform. Step 2: Wait until the next object sync or manually trigger a metadata crawl using . Step 3: If the parent schema has with |
Bulk create data sources | Step 1: Ensure your system user has been granted access to the relevant object in the data platform. Step 2: Wait until the next object sync or manually trigger a metadata crawl using . Step 3: If the parent schema has with |
Edit a data source connection | No substitute. Data sources no longer have their own separate connection details but are tied to the parent connection. |
Bulk edit data source's connections | No substitute. Data sources no longer have their own separate connection details but are tied to the global connection. |
Run schema detection (object sync) |
Delete a data source |
Bulk delete data sources |
Enable a single data source | with |
Bulk enable data sources | with |
Disable a single data source | with |
Bulk disable data sources | with |
Edit a data source name | No substitute. Data source names are automatically generated based on information from your data platform. |
Edit a connection key | No substitute. Data sources no longer have their own separate connection details but are tied to the global connection. |
Override a host name | No substitute. Data sources no longer have their own separate connection details but are tied to the global connection. |
Create an integration/connection |
Update an integration/connection |
Delete an integration/connection |
Delete and update a data dictionary |
| No substitute. Data source dictionaries are automatically generated based on information from your data platform. |
Update a data source owner | with |
Response to a data source owner request | with |
Most likely, since there are a number of API changes in regard to data sources and integrations. See the for details about each affected API endpoint and the substitute.
No, the Immuta system user still requires the same privileges in your data platform. See the for more details.
Connections support Snowflake or Databricks Unity Catalog technologies. See the for more details and reach out to your Immuta support professional if you are interested in the upgrade.