> For the complete documentation index, see [llms.txt](https://documentation.immuta.com/saas/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://documentation.immuta.com/saas/configuration/integrations/data-and-integrations/registering-metadata/data-source-overview.md).

# Data Sources Reference Guide

Data owners expose their data across their organization to other users by registering that data in Immuta as a data source. Data sources are collections of metadata about your tables or data objects and allow for Immuta actions like the following:

* Apply tags to data sources to enforce access controls
* Apply data policies to a data source's columns
* Restrict the users who can query a data source with subscription policies
* Gather your data sources into various domains for delegation
* Publish data products containing your data sources in the Request app

When data is registered, Immuta does not affect existing policies on those tables in the remote system for non-Immuta users, so users who had access to a table before it was registered can still access that data without interruption. However, this behavior is different for Immuta users on an integration-by-integration basis, so see the [integration reference guides](/saas/configuration/integrations.md#integrations) for more details.

## Data source registration

For policies to properly apply to data sources, there must be an integration configured in Immuta, and that integration's connection details must match the data source's connection details. This allows for Immuta to natively enforce policies on that table in your data platform. There are two ways to configure integrations and register data based on your technology.

* **Connections data platforms**: Connections combine integration configuration and data source registration for select technologies. After using connections to create the integration and data objects with the same credentials, [enable the data object for your tables, views, etc.](/saas/configuration/integrations/data-and-integrations/registering-a-connection/how-to-guides/manage-connection-settings.md#assign-domain-permissions) to create the data sources. See the [Connections section of documentation](/saas/configuration/integrations/data-and-integrations/registering-a-connection.md) for a list of technologies that are supported.
* **Non-connection data platforms**: For the technologies listed below, integration configuration and data source registration happen separately. After you configure your integration, ensure that the host, port, and other integration details match the data source details you enter during registration so that policies will properly apply:
  * [Amazon Redshift Spectrum data sources](/saas/configuration/integrations/data-and-integrations/registering-metadata/register-data-sources/redshift-tutorial.md)
  * [Amazon S3 data sources](/saas/configuration/integrations/data-and-integrations/registering-metadata/register-data-sources/s3-tutorial.md)
  * [Azure Synapse Analytics data sources](/saas/configuration/integrations/data-and-integrations/registering-metadata/register-data-sources/synapse-tutorial.md)
  * [Databricks Spark data sources](/saas/configuration/integrations/data-and-integrations/registering-metadata/register-data-sources/databricks-tutorial.md)
  * [Google BigQuery view-based data sources](/saas/configuration/integrations/data-and-integrations/registering-metadata/register-data-sources/bigquery-tutorial.md)

### Schema projects

Schema projects are automatically created and managed by Immuta when data sources are created. They group all the data sources of the schema, and when new data sources are created, manually or with [schema monitoring](/saas/configuration/integrations/data-and-integrations/registering-metadata/schema-monitoring.md), they are automatically added to the schema project. They work as a tool to organize all the data sources within a schema, which is particularly helpful with schema monitoring enabled.

Schema projects are created when tables are registered as data sources in Immuta. The user creating the data source does not need the CREATE\_PROJECT permission to have the project auto-create because no data sources can be added by the owner. Instead, new data sources are managed by Immuta. The user can manage subscription policies for schema projects, but they cannot apply data policies or purposes to them.

The schema settings, such as schema evolution and connection information, can be edited from the project overview tab. *Note: Deleting the project will delete all of the data sources within it as well.*

#### Schema project actions

Schema settings are edited from the project overview tab:

* [**Schema project connection details**](/saas/configuration/integrations/data-and-integrations/registering-metadata/schema-monitoring/manage-schema-monitoring.md#edit-schema-project-connection): Editing these details will update them for all the data sources within the schema project.
* [**Data source naming convention**](/saas/configuration/integrations/data-and-integrations/registering-metadata/schema-monitoring/manage-schema-monitoring.md#edit-schema-monitoring-naming-convention): When schema monitoring is enabled, new data sources will be automatically detected and added to the schema project. Updating the naming convention will change how these newly detected data sources are named by Immuta.
* [**Schema detection owner**](/saas/configuration/integrations/data-and-integrations/registering-metadata/schema-monitoring/manage-schema-monitoring.md#add-new-schema-detection-owner): When schema monitoring is enabled, a user is assigned to be the owner of any detected and Immuta created data source.
* [**Disable or delete your schema project**](/saas/govern/secure-your-data/projects-and-purpose-based-access-control/purpose-index/how-to-guides/project-management/manage-projects.md#disable-enable-or-delete-a-project): Deleting the project will delete all of the data sources within it as well.

### Data source health checks

When an Immuta data source is created, background jobs use the connection information provided to compute health checks dependent on the type of data source created and how it was configured. These data source health checks include the following options depending on the technology and registration method:

<table data-header-hidden="false" data-header-sticky><thead><tr><th width="186.875">Status</th><th width="140">Description</th><th>Action</th><th>Supported technologies</th><th>Registration type</th></tr></thead><tbody><tr><td><strong>External catalog</strong></td><td>Indicates whether or not the external catalog was successfully linked to the data source. This status will appear if you have an <a href="/pages/6UKor3iIwdrjwhinWhwi">external catalog configured</a>.</td><td><strong>Re-run</strong></td><td>All</td><td>All</td></tr><tr><td><strong>Fingerprint</strong></td><td>Indicates whether or not the data source fingerprint was successfully generated.</td><td><strong>Re-run</strong></td><td>Snowflake</td><td>Connections</td></tr><tr><td><strong>Identification</strong></td><td>Indicates whether identification was successfully run on the data source.</td><td><strong>Re-run</strong></td><td>All</td><td>All</td></tr><tr><td><strong>Row count</strong></td><td>Indicates whether the number of rows in the data source was successfully calculated.</td><td><strong>Re-run</strong></td><td>All</td><td>All</td></tr><tr><td><strong>High cardinality column</strong></td><td>Indicates whether the data source's high cardinality column was successfully calculated.</td><td><strong>Re-run</strong></td><td>All</td><td>All</td></tr><tr><td><strong>Global policies applied</strong></td><td>Indicates whether global policies were successfully applied to the data source.</td><td>No action</td><td>See the <a href="/pages/pbSsqyEoeXHCgNbYYQnG#integrations-overview">integrations support matrix</a> for the technologies that support policies.</td><td>All</td></tr><tr><td><strong>Object sync</strong></td><td>Indicates whether object sync has successfully run on the data source data object.</td><td><strong>Re-run</strong></td><td>See the <a href="/pages/pbSsqyEoeXHCgNbYYQnG#integrations-overview">integrations support matrix</a> for the technologies on connections.</td><td>Connections</td></tr><tr><td><strong>Schema detection</strong></td><td>Indicates whether the job run to determine if a remote table was added or removed from the schema was successful.</td><td>No action</td><td><ul><li>Redshift Spectrum</li><li>Azure Synapse Analytics</li><li>Google BigQuery</li></ul></td><td>Legacy onboarding</td></tr><tr><td><strong>Column detection</strong></td><td>Indicates whether the job run to determine if a column was added or removed from the remote table registered as an Immuta data source was successful.</td><td><strong>Trigger Detection</strong></td><td><ul><li>Redshift Spectrum</li><li>Azure Synapse Analytics</li><li>Google BigQuery</li></ul></td><td>Legacy onboarding</td></tr><tr><td><strong>Framework classification</strong></td><td>Indicates whether classification was successfully run on the data source to determine the sensitivity of the data source.</td><td><strong>Re-run</strong></td><td>All</td><td>All</td></tr><tr><td><strong>Databricks</strong></td><td>Indicates whether Databricks governance policies have been successfully synced.</td><td><strong>Sync all policies</strong></td><td>Databricks</td><td>Connections</td></tr><tr><td><strong>Snowflake</strong></td><td>Indicates whether Snowflake governance policies have been successfully synced.</td><td><strong>Sync data policies</strong></td><td>Snowflake</td><td>Connections</td></tr><tr><td><p></p><ul><li><strong>Redshift View</strong></li><li><strong>Azure Synapse Analytics View</strong></li><li><strong>Google BigQuery View</strong></li></ul></td><td><p></p><p>Indicates whether views were properly created for tables registered in Immuta.</p></td><td><strong>Create</strong></td><td><ul><li>Redshift Spectrum</li><li>Azure Synapse Analytics</li><li>Google BigQuery</li></ul></td><td>Legacy onboarding</td></tr></tbody></table>

After these jobs complete, the health status for each is updated to indicate whether the status check passed, was skipped, is unknown, or failed.

These background jobs can be disabled during data source creation by adding a specific tag to prevent automatic table statistics. This prevent statistics tag can be set on the [app settings page](/saas/configuration/application-configuration/how-to-guides/config-builder-guide.md#prevent-automatic-table-statistics) by a system administrator. However, with automatic table statistics disabled these policies will be unavailable until the data source owner [manually generates the fingerprint (available only for Snowflake data sources)](/saas/govern/secure-your-data/data-consumers/subscribe-to-data-source.md#manually-run-health-jobs):

* Masking with format preserving masking
* Masking using randomized response

#### Unhealthy Databricks data sources

Unhealthy data sources may fail their row count queries if they run against a cluster that has the Databricks query watchdog enabled.

#### Health check limitations

Data sources with over 1600 columns will not have health checks run, but will still appear as healthy. The health check cannot be run automatically or manually.

## Data source management

### Schema monitoring

With schema monitoring enabled, Immuta monitors your organization's servers to find when new tables or columns are created or deleted and automatically registers (or disables) those tables in Immuta.

See the [Schema monitoring section](/saas/configuration/integrations/data-and-integrations/registering-metadata/schema-monitoring.md) for more details.

### Data source user roles

There are various roles users and groups can play relating to each data source. These roles are managed through the members tab of the data source. Roles include the following types:

* **Owners**: Those who create and manage new data sources and their users, documentation, column tags, and column descriptions.
* **Subscribers**: Those who have access to the data source data. With the appropriate data accesses and attributes, these users and groups can view files, run queries, and generate analytics against the data source data. All users and groups granted access to a data source have subscriber status.
* **Experts**: Those who are knowledgeable about the data source data and can elaborate on it. They are responsible for managing the data source's documentation, column tags, and column descriptions.

See [Manage data source members](/saas/configuration/integrations/data-and-integrations/registering-metadata/data-source-settings/manage-members.md#modify-user-or-group-roles-within-a-data-source) for a tutorial on modifying user roles.

### Data source columns

The columns tab provides information about the columns within the data source, including column names and value types.

Columns are automatically generated when the data source is created. However, data owners and experts can [tag columns](/saas/configuration/integrations/data-and-integrations/registering-metadata/data-source-settings/manage-column-descriptions-and-tags.md#manage-column-tags) and [add descriptions to these entries](/saas/configuration/integrations/data-and-integrations/registering-metadata/data-source-settings/manage-column-descriptions-and-tags.md#manage-column-descriptions).

#### Column icons

The columns tab displays icons on columns that have a masking policy applied to them. The appearance of these icons varies depending on the permission of the user.

**Governors and data owners**

If you have the `GOVERNANCE` permission or are the data source owner, the column icons will appear in these ways:

* **No icon**: No masking policy applies to the column.
* **Yellow eye**: A masking policy applies to the column, but the column is unmasked for the current user because they meet the exception criteria for the policy.
* **Red eye**: A policy on the column masks it for the current user.

**All other users**

The column icons will appear in these ways for all other users:

* **No icon**: Either no masking policy applies to the column or a masking policy applies to the column, but the column is unmasked for the current user because they meet the exception criteria for the policy.
* **Red eye**: A policy on the column masks it for the current user.

#### Nested columns

You can create Databricks Spark data sources with nested columns when you enable [complex data types](/saas/configuration/application-configuration/how-to-guides/config-builder-guide.md#preview-features). When complex types are enabled, Databricks Spark data sources can have columns that are arrays, maps, or structs that can be nested.

## Audit

The following events related to data sources are [audited](/saas/govern/detect-your-data/audit/reference-guides/index/uam-schema.md) and can be found on the [audit page in the UI](/saas/govern/detect-your-data/audit/how-to-guides/use-immuta-audit.md):

* [DatasourceCreated](/saas/govern/detect-your-data/audit/reference-guides/index/uam-schema.md#datasourcecreated-event): A data source is created.
* [DatasourceDeleted](/saas/govern/detect-your-data/audit/reference-guides/index/uam-schema.md#datasourcedeleted-event): A data source is deleted.
* [DatasourceDisabled](/saas/govern/detect-your-data/audit/reference-guides/index/uam-schema.md#datasourcedisabled-event): A data source is disabled.
* [DatasourceUpdated](/saas/govern/detect-your-data/audit/reference-guides/index/uam-schema.md#datasourceupdated-event): A data source is updated.
* [DatasourceAppliedToProject](/saas/govern/detect-your-data/audit/reference-guides/index/uam-schema.md#datasourceappliedtoproject-event): A data source is added to a project.
* [DatasourceRemovedFromProject](/saas/govern/detect-your-data/audit/reference-guides/index/uam-schema.md#datasourceremovedfromproject-event): A data source is removed from a project.
* [DatasourceCatalogSynced](/saas/govern/detect-your-data/audit/reference-guides/index/uam-schema.md#datasourcecatalogsynced-event): An external catalog is linked and synced for the data source.
* [DatasourceGlobalPolicyApplied](/saas/govern/detect-your-data/audit/reference-guides/index/uam-schema.md#datasourceglobalpolicyapplied-event): A global policy is applied to a data source.
* [DatasourceGlobalPolicyConflictResolved](/saas/govern/detect-your-data/audit/reference-guides/index/uam-schema.md#datasourceglobalpolicyconflictresolved-event): A policy conflict between two global policies on a data source is resolved.
* [DatasourceGlobalPolicyDisabled](/saas/govern/detect-your-data/audit/reference-guides/index/uam-schema.md#datasourceglobalpolicydisabled-event): A global policy is disabled on a data source.
* [DatasourceGlobalPolicyRemoved](/saas/govern/detect-your-data/audit/reference-guides/index/uam-schema.md#datasourceglobalpolicyremoved-event): A global policy is removed from a data source.
* [LocalPolicyCreated](/saas/govern/detect-your-data/audit/reference-guides/index/uam-schema.md#localpolicycreated-event): A local policy is created on a data source.
* [LocalPolicyUpdated](/saas/govern/detect-your-data/audit/reference-guides/index/uam-schema.md#localpolicyupdated-event): A local policy is updated on a data source.
* [SubscriptionCreated](/saas/govern/detect-your-data/audit/reference-guides/index/uam-schema.md#subscriptioncreated-event): A user is subscribed to a data source or project.
* [SubscriptionDeleted](/saas/govern/detect-your-data/audit/reference-guides/index/uam-schema.md#subscriptiondeleted-event): A user's subscription to a data source or project is removed.
* [SubscriptionRequestApproved](/saas/govern/detect-your-data/audit/reference-guides/index/uam-schema.md#subscriptionrequestapproved-event): A user's request to subscribe to a data source or project is approved.
* [SubscriptionRequestDenied](/saas/govern/detect-your-data/audit/reference-guides/index/uam-schema.md#subscriptionrequestdenied-event): A user's request to subscribe to a data source or project is denied.
* [SubscriptionRequested](/saas/govern/detect-your-data/audit/reference-guides/index/uam-schema.md#subscriptionrequested-event): A user requests to subscribe to a data source or project.
* [SubscriptionUpdated](/saas/govern/detect-your-data/audit/reference-guides/index/uam-schema.md#subscriptionupdated-event): A user's subscription to a data source or project is updated.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://documentation.immuta.com/saas/configuration/integrations/data-and-integrations/registering-metadata/data-source-overview.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.