Immuta can scan your data sources and apply relevant tags when data is recognized. This eliminates a manual tagging process for your data, saving you time and providing standard taxonomy across all your data sources.

Requirements

• Registered Snowflake, Databricks, Redshift, or Starburst (Trino) data sources

• Immuta permission GOVERNANCE

Implement SDD

Sensitive data discovery (SDD) is an Immuta Discover feature that identifies your data sources and applies relevant tags when data is recognized. This eliminates a manual tagging process for your data, saving you time and providing standard taxonomy across all your data sources.

To learn more, see the Data discovery page.

Enable sensitive data discovery

Enable sensitive data discovery on your tenant. Opt to have SDD run automatically for new data sources by setting a global framework, or run SDD granularly by applying data sources to specific frameworks.

Configure SDD for your data

For additional control, create your own identifiers to recognize the data that matters to you. Add these identifiers to new frameworks and specify the data sources that need this framework. This fine-level control creates automatic tagging that is relevant and accurate to your data, requiring fewer manual adjustments to the resulting tags.

1. Customize SDD for your data:

   1. Create identifiers

   2. Create a framework

2. Assign your framework to specific data sources

Adjust Discovered tags

If you have any tags that are applied to your data sources by SDD that you don't want, you can easily disable these tags for each data source. This ensures that they will not be applied to the data source again if identification is re-run.

1. Verify tags

2. Disable Discovered tags

Reference pages:

Immuta comes with a default framework containing built-in Discovered tags and built-in identifiers. These identifiers and tags can be used in your own frameworks.

1. Built-in Discovered tags

2. Built-in identifiers

Implement classification

Classification is an Immuta Discover feature that categorizes your data based on the content and the associated risk the data poses. This increases your understanding of your data and allows you to make faster decisions about it.

Configure classification for your data

To create or manage a framework using the Immuta API, see the Frameworks API reference page.

Adjust classification tags

If you have any tags that are applied to your data sources by classification that you don't want, you can easily disable these tags for each data source. This ensures that they will not be applied to the data source again when classification is re-run.

1. Assess your data source tags.

2. Tune your data dictionaries.

Identifiers in domains allows you to use the same domains you already organize your data in to hold identifiers and run sensitive data discovery (SDD) without having to use identification frameworks. See the for more information about the feature and limitations.

Prerequisites

Identifiers can be added and SDD can be run in any of your current domains. However, if you are not already using domains, set up a domain specifically to run SDD:

1. .

2. .

3. .

4. .

Add identifiers to a domain

1. Navigate to the Identifiers tab of your domain.

2. Click Get Started.

3. Add reference identifiers to your domain that are relevant to your data by clicking the checkboxes. Note: When added to your domain, the identifier is a point-in-time copy of the reference identifier. It has the same name, pattern, and tags.

4. Click Add Identifiers.

Create a new identifier

This can be done within a domain from the Identifiers tab to create a domain-specific identifier, or it can be done from the Discover Identifiers page to create a reference identifier.

1. Click Create New.

2. Enter a name and description for your identifier.

3. Click Next.

4. 1. For regex, enter a regex to be matched against column values. The default criteria encoding is case-sensitive. You can change this encoding using the regex criteria. The regex must use RE2.

   2. For column name regex, enter a regex to be matched against column names. The default criteria encoding is case-insensitive. You can change this encoding using the regex criteria. The regex must use RE2 syntax.

   3. For a dictionary, enter the values in a comma-separated list to match against column values. Opt to toggle the Case insensitive switch to on if you want the dictionary to be case sensitive.

5. Click Next.

6. Select the tags to apply: Use the text box to search for a tag under the "Discovered" hierarchy or type a tag name to create a new tag under the "Discovered.Entity" hierarchy to apply to columns that match your identifier.

7. Click Next to review your new identifier and click Create Identifier to create it.

Requirements:

• Immuta permission GOVERNANCE

Create an identification framework

Create an identification framework with no identifiers

1. Click the Discover icon in the navigation menu and select the Identification tab.

2. Click Create New.

3. Enter a Name and Description for the identification framework.

4. Select the option to Create empty framework.

5. Click Create.

After you create the identification framework, you can .

Copy an existing identification framework and its identifiers

1. Click the Discover icon in the navigation menu and select the Identification tab.

2. Click Create New.

3. Enter a Name and Description for the identification framework.

4. Select the option to Create identifiers from an existing framework.

5. Select the checkbox for the framework you want to copy. You can only copy a single framework. For more information about a framework, click the framework name to open a new tab with details about the framework.

6. Click Create.

Manage an identification framework's identifiers

Add an identifier to a framework

To add an identifier to a framework,

1. Click the Discover icon in the navigation menu and select the Identification tab.

2. Select the framework name for the identification framework you want to edit.

3. Click Add Identifier.

4. Choose in the dropdown to add an identifier from those already in Immuta or create a new identifier for the framework.

   • For existing identifiers: Opt to edit the tags. Then click Add Identifier.

   • For new identifiers:

     1. Fill out a Name and Description.

     2. 1. For regex, enter a regex to be matched against column values. The default criteria encoding is case-sensitive. You can change this encoding using the regex criteria. The regex must use RE2.

        2. For column name regex, enter a regex to be matched against column names. The default criteria encoding is not case-sensitive. You can change this encoding using the regex criteria. The regex must use RE2 syntax.

        3. For a dictionary, enter the values in a comma-separated list to match against column values. Opt to toggle the Case insensitive switch to on if you want the dictionary to be case sensitive.

     3. Select the tags to apply: Use the text box to search for a tag under the "Discovered" hierarchy or type a tag name to create a new tag under the "Discovered" hierarchy to apply to columns that match your identifier.

     4. Click Next to review your new identifier and click Create Identifier to create it.

Edit an identifier in a framework

To edit the tags applied by an identifier for a framework,

1. Click the Discover icon in the navigation menu and select the Identification tab.

2. Select the framework name for the identification framework you want to edit.

3. Click the more actions icon for an identifier and select Edit tags.

4. Remove the tags or type a tag name to add tags.

5. Click Save.

Delete an identifier from a framework

1. Click the Discover icon in the navigation menu and select the Identification tab.

2. Select the framework name for the identification framework you want to edit.

3. Click the more actions icon for an identifier and select Delete.

4. Click Delete again in the modal.

Manage an identification framework's data sources

Assign an identification framework to data sources

To assign a framework to run on specific data sources,

1. Click the Discover icon in the navigation menu and select the Identification tab.

2. Select the framework you want to assign and navigate to the Data Sources tab.

3. Click Add Data Sources.

4. Select the checkbox for the data source you want this framework to run on. You may select more than one.

5. Click Add Data Source(s).

Remove data sources from an identification framework

After a data source is removed from a framework, it will use the global framework for any SDD scans and the tags applied by the removed framework will be replaced. The global framework is signified by the globe icon.

To remove data sources from a framework,

1. Click the Discover icon in the navigation menu and select the Identification tab.

2. Select the framework you want to remove data sources from and navigate to the Data Sources tab.

3. Select the checkbox for the data source you want to remove from the framework. You may select more than one.

4. Select Remove and click Remove again in the modal.

Delete an identification framework

Requirement: No data sources assigned to the framework

To delete a framework,

1. Click the Discover icon in the navigation menu and select the Identification tab.

3. Select Delete and click Delete again in the modal.

Sensitive data discovery (SDD) is an Immuta feature that uses data patterns to determine what type of data your column represents. Using identification frameworks and identifiers, Immuta evaluates your data and can assign the appropriate tags to your data dictionary based on what it finds. This saves the time of identifying your data manually and provides the benefit of a standard taxonomy across all your data sources in Immuta.

Supported technologies

Sensitive data discovery is supported for from the following technologies:

• or

• : Sensitive data discovery for Starburst (Trino) is currently in public preview and available to all accounts. Reach out to your Immuta representative to enable it on your tenant.

• : Sensitive data discovery for Redshift is currently in private preview and available to all accounts. Reach out to your Immuta representative to enable it on your tenant.

Architecture

To evaluate your data, SDD generates a SQL query using the identification framework's identifiers; the Immuta system account then executes that query in the native technology. Immuta receives the query result, containing the column name and the matching identifiers but no raw data values. These results are then used to apply the resulting tags to the appropriate columns.

This evaluating and tagging process occurs when identification runs and happens automatically from the following events, if a global framework is set:

• A new data source is created.

• Schema monitoring is enabled, and a new data source is detected.

The following actions will also trigger identification:

• Column detection is enabled, and new columns are detected. Here, SDD will only run on new columns, and no existing tags will be removed or changed. Note, this will use the identification framework that already ran on the data source.

• A user manually triggers it from the data source health check menu. Note, this will use the identification framework that already applies to the data source or the global framework, if set.

• A user manually triggers it from the identification frameworks page.

• A user manually triggers it through the API.

Components

Identification framework

An identification framework is a group of identifiers that will look for particular criteria and tag any columns where those conditions are met.

Global framework

Each organization can set a global framework to apply to all the data sources in Immuta by default unless they have a different framework assigned. It is labeled on the frameworks page with a globe icon. If a global framework is set, identification will run on all new data sources. If a global framework is not set, identification will only run on data sources manually applied to an identification framework.

Identifier

An identifier is a criteria and the tags to apply to data that matches the criteria. When Immuta recognizes that criteria, it can tag the data to describe the type.

Criteria

Criteria are the conditions that need to be met for resulting tags to be applied to data.

SDD only supports regular expressions (regex) written in RE2 syntax.

Supported criteria types for identifiers

• • Regex: This criteria contains a case-insensitive regular expression that searches for matches against column values.

  • Dictionary: This criteria contains a list of words and phrases to match against column values.

• Column name: This criteria includes a case-insensitive regular expression matched against column names, not against the values in the column. The identifier's tags will be applied to the column where the name is found. Multiple column name identifiers can match a column and be applied.

Configuration

Tag mutability

Performance

The amount of time it takes to run identification on a data source depends on several factors:

• Columns: The time to run identification grows nearly linearly with the number of text columns in the data source.

• Row count: Performance of identification may vary depending on the sampling method used by each technology. For Snowflake, the number of rows has little impact on the time because data sampling has near-constant performance.

• Views: Performance on views is limited by the performance of the query that defines the view.

Testing

Considerations

Supported data types and casing

*Two built-in patterns support and match based on additional data types:

• DATE: Columns will match this identifier if they are string and the regex matches or if the data type is date, date+time, or timestamp.

• TIME: Columns will match this identifier if they are string and the regex matches or if the data type is time. Note that if the date is included in the data, it will not match this identifier.

Limitations with dictionary patterns

Immuta compiles dictionary patterns into a regex that is sent in the body of a query.

Databricks limitation

Starburst (Trino) limitation

Redshift limitations

• Redshift Spectrum is not supported with SDD.

• The Redshift cluster must be up and running for SDD to successfully run.

Redshift supported authentication methods

• The username and password auth method is fully supported with SDD.

• AWS access key is supported with limitations with SDD:

  • • redshift-data:BatchExecuteStatement

    • redshift-data:CancelStatement

    • redshift-data:DescribeStatement

    • redshift-data:ExecuteStatement

    • redshift-data:GetStatementResult

    • redshift-data:ListStatements

  • The AWS access key used to register the data source must have redshift:GetClusterCredentials for the cluster, user, and database that they onboard their data sources with.

  • Redshift Serverless data sources are not supported for native SDD with the AWS access key authentication method.

Migrating from legacy to native SDD

Immuta has improved the performance and behavior of sensitive data discovery (SDD), so references to two types of SDD can be found in the product:

• Legacy SDD was available before October 2023. It is no longer available, but some users may still see the term "legacy SDD" in the context of their data tags.

• Native SDD was released to Snowflake and Databricks in May 2023. It was released to Starburst (Trino) and Redshift in April 2024. Native SDD is the only type of SDD available. It is often just referred to as SDD.

If you had legacy SDD enabled, running native SDD can result in different tags being applied because native SDD is more accurate and has fewer false positives than legacy SDD. Running a new SDD scan against a table will change the context of the resulting tags, but no Discovered tags previously applied by legacy SDD will be removed.