Requirement: Immuta permission GOVERNANCE

This how-to guide is for enabling sensitive data discovery (SDD). For additional information on sensitive data discovery and classification, see the Discover architecture page.

1. Navigate to the App Settings page and scroll to the Sensitive Data Discovery section.

2. Select the Enable Sensitive Data Discovery (SDD) checkbox to enable SDD.

3. Click Save and then click Confirm to apply your changes. Note that the Immuta tenant will have a system restart.

4. Run SDD for a select group of data sources; use one of the following options to run SDD on specific data sources:

   1. Run SDD on a data source in the UI.

   2. Run SDD using a specific framework in the UI.

   3. Make the following request specifying the data sources in the request using the Immuta API.

      Copy

      curl \
          --request 'POST' \
          'https://your-immuta-url.immuta.com/sdd/run' \
          --header 'Content-Type: application/json' \
          --header 'Authorization: 438a3096966c4a5188b3b468cedb213e' \
          --data '{"sources":["Example Data Source Name", "Example Data Source 2 Name"]}'

      A successful request will have the code 200 and a body with the number of jobs created from the request:

      Copy

      {
          "jobCount": 2
      }

5. Navigate to the data source overview page of the data source you listed in the payload.

6. Click the Data Dictionary tab.

7. Assess whether the Discovered and classification tags applied are accurate.

8. If they are, then repeat the steps above for more of your data sources. Once a majority of your data sources appear to have accurate tags, run SDD on all your data sources. If the tags are not accurate, you will need to tune SDD and classification frameworks. See the Adjust frameworks and tags guide for instructions.

Run SDD on all data sources

1. Click the Discover icon and the Identification tab in the navigation menu.

2. Select the more actions icon.

3. Select Run SDD and then select it again in the modal.

Run SDD on all data sources using the API

Requirement: Immuta permission GOVERNANCE

Make the following request using the Immuta API to run SDD for all data sources, specifying all as true:

curl \
    --request 'POST' \
    'https://your-immuta-url.immuta.com/sdd/run' \
    --header 'Content-Type: application/json' \
    --header 'Authorization: 438a3096966c4a5188b3b468cedb213e' \
    --data '{"all": true}'

A successful request will have the code 200 and a body with the number of jobs created from the request:

{
    "jobCount": 12
}

Requirements:

• Native SDD enabled and turned on

• Immuta permission GOVERNANCE

Create a pattern

1. Click the Discover icon in the navigation menu and select the Patterns tab.

2. Click Create New.

3. In the modal, enter a name for the new pattern.

4. Write a Description for the type of data the pattern will find.

5. Select the Type of pattern.

   1. For regex and column name regex, enter the regex.

   2. For dictionary, enter the values you want the pattern to match and toggle the switch on if you want them to be case-sensitive.

6. Click Create Pattern.

7. See the Manage rules page to add your new pattern to a framework.

Note that all user-created patterns must be a 90% match or greater for the contents of the column to be tagged.

Edit a pattern

Editing a pattern will affect any rule built off the pattern throughout Immuta. To edit a pattern,

1. Click the Discover icon in the navigation menu and select the Patterns tab.

2. Click the name of the pattern you want to edit.

3. Click Edit.

4. Edit the field you want to change. Note any field shadowed is not editable, and the pattern must be deleted and re-created to change them.

5. Click Save.

Built-in patterns cannot be edited.

Delete a pattern

Deleting a pattern will remove it from Immuta and remove all the rules that relied on it in the frameworks throughout Immuta. To delete a pattern,

1. Click the Discover icon in the navigation menu and select the Patterns tab.

2. Click the three dot menu in the Action column for the pattern you want to delete.

3. Select Remove.

4. Click Confirm.

Built-in patterns cannot be deleted.

Requirement: Immuta permission APPLICATION_ADMIN

Configure the global framework

1. Click the App Settings icon in the left sidebar.

2. Click Sensitive Data Discovery in the left panel to navigate to that section.

3. Enter the request-friendly name of your global template in the Global SDD Template Name field. This name can be found in the tooltip on the framework's detail page.

4. Click Save, and then Confirm your changes.

Requirements:

• Native SDD enabled and

• Immuta permission GOVERNANCE

Create a rule

You can only have one rule per pattern in the framework. If you do not see the pattern for the rule you want to create, then it already has a rule built off of it.

1. Click the Discover icon in the navigation menu and select the Framework tab.

2. Select the framework you want to edit and navigate to the Discovery Rules tab.

3. Click Create New.

4. Select the Tags to apply from the dropdown. The tags you select are the tags applied when the pattern is matched. Note that resulting tags must be under the Discovered parent tag and cannot be parent tags themselves unless they have already been manually applied to a data source.

5. Select the Criteria type from the dropdown. See the .

   1. Competitive pattern analysis is for regex and dictionary patterns.

   2. Column name is for column name patterns.

6. Select the Pattern from the dropdown.

7. Click Create Rule.

Edit a rule

1. Click the Discover icon in the navigation menu and select the Frameworks tab.

2. Select the framework of the rule you want to edit and navigate to the Discovery Rules tab.

3. Select the rule you want to edit.

4. Click Edit.

5. Edit the field you want to change. Note any field shadowed is not editable, and the rule must be deleted and re-created to change them.

6. Click Save.

Delete a rule

Deleting a rule removes the tags once applied by that rule the next time SDD runs on a data source. To delete a rule,

1. Click the Discover icon in the navigation menu and select the Frameworks tab.

2. Select the framework you want to edit and navigate to the Discovery Rules tab.

3. Click the three dot menu in the Action column for the rule you want to delete.

4. Select Remove.

5. Click Confirm.

This guide provides information and best practices for migrating from the deprecated legacy sensitive data discovery (SDD) option to the improved native SDD. This guide is for users who have already enabled SDD on their tenant and have Discovered tags on their data sources.

Before you begin

Native vs legacy SDD

Legacy SDD is deprecated. It will be removed and replaced by native SDD. Native SDD is significantly improved from legacy SDD for discovering and tagging your data with upgrades to the built-in patterns. Additionally, the greatest benefit is the respect for data residency. Native SDD doesn't move any of your data when running. The discovery is done right in your data platform, and the platform only returns the matching patterns and column names to Immuta.

See the for more information on native SDD.

Requirements

• Native SDD requires Snowflake, Databricks, Redshift, or Starburst (Trino) data sources

• Legacy SDD enabled on your tenant

• Legacy SDD tags applied to your data sources: To find out if you have legacy SDD tags applied, create a governance report as described in the .

Enable native SDD

Contact your Immuta representative to enable native SDD on your Immuta tenant. Note that unless specifically disabled, all Immuta installations after the 2024.2 LTS have native SDD automatically enabled. Proceed to if you want to self-service check if native SDD is already running and tagging your data before you reach out to the representative.

This action will not change anything immediately on your tenant; however, anytime SDD runs in the future, it will be native SDD instead of the legacy version.

To assess native SDD for your data, proceed with the steps below. If you do not review native SDD, the legacy SDD tags will all remain on your data source columns. However, when on new data sources and columns, it will apply native SDD tags, and because of the improvements to SDD, it may tag different data than legacy SDD.

Understand the context of your tags

Requirement: Immuta permission GOVERNANCE

2. To check the tags on an individual data source, navigate to the data source data dictionary and select a Discovered tag. On the tag side sheet, you can determine the context of the tag. When patterns match data, native SDD will apply tags, and their tag context will be Sensitive Data Discovery. Any tags with the context Legacy Sensitive Data Discovery were not matched by native SDD but will remain on the data source.

3. To check your tags globally, navigate to the governance reports page and build a report for sensitive data discovery. This report will present the legacy tags on your data sources' columns and native SDD tags that are also on those columns. Use this report to assess the context of the Discovered tags and understand if native SDD is matching the data you want it to.

These actions will allow you to understand the differences between how native SDD and legacy SDD tag your data and whether your data is recognized as expected by native SDD or if legacy SDD was over-tagging your data. This way you can better tune SDD to your data.

If there are any legacy SDD tags that you want native SDD to catch, you need to tune native SDD so that this type of data is discovered in future tables and columns; see guidance on that in the next section.

Tune SDD

Requirement: Immuta permission GOVERNANCE

Using the report you built above, complete these actions to tune SDD:

1. Focus on a legacy SDD tag properly applied to your data. Assess whether the native SDD tag on the column instead was applied more accurately than the legacy tag. If it is applied incorrectly, proceed to the next step.

4. Complete the steps above for all legacy SDD tags.

Completing the actions above will create parity between what legacy SDD was tagging your data and what native SDD will tag in the future.

Requirements:

• Native SDD enabled and

• Registered

• Immuta permission GOVERNANCE

Create a framework

Create a framework with no rules

1. Click the Discover icon in the navigation menu and select the Frameworks tab.

2. Click Create New.

3. Enter a Name for the framework.

4. Enter a Description for the framework.

5. Select the option to Create empty framework.

6. Click Create.

After you create the framework, you can .

Copy an existing framework and its rules

1. Click the Discover icon in the navigation menu and select the Frameworks tab.

2. Click Create New.

3. Enter a Name for the framework.

4. Enter a Description for the framework.

5. Select the option to Create rules from an existing framework.

6. Select the checkbox for the framework you want to copy. You can only copy a single framework. For more information about a framework, click the framework name to open a new tab with details about the framework.

7. Click Create.

Assign a framework to data sources

To assign a framework to run on specific data sources,

1. Click the Discover icon in the navigation menu and select the Frameworks tab.

2. Select the framework you want to assign and navigate to the Data Sources tab.

3. Click Add Data Sources.

4. Select the checkbox for the data source you want this framework to run on. You may select more than one.

5. Click Add Data Source(s).

Remove data sources from a framework

After a data source is removed from a framework, it will use the global framework for any SDD scans and the tags applied by the removed framework will be replaced. To remove data sources from a framework,

1. Click the Discover icon in the navigation menu and select the Frameworks tab.

2. Select the framework you want to remove data sources from and navigate to the Data Sources tab.

3. Select the checkbox for the data source you want to remove from the framework. You may select more than one.

4. Select the Bulk Actions more options.

5. Select Remove Data Sources.

6. Click Confirm.

Delete a framework

Deleting a framework will remove it from any data sources. Those data sources will then use the global framework for any SDD scans and the tags applied by the deleted framework will be replaced. Governors can delete any framework, and users with the CREATE_DATA_SOURCE or CREATE_DATA_SOURCE_IN_PROJECT permissions can only delete frameworks they created. To delete a framework,

1. Click the Discover icon in the navigation menu and select the Frameworks tab.

3. Select Remove.

4. Click Confirm.

Requirements:

• Native SDD enabled and

• Registered

• Immuta permission GOVERNANCE

Run SDD using a specific framework

SDD runs automatically, but if you want to re-run SDD when a new global framework is set or when new rules have been added, you can or for specific frameworks through the UI:

1. Click the Discover icon and the Identification tab in the navigation menu.

2. Select the more actions icon.

3. Select Run SDD and then select it again in the modal.

Run SDD on a data source

SDD runs automatically, but if you want to re-run SDD when a new global framework is set or when new rules have been added, you can or for specific data sources through the UI:

1. Navigate to the data source overview page.

2. Click the health status.

3. Select Re-run next to Sensitive Data Discovery (SDD).

Verify discovered tags

Disable Discovered tags from the data dictionary

If a governor, data owner, or data source expert disables a Discovered tag from the data dictionary, the column will not be re-tagged when that data source's fingerprint is recalculated or SDD is re-run. When a Discovered tag is disabled, the tag will not completely disappear, so it can be manually enabled through the tag side sheet.

To disable a discovered tag,

1. Navigate to a data source and click the Data Dictionary tab.

2. Scroll to the column you want to remove the tag from and click the tag you want to remove.

3. Click Disable in the side sheet and then click Confirm.