Tags have several uses, mainly to drive policies, but they can be used for the following purposes:

• Use tags for global subscription or data policies that will apply to all data sources in the organization. In doing this, company-wide data security restrictions can be controlled by the administrators and governors, while the users and data owners need only to worry about tagging the data correctly.

• Generate Immuta reports from tags for anything from insider threat surveillance to data access monitoring.

• Drive search results with tags in the Immuta UI.

Every user within Immuta can see tags, but they will all interact with them differently as their roles require. Governors create, manage, and delete tags or import tags from external catalogs. Data owners, data source experts, and governors apply these tags to or remove them from projects, data sources, and columns within the data sources. Data users view tags and tag metadata on data sources they have access to.

Navigating tags

When navigating tags in the Immuta UI, there are several helpful features:

• Side sheets: Clicking on a tag in the data dictionary, on the data overview page, or on a project page will open the tag side sheet with valuable information about the tag. This information depends on the kind of tag it is and where it is applied. The side sheet can include a link to the tag details page, a description of the tag, the context of the tag (i.e., where the tag was created and added from), the columns the tag is applied to, and actions that can be done to the tag (e.g., disabling or deleting the tag from its object).

• Tooltips: When you hover over a tag, a tooltip will appear. It contains information about the tag, including where it was created (e.g., Immuta or an external catalog), whether the tag was applied by sensitive data discovery, and the full name of the tag.

• Simplified names: When fully articulated, tags are presented as Parent . Child . Grandchild with " . " between each level. However, tags will usually appear as the lowest name level (i.e. Discovered . Entity . Person Name will appear as Person Name) and the full name can be seen in the tooltip.

Sensitive data discovery

Sensitive data discovery (SDD) helps to ensure sensitive data is properly managed and governed, providing fast identification for entities in columns such as credit card numbers, names, locations, social security numbers, bitcoin wallets, US phone numbers, financial data, and more.

Create tags

1. Click the Governance icon in the navigation menu and select the Tags tab.

2. Click Add Tags in the top right corner.

3. Complete the Enter tag name field.

4. Additional nested tags are optional. These nested tags follow a tree structure. There are parent, sibling, and child tags. Click Remove Tag to remove a nested tag.

5. Click Save.

View data source tags

1. Click the Data icon in the navigation menu and select the Data Sources tab.

2. Select a data source.

3. Navigate to the Data Dictionary tab.

4. Hover over tags for metadata or click on a tag to open the side sheet with information about the tag.

View all tags

1. Click the Governance icon in the navigation menu and select the Tags tab.

2. A list of all top-level tags will be displayed. Click the expand arrow to view nested tags.

3. Click the tag itself or the icon in the Actions column to edit tags, generate tag reports, or delete tags.

Import tags from an external catalog

You can pull external tags that you had previously defined in the external catalog (e.g., Collibra, Snowflake, etc.).

1. Click the Governance icon in the navigation menu and select the Tags tab.

2. Click Refresh External Tags.

Link an external catalog to a data source

External tags will be automatically detected when you create a new data source that originates in an external catalog, or they can be linked directly from the data source overview page.

Custom REST catalog

When using custom REST catalogs, the GET/dataSource/page/{id} endpoint returns a human-readable information page from the REST catalog for the data source associated with {id}. Immuta provides this as a mechanism for allowing the REST catalog to provide additional information about the data source that may not be directly ingested by or visible within Immuta. This link is accessed in the Immuta UI when a user clicks the catalog logo associated with the data source on the data source overview page.

Add tags to data sources

1. Click the Data icon in the navigation menu and select the Data Sources tab.

2. Select a data source.

3. Click the Add Tags button at the bottom of the Overview tab.

4. Begin typing Environment.dev in the Search by Name field and select the tag from the dropdown list.

5. Click Add. A list of the applied tags will populate at the bottom of the Overview tab.

6. Repeat as necessary for other data sources and tags.

Verify discovered tags

If sensitive data discovery is enabled by an application admin on the Immuta app settings page, Immuta will automatically discover sensitive data and tag those columns with Discovered tags when the data source is created. For more information about this feature, see the Sensitive data discovery page.

Disable discovered tags from the data dictionary

If a governor, data owner, or data source expert disables a Discovered tag from the data dictionary, the column will not be re-tagged when the data source's fingerprint is recalculated or SDD is re-run. When a Discovered tag is disabled, the tag will not completely disappear, so it can be manually enabled from the tag side sheet.

To disable a discovered tag,

1. Navigate to a data source and click the Data Dictionary tab.

2. Scroll to the column you want to remove the tag from and click the tag you want to remove.

3. Click Disable in the side sheet and then click Confirm.

Remove tags from data sources

1. Click the Data icon in the navigation menu and select the Data Sources tab.

2. Select a data source.

3. Scroll to the Tags section at the bottom of the Overview tab, and click on the tag you want to remove.

4. Click Delete in the side sheet and then click Confirm.

Manage data dictionary tags

The data dictionary lists the columns within the data source and the value type of the data within each column. From this page, governors can add tags to or remove them from specific columns in a data source.

Add tags to the data dictionary

1. Navigate to a data source and click the Data Dictionary tab.

2. Scroll to the column you want to add a tag to and click Add Tags.

3. Begin typing in the Search by Name field and select the tag from the dropdown list.

4. Click Add. The applied tag will appear below the column name in the data dictionary.

Remove tags from the data dictionary

1. Navigate to a data source and click the Data Dictionary tab.

2. Scroll to the column you want to remove the tag from and click on the tag you want to delete.

3. Click Delete in the side sheet and then click Confirm.

Manage project tags

Add tags to projects

1. Click the Data icon and select Projects in the left sidebar.

2. Select a project.

3. Click the Add Tags button at the bottom of the Project Overview tab.

4. Begin typing in the Search by Name field that appears, and then select the tag from the dropdown list.

5. Click Add. A list of the applied tags will populate at the bottom of the project overview.

Remove tags from projects

1. Click the Data icon and select Projects in the left sidebar.

2. Select a project.

3. Scroll to the Tags section at the bottom of the Overview tab, and then click the tag you want to delete.

4. Click Delete in the side sheet and then click Confirm.

Only application admins can on the Immuta app settings page. Then, data source creators can disable SDD on a data-source-by-data-source basis. Additionally, governors, data source owners, and data source experts can disable any unwanted Discovered tags in the data dictionary to prevent them from being used and auto-tagged on that data source in the future.

Configurable global settings

Global template

When SDD is triggered on a data source, the job is run for the identifiers within the set template. If a template is not set, the identifier and template within the SDD job are defined by the global setting. By default, the global setting will run for all identifiers in the system. However, a system administrator can instead.

An active global template cannot be deleted.

Sample size

SDD uses a sample of data to assess the likelihood that a column contains data that fits the pattern specified in the configured identifiers.

The default for SDD is to sample 1000 records (the sample size) during this process. However, administrators can taken by SDD on the Immuta app settings page. In general, increasing the sample size increases the accuracy of SDD predictions, but decreasing the number of records sampled during SDD may be necessary to meet some organizations' compliance requirements.

Tag mutability

When SDD is triggered by a data owner, all column tags that were previously applied by SDD are removed and the tags prescribed by the latest run are applied. However, if SDD is triggered because a new column is detected by schema monitoring, tags will only be applied to the new column, and no tags will be modified on existing columns.

Dry run

Users can also configure SDD to do a dryRun, which allows them to see what tags would be applied to a data source without actually applying them. See the for details.

SDD workflow

Two common workflows for using SDD are outlined below. The first illustrates how to apply a single global template to all data sources, while the second outlines how users can create and apply templates to data sources they own.

Workflow 1: Apply a global template to all data sources

Workflow 2: Apply a template to a specific data source

1. Data governor creates one or more custom identifiers:

Attributes overview

Attributes of all custom identifiers and templates are provided on the . However, attributes specific to this section are outlined below.

Run SDD on data sources

1. Specify the data sources you would like to run SDD on, and save the payload in a .json file.

   Or choose to run SDD on all the data sources in Immuta, and save the payload in a .json file.

2. Trigger SDD using one of these methods:

If sensitive data discovery was successfully run, you will receive a response similar to this:

Additional tutorials

Test SDD on a data source

Users can test how SDD will apply tags to their data sources by completing a dryRun, which allows users to test templates and tags:

• test templates: If a template is specified in the payload when the dryRun is true, SDD will use this template instead of the template applied to the data source. Note: SDD will error if a template is specified here when dryRun is false.

• test tags: Instead of applying tags, SDD just returns the tags that would be applied to the data source. This allows users to evaluate whether or not identifiers or templates are applying tags correctly without updating the data source.

After evaluating whether or not the tags have been applied appropriately, users can then make necessary changes to a template before triggering SDD again.

To complete a dryRun,

1. Trigger SDD using one of these methods:

1. You will receive a response that illustrates tags that will be added, tags that will be removed, and the final SDD result:

1. Once you are satisfied with how tags are applied by SDD, set dryRun to false (or omit it from the payload).

1. Trigger SDD again:

1. If the request was successful, you will receive a response similar to this one:

Trigger SDD in the Immuta UI

1. Select a data source from your My Data Sources page.

2. Click the Health Check dropdown menu.

3. In the Sensitive Data Discovery (SDD) section, click Re-run.

What's next

Continue to one of the following tutorials:

• Create a custom identifier: Data governors can create custom identifiers to define their own regular expressions, dictionaries, and tags that SDD will use to discover and tag data.

Sensitive data discovery (SDD) is an Immuta feature that uses sensitive data patterns to determine what type of data your column represents. Using identification rules and data samples from your tables, Immuta matches your data and can assign the appropriate tags to your data dictionary. This saves the time of identifying your data manually and provides the benefit of a standard taxonomy across all your data sources in Immuta.

Architecture

SDD works by looking at a sample of data from each table that it checks against templates compiled of built-in or customized identifiers. If an identifier's pattern is matched with a column of the sampled data with an appropriate amount of confidence, then the corresponding tag is applied to that column, signifying the data it contains.

SDD queries a small sample of data for each data source in Immuta. This sample is temporarily held in memory to check for identifier matches. Then Immuta applies the relevant tags to those columns where matches were found.

This sampling and tagging process will happen anytime SDD is run. SDD can be triggered through the , through , or in the Immuta UI on the data sources overview page. SDD will also run automatically anytime one of the following events occurs:

• A new data source is created.

• Schema detection is enabled and a new data source is detected.

• Column detection is enabled and new columns are detected. Here, SDD will only run on new columns and no existing tags will be removed or changed.

Components

Sensitive data discovery (SDD) comprises two major elements: and .

Identifier

The identifier is the basic building block of SDD. Each identifier in Immuta is a unique pattern (e.g., a regex or a list of values) and a list of tags to apply to data that matches the pattern. When Immuta recognizes that pattern, it can understand the type of data and tag the data to describe the type. For example, Immuta has the built-in identifier US_SOCIAL_SECURITY_NUMBER. Immuta will use a regex to look for strings of exactly nine digits, with or without hyphens after the third and fifth digits, with a leading digit between 0 and 8. SDD then scores columns by the percentage of values that match the pattern defined. This score determines whether or not the configured tags will be applied to a column. Once it finds a column that fits the expected pattern of US_SOCIAL_SECURITY_NUMBER with a reasonable match score, it will know how to tag it.

There are two types of identifiers:

• Custom identifier: Custom identifiers allow data governors to create their own regular expressions, dictionaries, and tags that SDD will use to discover and tag data.

Supported identifier types

The three types of identifiers are described below:

Templates

A template is a collection of identifiers and settings that drive the configuration of SDD runs. The settings users can apply through templates include the following:

• classifiers (identifiers) are applied to data sources in the SDD run.

• tags is an optional override for the tags applied by the identifiers.

• minConfidence is an optional override for the minConfidence established in the identifier(s). When the detection confidence is at least the percentage defined in minConfidence, tags are applied.

• sampleSize is an optional override for how many records to sample from the data source.

Considerations

• SDD does not run on data sources with over 1600 columns.

• Deleting the built-in Discovered tags is not recommended: If you do delete built-in Discovered tags and use SDD, when the identifier is detected, the column will not be tagged. Tags can be disabled on a column-by-column basis from the data dictionary, or SDD can be turned off on a data-source-by-data-source basis when creating a data source.

Configure and customize SDD

Enable Sensitive Data Discovery

1. Click the App Settings icon in the left sidebar.

2. Click Sensitive Data Discovery in the left panel to navigate to that section.

3. Select the checkbox to enable SDD, and then click Save and Confirm to apply your changes.

Configure a Global Template

1. Click the App Settings icon in the left sidebar.

2. Click Sensitive Data Discovery in the left panel to navigate to that section.

3. Enter the name of your global template in the Global SDD Template Name field.

4. Click Save, and then Confirm your changes.

Configure a Default Sample Size

When a sample size is not specified in a template, SDD will use the default sample size of 1000 records. To adjust the sample size,

1. Click the App Settings icon in the left sidebar.

2. Click Sensitive Data Discovery in the left panel to navigate to that section.

3. Enter the number of rows in a data source you would like sampled when running SDD in the Default SDD Sample Size field.

4. Click Save, and then Confirm your changes.

Create a template

1. Generate your API key on the API Keys tab on your profile page and save the API key somewhere secure. You will include this API key in the authorization header when you make a request to the Immuta API.

2. Find identifiers to include in your template using one of these methods:

immuta api sdd/classifier?sortField=name&sortOrder=asc&limit=25&searchText=IDENTIFIER

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: 12345678900000" \
    https://your-immuta-url.immuta.com/sdd/classifier?sortField=name&sortOrder=asc&limit=25&searchText=IDENTIFIER

1. If the request was successful, you will receive a list of available identifiers.

{
  "count": 3,
  "hits": [
    {
      "createdBy": {
        "id": 1,
        "name": "John",
        "email": "john@example.com"
      },
      "name": "ACCOUNT_NUMBER_IDENTIFIER",
      "displayName": "Account Number Identifier",
      "description": "This identifier recognizes account numbers using a regex",
      "type": "regex",
      "config": {
        "tags": [
          "Discovered.account-number"
        ],
        "regex": "[0-9]{9}-[0-9]{3}-[0-9]{1}",
        "minConfidence": 0.5
      },
      "id": 104,
      "createdAt": "2021-10-20T19:12:24.889Z",
      "updatedAt": "2021-10-20T19:12:24.889Z"
    },
    {
      "createdBy": {
        "id": 1,
        "name": "John",
        "email": "john@example.com"
      },
      "name": "EMPLOYEE_DESK_LOCATION_IDENTIFIER",
      "displayName": "Employee Desk Location Identifier",
      "description": "This identifier detects when an employee's desk location appears in a dataset.",
      "type": "dictionary",
      "config": {
        "tags": [
          "Discovered.desk-location"
        ],
        "values": [
          "Research Lab",
          "Blue Room",
          "Purple Room"
        ],
        "caseSensitive": false,
        "minConfidence": 0.6
      },
      "id": 68,
      "createdAt": "2021-10-20T17:57:51.696Z",
      "updatedAt": "2021-10-20T17:57:51.696Z"
    },
    {
      "createdBy": {
        "id": 1,
        "name": "John",
        "email": "john@example.com"
      },
      "name": "SOCIAL_SECURITY_NUMBER_COLUMNS_IDENTIFIER",
      "displayName": "Social Security Number Columns Identifier",
      "description": "This identifier recognizes column names that match the defined regex pattern.",
      "type": "columnNameRegex",
      "config": {
        "tags": [
          "Discovered.Social Security Numbers"
        ],
        "columnNameRegex": "ssn|social ?security"
      },
      "id": 67,
      "createdAt": "2021-10-20T17:57:17.930Z",
      "updatedAt": "2021-10-20T17:57:17.930Z"
    }
  ]
}

1. Save the template payload in a .json file. Use the tabs below to see different examples of templates.

{
  "name": "ACCOUNT_NUMBERS_TEMPLATE",
  "displayName": "Account Numbers Template",
  "description": "This template contains the identifier that recognizes account numbers.",
  "classifiers": [
    {
      "name": "ACCOUNT_NUMBER_IDENTIFIER"
    }
  ],
  "sampleSize": 100
}

{
  "name": "EMPLOYEE_DESK_LOCATION_TEMPLATE",
  "displayName": "Employee Desk Location Template",
  "description": "This template contains the identifier that detects when the name of the room an employee's desk is in appears in a dataset.",
  "classifiers": [
    {
      "name": "EMPLOYEE_DESK_LOCATION_IDENTIFIER"
    }
  ],
  "sampleSize": 100
}

{
  "name": "SOCIAL_SECURITY_NUMBERS_TEMPLATE",
  "displayName": "Social Security Numbers Template",
  "description": "This template contains the identifier that matches social security number column names with the defined regex.",
  "classifiers": [
    {
      "name": "SOCIAL_SECURITY_NUMBER_COLUMNS_IDENTIFIER"
    }
  ],
  "sampleSize": 100
}

{
  "name": "STUDENT_LOCATION_TEMPLATE",
  "displayName": "Student Location Template",
  "description": "This template contains the identifier that detects when a student's residence hall, floor, or room appears in a dataset.",
  "classifiers": [
    {
      "name": "STUDENT_LOCATION_IDENTIFIER"
    }
  ],
  "sampleSize": 100
}

1. Create the template:

immuta api sdd/template -X POST --input ./example-payload.json

curl \
    --request POST \
    --header "Content-Type: application/json" \
    --header "Authorization: 12345678900000" \
    --data @example-payload.json \
    https://your-immuta-url.immuta.com/sdd/template

1. If the request is successful, you will receive a response that contains details about the template. Use the tabs below to see different responses for different templates.

{
  "name": "ACCOUNT_NUMBERS_TEMPLATE",
  "displayName": "Account Numbers Template",
  "description": "This template contains the identifier that recognizes account numbers.",
  "sampleSize": 100,
  "createdBy": {
    "id": 1,
    "name": "John",
    "email": "john@example.com"
  },
  "id": 1,
  "createdAt": "2021-10-21T19:12:22.092Z",
  "updatedAt": "2021-10-21T19:12:22.092Z",
  "classifiers": [
    {
      "name": "ACCOUNT_NUMBER_IDENTIFIER",
      "overrides": {}
    }
  ]
}

{
  "name": "EMPLOYEE_DESK_LOCATION_TEMPLATE",
  "displayName": "Employee Desk Location Template",
  "description": "This template contains the identifier that detects when the name of the room an employee's desk is in appears in a dataset.",
  "sampleSize": 100,
  "createdBy": {
    "id": 1,
    "name": "John",
    "email": "john@example.com"
  },
  "id": 1,
  "createdAt": "2021-10-21T18:03:58.967Z",
  "updatedAt": "2021-10-21T18:03:58.967Z",
  "classifiers": [{
    "name": "EMPLOYEE_DESK_LOCATION_IDENTIFIER",
    "overrides": {}
  }]
}

{
  "name": "SOCIAL_SECURITY_NUMBERS_TEMPLATE",
  "displayName": "Social Security Numbers Template",
  "description": "This template contains the identifier that matches social security number column names with the defined regex.",
  "sampleSize": 100,
  "createdBy": {
    "id": 1,
    "name": "John",
    "email": "john@example.com"
  },
  "id": 2,
  "createdAt": "2021-10-21T19:12:22.092Z",
  "updatedAt": "2021-10-21T19:12:22.092Z",
  "classifiers": [
    {
      "name": "SOCIAL_SECURITY_NUMBER_COLUMNS_IDENTIFIER",
      "overrides": {}
    }
  ]
}

{
  "name": "STUDENT_LOCATION_TEMPLATE",
  "displayName": "Student Location Template",
  "description": "This template contains the identifier that detects when a student's residence hall, floor, or room appears in a dataset.",
  "sampleSize": 100,
  "createdBy": {
    "id": 1,
    "name": "John",
    "email": "john@example.com"
  },
  "id": 1,
  "createdAt": "2021-10-21T18:03:58.967Z",
  "updatedAt": "2021-10-21T18:03:58.967Z",
  "classifiers": [{
    "name": "STUDENT_LOCATION_IDENTIFIER",
    "overrides": {}
  }]
}

Apply a template to data sources

Attributes of all custom identifiers and templates are provided on the Sensitive data discovery API page. However, attributes specific to this section are outlined in the table below.

1. Find templates to apply to your data sources:

immuta api sdd/template

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: 12345678900000" \
    https://your-immuta-url.immuta.com/sdd/template

1. If the request was successful, you will receive a list of available templates.

{
  "count": 3,
  "hits": [
    {
      "name": "ACCOUNT_NUMBERS_TEMPLATE",
      "displayName": "Account Numbers Template",
      "description": "This template contains the identifier that recognizes account numbers.",
      "sampleSize": 100,
      "createdBy": {
        "id": 1,
        "name": "John",
        "email": "john@example.com"
      },
      "id": 2,
      "createdAt": "2021-10-20T19:13:35.319Z",
      "updatedAt": "2021-10-20T19:13:35.319Z",
      "classifiers": [
        {
          "name": "ACCOUNT_NUMBER_IDENTIFIER",
          "overrides": {}
        }
      ]
    },
    {
      "name": "EMPLOYEE_DESK_LOCATION_TEMPLATE",
      "displayName": "Employee Desk Location Template",
      "description": "Contains identifier that detects when the name of a room a desk is in appears in a dataset.",
      "sampleSize": 100,
      "createdBy": {
        "id": 1,
        "name": "John",
        "email": "john@example.com"
      },
      "id": 1,
      "createdAt": "2021-10-20T18:03:58.967Z",
      "updatedAt": "2021-10-20T18:03:58.967Z",
      "classifiers": [
        {
          "name": "EMPLOYEE_DESK_LOCATION_IDENTIFIER",
          "overrides": {}
        }
      ]
    },
    {
      "name": "SOCIAL_SECURITY_NUMBERS_TEMPLATE",
      "displayName": "Social Security Numbers Template",
      "description": "Contains identifier that matches ssn column names with the defined regex.",
      "sampleSize": 100,
      "createdBy": {
        "id": 1,
        "name": "John",
        "email": "john@example.com"
      },
      "id": 3,
      "createdAt": "2021-10-20T19:13:58.359Z",
      "updatedAt": "2021-10-20T19:13:58.359Z",
      "classifiers": [
        {
          "name": "SOCIAL_SECURITY_NUMBER_COLUMNS_IDENTIFIER",
          "overrides": {}
        }
      ]
    }
  ]
}

1. Select an appropriate template to apply to your data sources, and save the payload in a .json file:

{
  "template": "ACCOUNT_NUMBERS_TEMPLATE",
  "sources": [
    "Insurance Data"
  ]
}

1. Apply the template to your data source(s):

immuta api sdd/template/apply -X PUT --input ./example-payload.json

curl \
    --request PUT \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer dea464c07bd07300095caa8" \
    --data @example-payload.json \
    https://your-immuta-url.immuta.com/sdd/template/apply

1. You will receive a response that indicates whether or not the template was successfully applied to your data sources.

{
  "success": true
}

Additional tutorials

Clone a template

Users cannot modify templates created by other data owners, but they can clone templates and make changes to the clone.

1. Get a list of templates to determine the template you want to clone using one of these methods:

immuta api sdd/sdd/template?sortField=name&sortOrder=asc&offset=5&limit=5

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: 12345678900000" \
    https://your-immuta-url.immuta.com/sdd/template?sortField=name&sortOrder=asc&offset=5&limit=5

1. Save the template clone name and details in a .json file.

{
  "name": "INSURANCE_ACCOUNT_NUMBERS",
  "displayName": "Insurance Account Numbers",
  "description": "This template is specific to insurance accounts."
}

1. Clone the template:

immuta api sdd/template/ACCOUNT_NUMBERS_TEMPLATE/clone -X POST --input ./example-payload.json

curl \
    --request POST \
    --header "Content-Type: application/json" \
    --header "Authorization: 12345678900000" \
    --data @example-payload.json \
    https://your-immuta-url.immuta.com/sdd/template/ACCOUNT_NUMBERS_TEMPLATE/clone

1. If the request was successful, you will receive a response that provides details about the template clone.

{
  "name": "INSURANCE_ACCOUNT_NUMBERS",
  "displayName": "Insurance Account Numbers",
  "description": "This template is specific to insurance accounts.",
  "sampleSize": 100,
  "createdBy": {
    "id": 1,
    "name": "John",
    "email": "john@example.com"
  },
  "id": 4,
  "createdAt": "2021-10-20T20:48:37.805Z",
  "updatedAt": "2021-10-20T20:48:37.805Z",
  "classifiers": [
    {
      "name": "ACCOUNT_NUMBER_IDENTIFIER",
      "overrides": {}
    }
  ]
}

You can now modify the template, such as changing the identifiers (classifiers) included and the sampleSize.

Configure entity tags and confidence

To disable entity tags from being set, you can create a template to that configures the identifier that contains that tag.

For example, the built-in PERSON_NAME identifier contains the following tags: Discovered.PHI, Discovered.PII, Discovered.Entity.Person Name, and Discovered.Identifier Indirect. However, your organization doesn't have any health data, so you don't want the PHI tag to be applied to your data sources but you do want all the other tags within that identifier.

To override the Discovered.PHI tag, you would create a template that includes the PERSON_NAME identifier and removes the Discovered.PHI from the list of tags in the template payload.

1. View the details about the PERSON_NAME identifier so you know what to include in your template using one of these methods:

immuta api sdd/classifier?sortField=name&sortOrder=asc&limit=25&searchText=PERSON_NAME

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: 12345678900000" \
    https://your-immuta-url.immuta.com/sdd/classifier?sortField=name&sortOrder=asc&limit=25&searchText=PERSON_NAME

1. If the request was successful, the response will include details about the PERSON_NAME identifier.

{
  "createdBy": {
    "id": 21,
    "name": "Immuta System Account",
    "email": "immuta_system@immuta.com"
  },
  "name": "PERSON_NAME",
  "displayName": "Person Name",
  "description": "Detects strings consistent with a dictionary of people's names.",
  "type": "builtIn",
  "config": {
    "tags": [
      "Discovered.PHI",
      "Discovered.PII",
      "Discovered.Entity.Person Name",
      "Discovered.Identifier Indirect"
    ],
    "minConfidence": 0.3
  },
  "id": 54,
  "createdAt": "2021-10-21T07:35:14.416Z",
  "updatedAt": "2021-10-21T12:57:43.919Z"
}

1. Remove the Discovered.PHI tag from the list of tags in the identifier config, and save the template payload in a .json file.

{
  "name": "PERSON_NAME_OVERRIDE",
  "displayName": "Person Name Override",
  "description": "This template removes the PHI tag from the PERSON_NAME identifier.",
  "classifiers": [
    {
      "name": "PERSON_NAME",
        "overrides": {
          "tags": [
            "Discovered.PII",
            "Discovered.Entity.Person Name",
            "Discovered.Identifier Indirect"
          ]
        }
      }
    ],
  "sampleSize": 100
}

1. Create the template:

immuta api sdd/template -X POST --input ./example-payload.json

curl \
    --request POST \
    --header "Content-Type: application/json" \
    --header "Authorization: 12345678900000" \
    --data @example-payload.json \
    https://your-immuta-url.immuta.com/sdd/template

If the request is successful, you will receive a response that details the new template:

{
  "name": "PERSON_NAME_OVERRIDE",
  "displayName": "Person Name Override",
  "description": "This template removes the PHI tag from the PERSON_NAME identifier.",
  "sampleSize": 100,
  "createdBy": {
    "id": 1,
    "name": "John",
    "email": "john@example.com"
  },
  "id": 1,
  "createdAt": "2021-10-21T17:11:18.057Z",
  "updatedAt": "2021-10-21T17:11:18.057Z",
  "classifiers": [
    {
      "name": "PERSON_NAME",
      "overrides": {
        "tags": [
          "Discovered.PII",
          "Discovered.Entity.Person Name",
          "Discovered.Identifier Indirect"
        ]
      }
    }
  ]
}

What's next

Now that you've created a template, continue to one of the following tutorials:

• SDD global settings: Opt to add your template to the SDD global settings so that Immuta will use this template to run SDD for all data sources.

• Run sensitive data discovery on a data source

In previous documentation, identifier is referred to as classifier. The language is being updated to identifier to be more accurate and not conflate meaning with the Immuta data classification and frameworks feature.

Attributes of the custom column name regex identifier

Attributes of all custom identifiers are provided on the Sensitive data discovery API page. However, attributes specific to the custom column name regex identifier are outlined in the table below.

Create a custom column name regex identifier

1. Generate your API key on the API Keys tab on your profile page and save the API key somewhere secure. You will include this API key in the authorization header when you make a request to the Immuta API or use it to configure your instance with the Immuta CLI.

2. Save the custom column name regex identifier payload in a .json file. The regex ^ssn|social ?security$ looks for column names that match ssn, socialsecurity, or social security.

   Copy

   {
     "name": "SOCIAL_SECURITY_NUMBER_COLUMNS_IDENTIFIER",
     "displayName": "Social Security Number Columns Identifier",
     "description": "This identifier recognizes column names that match the defined regex pattern.",
     "type": "columnNameRegex",
     "config": {
       "columnNameRegex": "^ssn|social ?security$",
       "tags": ["Discovered.Social Security Numbers"]
     }
   }

3. Create the identifier using one of these methods:

immuta api sdd/classifier -X POST --input ./example-payload.json

curl \
    --request POST \
    --header "Content-Type: application/json" \
    --header "Authorization: 12345678900000" \
    --data @example-payload.json \
    https://your-immuta-url.immuta.com/sdd/classifier

1. If the request is successful, you will receive a response that contains details about the identifier.

{
  "createdBy": {
    "id": 1,
    "name": "John",
    "email": "john@example.com"
  },
  "name": "SOCIAL_SECURITY_NUMBER_COLUMNS_IDENTIFIER",
  "displayName": "Social Security Number Columns Identifier",
  "description": "This identifier recognizes column names that match the defined regex pattern.",
  "type": "columnNameRegex",
  "config": {
    "tags": [
      "Discovered.Social Security Number"
    ],
    "columnNameRegex": "^ssn|social ?security$"
  },
  "id": 2,
  "createdAt": "2021-10-14T18:48:56.289Z",
  "updatedAt": "2021-10-14T18:48:56.289Z"
}

What's Next

Continue to one of the following tutorials:

• Run sensitive data discovery on data sources: Trigger SDD to run on specified data sources.

• Create a template: Although only data governors can create identifiers, data owners can add identifiers to templates, which they then apply to their data sources to override minConfidence or tags for identifiers within the template.

Attributes

Attributes of identifiers and templates are provided on the Sensitive data discovery API page. However, attributes specific to listing identifiers are outlined in the table below.

Response details

The response lists all built-in identifiers that are currently supported in Immuta SDD and their details, including their name and description. For example,

{
  "count": 67,
  "hits": [
    {
      "createdBy": {
        "id": 21,
        "name": "Immuta System Account",
        "email": "immuta_system@immuta.com"
      },
      "name": "AGE",
      "displayName": "Age",
      "description": "Detects numeric strings between 10 and 199, provided the column header contains text such as `age`, `year`, `years`, `yr`, or `yrs`.",
      "type": "builtIn",
      "config": {
        "minConfidence": 0.7,
        "tags": [
          "Discovered.PII",
          "Discovered.Identifier Indirect",
          "Discovered.PHI",
          "Discovered.Entity.Age"
        ],
        "conditionalTags": {}
      },
      "id": 3,
      "createdAt": "2021-10-28T07:34:58.761Z",
      "updatedAt": "2021-10-28T07:34:58.761Z"
    }
  ]
}

List built-in identifiers

1. Generate your API key on the API Keys tab on your profile page and save the API key somewhere secure. You will include this API key in the authorization header when you make a request to the Immuta API or use it to configure your instance with the Immuta CLI.

2. List built-in identifiers using one of these methods:

immuta api sdd/classifier?sortField=name&sortOrder=asc&limit=100&type=builtIn

curl \
    --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: 12345678900000" \
    https://your-immuta-url.immuta.com/sdd/classifier?sortField=name&sortOrder=asc&limit=100&type=builtIn

If the request was successful, you will receive a list of built-in identifiers.

What's next

• Run sensitive data discovery on data sources: Trigger SDD to run on specified data sources.

• Create a template: Although only data governors can create identifiers, data owners can add identifiers to templates, which they then apply to their data sources to override minConfidence or tags for identifiers within the template.

• Create a custom identifier: Data governors can create custom identifiers to define their own regular expressions, dictionaries, and tags that SDD will use to discover and tag data.

  • Create a regex identifier

  • Create a column name regex identifier

  • Create a dictionary identifier

In previous documentation, identifier is referred to as classifier. The language is being updated to identifier to be more accurate and not conflate meaning with the Immuta data classification and frameworks feature.

Attributes of the custom regex identifier

Attributes of all custom identifiers are provided on the Sensitive data discovery API page. However, attributes specific to the custom regex identifier are outlined in the table below.

Create a custom regex identifier

1. Generate your API key on the API Keys tab on your profile page and save the API key somewhere secure. You will include this API key in the authorization header when you make a request to the Immuta API or use it to configure your instance with the Immuta CLI.

2. Save the custom regex identifier payload in a .json file.

   Copy

   {
     "name": "ACCOUNT_NUMBER_IDENTIFIER",
     "displayName": "Account Number Identifier",
     "description": "This identifier recognizes account numbers using a regex",
     "type": "regex",
     "config": {
       "regex": "^[0-9]{9}-[0-9]{3}-[0-9]{1}$",
       "minConfidence": 0.5,
       "tags": ["Discovered.account-number"]
     }
   }

3. Create the identifier using one of these methods:

immuta api sdd/classifier -X POST --input ./example-payload.json

curl \
    --request POST \
    --header "Content-Type: application/json" \
    --header "Authorization: 12345678900000" \
    --data @example-payload.json \
    https://your-immuta-url.immuta.com/sdd/classifier

1. If the request is successful, you will receive a response that contains details about the identifier.

{
  "createdBy": {
    "id": 1,
    "name": "John",
    "email": "john@example.com"
  },
  "name": "ACCOUNT_NUMBER_IDENTIFIER",
  "displayName": "Account Number Identifier",
  "description": "This identifier recognizes account numbers using a regex",
  "type": "regex",
  "config": {
    "tags": [
      "Discovered.account-number"
    ],
    "regex": "[0-9]{9}-[0-9]{3}-[0-9]{1}",
    "minConfidence": 0.5
  },
  "id": 1,
  "createdAt": "2021-10-14T18:48:56.289Z",
  "updatedAt": "2021-10-14T18:48:56.289Z"
}

What's next

Continue to one of the following tutorials:

• Run sensitive data discovery on data sources: Trigger SDD to run on specified data sources.

• Create a template: Although only data governors can create identifiers, data owners can add identifiers to templates, which they then apply to their data sources to override minConfidence or tags for identifiers within the template.

In previous documentation, identifier is referred to as classifier. The language is being updated to identifier to be more accurate and not conflate meaning with the Immuta data classification and frameworks feature.

Attributes of the custom dictionary identifier

Attributes of all custom identifiers are provided on the Sensitive data discovery API page. However, attributes specific to the custom dictionary identifier are outlined in the table below.

Create a custom dictionary identifier

1. Generate your API key on the API Keys tab on your profile page and save the API key somewhere secure. You will include this API key in the authorization header when you make a request to the Immuta API or use it to configure your instance with the Immuta CLI.

2. Save the custom dictionary identifier payload in a .json file. The dictionary below contains the words Research Lab, Blue Room, and Purple Room.

   Copy

   {
     "name": "EMPLOYEE_DESK_LOCATION_IDENTIFIER",
     "displayName": "Employee Desk Location Identifier",
     "description": "This identifier detects when an employee's desk location appears in a dataset.",
     "type": "dictionary",
     "config": {
       "values": ["Research Lab", "Blue Room", "Purple Room"],
       "caseSensitive": false,
       "minConfidence": 0.6,
       "tags": ["Discovered.desk-location"]
     }
   }

3. Create the identifier using one of these methods:

immuta api sdd/classifier -X POST --input ./example-payload.json

curl \
    --request POST \
    --header "Content-Type: application/json" \
    --header "Authorization: 12345678900000" \
    --data @example-payload.json \
    https://your-immuta-url.immuta.com/sdd/classifier

1. If the request is successful, you will receive a response that contains details about the identifier.

{
  "createdBy": {
    "id": 1,
    "name": "John",
    "email": "john@example.com"
  },
  "name": "EMPLOYEE_DESK_LOCATION_IDENTIFIER",
  "displayName": "Employee Desk Location Identifier",
  "description": "This identifier detects when an employee's desk location appears in a dataset.",
  "type": "dictionary",
  "config": {
    "tags": [
      "Discovered.desk-location"
    ],
    "values": [
      "Research Lab",
      "Blue Room",
      "Purple Room"
    ],
    "caseSensitive": false,
    "minConfidence": 0.6
  },
  "id": 68,
  "createdAt": "2021-10-20T17:57:51.696Z",
  "updatedAt": "2021-10-20T17:57:51.696Z"
}

What's next

Continue to one of the following tutorials:

• Run sensitive data discovery on data sources: Trigger SDD to run on specified data sources.

• Create a template: Although only data governors can create identifiers, data owners can add identifiers to templates, which they then apply to their data sources to override minConfidence or tags for identifiers within the template.

Immuta is pre-configured with a set of tags that can be used to write global policies before data sources even exist. See a list of the built-in Discovered tags below and the Built-in identifier reference page for information about where these tags will be applied.

Country tags

All the tags below belong to the Country parent. For example, the full tag name will appear as Discovered . Country . Argentina.

Entity tags

All the tags below belong to the Entity parent. For example, the full tag name will appear as Discovered . Entity . Aadhaar Individual.

Identifier tags

None of the tags below have an additional parent or child tag. For example, the full tag name will appear as Discovered . Identifier Direct.

Personal information tags

None of the tags below have an additional parent or child tag. For example, the full tag name will appear as Discovered . PCI.