Loading...
Loading...
Loading...
Loading...
Loading...
Audience: Data Engineers
Content Summary: This page contains example request payloads for creating policies.
Sample data is processed during computation of k-anonymization policies
When a k-anonymization policy is applied to a data source, the columns targeted by the policy are queried under a fingerprinting process that generates rules enforcing k-anonymity. The results of this query, which may contain data that is subject to regulatory constraints such as GDPR or HIPAA, are stored in Immuta's metadata database.
The location of the metadata database depends on your deployment:
Self-managed Immuta deployment: The metadata database is located in the server where you have your external metadata database deployed.
SaaS Immuta deployment: The metadata database is located in the AWS global segment you have chosen to deploy Immuta.
To ensure this process does not violate your organization's data localization regulations, you need to first activate this masking policy type before you can use it in your Immuta tenant. To enable k-anonymization for your account, see the k-anonymization section on the app settings how-to guide.
Sample data is processed during computation of k-anonymization policies
When a k-anonymization policy is applied to a data source, the columns targeted by the policy are queried under a fingerprinting process that generates rules enforcing k-anonymity. The results of this query, which may contain data that is subject to regulatory constraints such as GDPR or HIPAA, are stored in Immuta's metadata database.
The location of the metadata database depends on your deployment:
Self-managed Immuta deployment: The metadata database is located in the server where you have your external metadata database deployed.
SaaS Immuta deployment: The metadata database is located in the AWS global segment you have chosen to deploy Immuta.
To ensure this process does not violate your organization's data localization regulations, you need to first activate this masking policy type before you can use it in your Immuta tenant. To enable k-anonymization for your account, see the k-anonymization section on the app settings how-to guide.
Sample data is processed during computation of k-anonymization policies
When a k-anonymization policy is applied to a data source, the columns targeted by the policy are queried under a fingerprinting process that generates rules enforcing k-anonymity. The results of this query, which may contain data that is subject to regulatory constraints such as GDPR or HIPAA, are stored in Immuta's metadata database.
The location of the metadata database depends on your deployment:
Self-managed Immuta deployment: The metadata database is located in the server where you have your external metadata database deployed.
SaaS Immuta deployment: The metadata database is located in the AWS global segment you have chosen to deploy Immuta.
To ensure this process does not violate your organization's data localization regulations, you need to first activate this masking policy type before you can use it in your Immuta tenant. To enable k-anonymization for your account, see the k-anonymization section on the app settings how-to guide.
Audience: Data Engineers
Content Summary: This page contains details and examples of payload attributes for creating data sources.
connectionKey
The connectionKey
is a unique identifier for the collection of data sources being created. If an existing connectionKey
is used with new connection information, it will delete the old data sources and create new ones from the new information in the payload.
connection
Athena: Also requires region
and queryResultLocationBucket
. queryResultLocationDirectory
is optional. authenticationMethod
can be none
, accessKey
(default: username = access key, password = secret key), or instanceRole
.
BigQuery: Does not require hostname
and password
. Requires sid
, which is the GCP project ID, and userFiles
with the keyName
of KeyFilePath
and the base64-encoded keyfile.json
.
Databricks: Also requires httpPath
. No username
is required.
Hadoop: authenticationMethod
can be none
, userPassword
, hdInsight
, kerberos
, or kerberosHdInsight
.
Trino: authenticationMethod
can be No Authentication
, LDAP Authentication
, or Kerberos Authentication
.
Snowflake: Also requires warehouse
. authenticationMethod
can be userPassword
or PRIV_KEY_FILE
. If using PRIV_KEY_FILE
, do not specify a password
; userFiles
is required with the keyName
of PRIV_KEY_FILE
and the base64-encoded Snowflake key.
nameTemplate
Available templates include
<tablename>
<schema>
<database>
All cases of the name in Immuta should be lowercase.
For example, consider a table TPC.CUSTOMER
that is given the following nameTemplate
:
This nameTemplate
will produce a data source named tpc.customer
in a schema project named tpc
.
options
owners
sources
Best practice: Use Subscription Policies to Control Access
If you are not tagging individual columns, omit sources
to create data sources for all tables in the schema or database, and then use Subscription Policies to control access to the tables instead of excluding them from Immuta.
This attribute configures which sources are created. If sources
is not provided, all sources from the given connection will be created.
There are 3 types of sources than can be specified:
If you specify any sources (either tables or queries), but you still want to create data sources for the rest of the tables in the schema or database, you can specify all
as a source:
Best practice: Use schema monitoring
Excluding sources or specifying all: true
will turn on automatic schema monitoring in Immuta. As tables are added or removed, Immuta will look for those changes on a schedule (by default, once a day) and either disable or delete data sources for removed tables or create data sources for new tables. New tables will be tagged New
so that you can build a policy to restrict access to new tables until they are evaluated by data owners. Data owners will be notified of new tables, and all subscribers will be notified if data sources are disabled or deleted.
Immuta recommends creating a view in your native database instead of using this option, but if that is not possible, you can create data sources based on SQL statements:
If you want to select specific tables to be created as data sources, or if you want to tag individual data sources or columns within a data source, you need to leverage this parameter:
When specifying a table or query there are other options that can be specified:
If any columns are specified, those are the only columns that will be available in the data source.
If no columns are specified, Immuta will look for new or removed columns on a schedule (by default, once a day) and add or remove columns from the data sources automatically as needed.
New columns will be tagged New
, so you can build a policy to automatically mask new columns until they are approved.
Data Owners will be notified when columns are added or removed.
columns
is an array of objects for each column:
You can add descriptions to columns without having to specify all the columns in the data source. columnDescriptions
is an array of objects with the following schema:
You can add tags to columns or data sources. tags
is an object with the following schema:
Attribute | Description |
---|
Attribute | Description |
---|
Attribute | Description |
---|
Attribute | Description |
---|
Option | Description |
---|
Attribute | Description |
---|
Attribute | Description |
---|
Attribute | Description |
---|
dataSourceFormat |
|
schemaFormat |
|
tableFormat |
|
schemaProjectNameFormat |
|
type |
|
name |
|
iam (optional) |
|
name | The column name. |
dataType | The data type. |
nullable | Whether or not the column contains |
remoteType | The actual data type in the remote database. |
primaryKey | Specify whether this is the primary key of the remote table. |
description | Describe the column. |
columnName |
|
description |
|
table |
|
columns |
|
Audience: Data Engineers
Content Summary: This page contains example request payloads for creating projects.
Audience: Data Engineers
Content Summary: This page contains example request payloads for creating data sources.
userFile
)Your nativeSchemaFormat
must contain _immuta
to avoid schema name conflicts.
handler |
|
ssl |
|
database |
|
schema |
|
userFiles |
|
connectionStringOptions |
|
hostname |
|
port |
|
authenticationMethod |
username |
|
password |
|
staleDataTolerance |
|
expiration |
|
disableSensitiveDataDiscovery |
hardDelete |
|
tableTags |
|
columnDescriptions |
description | A short description for the data source. |
documentation | Markdown-supported documentation for the data source. |
naming |
owners | Specify owners for an individual data source. The payload is the same as owners at the root level. |
tags |
See below.
boolean
If true, Immuta will not perform . Default: false
.
.
See the example above in . This is required for query-based sources, but is optional for table-based sources and can be used to override the nameTemplate
provided for the whole database/schema.
.