Skip to content

Immuta v2021.3.0 Release Notes

Immuta v2021.3.0

Immuta version 2021.3.0 was released August 13, 2021.

v2021.3.0 New Features

General Availability

  • Databricks:

    • Adaptive Query Execution Support: Adaptive Query Execution is a Spark 3.0 feature that improves query performance by dynamically re-optimizing query plans during execution, specifically for queries with joins, aggregates, window functions, or subqueries. Adaptive Query Execution is enabled by default on Databricks runtime 7.3+ and Immuta 2021.3.0+. No special actions are required.

    • ADLS Gen 2 Support through S3 Access Pattern: Databricks users can connect to ADLS Gen 2 data sources using Immuta’s is3:// file system and S3 access pattern.

    • Hide the immuta database in Databricks: Users can hide the immuta database in Databricks. However, hiding the immuta database will not prevent someone from using it in queries. Instead, Immuta will just remove the database from the list of databases returned during a show databases command.

    • Masking improvements.

    • Simplified Databricks Configuration: This feature automates the configuration of Databricks clusters through the Immuta UI. Note: The immuta_conf.xml file is now optional and, therefore, no longer staged as a deployment artifact for Databricks installations. You can use the snippets from the Databricks configuration guide if you want to deploy an immuta_conf.xml file to set properties. If you have an existing immuta_conf.xml file, you can continue using it, but Immuta recommends to delete any default properties from the file that you have not explicitly overridden, or remove the file completely and rely on Spark environment variables. Either method will ensure that any property defaults changed in upcoming Immuta releases are propagated to your environment.

    • Snowflake Input Format for Databricks: This feature allows secure access to Snowflake from Databricks.

    • Support for Struct Data Types: Immuta transforms struct definitions into nested dictionaries. This feature is for Databricks only.

  • Enforce Minimum Password Complexity: This feature is for users accessing data through the Query Engine.

  • Global Policy improvements: Improved scalability.

  • Multiple Native Integrations: Users can configure multiple integrations for each type of native access pattern in a single instance of Immuta.

  • Overlapping Schemas/Names in Native Integrations: Immuta uses the nativeSchemaName and nativeViewName when creating views in native integrations to allow two native integrations to have overlapping schemas/names. When an Immuta data source has an associated native view, the schema/view name are displayed in the Immuta UI on the data source overview page.

  • Tag Enhancements: Periods can be used in tag names and non-leaf tags can be added to data sources and columns.

  • Data source documentation improvements: Documentation can be updated from an external catalog.

v2021.3.0 Bug Fixes

  • Spark session did not work when HBase authentication was turned on.
  • The NameNode plugin on Hadoop 2.7+ instantiated Configuration repeatedly.
  • SCIM responses for a user only contained the first attribute value.
  • SCIM users endpoint was not returning a user's groups.
  • Bulk connections could not be edited in the UI when searching by connection string.
  • Problems generating masked query plans.
  • Expanding and collapsing IAM issues in the UI.

v2021.3.0 Known Bugs

  • V2 API: Adding tags to an existing data source does not kick off the Global Policy update job.
  • Struct Support:
    • Error querying a struct data source when all columns are masked through a Global Policy.
    • Users can't query non-struct data sources with column names containing a period (.) if that column is masked by hashing.
  • Security Manager error on AWS metadata service.
  • Intermittent failures on GCP Databricks on job clusters.
  • Spark: Format preserving masking is broken on integer columns.
  • Native SQL Integrations:
    • Query snippets do not include database, schema, or table.
    • Multiple data sources pointing at the same tables will not work.

v2021.3.0 Migration Notes

  • If tokenExpirationInSeconds was being set manually in Advanced Configuration, users will need to reset the timeout using the new Session Token Expiration setting on the App Settings page.
  • All users must be on Immuta version 2020.2 or greater to migrate directly to 2021.3.

Immuta v2021.3 Patch Releases

Immuta v2021.3.1

Immuta 2021.3.1 was released August 24, 2021.

v2021.3.1 Bug Fixes

  • V2 API: Adding tags to an existing data source did not kick off the Global Policy update job.
  • Intermittent failures on GCP Databricks on job clusters.
  • Native SQL Integrations:
    • Query snippets did not include database, schema, or table.
    • Multiple data sources pointing at the same tables did not work.
  • Allow users to configure fingerprint sampleSize.
  • Immuta SaaS: Could not generate a System API key when the HDFS handler was disabled.

v2021.3.1 Known Bugs

  • Struct Support:
    • Error querying a struct data source when all columns are masked through a Global Policy.
    • Users can't query non-struct data sources with column names containing a period (.) if that column is masked by hashing.
  • Security Manager error on AWS metadata service.
  • Spark: Format preserving masking is broken on integer columns.

Immuta v2021.3.2

Immuta 2021.3.2 was released September 9, 2021.

v2021.3.2 Bug Fixes

  • Azure SCIM reported failures on successful group syncs.
  • dbt: a large dbt Snowflake project could cause the Query Engine pod to run out of memory.
  • Databricks:
    • CVE-2021-27568 and CVE-2017-18640.
    • Support storage auto scaling on GCP Databricks.
    • Support for data backed by ElasticSearch.
    • Users could SHOW PARTITIONS of tables to which they didn't have access.
    • Simplified Databricks configuration fixes.
    • /tmp directory was blocked under local_disk0 for R when using S3.
    • Struct Support:
      • Error querying a struct data source when all columns were masked through a Global Policy.
      • Users couldn't query non-struct data sources with column names containing a period (.) if that column was masked by hashing.
  • External Catalog and Immuta tags applied to the same data source sometimes led to Immuta tags getting dropped on catalog updates.
  • Global Row Redaction Policy creation error.
  • Race condition occurred during bulk data source creation/deletion.
  • Schema Evolution handlerMetadata was not getting updated.
  • Set ssl_min_protocol_version to TLSv1.2 by default; ssl_min_protocol_version can be set using an environment variable.
  • Spark: Format preserving masking was broken on integer columns.
  • Synapse manual bootstrap: invalid declare statement error.
  • Updated Patroni to the latest release.
  • Upgraded to the latest version of the Snowflake ODBC driver.

v2021.3.2 Known Bugs

  • SecurityManager error on AWS metadata service.

Immuta v2021.3.3

Immuta 2021.3.3 was released September 17, 2021.

v2021.3.3 Bug Fixes

  • LDAP Sync fixes to sync users more quickly.
  • Customers with thousands of users could have generated a delete group-user query that could have potentially crashed Postgres.
  • SaaS Configuration: Saving configuration on SaaS instances was unstable.

v2021.3.3 Known Bugs

  • SecurityManager error on AWS metadata service.

Immuta v2021.3.4

Immuta 2021.3.4 was released September 19, 2021.

v2021.3.4 Bug Fixes

  • Some LDAP instances didn't use memberOf to represent group membership.

v2021.3.4 Known Bugs

  • SecurityManager error on AWS metadata service.

Immuta v2021.3.5

Immuta 2021.3.5 was released October 22, 2021.

v2021.3.5 Fixes

  • Improved performance of LDAP Sync and automatic data source subscriptions.
  • Added support for Databricks Runtime 8.3.
  • Fixed issue that caused some GCP Delta commands to fail with Security Manager errors in DBR 7.3 and 8.3.
  • Added the option for LDAP Sync to bring in users as disabled.
  • When a Users with Specific Groups or Attributes Global Subscription policy is edited, users are no longer required to take action to subscribe or unsubscribe from the data source; updates happen automatically.

v2021.3.5 Known Bugs

  • SecurityManager error on AWS metadata service.