Skip to content

Discover Introduction

Discover is one of the Immuta flagship modules and is how you can automate discovering and tagging data across your data platform. Tagging is critical for two reasons:

  • It allows you define data sensitivity, which in turn allows you to monitor where you have potential data security issues and gaps in your security posture using Immuta Detect.

  • It allows you to abstract your physical structure from your access policy logic. In other words, using Immuta Secure, you can build access policies like mask all column tagged PII (where PII was auto-tagged by Discover) rather than much less scalable policies that must be knowledgeable of your physical layers like mask column x in database y in data platform z.

Challenge and goals

Today’s sensitive data discovery tools give you a shallow overview of your data corpus across a long list of platforms. They give you pointers on where you have sensitive data without the granularity to drive your column- or row-level access controls. They help you understand what data you possess according to a regulatory framework, like HIPAA or PCI, but without the details needed to automate your audits or compliance reporting. Knowing that you need to drive east to west on a road map from New York to California is helpful, but ultimately insufficient to get you from a specific location to another.

Existing tools promise a high degree of automation, yet their many false-positives result in painful manual work that never stops. Although data gets scanned automatically, performance breaks down at scale, or you manually need to fine-tune the computing resources of the scanners. Last but not least, your security team objects to the agent-based processing that requires taking data out of your data platform, and the associated data residency concerns may give you pause.

At Immuta, we believe that data security should not be painful. We believe that you can innovate and move quickly, while at the same time protecting your data and adhering to your internal policies and external regulations. Technology and automation allow you to make the right trade-off decisions quickly. It all starts with highly accurate and actionable metadata. If you trust your metadata and if it’s actionable, you can leverage it to automatically grant access to data, mask sensitive information, and automate your audit reporting.

Immuta Discover was built to tackle those challenges and address them through a unique architecture that was designed in collaboration with the largest financial institutions, healthcare companies, and government agencies in the world. The cloud and AI paradigm requires a fundamentally different approach. You must assume that your data is dynamic, unique, and collected in a multitude of different geographies and legal jurisdictions. Immuta Discover is built for this new world and its specific demands.

How does it work?

Scalability through in-platform processing

Identifying and classifying data requires analyzing and looking at the data - there’s no way around it. Immuta Discover does all the analysis and processing inside any data platform, including Snowflake and Databricks. It takes advantage of those platforms’ inherent scalability to enable you to analyze large amounts of data quickly, efficiently, and without the need for separate resource optimization for containers or virtual machines.

Data residency compliance by design

By processing data directly inside the data platform, Immuta Discover automatically adheres to data residency and locality requirements. If you run your data warehouse or lake globally - across North America, the European Union, and Asia - Immuta processes the data in the region where your data is stored. No data ever leaves the data platform, and it will never move around across different cloud regions.

Improved security and simplicity due to agentless scanning

In-platform processing greatly reduces risk and improves your data security posture. Provisioning agents, whether they’re in a container, virtual machine, or AMI, create complexity and an unnecessary security risk. Not only can those agents become compromised, but their misconfiguration might lead to data leaks to other parts of your cloud infrastructure. An agentless approach can better leverage data platform optimizations to process data instead of transferring it out to re-optimize and analyze. This simplifies operations and increases efficiency for your infrastructure teams.

Cross-platform consistency

The advantages of in-platform processing are obvious, but implementing it across a multitude of platforms is challenging. Immuta helps bypass the obstacles by doing all the heavy lifting for you and building in specific implementations for each technology. Although all those implementations are ultimately different, Immuta abstracts the results to one standardized taxonomy, so you can have consistently accurate and granular metadata across all your data stores.

Granular query-level classification

Immuta Discover classifies on a column level and instantaneously identifies schema changes. Only with that level of granularity and automation can you adhere to your audit requirements and understand what actions have been taken on your data. For example, if non-sensitive data is joined with sensitive data at query time, Immuta Discover will monitor and record that for your review. Continuous schema monitoring ensures schema changes never result in holes in your access controls and data security posture.

Highly accurate and actionable metadata

Trust in your metadata is critical for data security. To unblock your data consumers, you need to automate your data access controls. This requires first knowing that your classification and metadata are accurate and actionable. Immuta Discover provides you with highly accurate metadata and tags out-of-the-box and assists you in fine-tuning the classification mechanism to deal with false-positives quickly. That enables you to build policies that dynamically grant or restrict access to PII or PHI, depending on who is accessing it and what protections (like masking policies) you want to apply.

Components of Discover

Immuta Discover works in three phases:

  1. Identification: In the first phase, data is identified by its kind – for example, a name or an age. This identification can be manually performed, externally provided, or automatically determined by Immuta Discover through column-level analysis.

  2. Categorization: In the second phase, data is categorized in the context where it appears, subject to any active data compliance or security frameworks. For example, a record occurring in a clinical context containing both a name and individual health data is protected health information under HIPAA.

    Though entirely customizable, for this purpose, Immuta provides a default framework known as the Immuta Data Security Framework (or Immuta DSF). Immuta DSF gives a fined-grained categorization into a consistent set of security and compliance concepts, including things like whether or not a record pertains to an individual, the composition and kinds of any identifiers that present, the subject matter of the data, or whether it belongs to any commonly controlled data categories.

    The categorization provided by the Immuta DSF may be used directly. Still, it is best leveraged as a starting point for purpose-built compliance frameworks implementing organization-specific compliance categories or other relevant high-level regulatory or compliance frameworks, such as those for categorizing data into categories defined under CCPA, GDPR, GLBA, or HIPAA.

  3. Classification: In the third and final phase, data is classified according to its sensitivity level (e.g., Customer financial data is highly sensitive). Again, Immuta supplies sensitivity classification defaults based on standard industry practice. However, customers are free to customize this classification under their respective views.