Enabling Legacy Query Engine and Fingerprint

The query engine and fingerprint services are no longer installed by default. This guide demonstrates how to enable the query engine and fingerprint services using the Immuta Enterprise Helm chart (IEHC).

If you are using any of the data platforms below, you must enable the query engine:

• Amazon Redshift

• Azure Synapse Analytics

• Google BigQuery

• Any legacy database

If you are using the legacy sensitive data discovery (SDD) feature, you must enable the query engine and fingerprint services.

Kubernetes namespace

The following section(s) presume the IEHC was deployed into namespace immuta, and that the current namespace is immuta.

Prerequisites

When migrating from the IHC to IEHC, query engine state is not retained. You must enable query engine rehydration to restore existing data source tables. If SQL credentials are used, they must be recreated by using LDAP sync or manually with the following command executed in the bometadata database:

TRUNCATE bometadata."profile-sql";

• The Immuta in production guide must be completed before proceeding.

• Validate that secret immuta-secret exists in the current namespace.

  kubectl get secret/immuta-secret

Create Kubernetes secret

1. Create a file named secret-data.env with the following content.

   # query-engine
   IMMUTA_FEATURE_PASSWORD=<immuta-feature-password>
   PATRONI_SUPERUSER_PASSWORD=<patroni-superuser-password>
   PATRONI_REPLICATION_PASSWORD=<patroni-replication-password>
   PATRONI_RESTAPI_PASSWORD=<patroni-api-password>

2. Create secret named immuta-legacy-secret from file secret-data.env

   kubectl create secret generic immuta-legacy-secret --from-env-file=secret-data.env

3. Delete file secret-data.env, as it's no longer needed.

   rm -i secret-data.env

Edit Helm values

1. Edit the immuta-values.yaml file to include the following Helm values.

   legacy:
     enabled: true
   
     queryEngine:
       statefulset:
         extraEnvVars:
         - name: IMMUTA_FEATURE_PASSWORD
           valueFrom:
             secretKeyRef:
               name: immuta-legacy-secret
               key: IMMUTA_FEATURE_PASSWORD
         - name: PATRONI_SUPERUSER_PASSWORD
           valueFrom:
             secretKeyRef:
               name: immuta-legacy-secret
               key: PATRONI_SUPERUSER_PASSWORD
         - name: PATRONI_REPLICATION_PASSWORD
           valueFrom:
             secretKeyRef:
               name: immuta-legacy-secret
               key: PATRONI_REPLICATION_PASSWORD
         - name: PATRONI_RESTAPI_PASSWORD
           valueFrom:
             secretKeyRef:
               name: immuta-legacy-secret
               key: PATRONI_RESTAPI_PASSWORD
   
       postgres:
         # Query Engine feature user
         # Instead use queryEngine.statefulset.extraEnvVars[].name[IMMUTA_FEATURE_PASSWORD]
         # password: <immuta-feature-password>
   
         # Query Engine superuser user
         # Instead use queryEngine.statefulset.extraEnvVars[].name[PATRONI_SUPERUSER_PASSWORD]
         # superuserPassword: <patroni-superuser-password>
   
         # Query Engine replication user
         # Instead use queryEngine.statefulset.extraEnvVars[].name[PATRONI_REPLICATION_PASSWORD]
         # replicationPassword: <patroni-replication-password>
   
         # Query Engine patroni api user
         # Instead use queryEngine.statefulset.extraEnvVars[].name[PATRONI_RESTAPI_PASSWORD]
         # patroniApiPassword: <patroni-api-password>
       immutaSecurity:
         # Each Kubernetes Service has a DNS record associated with it. See: https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/
         # The anatomy of a domain name is as followed:
         #   <service>.<namespace>.svc.<cluster-domain>
         #
         # Where the default cluster domain is: cluster.local
         authEndpoint: "http://immuta-secure.immuta.svc.cluster.local:8823"
   
   secure:
     extraEnvVars:
     - name: IMMUTA_DATABASES_IMMUTA_CONNECTIONS_FEATURESTOREDB_PASSWORD
       valueFrom:
         secretKeyRef:
           name: immuta-legacy-secret
           key: IMMUTA_FEATURE_PASSWORD
   
     extraConfig:
       :
         enabled: true
       disableFeatureStore: false
       databases:
         immuta:
           connections:
             featureStoreDb:
               # Each Kubernetes Service has a DNS record associated with it. See: https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/
               # The anatomy of a domain name is as followed:
               #   <service>.<namespace>.svc.<cluster-domain>
               #
               # Where the default cluster domain is: cluster.local
               host: "immuta-legacy-query-engine-service.immuta.svc.cluster.local"
               port: 5432
               ssl: false
               # Query Engine feature user
               # Instead use secure.extraEnvVars[].name[IMMUTA_DATABASES_IMMUTA_CONNECTIONS_FEATURESTOREDB_PASSWORD]
               # password: <immuta-feature-password>
       fingerprints:
         # Each Kubernetes Service has a DNS record associated with it. See: https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/
         # The anatomy of a domain name is as follows:
         #   <service>.<namespace>.svc.<cluster-domain>
         #
         # Where the default cluster domain is: cluster.local
         uri: "http://immuta-legacy-fingerprint-service.immuta.svc.cluster.local:5001/"
         queryEngineHost: "immuta-legacy-query-engine-service.immuta.svc.cluster.local"
         queryEnginePort: 5432

2. Update all placeholder values in the immuta-values.yaml file.

Avoid these special characters in generated passwords

whitespace, $, &, :, \, /, '

Apply Helm values

Perform a Helm upgrade to apply the changes made to immuta-values.yaml.

helm upgrade <release-name> oci://ocir.immuta.com/stable/immuta-enterprise --values immuta-values.yaml --version 2024.2.20