Skip to content

Kubernetes Helm Installation

Audience: System Administrators

Content Summary: This page outlines how to install Immuta on Kubernetes using Helm.

Prerequisites:

For more information, see Helm Installation Prerequisites

Backup and Restore Upgrade Required for 2021.2

Postgres has been upgraded to Postgres 12, so you must do a complete back-up and restore upgrade for this release.

If using a Kubernetes namespace...

If deploying Immuta into a Kubernetes namespace other than the default, you must include the --namespace option into all helm and kubectl commands provided throughout this section.

Installation

In order to conduct an installation of Immuta in a Kubernetes environment, you will need Helm access to the Kubernetes cluster. To do this you must,

  1. Configure the Environment
  2. Configure Immuta Helm Values
  3. Configure Persistence
  4. Configure Backup and Restoration Values
  5. Deploy Immuta

1 - Configure the Environment

1.1 - Check Helm Version

Immuta's Helm Chart requires Helm version 3+.

Run helm version to verify the version of Helm you are using:

helm version
Example Output
version.BuildInfo{Version:"v3.2.3", GitCommit:"8f832046e258e2cb800894579b1b3b50c2d83492", GitTreeState:"clean", GoVersion:"go1.13.12"}

1.2 - Configure Immuta's Helm Chart Repo

In order to deploy Immuta to your Kubernetes cluster, you must be able to access the Immuta Helm Chart Repository and the Immuta Docker Registry. You can obtain credentials and instructions to set up both of these by accessing the Immuta Release Portal.

Run helm repo list to ensure Immuta's Helm Chart repository has been successfully added:

helm repo list
Example Output
NAME            URL
stable          https://kubernetes-charts.storage.googleapis.com
local           http://127.0.0.1:8879/charts
immuta          https://archives.immuta.com/charts
Don't forget the image pull secret!

As detailed in Immuta Release Portal, you must create a Kubernetes Image Pull Secret in the namespace that you are deploying Immuta in, or the Pods will fail to start due to ErrImagePull.

Run kubectl get secrets to confirm your Kubernetes image pull secret is in place:

kubectl get secrets
Example Output
NAME                  TYPE                                  DATA   AGE
immuta-registry       kubernetes.io/dockerconfigjson        1      5s

1.3 - Check/Update Your Local Immuta Helm Chart Version

Run helm search repo immuta to check the version of your local copy of Immuta's Helm Chart:

helm search repo immuta
Example Output
NAME            CHART VERSION   APP VERSION   DESCRIPTION
immuta/immuta   4.5.0           2020.2.4      Immuta: Automated Data Governance

Update your local Chart by running helm repo update.

To perform an upgrade without upgrading to the latest version of the Chart, run helm list to determine the Chart version of the installed release, and then specify that version using the --version argument of helm repo update.

2 - Configure Immuta Helm Values

Once you have the Immuta Docker Registry and Helm Chart Repository configured, download the immuta-values.yaml file below. This values file is a recommended starting point for your installation.

Download immuta-values.yaml

Fill in and modify immuta-values.yaml based on the determined configuration for your Kubernetes cluster and the desired Immuta installation. Replace the placeholder password value "<SPECIFY_PASSWORD_THAT_MEETS_YOUR_ORGANIZATIONS_POLICIES>" with a secure password that meets your organization's password policies.

Avoid these special characters in generated passwords

whitespace, $, &, : , \, /, '

3 - Configure Persistence

If you would like to disable persistence to disk for the database and query-engine components, you can do so by configuring database.persistence.enabled=false and/or queryEngine.persistence.enabled=false in immuta-values.yaml. Disabling persistence can be done for test environments. However, we strongly discourage disabling persistence in production environments as this leaves your database in ephemeral storage.

By default, database.persistence.enabled and queryEngine.persistence.enabled are set to true and request 120Gi of storage for each component. Recommendations for the Immuta Metadata Database storage size for POV, Staging, and Production deployments are provided in the immuta-values.yaml as shown below. However, the actual size needed is a function of the number of data sources you intend to create and the amount of logging/auditing (and its retention) that will be used in your system.

Provide Room for Growth

Provide plenty of room for growth here, as Immuta's operation will be severely impacted should database storage reach capacity.

database:
  persistence:
    enabled: true
    volumeClaimSpec:
      resources:
        requests:
          # Set the requested storage size for database persistence.
          # Recommended sizing:
          # - 20Gi: POV or other small sized deployments, such as development or test environments.
          # - 60Gi: Staging environments that mimic production, but see less use.
          # - 120Gi: Production deployments.
          storage: 120Gi

While the Immuta Query Engine persistent storage size is configurable as well, the default size of 20Gi should be sufficient for operations in nearly all environments.

queryEngine:
  persistence:
    enabled: true
    volumeClaimSpec:
      resources:
        requests:
          # Set the requested storage size for database persistence.
          # Recommended sizing:
          # - 20Gi: Usually sufficient for most deployments
          storage: 20Gi
Limitations on modifying database and query-engine persistence

Once persistence is set to either true or false for database or query-engine, it cannot be changed for the deployment. Modifying persistence will require a fresh installation or a full backup and restore procedure as per 4.2 - Restore From Backup - Immuta Kubernetes Re-installation.

4 - Configure Backup and Restoration Values

At this point this procedure forks depending on whether you are installing with the intent of restoring from a backup or not. Use the bullets below to determine which step to follow.

  • If this is a new installation with no restoration needed, follow Step 4.1.
  • If you are upgrading a previous installation using the full backup and restore (Method B), follow Step 4.2.

4.1 - Initial Immuta Kubernetes Installation -- No Backup Restoration

Immuta's Helm Chart has support for taking backups and storing them in a PersistentVolume or copying them directly to cloud provider blob storage including AWS S3, Azure Blob Storage, and Google Storage.

To configure backups with blob storage, reference the backup section in immuta-values.yaml and consult the subsections of this section of the documentation that are specific to your cloud provider for assistance in configuring a compatible resource. If your Kubernetes environment is not represented there, or a workable solution does not appear available, please contact your Immuta representative to discuss options.

If using volumes, the Kubernetes cluster Immuta is being installed into must support PersistentVolumes with an access mode of ReadWriteMany. If such a resource is available, Immuta's Helm Chart will set everything up for you if you enable backups and comment out the volume and claimName.

backup:
  # set to true to enable backups
  enabled: true
  #volume:
    # if claimName is set to an existing PVC no PV/PVC will be created
    #claimName: <YOUR ReadWriteMany PersistentVolumeClaim NAME>

4.2 - Restore From Backup - Immuta Kubernetes Re-Installation

If you are upgrading a previous installation using the full backup and restore procedure (Method B), a valid backup configuration must be available in the Helm values. Enable the functionality to restore from backups by setting the restore.enabled option to true in immuta-values.yaml.

backup:
  # set to true to enable backups
  enabled: true
  restore:
    # set to true to enable restoring from backups
    enabled: true

If using the volume backup type, an existing PersistentVolumeClaim name needs to be configured in your immuta-values.yaml because the persistentVolumeClaimSpec is only used to create a new, empty volume.

backup:
  # set to true to enable backups. requires RWX persistent volume support
  enabled: true
  restore:
    # set to true to enable restoring from backups
    enabled: true
  volume:
    # if claimName is set to an existing PVC no PV/PVC will be created
    claimName: <YOUR ReadWriteMany PersistentVolumeClaim NAME>

If you are unsure of the value for <YOUR ReadWriteMany PersistentVolumeClaim NAME>, the command kubectl get pvc will list it for you

kubectl get pvc
Example Output
NAME                 STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
test-immuta-backup   Bound    pvc-2ca50bff-8c2c-487c-b7bf-75c3764c470a   5Gi        RWX            standard       39h

5 - Deploy Immuta

helm install <YOUR RELEASE NAME> immuta/immuta \
  --values immuta-values.yaml
Troubleshooting

If you encounter errors while deploying the Immuta Helm Chart, see Troubleshooting.

Advanced Installations

Manage TLS

HTTP communication using TLS certificates is enabled by default in Immuta's Helm Chart for both internal (inside the Kubernetes cluster) and external (between the Kubernetes ingress and the outside world) communications. This is accomplished through the generation of a local certificate authority (CA) which signs certificates for each service - all handled automatically by the Immuta installation. While not recommended, if TLS must be disabled for some reason, this can be done by setting tls.enabled to false in the values file.

Use Your Own TLS Certificate(s)

Best Practice: TLS Certification

Immuta recommends to use your own TLS certificate for external (outside the Kubernetes cluster) communications for Immuta production deployments.

Using your own certificates requires you to create a Kubernetes Secret containing the private key, certificate, and certificate authority certificate. This can be easily done using kubectl:

kubectl create secret generic immuta-external-tls \
    --from-file=tls.crt=<path to your.pem cert> \
    --from-file=tls.key=<path to your-key.pem key> \
    --from-file=ca.crt=<path to your-ca.pem cert>
Make sure your certificates are correct

Make sure the certificate's Common Name (CN) and/or Subject Alternative Name (SAN) matches the specified externalHostname or contains an appropriate wildcard.

After creating the Kubernetes Secret, specify its use in the external ingress by setting tls.externalSecretName = immuta-external-tls in your immuta-values.yaml file:

tls:
  externalSecretName: immuta-external-tls

Resource Limits

The Immuta helm chart supports Resource Limits for all components. The resource recommendations below are for a Kubernetes cluster running 3 worker nodes and 1 master, where the worker nodes are a m4.xlarge (4 vCPU, 16GB RAM) or equivalent:

web:
  resources:
    requests:
      memory: "512Mi"
    limits:
      memory: "2Gi"

database:
  resource:
    requests:
      memory: "512Mi"
    limits:
      memory: "2Gi"

queryEngine:
  resource:
    requests:
      memory: "512Mi"
    limits:
      memory: "4Gi"

fingerprint:
  resources:
    requests:
      memory: "512Mi"
    limits:
      memory: "2Gi"

cache:
  memcached:
    maxItemMemory: 256
  resources:
    requests:
      memory: "256Mi"
    limits:
      memory: "256Mi"

Similarly, for an m4.large (2 vCPU, 8GB RAM) or equivalent:

web:
  resources:
    requests:
      memory: "256Mi"
    limits:
      memory: "1Gi"

database:
  resources:
    requests:
      memory: "512Mi"
    limits:
      memory: "2Gi"

queryEngine:
  resources:
    requests:
      memory: "512Mi"
    limits:
      memory: "2Gi"

fingerprint:
  resources:
    requests:
      memory: "256Mi"
    limits:
      memory: "1Gi"

cache:
  memcached:
    maxItemMemory: 256
  resources:
    requests:
      memory: "256Mi"
    limits:
      memory: "256Mi"

Using Argo CD

The Immuta Helm Chart (version 4.5.0+) can be deployed using Argo CD.

For Argo CD versions older than 1.7.0 you must use the following Helm values in order for the TLS generation hook to run successfully.

hooks:
  tlsGeneration:
    hookAnnotations:
      helm.sh/hook-delete-policy: "before-hook-creation"

Starting with Argo CD version 1.7.0 the default TLS generation hook values can be used.

tls.manageGeneratedSecret must be set to true when using Argo CD to deploy Immuta; otherwise, the generated TLS secret will be shown as OutOfSync (requires pruning) in Argo CD. Pruning the Secret would break TLS for the deployment, so it is important to set this value to prevent that from happening.

tls:
  manageGeneratedSecret: true

Troubleshoot

For detailed assistance in troubleshooting your installation, please contact your Immuta Support Professional or see Helm Troubleshooting.

What's Next

Now that you've installed Immuta, continue to the next page or to one of these tutorials:

Helm Troubleshooting Add a License Key