It is possible to deploy Immuta without the use of cloud provided managed services by using enterprise-ready tools for kubernetes.
This article describes deploying a PostgreSQL cluster in the Immuta installation namespace and pointing the Immuta application at this cluster for deployment
Crunchy PostgreSQL for Kubernetes
This section relies heavily on Crunchydata's installation documentation guide found at the link below and assumes you have forked and cloned their examples repository.
Install the Operator
Copy IMMUTA_NAMESPACE=immuta
PG_RELEASE_NAME=immuta-pg
helm install -n ${IMMUTA_NAMESPACE} postgres helm/install
Create a cluster
Copy helm install ${PG_RELEASE_NAME} -n ${IMMUTA_NAMESPACE} helm/postgres -f immuta-pg-values.yaml
immuta-pg-values.yaml
Copy postgresVersion: 16
instanceSize: 100Gi
metadata:
annotations:
eks.amazonaws.com/role-arn: "arn:aws:iam::231431240278:role/immuta-lts-crunchy-postgres-backup"
s3:
bucket: immuta-crunchy-backup
endpoint: s3.us-east-1.amazonaws.com
region: us-east-1
keyType: web-id
Connect to the cluster
Copy PG_CLUSTER_PRIMARY_POD=$(kubectl get pod -n ${IMMUTA_NAMESPACE} -o name -l postgres-operator.crunchydata.com/cluster=${PG_RELEASE_NAME},postgres-operator.crunchydata.com/role=master)
kubectl -n ${IMMUTA_NAMESPACE} port-forward "${PG_CLUSTER_PRIMARY_POD}" 5432:5432
Copy PG_CLUSTER_USER_SECRET_NAME=${PG_RELEASE_NAME}-pguser-${PG_RELEASE_NAME}
export PGHOSTNAME=$(kubectl get secrets -n ${IMMUTA_NAMESPACE} "${PG_CLUSTER_USER_SECRET_NAME}" -o go-template='{{.data.host | base64decode}}')
export PGPASSWORD=$(kubectl get secrets -n ${IMMUTA_NAMESPACE} "${PG_CLUSTER_USER_SECRET_NAME}" -o go-template='{{.data.password | base64decode}}')
export PGUSER=$(kubectl get secrets -n ${IMMUTA_NAMESPACE} "${PG_CLUSTER_USER_SECRET_NAME}" -o go-template='{{.data.user | base64decode}}')
export PGDATABASE=$(kubectl get secrets -n ${IMMUTA_NAMESPACE} "${PG_CLUSTER_USER_SECRET_NAME}" -o go-template='{{.data.dbname | base64decode}}')
psql -h localhost
Enable pgcrypto and set search path
Copy CREATE EXTENSION pgcrypto;
alter role "immuta-pg" set search_path to bometadata,public;
Immuta LTS Installation
Install Immuta with the db information in your environment.
Copy env |grep PG
43:PGDATABASE=immuta-pg
44:PGUSER=immuta-pg
45:PGPASSWORD=;V<GaAQ+cP(pUT6M<jR2zi^y
46:PGHOSTNAME=immuta-primary.immuta-lts.svc
immuta-crunchy-2024.2.yaml
Copy global:
imageRegistry: 231431240278.dkr.ecr.us-east-1.amazonaws.com
imageTag: 2024.2.0
audit:
enabled: false
secure:
extraEnvVars:
- name: FeatureFlag_AuditService
value: "false"
- name: FeatureFlag_detect
value: "false"
- name: FeatureFlag_auditLegacyViewHide
value: "false"
ingress:
hostname: crunchy.immuta.us
ingressClassName: alb
annotations:
alb.ingress.kubernetes.io/group.name: immuta-trino
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/target-type: ip
alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]'
alb.ingress.kubernetes.io/ssl-redirect: '443'
alb.ingress.kubernetes.io/backend-protocol: HTTP
tls: true
postgresql:
host: immuta-pg-primary.immuta-lts.svc
port: 5432
database: immuta-pg
username: immuta-pg
password: ;V<GaAQ+cP(pUT6M<jR2zi^y
ssl: true
Backups
In order to configure backups, edit the PostgresCluster
resource to look approximately like the following:
Copy backups:
pgbackrest:
configuration:
- secret:
name: immuta-pg-pgbackrest-secret
global:
repo1-path: /pgbackrest/immuta/immuta-pg/repo1
repo1-retention-full: "14"
repo1-retention-full-type: time
manual:
options:
- --type=full
repoName: repo1
repos:
- name: repo1
schedules:
full: "0 0 * * *"
s3:
bucket: immuta-crunchy-backup
endpoint: s3.us-east-1.aamazonaws.com
region: us-east-1
This config creates a cronjob for taking a nightly backup and allows an adhoc backup via annotation:
Copy kubectl annotate -n ${IMMUTA_NAMESPACE} postgrescluster ${PG_RELEASE_NAME} postgres-operator.crunchydata.com/pgbackrest-backup="$(date)" --overwrite