In-cluster PostgreSQL using Crunchydata
It is possible to deploy Immuta without the use of cloud provided managed services by using enterprise-ready tools for kubernetes.
This article describes deploying a PostgreSQL cluster in the Immuta installation namespace and pointing the Immuta application at this cluster for deployment
Crunchy PostgreSQL for Kubernetes
This section relies heavily on Crunchydata's installation documentation guide found at the link below and assumes you have forked and cloned their examples repository.
Install the Operator
IMMUTA_NAMESPACE=immuta
PG_RELEASE_NAME=immuta-pg
helm install -n ${IMMUTA_NAMESPACE} postgres helm/install
Create a cluster
helm install ${PG_RELEASE_NAME} -n ${IMMUTA_NAMESPACE} helm/postgres -f immuta-pg-values.yaml
Connect to the cluster
PG_CLUSTER_PRIMARY_POD=$(kubectl get pod -n ${IMMUTA_NAMESPACE} -o name -l postgres-operator.crunchydata.com/cluster=${PG_RELEASE_NAME},postgres-operator.crunchydata.com/role=master)
kubectl -n ${IMMUTA_NAMESPACE} port-forward "${PG_CLUSTER_PRIMARY_POD}" 5432:5432
PG_CLUSTER_USER_SECRET_NAME=${PG_RELEASE_NAME}-pguser-${PG_RELEASE_NAME}
export PGHOSTNAME=$(kubectl get secrets -n ${IMMUTA_NAMESPACE} "${PG_CLUSTER_USER_SECRET_NAME}" -o go-template='{{.data.host | base64decode}}')
export PGPASSWORD=$(kubectl get secrets -n ${IMMUTA_NAMESPACE} "${PG_CLUSTER_USER_SECRET_NAME}" -o go-template='{{.data.password | base64decode}}')
export PGUSER=$(kubectl get secrets -n ${IMMUTA_NAMESPACE} "${PG_CLUSTER_USER_SECRET_NAME}" -o go-template='{{.data.user | base64decode}}')
export PGDATABASE=$(kubectl get secrets -n ${IMMUTA_NAMESPACE} "${PG_CLUSTER_USER_SECRET_NAME}" -o go-template='{{.data.dbname | base64decode}}')
psql -h localhost
Enable pgcrypto and set search path
CREATE EXTENSION pgcrypto;
alter role "immuta-pg" set search_path to bometadata,public;
Immuta LTS Installation
Install Immuta with the db information in your environment.
env |grep PG
43:PGDATABASE=immuta-pg
44:PGUSER=immuta-pg
45:PGPASSWORD=;V<GaAQ+cP(pUT6M<jR2zi^y
46:PGHOSTNAME=immuta-primary.immuta-lts.svc
Backups
In order to configure backups, edit the PostgresCluster
resource to look approximately like the following:
backups:
pgbackrest:
configuration:
- secret:
name: immuta-pg-pgbackrest-secret
global:
repo1-path: /pgbackrest/immuta/immuta-pg/repo1
repo1-retention-full: "14"
repo1-retention-full-type: time
manual:
options:
- --type=full
repoName: repo1
repos:
- name: repo1
schedules:
full: "0 0 * * *"
s3:
bucket: immuta-crunchy-backup
endpoint: s3.us-east-1.aamazonaws.com
region: us-east-1
This config creates a cronjob for taking a nightly backup and allows an adhoc backup via annotation:
kubectl annotate -n ${IMMUTA_NAMESPACE} postgrescluster ${PG_RELEASE_NAME} postgres-operator.crunchydata.com/pgbackrest-backup="$(date)" --overwrite