AWS EKS Installation

This guide covers installing one Institutional Vault instance on Amazon Elastic Kubernetes Service (EKS). Complete Cluster Prerequisites first.

Prerequisites

Before starting, confirm you have:

An EKS cluster with the OIDC provider enabled (required for IRSA)
An Amazon ECR repository with the Blockdaemon images mirrored (see Image setup below)
AWS Secrets Manager entries provisioned for this instance
A Route 53 hosted zone (or another DNS provider) for the instance domain
Helm 3.12+, kubectl, and aws CLI configured
The mpc chart version and image versions provided by your Blockdaemon account team

Step 1: Mirror images to ECR

AWS_ACCOUNT=<your-aws-account-id>
AWS_REGION=us-east-1
DST=${AWS_ACCOUNT}.dkr.ecr.${AWS_REGION}.amazonaws.com/mpa-wallet

# Image tags from your Blockdaemon delivery (examples).
# Confirm current tags in the Institutional Vault changelog:
# https://vault.docs.blockdaemon.com/changelog
IV_VERSION=v3.6.0              # mothership, wallet-frontend, evm-tracker, nats
POLICY_NODE_VERSION=v9.10.0
CONFIGMAP_INIT_VERSION=v0.2.0

# Authenticate to the Blockdaemon source registry and to ECR
docker login iv.sepior.net
aws ecr get-login-password --region $AWS_REGION | \
  docker login --username AWS --password-stdin \
  ${AWS_ACCOUNT}.dkr.ecr.${AWS_REGION}.amazonaws.com

# Create repositories (once per image)
for image in mothership policy-node wallet-frontend evm-tracker configmap-init nats; do
  aws ecr create-repository --repository-name mpa-wallet/${image} --region $AWS_REGION
done

mirror() {
  local image=$1 version=$2
  docker pull iv.sepior.net/${image}:${version}
  docker tag  iv.sepior.net/${image}:${version} ${DST}/${image}:${version}
  docker push ${DST}/${image}:${version}
}

# Institutional Vault release tag
for image in mothership wallet-frontend evm-tracker nats; do
  mirror "$image" "$IV_VERSION"
done

# Independently versioned images
mirror policy-node "$POLICY_NODE_VERSION"
mirror configmap-init "$CONFIGMAP_INIT_VERSION"

Set global.imageRegistry: <account>.dkr.ecr.<region>.amazonaws.com/mpa-wallet in your values file. Use the same tags in the versions block (see Step 4).

Step 2: Create IAM roles with IRSA

Each Kubernetes ServiceAccount that needs to read from Secrets Manager is annotated with an IAM role ARN. EKS IRSA issues short-lived AWS credentials to the pod without requiring static access keys.

Create one IAM role per component group, or a single shared role for all components (simpler, but less fine-grained):

CLUSTER_NAME=<your-eks-cluster>
AWS_ACCOUNT=<your-aws-account-id>
AWS_REGION=us-east-1
NAMESPACE=<tenant-namespace>
ENV=<env>   # e.g. testnet

# Get the OIDC issuer URL for this cluster
OIDC_URL=$(aws eks describe-cluster --name $CLUSTER_NAME \
  --region $AWS_REGION --query "cluster.identity.oidc.issuer" --output text)
OIDC_PROVIDER=${OIDC_URL#https://}

# Create the trust policy document
cat > trust-policy.json <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {"Federated": "arn:aws:iam::${AWS_ACCOUNT}:oidc-provider/${OIDC_PROVIDER}"},
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "${OIDC_PROVIDER}:aud": "sts.amazonaws.com",
          "${OIDC_PROVIDER}:sub": [
            "system:serviceaccount:${NAMESPACE}:wallet",
            "system:serviceaccount:${NAMESPACE}:policy-node",
            "system:serviceaccount:${NAMESPACE}:nats"
          ]
        }
      }
    }
  ]
}
EOF

# Create the role
ROLE_NAME=mpa-${ENV}-${NAMESPACE}
aws iam create-role \
  --role-name $ROLE_NAME \
  --assume-role-policy-document file://trust-policy.json

# Attach the Secrets Manager read policy
aws iam put-role-policy \
  --role-name $ROLE_NAME \
  --policy-name SecretsManagerRead \
  --policy-document '{
    "Version": "2012-10-17",
    "Statement": [{
      "Effect": "Allow",
      "Action": ["secretsmanager:GetSecretValue", "secretsmanager:DescribeSecret"],
      "Resource": "arn:aws:secretsmanager:'"${AWS_REGION}"':'"${AWS_ACCOUNT}"':secret:'"${NAMESPACE}"'-*"
    }]
  }'

ROLE_ARN=$(aws iam get-role --role-name $ROLE_NAME --query "Role.Arn" --output text)
echo "Role ARN: $ROLE_ARN"

Step 3: Populate AWS Secrets Manager

The wallet reads its runtime configuration from Secrets Manager at pod startup. The secret names must match the {{ awsSecret "name#key" }} references in your config templates.

The secret names must match the {{ awsSecret "name#key" }} references in your config templates. Typical secrets for one instance include:

Secret name pattern	Contents
`<ns>.wallet-db-info`	Database connection string
`<ns>.policy-node{0,1,2}.encryptor-master-password`	Encryptor master password per node
`<ns>.policy-node{0,1,2}.broker-info`	NATS broker credentials (JSON: `{"PASSWORD":"..."}`)
`<ns>.nats-info`	NATS operator/system account credentials
`<ns>.facade-oidc-secret`	OIDC client secret for IdP integration

🚧
Caution:
EncryptorMasterPassword must never change after the initial deployment. Back it up to a separate, offline-accessible location immediately after generation.

Create secrets with the AWS CLI:

aws secretsmanager create-secret \
  --name "${NAMESPACE}.policy-node0.encryptor-master-password" \
  --region $AWS_REGION \
  --secret-string "<generated-password>"

Step 4: Create your values file

Start from the aws.yaml example in the chart repository and fill in all REPLACE placeholders:

# my-instance-values.yaml

global:
  imageRegistry: <account>.dkr.ecr.<region>.amazonaws.com/mpa-wallet
  environment: <env>
  domain: <base-domain>     # e.g. wallet.example.com
  awsRegion: us-east-1
  env:
    - name: AWS_REGION
      value: us-east-1
    - name: SECRET_STORE
      value: aws

secrets:
  provider: aws

versions:
  wallet: v3.6.0              # mothership / wallet-frontend / nats / evm-tracker
  mpa: v9.10.0                # policy-node
  # configmap-init uses CONFIGMAP_INIT_VERSION (e.g. v0.2.0) from the mirror step

wallet:
  serviceAccount:
    create: true
    annotations:
      eks.amazonaws.com/role-arn: <ROLE_ARN>
  config:
    approvalHostUrl: ""
  ingress:
    enabled: true
    className: nginx   # or alb if using AWS Load Balancer Controller
    annotations:
      cert-manager.io/cluster-issuer: letsencrypt-prod
      nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
      nginx.ingress.kubernetes.io/ssl-redirect: "true"
    hosts:
      - host: api.<env>.<base-domain>
        paths:
          - path: /approval
            pathType: Prefix
            portName: approval
          - path: /event-streaming
            pathType: Prefix
            portName: webhook
          - path: /
            pathType: Prefix
            portName: http
    tls:
      - hosts: [api.<env>.<base-domain>]
        secretName: api.<env>.<base-domain>-tls

frontend:
  enabled: true
  apiPath: https://api.<env>.<base-domain>/
  serviceAccount:
    create: true
    annotations:
      eks.amazonaws.com/role-arn: <ROLE_ARN>
  ingress:
    enabled: true
    className: nginx
    host: <env>.<base-domain>
    tls:
      enabled: true

policyNodes:
  enabled: true
  nodeCount: 3
  serviceAccount:
    create: true
    annotations:
      eks.amazonaws.com/role-arn: <ROLE_ARN>
  config:
    configFile: "/config/policy-node.conf"
    nodes:
      - templateData:
          policy-node.conf: |
            [Identity]
            NodeId = "route0"
            PlayerIndex = 0
            [Secrets]
            EncryptorMasterPassword = "{{ awsSecret `<ns>.policy-node0.encryptor-master-password` }}"
            [Broker]
            Username = "node0"
            Password = "{{ awsSecret `<ns>.policy-node0.broker-info#PASSWORD` }}"
            URL = "nats://mpc-nats.<namespace>.svc.cluster.local:4222"
            [Service]
            HealthCheckPort = 8080
            CloudWatchLogging = false
      # Repeat for node1 (PlayerIndex 1, NodeId route1, Username node1) and
      # node2 (PlayerIndex 2, NodeId route2, Username node2), referencing
      # policy-node1 / policy-node2 secret names.

nats:
  replicaCount: 3
  persistence:
    storageClass: gp3
    size: 5Gi
  serviceAccount:
    create: true
    annotations:
      eks.amazonaws.com/role-arn: <ROLE_ARN>

configmapInit:
  enabled: false   # AWS Policy Nodes resolve secrets natively; configmap-init is not needed

dbSetup:
  enabled: true
  network: <rds-endpoint>

natsSetup:
  certProvider: none

Step 5: Create the namespace

kubectl create namespace <tenant-namespace>

Step 6: Install the chart

helm install mpc oci://iv.sepior.net/charts/mpc \
  --version <chart-version> \
  --namespace <tenant-namespace> \
  -f my-instance-values.yaml

Passing policy node configs as external files

The policy node config templates shown in Step 4 are embedded directly in the values file under templateData. If you prefer to keep per-node config files separate (for example, to manage them outside version control), use --set-file instead:

helm install mpc oci://iv.sepior.net/charts/mpc \
  --version <chart-version> \
  --namespace <tenant-namespace> \
  -f my-instance-values.yaml \
  --set-file "policyNodes.config.nodes[0].templateData.policy-node\.conf=./node0.conf" \
  --set-file "policyNodes.config.nodes[1].templateData.policy-node\.conf=./node1.conf" \
  --set-file "policyNodes.config.nodes[2].templateData.policy-node\.conf=./node2.conf"

Each file (node0.conf, node1.conf, node2.conf) contains the rendered TOML template for that node, with {{ awsSecret "..." }} placeholders. Remove the templateData block from the values file when using this approach.

Step 7: Verify the deployment

kubectl get pods -n <tenant-namespace>

Check the wallet API health endpoint:

kubectl port-forward -n <tenant-namespace> svc/mpc-wallet 8080:80 &
curl http://localhost:8080/ready

Check that the certificate issued:

kubectl get certificate -n <tenant-namespace>

Troubleshooting

Pod fails with `NoCredentialProviders`

The pod's ServiceAccount annotation is missing or the OIDC trust policy condition doesn't match the namespace:serviceaccount pair. Verify:

kubectl describe serviceaccount wallet -n <tenant-namespace>
# annotation should show: eks.amazonaws.com/role-arn: arn:aws:iam::...

Secrets Manager `AccessDeniedException`

The IAM role policy does not cover the secret's ARN. Check that the Resource in the policy matches the actual secret name (including any suffix AWS appends, e.g. -AbCdEf). Use a wildcard suffix: arn:aws:secretsmanager:...:secret:<namespace>-*.

Database setup job failing

Check that the RDS instance is reachable from the cluster's VPC, the security group allows ingress on port 5432 from the EKS node security group, and the dbSetup.network value matches the RDS endpoint.

Certificate not issuing

Ensure external-dns (or manual Route53 records) has created the A record for the ingress host. Use dig api.<env>.<domain> to confirm DNS resolution before the ACME HTTP-01 challenge fires.

Network requirements

Internal cluster communication

Source	Destination	Port	Protocol	Purpose
`wallet`	`nats`	4222	TCP	NATS client — MPC coordination
`wallet`	PostgreSQL	5432	TCP	Application database
`frontend`	`wallet`	80	TCP	Internal API proxy
`nats` pod	`nats` pod	6222	TCP	NATS inter-pod cluster routing
`db-setup` job	PostgreSQL	5432	TCP	Schema migrations
All pods	CoreDNS	53	UDP/TCP	DNS resolution

Policy nodes expose port 8080 for health checks.

Outbound to AWS services

AWS VPC Interface Endpoints are strongly recommended for Secrets Manager, STS, and ECR to avoid public internet egress.

Destination	Port	Purpose	Recommended endpoint
AWS Secrets Manager	443	Secret resolution at pod startup	VPC Interface Endpoint (`com.amazonaws.<region>.secretsmanager`)
AWS STS	443	IRSA token exchange	VPC Interface Endpoint (`com.amazonaws.<region>.sts`)
Amazon ECR	443	Image pull	VPC Interface Endpoints (`ecr.api`, `ecr.dkr`) + S3 Gateway Endpoint
Amazon Route 53	443	DNS record management (external-dns)	VPC Endpoint or public
Let's Encrypt (`acme-v02.api.letsencrypt.org`)	443	TLS certificate issuance (cert-manager)	Public internet

Inbound

Source	Destination	Port	Protocol	Purpose
External clients	Ingress → `wallet`	443	HTTPS	Wallet REST API
External clients	Ingress → `frontend`	443	HTTPS	Web UI (if `frontend.enabled: true`)

NetworkPolicy

The chart includes optional egress-only NetworkPolicy resources, disabled by default. Enable with:

security:
  networkPolicies:
    enabled: true

When enabled, egress is allowed from all pods in the namespace to: CoreDNS (53 UDP/TCP), any HTTPS (443/TCP), PostgreSQL (5432/TCP), and NATS internally (4222, 8222, 8443/TCP). No ingress NetworkPolicy rules are rendered by the chart.

Prerequisites

Step 1: Mirror images to ECR

Step 2: Create IAM roles with IRSA

Step 3: Populate AWS Secrets Manager

Caution:

Step 4: Create your values file

Step 5: Create the namespace

Step 6: Install the chart

Passing policy node configs as external files

Step 7: Verify the deployment

Troubleshooting

Pod fails with NoCredentialProviders

Secrets Manager AccessDeniedException

Database setup job failing

Certificate not issuing

Network requirements

Internal cluster communication

Outbound to AWS services

Inbound

NetworkPolicy

Pod fails with `NoCredentialProviders`

Secrets Manager `AccessDeniedException`