Air-Gapped Installation Preparation for Helm

Before starting an air-gapped installation of Wallaroo, complete the following preparation steps.

The general process follows these steps:

• Pre Setup Checklist for Helm: The necessary installation files are available and values set for installation.
• Wallaroo Image Retrieval and Installation Preparation for Helm: Retrieve the installation images from Wallaroo and store them in a private container registry available from the Target Cluster.

Pre Setup Checklist for Helm

The following checklist ensures that required items are ready before starting the process.

• The installation environment meets the general Wallaroo Installation Prerequisites.
• The Domain Name for the Wallaroo instance is registered in a private DNS accessible from the air-gapped installation.
• TLS certificate and private key matching the Domain Name available for the Target Cluster. This can be registered to a private certificate service.
• Access to private image registry that hosts the Wallaroo install images with the following permissions:
  • Read
  • Write
  • List
• The following Bash scripts:
  • load-images.bash: Loads the Wallaroo install images into the private image registry and generates the image-values.yaml file. Available here: load-images.bash
  • install-nvidia-driver.bash (Optional): Installs Nvidia drivers for the Target Cluster. Available here: install-nvidia-driver.bash
• Wallaroo Image Download Details: This is provided by a Wallaroo Support Representative and are stored as the following variables environmental variables for the installation scripts:
  • LICENSE_CHANNEL: The registry channel used based on the version of Wallaroo being installed, currently 2025-1.
  • VERSION: The Wallaroo version to be installed. For example: 2025.1.0-6290.
  • WALLAROO_LICENSE: The Wallaroo license identifier.
  • WALLAROO_LICENSE_USERNAME: The username associated with the Wallaroo license.
  • WALLAROO_LICENSE_PASSWORD: The password associated with the Wallaroo license.
• The following environmental variables for connecting to the private model registry:
  • REGISTRY_HOST: The fully qualified domain name of the private image registry. For example: registry.wallaroo.ai.
  • REGISTRY_NAMESPACE: Namespace where the Wallaroo images are stored.
  • REGISTRY_USERNAME: Username for authentication to the registry.
  • REGISTRY_PASSWORD: Authentication credential for private registry. Often this is either a password or a token.
• Administration Host Software Requirements: The administrative host that submits the installation commands to the target cluster requires the following software.
  • docker
  • curl
  • jq version 1.7.1
  • helm
  • kubectl version 1.31

Note that the convenience variables are used for helm based installations of Wallaroo.

Wallaroo Image Retrieval and Installation Preparation for Helm

For air-gapped installations, the Wallaroo installation images are downloaded and stored in a private registry through the following process.

The image load script below expects ECR root level access and add the prefix wallaroo/ prefix for all the Wallaroo images. Access to the ECR must include the following permissions:

• Read
• List
• Write

From a terminal with access to the Kubernetes cluster hosting the Wallaroo instance and read/write access to the private model registry, use the following procedure.

1. Set the following convenience variables.

   REGISTRY_HOST=YOUR PRIVATE IMAGE REGISTRY URL
   REGISTRY_NAMESPACE=YOUR WALLAROO CONTAINER NAMESPACE
   LICENSE_CHANNEL=YOUR LICENSE CHANNEL
   VERSION=YOUR VERSION
   WALLAROO_LICENSE=YOUR WALLAROO LICENSE ID
   WALLAROO_LICENSE_USERNAME=YOUR WALLAROO INSTALL USER NAME
   WALLAROO_LICENSE_PASSWORD=YOUR WALLAROO INSTALL USER PASSWORD
   REGISTRY_USERNAME=YOUR REGISTRY AUTHENTICATION USER NAME
   REGISTRY_PASSWORD=YOUR REGISTRY AUTHENTICATION CREDENTIAL
   

   For example:

   REGISTRY_HOST=registry.wallaroo.ai
   REGISTRY_NAMESPACE=wallaroo
   LICENSE_CHANNEL=2025-1
   VERSION=2025.1.0-6290
   WALLAROO_LICENSE=99999xxxyyyzzz0000
   WALLAROO_LICENSE_USERNAME=abcdefg
   WALLAROO_LICENSE_PASSWORD=12345679
   REGISTRY_USERNAME=ABC123
   REGISTRY_PASSWORD=ZYX987
   

2. Login to the private image registry via docker using the following command:

   echo $REGISTRY_PASSWORD | docker login -u $REGISTRY_USERNAME --password-stdin $REGISTRY_HOST
   

   Upon successful login, the following is returned.

   Login Succeeded
   

3. Load images to the private registry using the load-images.bash script. When complete, this outputs the file image-values.yaml with the relevant installation data.

   bash load-images.bash \
       --wallaroo-version $WALLAROO_VERSION \
       --wallaroo-license $WALLAROO_LICENSE \
       --wallaroo-username $WALLAROO_USERNAME \
       --registry-namespace $REGISTRY_NAMESPACE \
       --registry-host $REGISTRY_HOST
   

Save the file image-values.yaml and use it for the step Install Wallaroo.

Once complete, the Wallaroo images are stored and ready for installation.

Install Wallaroo

Wallaroo air-gapped installations for clusters use helm.

Helm Based Installation of Wallaroo

Installation Client Helm Requirements

The following software is required for the client with administrative access to the Kubernetes cluster that will host Wallaroo via Helm.

• For Helm installs:
  • helm: Install Helm
    • Minimum supported version: Helm 3.11.2
  • krew: Install Krew
  • krew preflight and krew support-bundle. Install with the following commands:
    • kubectl krew install support-bundle
    • kubectl krew install preflight

The following details how to install Wallaroo via helm. Note that these procedures require the Air-Gapped Installation Preparation be completed.

Registration Login

The first step in the Wallaroo installation process via Helm is to connect to the Kubernetes environment that will host the Wallaroo Enterprise instance and login to the Wallaroo container registry through the command provided by the Wallaroo support staff. The command will take the following format, replacing $WALLAROO_LICENSE_USERNAME and $WALLAROO_LICENSE_PASSWORD with the respective username and password provided.

helm registry login registry.replicated.com --username $WALLAROO_LICENSE_USERNAME --password $WALLAROO_LICENSE_PASSWORD

Preflight Verification

Preflight verification is performed with the following command format. The variables LICENSE_CHANNEL and VERSION is supplied by your Wallaroo support representative.

helm template oci://registry.replicated.com/wallaroo/$LICENSE_CHANNEL/wallaroo --version $VERSION | kubectl preflight -

For example, the LICENSE_CHANNEL=2025-1 and the VERSION=2025.1.0-6290

helm template oci://registry.replicated.com/wallaroo/2025-1/wallaroo --version 2025.1.0-6290 | kubectl preflight -

This displays the Preflight Checks report.

The following commands are available:

• s: Save the report to a text file as the file preflight-checks-results-DATETIME.txt. For example: preflight-checks-results-2024-03-19T13_30_41.txt.
• q: Exit the preflight report.
• Up Arrow or Down Arrow: Scroll through the preflight elements and view the report details.

The following example shows a successful preflight test.

Preflight Checks Preflight Checks

Check PASS
Title: Required Kubernetes Version
Message: Your cluster meets the recommended and required versions of Kubernetes.

------------
Check PASS
Title: Container Runtime
Message: Containerd container runtime was found.

------------
Check PASS
Title: Check Kubernetes environment.
Message: GKE is a supported distribution

------------
Title: Cluster CPU Resources
Message: 

------------
Check PASS
Title: Cluster Resources
Message: Cluster resources are satisfactory

------------
Check PASS
Title: Every node in the cluster must have at least 12Gi of memory
Message: All nodes have at least 12 GB of memory capacity

------------
Check PASS
Title: Every node in the cluster must have at least 8 cpus allocatable.
Message: All nodes have at least 8 CPU capacity

------------

Prepare Helm Installation

The following instructions detail how to install Wallaroo Enterprise via Helm for Kubernetes cloud environments such as Microsoft Azure, Amazon Web Service, and Google Cloud Platform.

With the preflight checks and prerequisites met, Wallaroo can be installed via Helm through the following process:

1. Create namespace. By default, the namespace wallaroo is used:

   kubectl create namespace wallaroo
   

2. Set the new namespace as the current namespace:

   kubectl config set-context --current --namespace wallaroo
   

3. Set the TLS certificate secret in the Kubernetes environment:

   1. Create the certificate and private key. It is recommended to name it after the domain name of your Wallaroo instance. For example: wallaroo.example.com. For production environments, organizations are recommended to use certificates from their certificate authority. Note that the Wallaroo SDK will not connect from an external connection without valid certificates. For more information on using DNS settings and certificates, see the Wallaroo DNS Integration Guide.

   2. Create the Kubernetes secret from the certificates created in the previous step, replacing $TLS_CONFIG with the name of the Kubernetes secret, with the certificate file $TLS_CERT and the private key $TLS_KEY. Store the secret name for a the step Configure local values file.

      kubectl create secret tls $TLS_CONFIG --cert=$TLS_CERT --key=$TLS_KEY
      

      For example, if $TLS_CONFIG is cust-cert-secret with the certificate $TLS_CERT is fullchain.pem and key $TLS_KEY is privkey.pem, then the command would be translated as

      kubectl create secret tls cust-cert-secret --cert=fullchain.pem --key=privkey.pem
      

4. Create the image pull secret for the private image registry. IMPORTANT NOTE: This assumes that the docker login command was run earlier in the step Wallaroo Image Storage.

   kubectl create secret docker-registry regcred --from-file=$HOME/.docker/config.json
   

5. (Optional) If using NVIDIA drivers as per the step Optional: Load NVIDIA Gpu Drivers, create a secret in the kube-system namespace with the following command:

   kubectl create secret docker-registry regcred --from-file=$HOME/.docker/config.json --name kube-system
   

Default Helm Installation Settings

A default Helm install of Wallaroo contains various default settings. The local values file overwrites values based on the organization needs. The following represents the minimum mandatory values for a Wallaroo installation using certificates and the default LoadBalancer for a cloud Kubernetes cluster. The configuration details below is saved as values.yaml for these examples.

Note the following required settings:

• kubernetes_distribution: Must be set to openShift for OpenShift based installations.

• wallarooDomain: Used to set the DNS domain name for the Wallaroo instance. For more information, see the Wallaroo DNS Integration Guide.

• custTlsSecretName: Specify the Kubernetes secret created in the previous step. External connections through the Wallaroo SDK require valid certificates.

• ingress_mode: How the Wallaroo instance is reached through the Kubernetes network settings.

  If the environment has a load balancer controller, select external. If the environment does not have a load balancer controller, configure their load balancer as required to point at a K8s or OpenShift service, then select none as the networking setting in configuration. For any questions or other conditions not listed, contact the Wallaroo support representative.

  Options include:

  • internal (Default): An internal cloud load balancer and associated resources are created. Network users outside the Kubernetes cluster – but on the same internal network – can connect directly using DNS names, and do not need to use port forward or related configurations.
  • external: An external, Internet-facing cloud load balancer, public IP, and associated resources are created. This is highly discouraged. Public DNS is also required. This is the default for Wallaroo Community Edition.
  • none: Services are local to the Kubernetes cluster. kubectl-port forward or some other means is required to access them. If all work will be done in-cluster, select this option.

• dashboard: The name displayed when users login to the Wallaroo Ops center Dashboard. For example, “My Company” or “Sales Division”, etc.

• imageRegistry: This is required for air-gapped installations and must match the REGISTRY_HOST and REGISTRY_NAMESPACE of private container registry where the Wallaroo install images are installed as per step Wallaroo Image Retrieval and Installation Preparation. For example, if the REGISTRY_HOST=registry.wallaroo.ai and the REGISTRY_NAMESPACE=wallaroo, then the imageRegistry value is registry.wallaroo.ai/wallaroo.

• privatePypi (Optional): Enables private Python repositories for custom libraries and sources. For more details, see Python Private Repositories.

The following example shows the minimum required options.

This example shows the uncommented keys for the minimum required settings and additional commented optional settings. For full details on helm values for Wallaroo installations, see the Wallaroo Helm Reference Guides.

wallarooDomain: "wallaroo.example.com" # change to match the actual domain name

custTlsSecretName: cust-cert-secret

ingress_mode: internal # internal (Default), external,or none

dashboard:
  clientName: "Wallaroo Helm Example" # Insert the name displayed in the Wallaroo Dashboard

kubernetes_distribution: openShift

# Must be a full registry address calculated by:
# $REGISTRY_HOST/$REGISTRY_NAMESPACE
# For example:  
imageRegistry: registry.wallaroo.ai:1234/wallaroo

# enable Wallaroo assays.  Select **one** of the following: `v1` or `v2`, or leave commented out to disable assays.
# v2 is enabled by default.
#assays:
#  enabled: true
#  v1: false
#  v2: true

# Enable edge deployment
#ociRegistry: 
#  enabled: true # true enables the Edge Server registry information, false disables it.
#  registry: ""# The registry url. For example: reg.big.corp:3579.
#  repository: ""# The repository within the registry. This may include the cloud account, or the full path where the Wallaroo published pipelines should be kept. For example: account123/wallaroo/pipelines.
#  email: "" # Optional field to track the email address of the registry credential.
#  username: "" # The username to the registry. This may vary based on the provider. For example, GCP Artifact Registry with service accounts uses the username _json_key_base64 with the p`ass`word as a base64 processed token of the credential information.
#  password: "" # The password or token for the registry service.

# Enable edge deployment observability
# edgelb:
#     enabled: true

# Private PyPI repository configuration for installing Python packages from a private index.
# When enabled, this configures access to a private PyPI repository for Python package installation in BYOP and Workload Orchestrations.
# privatePypi:
#   enabled: false                    # If true, configure access to a private PyPI repository
#   secretName: private-pypi-secret
#   url: ""                           # Private PyPI repository URL, eg "https://pypi.example.com/simple/"
#   username: ""                      # Username for private PyPI authentication
#   password: ""                      # Password for private PyPI authentication
#   privateOnly: false                # If true, only use private PyPI repository (no fallback to public PyPI)

# The nodeSelector and tolerations for all components
# This does not apply to nats, fluent-bit, or minio so needs to be applied separately
# nodeSelector:
#   wallaroo.ai/reserved: true

# tolerations:
# - key: "wallaroo.ai/reserved"
#   operator: "Exists"
#   effect: "NoSchedule"

# To change the pipeline taint or nodeSelector, 
# best practice is to change engine, enginelb, and engineAux 
# together unless they will be in different pools.
# engine:
#   nodeSelector:
#     wallaroo.ai/node-purpose: pipelines
#   tolerations:
#     - key: "wallaroo.ai/pipelines"
#       operator: "Exists"
#       effect: "NoSchedule"

# enginelb:
#   nodeSelector:
#     wallaroo.ai/node-purpose: pipelines
#   tolerations:
#     - key: "wallaroo.ai/pipelines"
#       operator: "Exists"
#       effect: "NoSchedule"

# engineAux:
#   nodeSelector:
#     wallaroo.ai/node-purpose: pipelines
#   tolerations:
#     - key: "wallaroo.ai/pipelines"
#       operator: "Exists"
#       effect: "NoSchedule"

# For each service below, adjust the disk size and resources as required.
# If the nodeSelector or tolerations are changed for one service, 
# the other services nodeSelector and tolerations **must** be changed to match
#
#
# plateau:
#   diskSize: 100Gi
#   resources:
#     limits:
#       memory: 4Gi
#       cpu: 1000m
#     requests:
#       memory: 128Mi
#       cpu: 100m
#   nodeSelector:
#     wallaroo.ai/node-purpose: persistent
#   tolerations:
#     - key: "wallaroo.ai/persistent"
#       operator: "Exists"
#       effect: "NoSchedule"

# Jupyter has both hub and lab nodeSelectors and tolerations
# They default to the same persistent pool, but can be assigned to different ones
# jupyter:
#   nodeSelector:                 # Node placement for Hub administrative pods
#     wallaroo.ai/node-purpose: persistent
#   tolerations:
#     - key: "wallaroo.ai/persistent"
#       operator: "Exists"
#       effect: "NoSchedule"
#   labNodeSelector:              # Node placement for Hub-spawned jupyterlab pods
#     wallaroo.ai/node-purpose: persistent
#   labTolerations:
#     - key: "wallaroo.ai/persistent"
#       operator: "Exists"
#       effect: "NoSchedule"
#   memory:
#     limit: "4"                  # Each Lab - memory limit in GB
#     guarantee: "2"              # Each Lab - lemory guarantee in GB
#   cpu:
#     limit: "2.0"                # Each Lab - fractional CPU limit
#     guarantee: "1.0"            # Each Lab - fractional CPU guarantee
#   storage:
#     capacity: "50"              # Each Lab - disk storage capacity in GB

# minio:
#   persistence:
#     size: 25Gi
#   nodeSelector:
#     wallaroo.ai/node-purpose: persistent
#   tolerations:
#   - key: wallaroo.ai/persistent
#     operator: "Exists"
#     effect: "NoSchedule"
#   resources:
#     requests:
#       memory: 1Gi

# postgres:
#   diskSize: 10Gi
#   nodeSelector:
#     wallaroo.ai/node-purpose: persistent
#   tolerations:
#     - key: "wallaroo.ai/persistent"
#       operator: "Exists"
#       effect: "NoSchedule"
#   resources:
#     limits:
#       memory: 2Gi
#       cpu: 500m
#     requests:
#       memory: 512Mi
#       cpu: 100m

# Prometheus has the usual persistent options, but also a retention size
# The the size on disk and time can be configured before removing it.
# prometheus:
#   storageRetentionSizeGb: "10"        # Retain this much data, in GB.
#   storageRetentionTimeDays: "15"     # When to remove old data. In days.
#   nodeSelector:
#     wallaroo.ai/node-purpose: persistent
#   tolerations:
#     - key: "wallaroo.ai/persistent"
#       operator: "Exists"
#       effect: "NoSchedule" 
#   resources:
#     limits:
#       memory: 6Gi
#       cpu: 2000m
#     requests:
#       memory: 512Mi
#       cpu: 100m

# nats:
#   podTemplate:
#     merge:
#       spec:
#         nodeSelector: 
#           wallaroo.ai/node-purpose: persistent
#         tolerations:
#         - key: "wallaroo.ai/persistent"
#           operator: "Exists"
#           effect: NoSchedule

# wallsvc:
#   nodeSelector:
#     wallaroo.ai/node-purpose: persistent
#   tolerations:
#     - key: "wallaroo.ai/persistent"
#       operator: "Exists"
#       effect: "NoSchedule"

persistentVolume Settings

Wallaroo services that have a persistentVolume have the following default nodeSelector label and tolerations:

• nodeSelector
  • Label: wallaroo.ai/node-purpose
  • Value: persistent
• tolerations
  • Key: wallaroo.ai/persistent
  • Operator: “Exists”
  • Effect: “NoSchedule”

For example:

nodeSelector:
  wallaroo.ai/node-purpose: persistent
tolerations:
  - key: wallaroo.ai/persistent
    operator: "Exists"
    effect: "NoSchedule"

If the nodeSelector or tolerations are changed for any service with a persistentVolume, all other services must be edited to match.

For additional information on taints and tolerations settings, see the Taints and Labels Guide.

Install Wallaroo with Helm

1. Install Wallaroo: The Wallaroo support representative will provide the installation command for the Helm install that will use the Wallaroo container registry. This assumes that the preflight checks were successful. This assumes some of these helper variables are set from the previous procedure Air-Gapped Installation Preparation.

   1. $RELEASE: The name of the Helm release. By default, wallaroo.
   2. $REGISTRY_URI: The URl for the Wallaroo container registry service.
   3. $VERSION: The version of Wallaroo to install. For this example, 2025.1.0-6290.
   4. $LOCALVALUES: The .yaml file containing the local values overrides. For this example, values.yaml.
   5. image-values.yaml file generated created in the step Wallaroo Image Retrieval and Installation Preparation.

   helm install wallaroo \
     oci://registry.replicated.com/wallaroo/2025-1/wallaroo \
     --version 2025.1.0-6290 \
     --values values.yaml \
     --values image-values.yaml \
     --timeout 10m \
     --wait \
     --wait-for-jobs
   

Verify Installation

If any required elements are missing from the values.yaml file, an error is displayed. For example, leaving out the kubernetes_distribution field returns the following:

Warning - kubernetes_distribution must be set in user provided values.yaml

Upon successful installation, notes are published indicating the installed version, where to find documentation, etc.:

NOTES:
.

Welcome to Wallaroo 2025.1.0

1. Deployment Information:
Name:             2025.1.0
Release notes:    https://docs.wallaroo.ai/wallaroo-release-notes/wallaroo-release-202501
Version:          v2025.1.0-5187

2. Accessing Wallaroo
Documentation:    https://docs.wallaroo.ai
Dashboard:        https://sample.wallaroocommunity.ninja

3. Useful Commands:

- Helm tests are available by using: `helm test wallaroo`.

- External load balancer hostname can be found by using:

    kubectl get svc api-lb-ext  -o jsonpath='{.status.loadBalancer.ingress[0].hostname}'

- List Wallaroo namespaces, including pipeline deployments, but not including the main `wallaroo` namespace:

    kubectl get namespaces -l wallaroo-managed=true

- In order to change any helm values:

    helm upgrade --install wallaroo oci://registry.replicated.com/wallaroo/uat-latest/wallaroo --version v2025.1.0-5187 --values $LOCALVALUES_YAML --timeout 10m --wait --wait-for-jobs

4. Uninstall:

 1. To uninstall/delete the Wallaroo deployment, run:

   kubectl delete ns wallaroo && kubectl delete \
     all,secret,configmap,clusterroles,clusterrolebindings,storageclass,crd \
     --selector app.kubernetes.io/part-of=wallaroo --selector kots.io/app-slug=wallaroo

 2. To delete all pipelines, run:

   kubectl delete ns -l wallaroo-managed=true
.

1. Once the installation is complete, verify the installation with the helm test $RELEASE command. A condensed display uses egrep to show only the test suite and phase status as follows. Replace wallaroo with the name of the helm release used.

   helm test wallaroo | egrep 'SUITE:|Phase:'
   

   A successful result shows the following:

   TEST SUITE:     wallaroo-fluent-bit-test-connection
   Phase:          Succeeded
   TEST SUITE:     nats-test-request-reply
   Phase:          Succeeded
   TEST SUITE:     wallaroo-wallaroo-test-connections-hook
   Phase:          Succeeded
   TEST SUITE:     wallaroo-test-objects-hook
   Phase:          Succeeded
   

   The following will show the full helm test output with notes.

   helm test wallaroo
   

   which displays the following:

   NAME: wallaroo
   LAST DEPLOYED: Fri May 17 14:00:04 2024
   NAMESPACE: wallaroo
   STATUS: deployed
   REVISION: 2
   TEST SUITE:     wallaroo-fluent-bit-test-connection
   Last Started:   Fri May 17 14:04:48 2024
   Last Completed: Fri May 17 14:04:51 2024
   Phase:          Succeeded
   TEST SUITE:     nats-test-request-reply
   Last Started:   Fri May 17 14:04:43 2024
   Last Completed: Fri May 17 14:04:48 2024
   Phase:          Succeeded
   TEST SUITE:     sample-wallaroo-test-connections-hook
   Last Started:   Fri May 17 14:04:24 2024
   Last Completed: Fri May 17 14:04:31 2024
   Phase:          Succeeded
   TEST SUITE:     sample-wallaroo-test-objects-hook
   Last Started:   Fri May 17 14:04:31 2024
   Last Completed: Fri May 17 14:04:43 2024
   Phase:          Succeeded
   NOTES:
   .
   
   Welcome to Wallaroo 2025.1.0
   
   1. Deployment Information:
     Name:             2025.1.0
     Release notes:    https://docs.wallaroo.ai/wallaroo-release-notes/wallaroo-release-202501
     Version:          v2025.1.0-5187
   
   2. Accessing Wallaroo
     Documentation:    https://docs.wallaroo.ai
     Dashboard:        https://sample.wallaroocommunity.ninja
   
   3. Useful Commands:
   
     - Helm tests are available by using: `helm test wallaroo`.
   
     - External load balancer hostname can be found by using:
   
         kubectl get svc api-lb-ext  -o jsonpath='{.status.loadBalancer.ingress[0].hostname}'
   
     - List Wallaroo namespaces, including pipeline deployments, but not including the main `wallaroo` namespace:
   
         kubectl get namespaces -l wallaroo-managed=true
   
     - In order to change any helm values:
   
         helm upgrade --install wallaroo oci://registry.replicated.com/wallaroo/2025-1/wallaroo --version 2025.1.0-6290 --values $LOCALVALUES_YAML --timeout 10m --wait --wait-for-jobs
   
    4. Uninstall:
   
     1. To uninstall/delete the Wallaroo deployment, run:
   
       kubectl delete ns wallaroo && kubectl delete \
         all,secret,configmap,clusterroles,clusterrolebindings,storageclass,crd \
         --selector app.kubernetes.io/part-of=wallaroo --selector kots.io/app-slug=wallaroo
   
     2. To delete all pipelines, run:
   
       kubectl delete ns -l wallaroo-managed=true
   .
   

At this point, the installation is complete and can be accessed through the fully qualified domain names set in the installation process above.

To add the initial users if they were not set up through Helm values, see the Wallaroo Enterprise User Management guide.

Before starting an air-gapped installation of Wallaroo using Kots, complete the following preparation steps.

The general process follows these steps:

• Pre Setup Checklist for Kots: The necessary installation files are available and values set for installation.
• Wallaroo Image Retrieval and Installation Preparation for Kots: Retrieve the installation images from Wallaroo and store them in a private container registry available from the Target Cluster.

Pre Setup Checklist for Kots

The following checklist ensures that required items are ready before starting the process.

• The installation environment meets the general Wallaroo Installation Prerequisites.
• The Domain Name for the Wallaroo instance is registered in a private DNS accessible from the air-gapped installation.
• TLS certificate and private key matching the Domain Name available for the Target Cluster. This can be registered to a private certificate service.
• Access to private image registry that hosts the Wallaroo install images with the following permissions:
  • Read
  • Write
  • List
• The following Bash scripts:
  • install-nvidia-driver.bash (Optional): Installs Nvidia drivers for the Target Cluster. Available here: install-nvidia-driver.bash
• Wallaroo Image Download Details: This is provided by a Wallaroo Support Representative and are stored as the following variables environmental variables:
  • LICENSE_CHANNEL: The registry channel used based on the version of Wallaroo being installed, currently 2025-1.
  • VERSION: The Wallaroo version to be installed. For example: 2025.1.0-6290.
  • KOTS_CLIENT_URL: The URL for downloading the Kots client file kots_linux_amd64.tar.gz with the kots version used for the Wallaroo air-gapped installation.
  • KOTS_ADMIN_URL: The URL for downloading the Kots admin bundle kotsadm.tar.gz with the kots version used for the Wallaroo air-gapped installation.
  • WALLAROO_BUNDLE_URL: The customer specific URL for downloading the Wallaroo installation package. IMPORTANT NOTE: This URL has an expiration time of around 60 minutes, and the final downloaded file is 30 GB in size.
• The following artifacts provided by the Wallaroo Support representative:
  • • License File: This is a yaml file that contains the Wallaroo license information for your organization. For this procedure, it is referred to as license.yaml.
• The following environmental variables for connecting to the private model registry:
  • REGISTRY_ADDRESS: The fully qualified domain name of the private image registry with the registry namespace used to install Wallaroo. For example: registry.wallaroo.ai/wallaroo. The wallaroo in the URL is the namespace.
  • REGISTRY_USERNAME: Username for authentication to the registry.
  • REGISTRY_PASSWORD: Authentication credential for private registry. Often this is either a password or a token.
• Administration Host Software Requirements: The administrative host that submits the installation commands to the target cluster requires the following software.
  • docker
  • curl
  • jq version 1.7.1
  • kubectl version 1.31

Wallaroo Image Retrieval and Installation Preparation for Kots

For air-gapped installations, the Wallaroo installation images are downloaded and stored in a private registry through the following process.

The image load script below expects root level access and add the prefix wallaroo/ prefix for all the Wallaroo images. Access to the registry must include the following permissions:

• Read
• List
• Write

From a terminal with access to the Kubernetes cluster hosting the Wallaroo instance and read/write access to the private model registry, use the following procedure.

1. This installation assumes the user will ssh into the target installation system with port forwarding enabled. For example, ssh -L 8800:localhost:8800 hostname. This is so later in the process the Kots Administrative Dashboard is access through the url http://localhost:8080.

2. Set the following convenience variables.

   REGISTRY_ADDRESS=YOUR PRIVATE IMAGE REGISTRY URL AND NAMESPACE
   LICENSE_CHANNEL=YOUR LICENSE CHANNEL
   REGISTRY_USERNAME=YOUR REGISTRY AUTHENTICATION USER NAME
   REGISTRY_PASSWORD=YOUR REGISTRY AUTHENTICATION CREDENTIAL
   KOTS_CLIENT_URL=YOUR KOTS CLIENT URL
   KOTS_ADMIN_URL=YOUR KOTS ADMIN URL
   WALLAROO_BUNDLE_URL=YOUR WALLAROO BUNDLE URL
   WALLAROO_DOWNLOAD_AUTHORIZATION=YOUR AUTHORIZATION CODE
   

   For example:

   REGISTRY_ADDRESS=registry.wallaroo.ai/wallaroo
   LICENSE_CHANNEL=2025-1
   REGISTRY_USERNAME=ABC123
   REGISTRY_PASSWORD=ZYX987
   KOTS_CLIENT_URL="https://github.com/replicatedhq/kots/releases/download/v1.124.18/kots_linux_amd64.tar.gz"
   KOTS_ADMIN_URL="https://github.com/replicatedhq/kots/releases/download/v1.124.18/kotsadm.tar.gz"
   WALLAROO_BUNDLE_URL="https://s3.amazonaws.com/airgap.replicated.com/abcdefg/2.airgap?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIA3UMYHRA5LHP4KTHN%2F20250708%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20250708T160517Z&X-Amz-Expires=3600&X-Amz-Security-Token=abcdefg"
   

3. Download the following via curl via the following commands:

   1. Kots CLI: curl -LO $KOTS_CLIENT_URL. This downloads the file kots_linux_amd64.tar.gz.
   2. Kots admin bundle: curl -LO $KOTS_ADMIN_URL. This downloads the file kotsadm.tar.gz.
   3. Wallaroo Installation Bundle: curl -Lo wallaroo.airgap $WALLAROO_BUNDLE_URL. This saves the file as wallaroo.airgap. IMPORTANT NOTE: This URL has an expiration time of around 60 minutes, and the final downloaded file is 30 GB in size.

4. Install Kots client via the following commands:

   tar zxf kots_linux_amd64.tar.gz
   sudo mv kots /usr/local/bin/kubectl-kots
   sudo chmod a+x /usr/local/bin/kubectl-kots
   

5. Push the Kots administrative bundle to the private registry via the following command:

   kubectl kots admin-console \
       push-images kotsadm.tar.gz \
       $REGISTRY_ADDRESS \
       --registry-username $REGISTRY_USERNAME \
       --registry-password $REGISTRY_PASSWORD
   

6. Install Wallaroo via the following command. This takes approximately 30 minutes, and will pause as each image is loaded. The following parameters are manually set:

   • --shared-password: The password to the Kots Administrative Dashboard after installation is complete. For example, --shared-password wallaroo
   • --license-file: The file name of the license file. For example: --license-file license.yaml

   kubectl kots install $CHANNEL \
       --namespace wallaroo --shared-password wallaroo \
       --airgap-bundle wallaroo.airgap --license-file license.yaml \
       --kotsadm-registry $REGISTRY_ADDRESS \
       --registry-username $REGISTRY_USERNAME  \
       --registry-password $REGISTRY_PASSWORD
   

7. Once complete, a prompt is displayed showing the Kots Administrative Dashboard is displayed. Use this URL in a browser to proceed to the next stage in the installation process.

   • Deploying Admin Console
   • Creating namespace ✓
   • Waiting for datastore to be ready ✓
       Enter a new password to be used for the Admin Console: •••••••••••••
     • Waiting for Admin Console to be ready ✓
   
   • Press Ctrl+C to exit
   • Go to http://localhost:8800 to access the Admin Console
   

Initial Wallaroo Config

Once Wallaroo has been installed via kots for the first time, we can perform initial configuration.

1. If the Kots Administrative Console has not started, launch it with the following command:

   ❯ kubectl kots admin-console --namespace wallaroo
     • Press Ctrl+C to exit
     • Go to http://localhost:8800 to access the Admin Console
   

2. Enter the Kots Administrative Console address into a browser. You will be prompted for the default password as set in the step above. Enter it and select Log in.

   

3. Upload your license file. If the license file was already selected from the Automated installation, this step is skipped.

   

4. The Kots Administrative Console will run the preflight checks to verify that all of the minimum requirements are met. This may take a few minutes. If there are any issues, Wallaroo can still be launched but may not function properly. When ready, select Continue.

   

5. The Configure Wallaroo page will be displayed which allows you to customize your Wallaroo environment. The following are the minimum required settings.

   1. Networking Configuration: Set whether the Wallaroo instance is available from outside the Kubernetes cluster, or only from within it.

      1. Ingress Mode for Wallaroo Endpoints: If a load balance controller is used, select External. If there is no load balance controller configured, the load balancer must be configured to point at a K8s or OpenShift service, then select None as the networking setting in configuration. For any questions or other conditions not listed, contact the Wallaroo support representative.
         1. None: Services are local to the Kubernetes cluster. kubectl-port forward or some other means is required to access them. If all work will be done in-cluster, select this option.
         2. Internal: An internal cloud load balancer and associated resources are created. Network users outside the Kubernetes cluster – but on the same internal network – can connect directly using DNS names, and do not need to use port forward or related configurations.
         3. External: An external, Internet-facing cloud load balancer, public IP, and associated resources are created. This is highly discouraged. Public DNS is also required. This is the default for Wallaroo Community Edition.
            1. Enable external URL inference endpoints: Creates pipeline inference endpoints. For more information, see Model Endpoints Guide.

   2. From the Wallaroo Dashboard, select Config and set the following:

      

   3. DNS Settings

      1. Wallaroo domain name (Required): The domain for the Wallaroo instance.

   4. TLS Certificates

      1. Use custom TLS Certs: Checked
      2. TLS Certificate: Enter your TLS Certificate (.crt file).
      3. TLS Private Key: Enter your TLS private key (.key file).

6. Once complete, scroll to the bottom of the Config page and select Deploy.

7. At this point, continue to Required Installation Configurations for the required configuration settings.