Air-Gapped Installation Preparation for Helm

Before starting an air-gapped installation of Wallaroo, complete the following preparation steps.

The general process follows these steps:

• Pre Setup Checklist for Helm: The necessary installation files are available and values set for installation.
• Wallaroo Image Retrieval and Installation Preparation for Helm: Retrieve the installation images from Wallaroo and store them in a private container registry available from the Target Cluster.

Pre Setup Checklist for Helm

The following checklist ensures that required items are ready before starting the process.

• The installation environment meets the general Wallaroo Installation Prerequisites.
• The Domain Name for the Wallaroo instance is registered in a private DNS accessible from the air-gapped installation.
• TLS certificate and private key matching the Domain Name available for the Target Cluster. This can be registered to a private certificate service.
• Access to private image registry that hosts the Wallaroo install images with the following permissions:
  • Read
  • Write
  • List
• The following Bash scripts:
  • load-images.bash: Loads the Wallaroo install images into the private image registry and generates the image-values.yaml file. Available here: load-images.bash
  • install-nvidia-driver.bash (Optional): Installs Nvidia drivers for the Target Cluster. Available here: install-nvidia-driver.bash
• Wallaroo Image Download Details: This is provided by a Wallaroo Support Representative and are stored as the following variables environmental variables for the installation scripts:
  • LICENSE_CHANNEL: The registry channel used based on the version of Wallaroo being installed, currently 2025-1.
  • VERSION: The Wallaroo version to be installed. For example: 2025.1.0-6290.
  • WALLAROO_LICENSE: The Wallaroo license identifier.
  • WALLAROO_LICENSE_USERNAME: The username associated with the Wallaroo license.
  • WALLAROO_LICENSE_PASSWORD: The password associated with the Wallaroo license.
• The following environmental variables for connecting to the private model registry:
  • REGISTRY_HOST: The fully qualified domain name of the private image registry. For example: registry.wallaroo.ai.
  • REGISTRY_NAMESPACE: Namespace where the Wallaroo images are stored.
  • REGISTRY_USERNAME: Username for authentication to the registry.
  • REGISTRY_PASSWORD: Authentication credential for private registry. Often this is either a password or a token.
• Administration Host Software Requirements: The administrative host that submits the installation commands to the target cluster requires the following software.
  • docker
  • curl
  • jq version 1.7.1
  • helm
  • kubectl version 1.31

Note that the convenience variables are used for helm based installations of Wallaroo.

Wallaroo Image Retrieval and Installation Preparation for Helm

For air-gapped installations, the Wallaroo installation images are downloaded and stored in a private registry through the following process.

The image load script below expects ECR root level access and add the prefix wallaroo/ prefix for all the Wallaroo images. Access to the ECR must include the following permissions:

• Read
• List
• Write

From a terminal with access to the Kubernetes cluster hosting the Wallaroo instance and read/write access to the private model registry, use the following procedure.

1. Set the following convenience variables.

   REGISTRY_HOST=YOUR PRIVATE IMAGE REGISTRY URL
   REGISTRY_NAMESPACE=YOUR WALLAROO CONTAINER NAMESPACE
   LICENSE_CHANNEL=YOUR LICENSE CHANNEL
   VERSION=YOUR VERSION
   WALLAROO_LICENSE=YOUR WALLAROO LICENSE ID
   WALLAROO_LICENSE_USERNAME=YOUR WALLAROO INSTALL USER NAME
   WALLAROO_LICENSE_PASSWORD=YOUR WALLAROO INSTALL USER PASSWORD
   REGISTRY_USERNAME=YOUR REGISTRY AUTHENTICATION USER NAME
   REGISTRY_PASSWORD=YOUR REGISTRY AUTHENTICATION CREDENTIAL
   

   For example:

   REGISTRY_HOST=registry.wallaroo.ai
   REGISTRY_NAMESPACE=wallaroo
   LICENSE_CHANNEL=2025-1
   VERSION=2025.1.0-6290
   WALLAROO_LICENSE=99999xxxyyyzzz0000
   WALLAROO_LICENSE_USERNAME=abcdefg
   WALLAROO_LICENSE_PASSWORD=12345679
   REGISTRY_USERNAME=ABC123
   REGISTRY_PASSWORD=ZYX987
   

2. Login to the private image registry via docker using the following command:

   echo $REGISTRY_PASSWORD | docker login -u $REGISTRY_USERNAME --password-stdin $REGISTRY_HOST
   

   Upon successful login, the following is returned.

   Login Succeeded
   

3. Load images to the private registry using the load-images.bash script. When complete, this outputs the file image-values.yaml with the relevant installation data.

   bash load-images.bash \
       --wallaroo-version $WALLAROO_VERSION \
       --wallaroo-license $WALLAROO_LICENSE \
       --wallaroo-username $WALLAROO_USERNAME \
       --registry-namespace $REGISTRY_NAMESPACE \
       --registry-host $REGISTRY_HOST
   

Save the file image-values.yaml and use it for the step Install Wallaroo.

Once complete, the Wallaroo images are stored and ready for installation.

Install Wallaroo

Wallaroo air-gapped installations for clusters use helm.

Helm Based Installation of Wallaroo

Installation Client Helm Requirements

The following software is required for the client with administrative access to the Kubernetes cluster that will host Wallaroo via Helm.

• For Helm installs:
  • helm: Install Helm
    • Minimum supported version: Helm 3.11.2
  • krew: Install Krew
  • krew preflight and krew support-bundle. Install with the following commands:
    • kubectl krew install support-bundle
    • kubectl krew install preflight

Before starting an air-gapped installation of Wallaroo using Kots, complete the following preparation steps.

The general process follows these steps:

• Pre Setup Checklist for Kots: The necessary installation files are available and values set for installation.
• Wallaroo Image Retrieval and Installation Preparation for Kots: Retrieve the installation images from Wallaroo and store them in a private container registry available from the Target Cluster.

Pre Setup Checklist for Kots

The following checklist ensures that required items are ready before starting the process.

• The installation environment meets the general Wallaroo Installation Prerequisites.
• The Domain Name for the Wallaroo instance is registered in a private DNS accessible from the air-gapped installation.
• TLS certificate and private key matching the Domain Name available for the Target Cluster. This can be registered to a private certificate service.
• Access to private image registry that hosts the Wallaroo install images with the following permissions:
  • Read
  • Write
  • List
• The following Bash scripts:
  • install-nvidia-driver.bash (Optional): Installs Nvidia drivers for the Target Cluster. Available here: install-nvidia-driver.bash
• Wallaroo Image Download Details: This is provided by a Wallaroo Support Representative and are stored as the following variables environmental variables:
  • LICENSE_CHANNEL: The registry channel used based on the version of Wallaroo being installed, currently 2025-1.
  • VERSION: The Wallaroo version to be installed. For example: 2025.1.0-6290.
  • KOTS_CLIENT_URL: The URL for downloading the Kots client file kots_linux_amd64.tar.gz with the kots version used for the Wallaroo air-gapped installation.
  • KOTS_ADMIN_URL: The URL for downloading the Kots admin bundle kotsadm.tar.gz with the kots version used for the Wallaroo air-gapped installation.
  • WALLAROO_BUNDLE_URL: The customer specific URL for downloading the Wallaroo installation package. IMPORTANT NOTE: This URL has an expiration time of around 60 minutes, and the final downloaded file is 30 GB in size.
• The following artifacts provided by the Wallaroo Support representative:
  • • License File: This is a yaml file that contains the Wallaroo license information for your organization. For this procedure, it is referred to as license.yaml.
• The following environmental variables for connecting to the private model registry:
  • REGISTRY_ADDRESS: The fully qualified domain name of the private image registry with the registry namespace used to install Wallaroo. For example: registry.wallaroo.ai/wallaroo. The wallaroo in the URL is the namespace.
  • REGISTRY_USERNAME: Username for authentication to the registry.
  • REGISTRY_PASSWORD: Authentication credential for private registry. Often this is either a password or a token.
• Administration Host Software Requirements: The administrative host that submits the installation commands to the target cluster requires the following software.
  • docker
  • curl
  • jq version 1.7.1
  • kubectl version 1.31

Wallaroo Image Retrieval and Installation Preparation for Kots

For air-gapped installations, the Wallaroo installation images are downloaded and stored in a private registry through the following process.

The image load script below expects root level access and add the prefix wallaroo/ prefix for all the Wallaroo images. Access to the registry must include the following permissions:

• Read
• List
• Write

From a terminal with access to the Kubernetes cluster hosting the Wallaroo instance and read/write access to the private model registry, use the following procedure.

1. This installation assumes the user will ssh into the target installation system with port forwarding enabled. For example, ssh -L 8800:localhost:8800 hostname. This is so later in the process the Kots Administrative Dashboard is access through the url http://localhost:8080.

2. Set the following convenience variables.

   REGISTRY_ADDRESS=YOUR PRIVATE IMAGE REGISTRY URL AND NAMESPACE
   LICENSE_CHANNEL=YOUR LICENSE CHANNEL
   REGISTRY_USERNAME=YOUR REGISTRY AUTHENTICATION USER NAME
   REGISTRY_PASSWORD=YOUR REGISTRY AUTHENTICATION CREDENTIAL
   KOTS_CLIENT_URL=YOUR KOTS CLIENT URL
   KOTS_ADMIN_URL=YOUR KOTS ADMIN URL
   WALLAROO_BUNDLE_URL=YOUR WALLAROO BUNDLE URL
   WALLAROO_DOWNLOAD_AUTHORIZATION=YOUR AUTHORIZATION CODE
   

   For example:

   REGISTRY_ADDRESS=registry.wallaroo.ai/wallaroo
   LICENSE_CHANNEL=2025-1
   REGISTRY_USERNAME=ABC123
   REGISTRY_PASSWORD=ZYX987
   KOTS_CLIENT_URL="https://github.com/replicatedhq/kots/releases/download/v1.124.18/kots_linux_amd64.tar.gz"
   KOTS_ADMIN_URL="https://github.com/replicatedhq/kots/releases/download/v1.124.18/kotsadm.tar.gz"
   WALLAROO_BUNDLE_URL="https://s3.amazonaws.com/airgap.replicated.com/abcdefg/2.airgap?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIA3UMYHRA5LHP4KTHN%2F20250708%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20250708T160517Z&X-Amz-Expires=3600&X-Amz-Security-Token=abcdefg"
   

3. Download the following via curl via the following commands:

   1. Kots CLI: curl -LO $KOTS_CLIENT_URL. This downloads the file kots_linux_amd64.tar.gz.
   2. Kots admin bundle: curl -LO $KOTS_ADMIN_URL. This downloads the file kotsadm.tar.gz.
   3. Wallaroo Installation Bundle: curl -Lo wallaroo.airgap $WALLAROO_BUNDLE_URL. This saves the file as wallaroo.airgap. IMPORTANT NOTE: This URL has an expiration time of around 60 minutes, and the final downloaded file is 30 GB in size.

4. Install Kots client via the following commands:

   tar zxf kots_linux_amd64.tar.gz
   sudo mv kots /usr/local/bin/kubectl-kots
   sudo chmod a+x /usr/local/bin/kubectl-kots
   

5. Push the Kots administrative bundle to the private registry via the following command:

   kubectl kots admin-console \
       push-images kotsadm.tar.gz \
       $REGISTRY_ADDRESS \
       --registry-username $REGISTRY_USERNAME \
       --registry-password $REGISTRY_PASSWORD
   

6. Install Wallaroo via the following command. This takes approximately 30 minutes, and will pause as each image is loaded. The following parameters are manually set:

   • --shared-password: The password to the Kots Administrative Dashboard after installation is complete. For example, --shared-password wallaroo
   • --license-file: The file name of the license file. For example: --license-file license.yaml

   kubectl kots install $CHANNEL \
       --namespace wallaroo --shared-password wallaroo \
       --airgap-bundle wallaroo.airgap --license-file license.yaml \
       --kotsadm-registry $REGISTRY_ADDRESS \
       --registry-username $REGISTRY_USERNAME  \
       --registry-password $REGISTRY_PASSWORD
   

7. Once complete, a prompt is displayed showing the Kots Administrative Dashboard is displayed. Use this URL in a browser to proceed to the next stage in the installation process.

   • Deploying Admin Console
   • Creating namespace ✓
   • Waiting for datastore to be ready ✓
       Enter a new password to be used for the Admin Console: •••••••••••••
     • Waiting for Admin Console to be ready ✓
   
   • Press Ctrl+C to exit
   • Go to http://localhost:8800 to access the Admin Console
   

Initial Wallaroo Config

Once Wallaroo has been installed via kots for the first time, we can perform initial configuration.

1. If the Kots Administrative Console has not started, launch it with the following command:

   ❯ kubectl kots admin-console --namespace wallaroo
     • Press Ctrl+C to exit
     • Go to http://localhost:8800 to access the Admin Console
   

2. Enter the Kots Administrative Console address into a browser. You will be prompted for the default password as set in the step above. Enter it and select Log in.

   

3. Upload your license file. If the license file was already selected from the Automated installation, this step is skipped.

   

4. The Kots Administrative Console will run the preflight checks to verify that all of the minimum requirements are met. This may take a few minutes. If there are any issues, Wallaroo can still be launched but may not function properly. When ready, select Continue.

   

5. The Configure Wallaroo page will be displayed which allows you to customize your Wallaroo environment. The following are the minimum required settings.

   1. Networking Configuration: Set whether the Wallaroo instance is available from outside the Kubernetes cluster, or only from within it.

      1. Ingress Mode for Wallaroo Endpoints: If a load balance controller is used, select External. If there is no load balance controller configured, the load balancer must be configured to point at a K8s or OpenShift service, then select None as the networking setting in configuration. For any questions or other conditions not listed, contact the Wallaroo support representative.
         1. None: Services are local to the Kubernetes cluster. kubectl-port forward or some other means is required to access them. If all work will be done in-cluster, select this option.
         2. Internal: An internal cloud load balancer and associated resources are created. Network users outside the Kubernetes cluster – but on the same internal network – can connect directly using DNS names, and do not need to use port forward or related configurations.
         3. External: An external, Internet-facing cloud load balancer, public IP, and associated resources are created. This is highly discouraged. Public DNS is also required. This is the default for Wallaroo Community Edition.
            1. Enable external URL inference endpoints: Creates pipeline inference endpoints. For more information, see Model Endpoints Guide.

   2. From the Wallaroo Dashboard, select Config and set the following:

      

   3. DNS Settings

      1. Wallaroo domain name (Required): The domain for the Wallaroo instance.

   4. TLS Certificates

      1. Use custom TLS Certs: Checked
      2. TLS Certificate: Enter your TLS Certificate (.crt file).
      3. TLS Private Key: Enter your TLS private key (.key file).

6. Once complete, scroll to the bottom of the Config page and select Deploy.

7. At this point, continue to Required Installation Configurations for the required configuration settings.

For organizations that use NVIDIA CUDA gpus for AI acceleration, install the images using the following commands.

1. Verify the drivers are installed via the kubectl describe node command, and verify the label nvidia.com/gpu is in the Allocatable section. If it is, then no further steps are required.
2. If the label is not present, install the NVIDIA drivers via the following steps.

   1. Load the Nvidia image into the registry.

      bash load-images \
        --wallaroo-version $VERSION \
        --wallaroo-license $WALLAROO_LICENSE \
        --wallaroo-username $WALLAROO_LICENSE_USERNAME \
        --registry-host $REGISTRY_HOST \
        --registry-namespace $REGISTRY_NAMESPACE \
        --load-image nvcr.io/nvidia/k8s-device-plugin:v0.17.0
      

   2. Install the Nvidia driver into the kube-system namespace.

      bash install-nvidia-driver.bash \
         --registry-host $REGISTRY_HOST \
         --registry-namespace $REGISTRY_NAMESPACE
      

   3. Verify by running the kubectl command below. If successful, the gpu should display allocatable: 1 after execution.

      kubectl describe pod
      

User management is managed through the Wallaroo Dashboard, via the Platform Admin Dashboard page. See the Wallaroo User Management for full guides on setting up users, identity providers, and other user configuration options.

The following is an abbreviated guide on setting up new Wallaroo users.

The process includes the following steps:

• Obtain the User Admin Credentials
• Create a New User with the Admin Role

Obtain the User Admin Credentials

Obtaining the admin User Credentials

The standard Wallaroo installation creates the user admin by default and assigns them a randomly generated password. The admin user credentials are obtained which may be obtained directly from Kubernetes with the following commands, assuming the Wallaroo instance namespace is wallaroo.

• Retrieve Admin Username

  kubectl -n wallaroo \
  get secret keycloak-admin-secret \
  -o go-template='{{.data.KEYCLOAK_ADMIN_USER | base64decode }}'
  

• Retrieve Admin Password

  kubectl -n wallaroo \
  get secret keycloak-admin-secret \
  -o go-template='{{.data.KEYCLOAK_ADMIN_PASSWORD | base64decode }}'
  

Create a New User with the Admin Role

Creating users is managed through the Platform Admin Dashboard. The following steps are used to create an initial user with the role admin.

1. Access the Wallaroo Dashboard through the DNS name set up in the DNS Services Integration step. For example, if the DNS name of the Wallaroo Ops center is wallaroo.example.com, the Wallaroo Dashboard is available at https://wallaroo.example.com.
2. Login with the username admin and the password retrieved in the step Obtaining the admin User Credentials.
3. Select Create Users and enter the following:
   1. User Email: The email address for the user. This must be in the valid email address format.
   2. Assign Type: Select Admin.
   3. Password: Enter the user’s password. This user password be sent to the new user.
   4. Temporary or Permanent:
      1. Temporary: The user will be forced to change their login password upon their next login (Recommended).
      2. Permanent: The user will keep their password.
4. Create any additional users as needed. When finished, select the Wallaroo icon drop down and select Logout.

At this point, users can log in to Wallaroo Dashboard with their provided identities. For guides on setting up Single Sign-On (SSO) and other features, see Wallaroo User Management for full guides on setting up users, identity providers, and other user configuration options.

Retrieve IP Address for Air-Gapped Installations

For air-gapped installations of Wallaroo, the IP address associated with the Kubernetes service api-lb-ext. As part of the air-gapped environment, the api-lb-ext IP address is not available on the public internet - just available to the air-gapped environment. This is retrieved through the following command:

kubectl get svc api-lb-ext -n wallaroo -o jsonpath='{.status.loadBalancer.ingress[0].ip}'

34.83.140.98