The following template scripts are provided as examples on how to create single node virtual machines that meet the requirements listed above in AWS, GCP, and Microsoft Azure environments, and open the required network ports.

# Open ports 22, 443, 30000
aws ec2 authorize-security-group-ingress --group-name $NAME --protocol tcp --port 22 --cidr $MY_IP --no-cli-pager
aws ec2 authorize-security-group-ingress --group-name $NAME --protocol tcp --port 443 --cidr $MY_IP --no-cli-pager
aws ec2 authorize-security-group-ingress --group-name $NAME --protocol tcp --port 30000 --cidr $MY_IP --no-cli-pager
# optional:  For Edge Observability
#aws ec2 authorize-security-group-ingress --group-name $NAME --protocol tcp --port 8081 --cidr $MY_IP --no-cli-pager

# Variables

# The name of the virtual machine
NAME=wallaroo-vm                     # eg bob-demo-vm

# The image used : ubuntu/images/2024.4/hvm-ssd/ubuntu-jammy-22.04-amd64-server-20230208
IMAGE_ID=ami-0557a15b87f6559cf

# Instance type meeting the Wallaroo requirements.
INSTANCE_TYPE=c6i.8xlarge # CPU only
#INSTANCE_TYPE=g4dn.4xlarge # T4 GPU
#INSTANCE_TYPE=g5.4xlarge # A10 GPU

# key name - generate keys using Amazon EC2 Key Pairs
# https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html
# Wallaroo people: https://us-east-1.console.aws.amazon.com/ec2/home?region=us-east-1#KeyPairs:v=3 - 
MYKEY=my-key-name-keygen


# We will whitelist the source IP for maximum security -- just use 0.0.0.0/0 if you don't care.
MY_IP=$(curl -s https://checkip.amazonaws.com)/32

# Create security group in the Default VPC
aws ec2 create-security-group --group-name $NAME --description "$USER demo" --no-cli-pager

# Open port 22 and 443
aws ec2 authorize-security-group-ingress --group-name $NAME --protocol tcp --port 22 --cidr $MY_IP --no-cli-pager
aws ec2 authorize-security-group-ingress --group-name $NAME --protocol tcp --port 443 --cidr $MY_IP --no-cli-pager

# Launch instance with a 650 GB Boot device.
aws ec2 run-instances --image-id $IMAGE_ID --count 1 --instance-type $INSTANCE_TYPE \
    --no-cli-pager \
    --key-name $MYKEY \
    --block-device-mappings '[{"DeviceName":"/dev/sda1","Ebs":{"VolumeSize":650,"VolumeType":"gp2"}}]'  \
    --tag-specifications "ResourceType=instance,Tags=[{Key=Name,Value=$NAME}]" \
    --security-groups $NAME

# SSH port is open by default. This adds an application port.
az vm open-port --resource-group $RESOURCEGROUP --name $NAME --port 443
az vm open-port --resource-group $RESOURCEGROUP --name $NAME --port 30000
# optional - for Edge observability
# az vm open-port --resource-group $RESOURCEGROUP --name $NAME --port 8081

#!/bin/bash

# Variables list.  Update as per your organization's settings
NAME=$USER-demo-vm                          # eg bob-demo-vm
RESOURCEGROUP=YOURRESOURCEGROUP
LOCATION=eastus
IMAGE=Canonical:0001-com-ubuntu-server-jammy:22_04-lts:22.04.202301140

# Pick a location
az account list-locations  -o table |egrep 'US|----|Name'

# Create resource group
az group create -l $LOCATION --name $USER-demo-$(date +%y%m%d)

# Create VM. This will create ~/.ssh/id_rsa and id_rsa.pub - store these for later use.
az vm create --resource-group $RESOURCEGROUP --name $NAME --image $IMAGE  --generate-ssh-keys \
   --size Standard_D32s_v4 --os-disk-size-gb 500 --public-ip-sku Standard

# Sample output
# {
#   "location": "eastus",
#   "privateIpAddress": "10.0.0.4",
#   "publicIpAddress": "20.127.249.196",    <-- Write this down as MYPUBIP
#   "resourceGroup": "mnp-demo-230213",
#   ...
# }

# SSH port is open by default. This adds an application port.
az vm open-port --resource-group $RESOURCEGROUP --name $NAME --port 443

# SSH to the VM - assumes that ~/.ssh/id_rsa and ~/.ssh/id_rsa.pub from above are availble.
# ssh $MYPUBIP

# Use this Stop the VM ("deallocate" frees resources and billing; "stop" does not)
# az vm deallocate --resource-group $RESOURCEGROUP --name $NAME

# Restart the VM
# az vm start --resource-group $RESOURCEGROUP --name $NAME

# Settings
NAME=YOUR VM NAME HERE            # For example: jcw-demo-vm
ZONE=YOUR ZONE HERE               # For a complete list, use `gcloud compute zones list | egrep ^us-`
PROJECT=YOUR PROJECT ID HERE      # Insert the GCP Project ID here. This is the one for Wallaroo.
NETWORK=default                   # default network is the default, substitute as needed
RULE_NAME=Wallaroo-admin-console  #port rule name for port 30000

# Create VM
IMAGE=projects/ubuntu-os-cloud/global/images/ubuntu-2204-jammy-v20231030

# Open port 30000. Port 22 and 443 open by default
gcloud compute firewall-rules create $RULE_NAME \
--action=ALLOW \
--direction=INGRESS \
--network=$NETWORK \
--priority=1000 \              #this can be modified to any priority as needed
--rules=tcp:30000 \
--source-ranges=0.0.0.0/0      #This allows access from any IP and can be limited as well

gcloud compute instances create $NAME \
--project=$PROJECT \
--zone=$ZONE \
--machine-type=e2-standard-32 \ # for GPU machines, ensure at least 16 CPUs
--network-interface=network-tier=STANDARD,subnet=default \
--maintenance-policy=MIGRATE \
--provisioning-model=STANDARD \
--no-service-account \
--no-scopes \
--tags=https-server \
--create-disk=boot=yes,image=${IMAGE},size=500,type=pd-standard \
--no-shielded-secure-boot \
--no-shielded-vtpm \
--no-shielded-integrity-monitoring \
--reservation-affinity=any

# Get the external IP address
gcloud compute instances describe $NAME --zone $ZONE --format='get(networkInterfaces[0].accessConfigs[0].natIP)'

Wallaroo Single Node Linux installation works with on premise installations. The procedure for setting up an on premise environment for Wallaroo differs, but are supported by any of the following.

• Bare metal installation: Any X86 device that meets the listed hardware, operating system, and network requirements above.
• Virtual Machines: Organizations that prefer hosted virtual environments can install Wallaroo in that environment provided is meets the requirements listed above. THis includes but is not limited to the following virtual machine environments:
  • Proxmox
  • Oracle Virtualbox
  • QEMU

Consult the setup procedures for the bare metal or virtual machine used to install Ubuntu 22.04 and proceed with the installation instructions below. For additional assistance, consult your Wallaroo support representative.

The following optional configurations to enable OCI Registry integration with Wallaroo for deploying models in edge and multi-cloud environments through Wallaroo. For more details, see Inference Anywhere.

To set the Edge Registry Settings through the Kots Administrative Dashboard:

1. Scroll to Edge Deployment.

2. Enable Provide OCI registry credentials for pipelines.

3. Enter the following:

   

   1. Registry URL: The address of the registry service. For example: us-west1-docker.pkg.dev.
   2. email: The email address of the user account used to authenticate to the service.
   3. username: The account used to authenticate to the registry service.
   4. password: The password or token used to authenticate to the registry service.
   5. To enable edge observability, enable Enable pipelines deployed on the edge to send data back to the OpsCenter.

4. Save the updated configuration, then deploy it. Once complete, the edge registry settings will be available.

To enable Wallaroo assays via kots:

1. From the Config page, scroll to Config -> Observability -> Enable Assays.

   

2. Select from one of the following options:

   1. Disable: Disable assays.
   2. v1: Enable Wallaroo Assays V1.
   3. v2: Enable Wallaroo Assays V2 (Default).

3. Complete the deployment as needed.

To enable ML Workload Orchestrations, use the following steps.

1. Scroll to Pipeline Orchestration.
2. Enable Enable Pipeline Orchestration service.

DNS services integration is required for Wallaroo Enterprise to provide access to the various supporting services that are part of the Wallaroo instance. These include:

• Simplified user authentication and management.
• Centralized services for accessing the Wallaroo Dashboard, Wallaroo SDK and Authentication.
• Collaboration features allowing teams to work together.
• Managed security, auditing and traceability.

The following guide is for standard DNS services.

Once integrated, users can access the following services directly from a URL starting with the Wallaroo Domain - this is the domain name the DNS enetry is set to. For example, if the Wallaroo Domain is wallaroo.example.com, then the Wallaroo Ops instance is accessed from https://wallaroo.example.com/{service}.

Note that even when accessing specific Wallaroo services directly that the user must authenticate through Wallaroo or another authorized authentication service.

DNS Prerequisites

• Install Wallaroo Enterprise into a qualified environment. For more details, see the Wallaroo Install Guides and the Wallaroo Enterprise Install Guides.
• Generate or update the the SSL Certificates
• Have access to internal corporate DNS configurations that can be updated. A subdomain for the Wallaroo instance will be created through this process.
• Have the IP address for the Wallaroo instance. Depending on the cloud platform, this is obtain in different ways.
  • AWS: How do I get the ID and IP address of an Amazon EC2 instance for an AWS Batch job?
  • GCP: Locating IP addresses for an instance
  • Azure: IP addresses in Azure Functions

DNS Entries

Create DNS the following entries based on the list above for the Wallaroo instance’s IP address. Select the cloud environment used for DNS services. Note that DNS services are not tied to the specific cloud platform; check with your organizations setup requirements before proceeding.

• Wallaroo Domain:
  • A record, NS (Name Server) record, SOA (Start Of Authority) record. For example: wallaroo.example.com.

User management is managed through the Wallaroo Dashboard, via the Platform Admin Dashboard page. See the Wallaroo User Management for full guides on setting up users, identity providers, and other user configuration options.

The following is an abbreviated guide on setting up new Wallaroo users.

The process includes the following steps:

• Obtain the User Admin Credentials
• Create a New User with the Admin Role

Obtain the User Admin Credentials

Obtaining the admin User Credentials

The standard Wallaroo installation creates the user admin by default and assigns them a randomly generated password. The admin user credentials are obtained through the following process:

To retrieve the admin user’s password:

1. From SSH session with the VM, enter the following command to access the Wallaroo shell.

   sudo ./wallaroo shell
   

2. Retrieve the Admin Password with the following:

   kubectl -n kotsadm \
   get secret keycloak-admin-secret \
   -o go-template='{{.data.KEYCLOAK_ADMIN_PASSWORD | base64decode }}'
   

Create a New User with the Admin Role

Creating users is managed through the Platform Admin Dashboard. The following steps are used to create an initial user with the role admin.

1. Access the Wallaroo Dashboard through the DNS name set up in the DNS Services Integration step. For example, if the DNS name of the Wallaroo Ops center is wallaroo.example.com, the Wallaroo Dashboard is available at https://wallaroo.example.com.
2. Login with the username admin and the password retrieved in Obtaining the admin User Credentials.
3. Select Create Users and enter the following:
   1. User Email: The email address for the user. This must be in the valid email address format.
   2. Assign Type: Select Admin.
   3. Password: Enter the user’s password. This user password be sent to the new user.
   4. Temporary or Permanent:
      1. Temporary: The user will be forced to change their login password upon their next login (Recommended).
      2. Permanent: The user will keep their password.
4. Create any additional users as needed. When finished, select the Wallaroo icon drop down and select Logout.

At this point, users can log in to Wallaroo Dashboard with their provided identities. For guides on setting up Single Sign-On (SSO) and other features, see Wallaroo User Management for full guides on setting up users, identity providers, and other user configuration options.

For cloud or on-premise systems leveraging NVIDIA CUDA 12 gpus, the following steps must be followed.

1. SSH into the VM or on-premise installation with an account with sudo Administrative access.

2. Install the Nvidia drivers as per the instructions from Installing the drivers on servers and/or for computing purposes for Cuda as follows.

   sudo ubuntu-drivers --gpgpu install
   

3. Install helm via the following command:

   curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 && chmod 700 get_helm.sh && ./get_helm.sh
   

4. Enter the Wallaroo Shell:

   sudo ./wallaroo shell
   

5. Create the namespace gpu-operator:

   kubectl create ns gpu-operator
   

6. Add the label pod-security.kubernetes.io/enforce=privileged to the new namespace:

   kubectl label --overwrite ns gpu-operator pod-security.kubernetes.io/enforce=privileged
   

7. Add Nvidia support via helm:

   helm repo add nvidia https://helm.ngc.nvidia.com/nvidia && helm repo update
   

8. Add the necessary toolkit settings to enable Nvidia support:

   helm install --wait --generate-name -n gpu-operator --create-namespace nvidia/gpu-operator --set driver.enabled=false --set operator.defaultRuntime=containerd --set toolkit.env[0].name=CONTAINERD_CONFIG --set toolkit.env[0].value=/etc/k0s/containerd.d/nvidia.toml --set toolkit.env[1].name=CONTAINERD_SOCKET --set toolkit.env[1].value=/run/k0s/containerd.sock --set toolkit.env[2].name=CONTAINERD_RUNTIME_CLASS --set toolkit.env[2].value=nvidia
   

For systems with one or more GPUs, a Kubernetes label is required. The following labels are provided by default for cloud based installations:

• AWS: kubernetes.io/hostname=ip-{EC2 Internal IP Address}.ec2.internal. For example, if the internal IP address is 127.31.30.77, then the Kubernetes label is kubernetes.io/hostname=ip-172-31-70-77.ec2.internal.
• Azure: kubernetes.io/hostname={HOSTNAME}. For example, if the hostname is wallaroo-linux, then the label is kubernetes.io/hostname=wallaroo-linux.
• GCP: kubernetes.io/hostname={HOSTNAME}. For example, if the hostname is wallaroo-linux, then the label is kubernetes.io/hostname=wallaroo-linux.

For details on model deployment with GPUs, see Deployment Configuration GPU Support.

To add a custom label to the installation:

1. SSH into the Linux installation as the same user that performed the installation.

2. Access the Wallaroo Shell by executing the following command in same directory where the Wallaroo installation script was downloaded in the step Install Embedded Wallaroo.

   sudo ./wallaroo shell
   

3. Get the current nodes and labels via the following command. Note the name of the single node - typically this is the same as the VM host name.

   kubectl get nodes --show-labels
   

   This returns an output like the following:

   NAME                          STATUS   ROLES           AGE     VERSION       LABELS
   wallaroo-linux-install   Ready    control-plane   6d23h   v1.30.5+k0s   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kots.io/embedded-cluster-role-0=controller,kots.io/embedded-cluster-role=total-1,kubernetes.io/arch=amd64,kubernetes.io/hostname=wallaroo-linux-install,kubernetes.io/os=linux,node-role.kubernetes.io/control-plane=true,node.k0sproject.io/role=control-plane
   

4. Add a label to the single nodes via the kubectl label nodes command in the following format, replacing the following:

   • $NODENAME: The name of the node. Typically this is the same as the VM’s host name.
   • $LABEL: The label name.
   • $VALUE: The label value.

   kubectl label nodes $NODENAME $LABEL=$VALUE
   

   For example, for the following values, the command is:

   • $NODENAME: wallaroo-linux-install
   • $LABEL: wallaroo.ai/accelerator
   • $VALUE: gpu

   label nodes wallaroo-linux-install wallaroo.ai/accelerator=gpu
   

Note that the deployment configuration label used by the Wallaroo SDK follows the Kubernetes Labels and Selectors format of label:value. For example, a typical deployment configuration for the above label would be:

import wallaroo

# create the deployment configuration with 4 cpus, 3 Gi RAM, and 1 GPU with the deployment label
wallaroo.DeploymentConfigBuilder()
    .cpus(4)
    .memory('3Gi')
    .gpus(1)
    .deployment_label('wallaroo.ai/accelerator:gpu')
    .build()

For more details on model deployment configurations, see Deployment Configuration with the Wallaroo SDK.

The Installer Is Unable to Autodetect the IP Address

If the IP address is not auto-detected by the installation script, use the hostname command as below to discover the true internal IP address and use that to choose which interface to select.

hostname -i

10.138.0.59

Then use the installation script:

curl -s https://kurl.sh/wallaroo-2024-4 | sudo bash
⚙  Running install with the argument(s):

The installer was unable to automatically detect the private IP address of this machine.
Please choose one of the following network interfaces:
[0] ens4        10.138.0.59
[1] vxlan.calico        10.244.154.64
Enter desired number (0-1): 0
The installer will use network interface 'ens4' (with IP address '10.138.0.59').

dpkg error: dpkg frontend lock was locked by another process

If the error dpkg: error: dpkg frontend lock was locked by another process appears after starting or restarting itself, the Linux installation updating itself after boot. In this scenario, apt or dpkg conflict with Kurl.

If these scenarios appears, verify with a ps command as below. If you see apt updating, wait until it’s finished and then try running Kurl again.

dpkg: error: dpkg frontend lock was locked by another process with pid 41232
Note: removing the lock file is always wrong, and can end up damaging the
locked area and the entire system. See <https://wiki.debian.org/Teams/Dpkg/FAQ>.
An error occurred on line 2866

Test with the ps command:

ps auwx|grep apt

As seen below, apt is running. Wait until finished.

root       41163  0.0  0.0   2892  1536 ?        Ss   20:27   0:00 /bin/sh /usr/lib/apt/apt.systemd.daily install
root       41173  0.0  0.0   2892  1536 ?        S    20:27   0:00 /bin/sh /usr/lib/apt/apt.systemd.daily lock_is_held install
root       43801 92.0  0.1 174852 159604 ?       SN   20:28   0:00 /usr/bin/python3 /usr/lib/update-notifier/apt-check --human-readable
mnp        43826  0.0  0.0   7008  2048 pts/1    S+   20:28   0:00 grep --color=auto apt