Wallaroo Enterprise Azure integration Overview
An overview of the Wallaroo Enterprise for Azure Cloud
Before starting, verify that the cluster hosting the Wallaroo service meets the installation prerequisites.
To install Wallaroo in your Microsoft Azure environment, the following prerequisites must be met:
az
application to your Microsoft Azure account.containerd
driver as the default.Standard_D8as_v5
.Instructions for creating a Kubernetes cluster in Microsoft Azure are detailed in Quickstart: Deploy an Azure Kubernetes Service (AKS) cluster using Azure portal. Modify the instructions there with the following recommendations:
Nodepool | Taints | Labels | Description |
---|---|---|---|
general | N/A | wallaroo.ai/node-purpose: general | For general Wallaroo services. No taints are applied to this nodepool to allow any process not assigned with a deployment label to run in this space. |
persistent | wallaroo.ai/persistent=true:NoSchedule | wallaroo.ai/node-purpose: persistent | For Wallaroo services with a persistentVolume settings, including JupyterHub, Minio, etc. |
pipesx86 | wallaroo.ai/pipelines=true:NoSchedule | wallaroo.ai/node-purpose: pipelines | For deploying pipelines for default x86 architectures. The taints and label must be applied to any nodepool used for model deployments. |
For nodepools with AI Accelerators, GPUs, ARM support, etc, see the Taints and Tolerations Guide for how to set up nodepools with the proper taints and labels. Note that Azure AKS nodepool names have naming and length limitations as detailed in Manage system node pools in Azure Kubernetes Service (AKS).
The following sample script is provided as an example of setting up the AKS nodepools. Modify the following variables to match your organization’s requirements. The script is available from the following link:
#!/bin/bash
WALLAROO_RESOURCE_GROUP=example-resource-group
WALLAROO_GROUP_LOCATION=eastus
WALLAROO_CLUSTER=wallaroo # rename as required
WALLAROO_SKU_TYPE=Basic
WALLAROO_VM_SIZE=Standard_D8as_v5
PIPELINE_VM_SIZE=Standard_D8s_v5
az group create --name $WALLAROO_RESOURCE_GROUP --location $WALLAROO_GROUP_LOCATION
az aks create \
--resource-group $WALLAROO_RESOURCE_GROUP \
--name $WALLAROO_CLUSTER \
--node-count 1 \
--min-count 1 \
--max-count 3 \
--enable-private-cluster \
--enable-cluster-autoscaler \
--generate-ssh-keys \
--node-vm-size $WALLAROO_VM_SIZE \
--nodepool-name general \
--nodepool-labels wallaroo.ai/node-purpose=general \
--kubernetes-version=1.29.4 \
--location $WALLAROO_GROUP_LOCATION
az aks nodepool add \
--resource-group $WALLAROO_RESOURCE_GROUP \
--cluster-name $WALLAROO_CLUSTER \
--mode User \
--name persistent \
--node-count 1 \
--min-count 1 \
--max-count 3 \
--enable-cluster-autoscaler \
--node-vm-size $WALLAROO_VM_SIZE \
--no-wait \
--labels wallaroo.ai/node-purpose=persistent \
--node-taints wallaroo.ai/persistent=true:NoSchedule \
--zones 1
az aks nodepool add \
--resource-group $WALLAROO_RESOURCE_GROUP \
--cluster-name $WALLAROO_CLUSTER \
--mode User \
--name pipesx86 \
--node-count 2 \
--min-count 0 \
--max-count 3 \
--enable-cluster-autoscaler \
--node-vm-size $PIPELINE_VM_SIZE \
--no-wait \
--labels wallaroo.ai/node-purpose=pipelines \
--node-taints wallaroo.ai/pipelines=true:NoSchedule
# ARM pipelines - optional, for model deployment- optional
# az aks nodepool add \
# --resource-group $WALLAROO_RESOURCE_GROUP \
# --cluster-name $WALLAROO_CLUSTER \
# --mode User \
# --name pipesarm \
# --node-count 0 \
# --min-count 0 \
# --max-count 3 \
# --enable-cluster-autoscaler \
# --node-vm-size Standard_D8plds_v5 \
# --no-wait \
# --labels wallaroo.ai/node-purpose=pipelines \
# --node-taints wallaroo.ai/pipelines=true:NoSchedule
# ARM general - optional, for model packaging and conversion to ARM
# az aks nodepool add \
# --resource-group $WALLAROO_RESOURCE_GROUP \
# --cluster-name $WALLAROO_CLUSTER \
# --mode User \
# --name generalarm \
# --node-count 0 \
# --min-count 0 \
# --max-count 3 \
# --enable-cluster-autoscaler \
# --node-vm-size Standard_D8plds_v5 \
# --no-wait \
# --labels wallaroo.ai/node-purpose=general
# GPU nodepools - optional, for model deployment with GPUs
# az aks nodepool add \
# --resource-group $WALLAROO_RESOURCE_GROUP \
# --cluster-name $WALLAROO_CLUSTER \
# --mode User \
# --name pipesa100 \
# --node-count 0 \
# --min-count 0 \
# --max-count 3 \
# --enable-cluster-autoscaler \
# --node-vm-size Standard_NC24ads_A100_v4 \
# --no-wait \
Wallaroo installations are available via two methods:
kots
based installations.helm
based installations.Before installing or upgrading Wallaroo, the administrative node managing the Kubernetes cluster will require these tools.
1.107.2
helm
: Install Helmkrew
: Install Krewkrew preflight
and krew support-bundle
. Install with the following commands:kubectl krew install support-bundle
kubectl krew install preflight
The following are quick guides for installing kubectl
for macOS.
To install kubectl
on a macOS system using Homebrew:
Issue the brew install
command:
brew install kubectl
Verify the installation:
kubectl version --client
Wallaroo kots
based installations are broken up into two formats:
The following are quick guides for installing kots
for different operating systems. For more details, see the instructions for your specific environment.
IMPORTANT NOTE
As of this time, Wallaroo requires
kots
version 1.107.2. Please verify that version is installed before starting the Wallaroo installation process.
Install kots
by downloading the script and piping it into the bash
shell, assuming curl
is installed:
curl https://kots.io/install/1.107.2 | REPL_USE_SUDO=y bash
For instructions on updating the kots
version for the Wallaroo Ops installation, see Updating KOTS.
A manual method to install KOTS is:
Download from https://github.com/replicatedhq/kots/releases/tag/v1.107.2. Linux and MacOS are supported.
Unpack the release
Rename the kots executable to kubectl kots.
Copy the renamed kubectl kots to anywhere on the PATH
.
Next, verify successful installation.
~ kubectl kots version
Replicated KOTS 1.107.2
To automatically install Wallaroo into the namespace wallaroo
, specify the administrative password and the license file during the installation as in the following format with the following variables:
NAMESPACE
: The namespace for the Wallaroo Enterprise install, typically wallaroo
.LICENSEFILE
: The location of the Wallaroo Enterprise license file.SHAREDPASSWORD
: The password of for the Kots Administrative Dashboard.kubectl kots install wallaroo/2024-1 -n $NAMESPACE --license-file $LICENSEFILE --shared-password $SHAREDPASSWORD
For example, the following settings translate to the following install command:
NAMESPACE
: wallaroo
.LICENSEFILE
: myWallaroolicense.yaml
SHAREDPASSWORD
: snugglebunnies
kubectl kots install wallaroo/2024-1 -n wallaroo --license-file myWallaroolicense.yaml --shared-password wallaroo
The Interactive Install process allows users to adjust the configuration settings before Wallaroo is deployed. It requires users be able to access the Kots Administrative Dashboard through a browser, typically on port 8800
.
IMPORTANT NOTE: Users who install Wallaroo through another node such as in the single node installation can port use SSH tunneling to access the Kots Administrative Dashboard. For example:
ssh IP -L8800:localhost:8800
Install the Wallaroo Enterprise Edition using kots install wallaroo/2024-1
, specifying the namespace to install Wallaroo into. For example, if wallaroo
is the namespace, then the command is:
kubectl kots install wallaroo/2024-1 --namespace wallaroo
Wallaroo Enterprise Edition will be downloaded and installed into your Kubernetes environment in the namespace specified. When prompted, set the default password for the Wallaroo environment. When complete, Wallaroo Enterprise Edition will display the URL for the Admin Console, and how to end the Admin Console from running.
• Deploying Admin Console
• Creating namespace ✓
• Waiting for datastore to be ready ✓
Enter a new password to be used for the Admin Console: •••••••••••••
• Waiting for Admin Console to be ready ✓
• Press Ctrl+C to exit
• Go to http://localhost:8800 to access the Admin Console
To relaunch the Kots Administrative Dashboard and make changes or updates, use the following command:
kubectl kots admin-console --namespace wallaroo
Once Wallaroo has been installed for the first time, we can perform initial configuration.
If Wallaroo Edition has not started, launch it with the following command:
❯ kubectl kots admin-console --namespace wallaroo
• Press Ctrl+C to exit
• Go to http://localhost:8800 to access the Admin Console
Enter the Kots Administrative Console address into a browser. You will be prompted for the default password as set in the step above. Enter it and select Log in.
Upload your license file. If the license file was already selected from the Automated installation, this step is skipped.
The Kots Administrative Console will run the preflight checks to verify that all of the minimum requirements are met. This may take a few minutes. If there are any issues, Wallaroo can still be launched but may not function properly. When ready, select Continue.
The Configure Wallaroo page will be displayed which allows you to customize your Wallaroo environment. The following are the minimum required settings.
kubectl-port forward
or some other means is required to access them. If all work will be done in-cluster, select this option.Once complete, scroll to the bottom of the Config page and select Deploy.
At this point, continue to Required Installation Configurations for the required configuration settings.
By default, the following taints and labels are applied to nodepools used for Wallaroo installations.
Nodepool | Taints | Labels | Description |
---|---|---|---|
general | N/A | wallaroo.ai/node-purpose: general | For general Wallaroo services. No taints are applied to this nodepool to allow any process not assigned with a deployment label to run in this space. |
persistent | wallaroo.ai/persistent=true:NoSchedule | wallaroo.ai/node-purpose: persistent | For Wallaroo services with a persistentVolume settings, including JupyterHub, Minio, etc. |
pipesx86 | wallaroo.ai/pipelines=true:NoSchedule | wallaroo.ai/node-purpose: pipelines | For deploying pipelines for default x86 architectures. The taints and label must be applied to any nodepool used for model deployments. |
The specific nodepool names may differ based on your cloud services naming requirements; check with the cloud services provider for the nodepool name requirements and adjust as needed.
For organizations that use custom Kubernetes taints and labels for their nodepools, the tolerations and node selectors for Wallaroo services will have to be modified to match. For full details, see the Custom Taints and Labels Guide.
To modify the tolerations and node selector labels to match the taints and labels set for the nodepools:
Node Affinities:
Node affinity type key: Verify that the node affinity key matches the label for the nodepools.
Engine affinity value: Set the engine affinity - the affinity used for pipeline deployment - to match the label.
Taints and Tolerations. Set the custom tolerations to match the taints applied to the nodepools, and the node selectors to match the labels used for the nodepools.
Node Selectors: Update the node selector to match the custom nodepools labels for each service.
The following software is required for the client with administrative access to the Kubernetes cluster that will host Wallaroo via Helm.
helm
: Install Helmkrew
: Install Krewkrew preflight
and krew support-bundle
. Install with the following commands:kubectl krew install support-bundle
kubectl krew install preflight
The following details how to install Wallaroo via helm
.
Members of the Wallaroo support staff will provide the following information:
The following steps are used with these command and configuration files to install Wallaroo Enterprise via Helm.
The first step in the Wallaroo installation process via Helm is to connect to the Kubernetes environment that will host the Wallaroo Enterprise instance and login to the Wallaroo container registry through the command provided by the Wallaroo support staff. The command will take the following format, replacing $YOURUSERNAME
and $YOURPASSWORD
with the respective username and password provided.
helm registry login registry.replicated.com --username $YOURUSERNAME --password $YOURPASSWORD
Preflight verification is performed with the following command format. The variables $LICENSE_CHANNEL
and $VERSION
is supplied by your Wallaroo support representative.
helm template oci://registry.replicated.com/wallaroo/$LICENSE_CHANNEL/wallaroo --version $VERSION | kubectl preflight -
For example, the $LICENSE_CHANNEL=2024-1
of ee
and the $VERSION=2024.1.0-5097
helm template oci://registry.replicated.com/wallaroo/2024-1/wallaroo --version 2024.1.0-5097 | kubectl preflight -
This displays the Preflight Checks report.
The following commands are available:
s
: Save the report to a text file as the file preflight-checks-results-DATETIME.txt
. For example: preflight-checks-results-2024-03-19T13_30_41.txt
.q
: Exit the preflight report.Up Arrow
or Down Arrow
: Scroll through the preflight elements and view the report details.The following example shows a successful preflight test.
Preflight Checks Preflight Checks
Check PASS
Title: Required Kubernetes Version
Message: Your cluster meets the recommended and required versions of Kubernetes.
------------
Check PASS
Title: Container Runtime
Message: Containerd container runtime was found.
------------
Check PASS
Title: Check Kubernetes environment.
Message: GKE is a supported distribution
------------
Title: Cluster CPU Resources
Message:
------------
Check PASS
Title: Cluster Resources
Message: Cluster resources are satisfactory
------------
Check PASS
Title: Every node in the cluster must have at least 12Gi of memory
Message: All nodes have at least 12 GB of memory capacity
------------
Check PASS
Title: Every node in the cluster must have at least 8 cpus allocatable.
Message: All nodes have at least 8 CPU capacity
------------
The following instructions detail how to install Wallaroo Enterprise via Helm for Kubernetes cloud environments such as Microsoft Azure, Amazon Web Service, and Google Cloud Platform.
apilb.serviceType
and edgelb.serviceType
settings have the following effects depending on whether they are installed on single node Linux installations, or part of a cloud Kubernetes installation.
Setting | Single Node Linux | Cloud Kubernetes |
---|---|---|
Internal Only Connections | ClusterIP | ClusterIP |
External Connections | NodePort | LoadBalancer |
Refer to the instructions for environment host for details on IP address allocation and support.
With the preflight checks and prerequisites met, Wallaroo can be installed via Helm through the following process:
Create namespace. By default, the namespace wallaroo
is used:
kubectl create namespace wallaroo
Set the new namespace as the current namespace:
kubectl config set-context --current --namespace wallaroo
Set the TLS certificate secret in the Kubernetes environment:
Create the certificate and private key. It is recommended to name it after the domain name of your Wallaroo instance. For example: wallaroo.example.com
. For production environments, organizations are recommended to use certificates from their certificate authority. Note that the Wallaroo SDK will not connect from an external connection without valid certificates. For more information on using DNS settings and certificates, see the Wallaroo DNS Integration Guide.
Create the Kubernetes secret from the certificates created in the previous step, replacing $TLSCONFIG
with the name of the Kubernetes secret, with the certificate file $TLSCERT
and the private key $TLSKEY
. Store the secret name for a the step Configure local values file.
kubectl create secret tls $TLSCONFIG --cert=$TLSCERT --key=$TLSKEY
For example, if $TLSCONFIG
is cust-cert-secret
with the certificate $TLSCERT
is fullchain.pem
and key $TLSKEY
is privkey.pem
, then the command would be translated as
kubectl create secret tls cust-cert-secret --cert=fullchain.pem --key=privkey.pem
A default Helm install of Wallaroo contains various default settings. The local values file overwrites values based on the organization needs. The following represents the minimum mandatory values for a Wallaroo installation using certificates and the default LoadBalancer for a cloud Kubernetes cluster. The configuration details below is saved as values.yaml
for these examples.
Note the following required settings:
domainPrefix
and domainSuffix
: Used to set the DNS settings for the Wallaroo instance. For more information, see the Wallaroo DNS Integration Guide.custTlsSecretName
: Specify the Kubernetes secret created in the previous step. External connections through the Wallaroo SDK require valid certificates.apilb
: Sets the apilb
service options including the following:serviceType: LoadBalancer
: Uses the default LoadBalancer setting for the Kubernetes cloud service the Wallaroo instance is installed into. Replace with the specific service connection settings as required.external_inference_endpoints_enabled: true
: This setting is required for performing external SDK inferences to a Wallaroo instance. For more information, see the Wallaroo Model Endpoints Guidekubernetes_distribution
: The cloud provider used. Each cloud provider has their own methods of handling storage classes, LoadBalancers (LB), etc. The following values are available; if your selected provider or distribution is not on the list, contact your Wallaroo Support representative for more options.aks
: Microsoft Azure Kubernetes Serviceeks
: Amazon Elastic Kubernetes Servicegke
: Google Kubernetes Engineoke
: Oracle Cloud Infrastructure Container Engine for Kuberneteskurl
: On premise aka stand alone Linux installations.ingress_mode
: How the Wallaroo instance is reached through the Kubernetes network settings. Options include:internal
(Default): An internal cloud load balancer and associated resources are created. Network users outside the Kubernetes cluster – but on the same internal network – can connect directly using DNS names, and do not need to use port forward or related configurations.external
: An external, Internet-facing cloud load balancer, public IP, and associated resources are created. This is highly discouraged. Public DNS is also required. This is the default for Wallaroo Community Edition.embedded
: A direct connection to the host. This option is available for embedded installs on “bare metal” or single node Linux installs.none
: Services are local to the Kubernetes cluster. kubectl-port forward
or some other means is required to access them. If all work will be done in-cluster, select this option.dashboard
: The name displayed when users login to the Wallaroo Ops center Dashboard. For example, “My Company” or “Sales Division”, etc.The following example shows the minimum required options.
This example shows the uncommented keys for the minimum required settings and additional commented optional settings. For full details on helm
values for Wallaroo installations, see the Wallaroo Helm Reference Guides.
domainPrefix: "" # optional if using a DNS Prefix
domainSuffix: "wallaroo.example.com"
custTlsSecretName: cust-cert-secret
apilb:
serviceType: LoadBalancer
external_inference_endpoints_enabled: true
ingress_mode: internal # internal (Default), external, embedded, or none
dashboard:
clientName: "Wallaroo Helm Example" # Insert the name displayed in the Wallaroo Dashboard
kubernetes_distribution: "" # Required. One of: aks, eks, gke, oke, or kurl.
# for edge and multicloud model publishing and deployment
# ociRegistry: #Sets the Edge Server registry information.
# enabled: true #enables the Edge Server registry information. true enables it, false disables it.
# registry: # The registry url. For example: reg.big.corp:3579
# repository: # The repository within the registry. This may include the cloud account, or the full path where the Wallaroo published pipelines should be kept. For example: account123/wallaroo/pipelines.
# email: # Optional field to track the email address of the registry credential.
# username: # The username to the registry. This may vary based on the provider. For example, GCP Artifact Registry with service accounts uses the username _json_key_base64 with the password as a base64 processed token of the credential information.
# password: #The password or token for the registry service.
# for edge and multicloud deployment observability
# edgelb:
# serviceType: LoadBalancer
# enabled: true
# opscenterHost: mitch4.edge.wallaroocommunity.ninja
# The nodeSelector and tolerations for all components
# This does not apply to nats, fluent-bit, or minio so needs to be applied separately
# nodeSelector:
# wallaroo.ai/reserved: true
# tolerations:
# - key: "wallaroo.ai/reserved"
# operator: "Exists"
# effect: "NoSchedule"
# To change the pipeline taint or nodeSelector,
# best practice is to change engine, enginelb, and engineAux
# together unless they will be in different pools.
# engine:
# nodeSelector:
# wallaroo.ai/node-purpose: pipelines
# tolerations:
# - key: "wallaroo.ai/pipelines"
# operator: "Exists"
# effect: "NoSchedule"
# enginelb:
# nodeSelector:
# wallaroo.ai/node-purpose: pipelines
# tolerations:
# - key: "wallaroo.ai/pipelines"
# operator: "Exists"
# effect: "NoSchedule"
# engineAux:
# nodeSelector:
# wallaroo.ai/node-purpose: pipelines
# tolerations:
# - key: "wallaroo.ai/pipelines"
# operator: "Exists"
# effect: "NoSchedule"
# For each service below, adjust the disk size and resources as required.
# If the nodeSelector or tolerations are changed for one service,
# the other services nodeSelector and tolerations **must** be changed to match
#
#
# plateau:
# diskSize: 100Gi
# resources:
# limits:
# memory: 4Gi
# cpu: 1000m
# requests:
# memory: 128Mi
# cpu: 100m
# nodeSelector:
# wallaroo.ai/node-purpose: persistent
# tolerations:
# - key: "wallaroo.ai/persistent"
# operator: "Exists"
# effect: "NoSchedule"
# Jupyter has both hub and lab nodeSelectors and tolerations
# They default to the same persistent pool, but can be assigned to different ones
# jupyter:
# nodeSelector: # Node placement for Hub administrative pods
# wallaroo.ai/node-purpose: persistent
# tolerations:
# - key: "wallaroo.ai/persistent"
# operator: "Exists"
# effect: "NoSchedule"
# labNodeSelector: # Node placement for Hub-spawned jupyterlab pods
# wallaroo.ai/node-purpose: persistent
# labTolerations:
# - key: "wallaroo.ai/persistent"
# operator: "Exists"
# effect: "NoSchedule"
# memory:
# limit: "4" # Each Lab - memory limit in GB
# guarantee: "2" # Each Lab - lemory guarantee in GB
# cpu:
# limit: "2.0" # Each Lab - fractional CPU limit
# guarantee: "1.0" # Each Lab - fractional CPU guarantee
# storage:
# capacity: "50" # Each Lab - disk storage capacity in GB
# minio:
# persistence:
# size: 25Gi
# nodeSelector:
# wallaroo.ai/node-purpose: persistent
# tolerations:
# - key: wallaroo.ai/persistent
# operator: "Exists"
# effect: "NoSchedule"
# resources:
# requests:
# memory: 1Gi
# postgres:
# diskSize: 10Gi
# nodeSelector:
# wallaroo.ai/node-purpose: persistent
# tolerations:
# - key: "wallaroo.ai/persistent"
# operator: "Exists"
# effect: "NoSchedule"
# resources:
# limits:
# memory: 2Gi
# cpu: 500m
# requests:
# memory: 512Mi
# cpu: 100m
# Prometheus has the usual persistent options, but also a retention size
# The the size on disk and time can be configured before removing it.
# prometheus:
# storageRetentionSizeGb: "10" # Retain this much data, in GB.
# storageRetentionTimeDays: "15" # When to remove old data. In days.
# nodeSelector:
# wallaroo.ai/node-purpose: persistent
# tolerations:
# - key: "wallaroo.ai/persistent"
# operator: "Exists"
# effect: "NoSchedule"
# resources:
# limits:
# memory: 6Gi
# cpu: 2000m
# requests:
# memory: 512Mi
# cpu: 100m
# nats:
# podTemplate:
# merge:
# spec:
# nodeSelector:
# wallaroo.ai/node-purpose: persistent
# tolerations:
# - key: "wallaroo.ai/persistent"
# operator: "Exists"
# effect: NoSchedule
# wallsvc:
# nodeSelector:
# wallaroo.ai/node-purpose: persistent
# tolerations:
# - key: "wallaroo.ai/persistent"
# operator: "Exists"
# effect: "NoSchedule"
Wallaroo services that have a persistentVolume have the following default nodeSelector label and tolerations:
nodeSelector
wallaroo.ai/node-purpose
persistent
tolerations
For example:
nodeSelector:
wallaroo.ai/node-purpose: persistent
tolerations:
- key: wallaroo.ai/persistent
operator: "Exists"
effect: "NoSchedule"
If the nodeSelector
or tolerations
are changed for any service with a persistentVolume
, all other services must be edited to match.
For additional information on taints and tolerations settings, see the Taints and Labels Guide.
The Wallaroo support representative will provide the installation command for the Helm install that will use the Wallaroo container registry. This assumes that the preflight checks were successful. This command uses the following format:
helm install $RELEASE $REGISTRYURL --version $VERSION --values $LOCALVALUES.yaml
Where:
$RELEASE
: The name of the Helm release. By default, wallaroo
.$REGISTRYURL
: The URl for the Wallaroo container registry service.$VERSION
: The version of Wallaroo to install. For this example, 2024.1.0-5097
.$LOCALVALUES
: The .yaml file containing the local values overrides. For this example, values.yaml
.For example, for the registration wallaroo
the command would be:
helm install wallaroo oci://registry.replicated.com/wallaroo/2024-1/wallaroo --version 2024.1.0-5097 --values values.yaml
If any required elements are missing from the values.yaml
file, an error is displayed. For example, leaving out the kubernetes_distribution
field returns the following:
Warning - kubernetes_distribution must be set in user provided values.yaml
Upon successful installation, notes are published indicating the installed version, where to find documentation, etc.:
NOTES:
.
Welcome to Wallaroo 2024.1.0
1. Deployment Information:
Name: 2024.1.0
Release notes: https://docs.wallaroo.ai/wallaroo-release-notes/wallaroo-release-202401
Version: v2024.1.0-5187
2. Accessing Wallaroo
Documentation: https://docs.wallaroo.ai
Dashboard: https://sample.wallaroocommunity.ninja
Jupyterhub: https://sample.jupyter.wallaroocommunity.ninja
Auth management: https://sample.keycloak.wallaroocommunity.ninja
3. Useful Commands:
- Helm tests are available by using: `helm test wallaroo`.
- External load balancer hostname can be found by using:
kubectl get svc api-lb-ext -n wallaroo -o jsonpath='{.status.loadBalancer.ingress[0].hostname}'
- List Wallaroo namespaces, including pipeline deployments, but not including the main `wallaroo` namespace:
kubectl get namespaces -l wallaroo-managed=true
- In order to change any helm values:
helm upgrade --install wallaroo oci://registry.replicated.com/wallaroo/uat-latest/wallaroo --version v2024.1.0-5187 --values $LOCALVALUES_YAML
4. Uninstall:
1. To uninstall/delete the Wallaroo deployment, run:
kubectl delete ns wallaroo && kubectl delete \
all,secret,configmap,clusterroles,clusterrolebindings,storageclass,crd \
--selector app.kubernetes.io/part-of=wallaroo --selector kots.io/app-slug=wallaroo
2. To delete all pipelines, run:
kubectl delete ns -l wallaroo-managed=true
.
Verify the Installation: Once the installation is complete, verify the installation with the helm test $RELEASE
command. A condensed display uses egrep
to show only the test suite and phase status as follows. Replace wallaroo
with the name of the helm
release used.
helm test wallaroo | egrep 'SUITE:|Phase:'
TEST SUITE: wallaroo-fluent-bit-test-connection
Phase: Succeeded
TEST SUITE: nats-test-request-reply
Phase: Succeeded
TEST SUITE: wallaroo-wallaroo-test-connections-hook
Phase: Succeeded
TEST SUITE: wallaroo-test-objects-hook
Phase: Succeeded
The following will show the full helm
test output with notes.
helm test wallaroo
which displays the following:
NAME: wallaroo
LAST DEPLOYED: Fri May 17 14:00:04 2024
NAMESPACE: wallaroo
STATUS: deployed
REVISION: 2
TEST SUITE: wallaroo-fluent-bit-test-connection
Last Started: Fri May 17 14:04:48 2024
Last Completed: Fri May 17 14:04:51 2024
Phase: Succeeded
TEST SUITE: nats-test-request-reply
Last Started: Fri May 17 14:04:43 2024
Last Completed: Fri May 17 14:04:48 2024
Phase: Succeeded
TEST SUITE: sample-wallaroo-test-connections-hook
Last Started: Fri May 17 14:04:24 2024
Last Completed: Fri May 17 14:04:31 2024
Phase: Succeeded
TEST SUITE: sample-wallaroo-test-objects-hook
Last Started: Fri May 17 14:04:31 2024
Last Completed: Fri May 17 14:04:43 2024
Phase: Succeeded
NOTES:
.
Welcome to Wallaroo 2024.1.0
1. Deployment Information:
Name: 2024.1.0
Release notes: https://docs.wallaroo.ai/wallaroo-release-notes/wallaroo-release-202401
Version: v2024.1.0-5187
2. Accessing Wallaroo
Documentation: https://docs.wallaroo.ai
Dashboard: https://sample.wallaroocommunity.ninja
Jupyterhub: https://sample.jupyter.wallaroocommunity.ninja
Auth management: https://sample.keycloak.wallaroocommunity.ninja
3. Useful Commands:
- Helm tests are available by using: `helm test wallaroo`.
- External load balancer hostname can be found by using:
kubectl get svc api-lb-ext -n wallaroo -o jsonpath='{.status.loadBalancer.ingress[0].hostname}'
- List Wallaroo namespaces, including pipeline deployments, but not including the main `wallaroo` namespace:
kubectl get namespaces -l wallaroo-managed=true
- In order to change any helm values:
helm upgrade --install wallaroo oci://registry.replicated.com/wallaroo/2024-1/wallaroo --version 2024.1.0-5097 --values $LOCALVALUES_YAML
4. Uninstall:
1. To uninstall/delete the Wallaroo deployment, run:
kubectl delete ns wallaroo && kubectl delete \
all,secret,configmap,clusterroles,clusterrolebindings,storageclass,crd \
--selector app.kubernetes.io/part-of=wallaroo --selector kots.io/app-slug=wallaroo
2. To delete all pipelines, run:
kubectl delete ns -l wallaroo-managed=true
.
At this point, the installation is complete and can be accessed through the fully qualified domain names set in the installation process above. Verify that the DNS settings are accurate before attempting to connect to the Wallaroo instance. For more information, see the Wallaroo DNS Integration Guide.
To add the initial users if they were not set up through Helm values, see the Wallaroo Enterprise User Management guide.
Once installed, the following actions are required settings are required to complete the setup process process.
DNS services integration is required for Wallaroo Enterprise to provide access to the various supporting services that are part of the Wallaroo instance. These include:
The following guide is for standard DNS services.
Once integrated, users can access the following services directly from a URL starting with the suffix domain
- this is the domain name where other DNS entries are appended to. For example, if the suffix domain
is sales.example.com
, then the other services would be referenced by https://api.sales.sample.com
, etc.
Note that even when accessing specific Wallaroo services directly that the user must still be authenticated through Wallaroo.
Service | DNS Entry | Description |
---|---|---|
Wallaroo Dashboard | suffix domain | Provides access to a user interface for updating workspaces, pipelines, and models. Also provides access to the integrated JupyterHub service. |
JupyterHub | jupyterhub | Allows the use of Jupyter Notebooks and access to the Wallaroo SDK. |
API | api | Provides access to the Wallaroo API. |
Keycloak | keycloak | Keycloak provides user management to the Wallaroo instance. |
Connections to Wallaroo services are provided as https://service.{suffix domain}. For example, if the domain suffix is wallaroo.example.com
then the URLs to access the various Wallaroo services would be:
https://wallaroo.example.com
https://jupyter.wallaroo.example.com
https://api.wallaroo.example.com
https://keycloak.wallaroo.example.com
The first step is to retrieve the IP address that is set to the DNS services.
For most organizations that install Wallaroo into a cloud based Kubernetes cluster such as Microsoft Azure, AWS, etc the external IP address is tied to Wallaroo LoadBalancer service. This can be retrieved with the kubectl
command as follows:
Retrieve the external IP address for your Wallaroo instance LoadBalancer. For example, this can be performed through the following kubectl
command:
kubectl get svc api-lb-ext -n wallaroo -o jsonpath='{.status.loadBalancer.ingress[0].ip}'
34.83.140.98
Verify that the IP address is the one that is used either for clusters in an internal or external mode.
An alternative to this command to show all services is the following:
kubectl get svc -A
Example Result:
kubectl get svc -A
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default kubernetes ClusterIP 10.25.64.1 <none> 443/TCP 50d
gmp-system alertmanager ClusterIP None <none> 9093/TCP 50d
gmp-system gmp-operator ClusterIP 10.25.70.18 <none> 8443/TCP,443/TCP 50d
kube-system default-http-backend NodePort 10.25.67.38 <none> 80:31378/TCP 50d
kube-system kube-dns ClusterIP 10.25.64.10 <none> 53/UDP,53/TCP 50d
kube-system metrics-server ClusterIP 10.25.67.8 <none> 443/TCP 50d
wallaroo alertconfig ClusterIP 10.25.67.141 <none> 9094/TCP 2m30s
wallaroo alertmanager ClusterIP 10.25.77.167 <none> 9093/TCP 2m30s
wallaroo api-lb ClusterIP 10.25.79.188 <none> 8080/TCP 2m31s
wallaroo api-lb-admin ClusterIP 10.25.70.0 <none> 9901/TCP 2m31s
wallaroo api-lb-ext LoadBalancer 10.25.69.9 34.83.140.98 80:30715/TCP,443:32664/TCP 2m31s
wallaroo dashboard ClusterIP 10.25.72.14 <none> 4567/TCP 2m31s
wallaroo edge-lb LoadBalancer 10.25.69.175 35.247.14.253 8081:32744/TCP,8443:30087/TCP 2m31s
wallaroo wallsvc ClusterIP 10.25.75.74 <none> 3040/TCP 2m30s
Some cloud providers may provide multiple external IP addresses for the api-lb
service. For some services such as Oracle, this is typically a 10 net address, with the second IP address being the actual external address. Verify the IP address before changing DNS settings.
For organizations that install Wallaroo other methods, such as Single Node Linux may find the kubectl get svc api-lb-ext
command only returns the internal IP address.
Depending on the instance, there are different methods of acquiring that IP address. The links below reference difference sources.
Refer to your Wallaroo support representative if further assistance is needed.
Create DNS the following entries based on the list above for the Wallaroo instance’s IP address. Select the cloud environment used for DNS services. Note that DNS services are not tied to the specific cloud platform; check with your organizations setup requirements before proceeding.
The following settings are used with AWS EKE clusters through Route 53. For more information, see Enabling DNS resolution for Amazon EKS cluster endpoints.
Set the following domain names for the Wallaroo instance.
For AWS Clusters:
wallaroo.example.com
.api.wallaroo.example.com
.jupyter.wallaroo.example.com
.keycloak.wallaroo.example.com
.For clusters managed by other cloud services:
A
record, NS
(Name Server) record, SOA
(Start Of Authority) record. For example: wallaroo.example.com
api.wallaroo.example.com
.jupyter.wallaroo.example.com
.keycloak.wallaroo.example.com
.The following settings are used with Microsoft Azure DNS Records. For more information, see Manage DNS records and record sets by using the Azure portal.
A
record, NS
(Name Server) record, SOA
(Start Of Authority) record. For example: wallaroo.example.com
api.wallaroo.example.com
.jupyter.wallaroo.example.com
.keycloak.wallaroo.example.com
.The following settings are used with Google Cloud Platform DNS Records. For more information, see GCP Add, modify, and delete records.
A
record, NS
(Name Server) record, SOA
(Start Of Authority) record. For example: wallaroo.example.com
api.wallaroo.example.com
.jupyter.wallaroo.example.com
.keycloak.wallaroo.example.com
.The following settings are used with Oracle Cloud Platform DNS Records. For more information, see Configuring DNS Servers for Kubernetes Clusters.
A
record, NS
(Name Server) record, SOA
(Start Of Authority) record. For example: wallaroo.example.com
. IMPORTANT NOTE: Oracle OKE clusters have two IP addresses for the load balancer endpoint. Use the public IP address, not the internal 10-net address.api.wallaroo.example.com
.jupyter.wallaroo.example.com
.keycloak.wallaroo.example.com
.Set the following domain names for the Wallaroo instance.
A
record, NS
(Name Server) record, SOA
(Start Of Authority) record. For example: wallaroo.example.com
api.wallaroo.example.com
.jupyter.wallaroo.example.com
.keycloak.wallaroo.example.com
.For organizations that deploy Wallaroo pipelines on edge devices as Wallaroo Servers, see the DNS settings from the Edge Deployment Registry Guide.
User management is managed through the Wallaroo Dashboard, via the Platform Admin Dashboard page. See the Wallaroo User Management for full guides on setting up users, identity providers, and other user configuration options.
The following is an abbreviated guide on setting up new Wallaroo users.
The process includes the following steps:
Obtaining the admin User Credentials
The standard Wallaroo installation creates the user admin
by default and assigns them a randomly generated password. The admin
user credentials are obtained which may be obtained directly from Kubernetes with the following commands, assuming the Wallaroo instance namespace is wallaroo
.
Retrieve Keycloak Admin Username
kubectl -n wallaroo \
get secret keycloak-admin-secret \
-o go-template='{{.data.KEYCLOAK_ADMIN_USER | base64decode }}'
Retrieve Keycloak Admin Password
kubectl -n wallaroo \
get secret keycloak-admin-secret \
-o go-template='{{.data.KEYCLOAK_ADMIN_PASSWORD | base64decode }}'
Creating users is managed through the Platform Admin Dashboard. The following steps are used to create an initial user with the role admin
.
wallaroo.example.com
, the Wallaroo Dashboard is available at https://wallaroo.example.com
.admin
and the password retrieved in Obtaining the admin User Credentials.At this point, users can log in to Wallaroo Dashboard with their provided identities. For guides on setting up Single Sign-On (SSO) and other features, see Wallaroo User Management for full guides on setting up users, identity providers, and other user configuration options.
An overview of the Wallaroo Enterprise for Azure Cloud