Uninstall Guides

The following is a short version of the uninstall procedure to remove a previously installed version of Wallaroo. For full details, see the How to Uninstall Wallaroo. These instructions assume administrative use of the Kubernetes command kubectl.

To uninstall a previously installed Wallaroo instance:

1. Delete any Wallaroo pipelines still deployed with the command kubectl delete namespace {namespace}. Typically these are the pipeline name with some numerical ID. For example, in the following list of namespaces the namespace ccfraud-pipeline-21 correspond to the Wallaroo pipeline ccfraud-pipeline. Verify these are Wallaroo pipelines before deleting.

     -> kubectl get namespaces
       NAME    STATUS    AGE
       default    Active    7d4h
       kube-node-lease    Active    7d4h
       kube-public    Active    7d4h
       ccfraud-pipeline-21    Active    4h23m
       wallaroo    Active    3d6h
   
     -> kubectl delete namespaces ccfraud-pipeline-21
   

2. Use the following bash script or run the commands individually. Warning: If the selector is incorrect or missing from the kubectl command, the cluster could be damaged beyond repair. For a default installation, the selector and namespace will be wallaroo.

   #!/bin/bash
   kubectl delete ns wallaroo && \ 
   kubectl delete all,secret,configmap,clusterroles,clusterrolebindings,storageclass,crd \
   --selector app.kubernetes.io/part-of=wallaroo --selector kots.io/app-slug=wallaroo
   

Wallaroo can now be reinstalled into this environment.

AWS Cluster for Wallaroo Enterprise Instructions

The following steps are guidelines to assist new users in setting up their AWS environment for Wallaroo. Feel free to replace these with commands with ones that match your needs.

• AWS Prerequisites

To install Wallaroo in your AWS environment based on these instructions, the following prerequisites must be met:

• Register an AWS account: https://aws.amazon.com/ and assign the proper permissions according to your organization’s needs.
• The Kubernetes cluster must include the following minimum settings:
  • Nodes must be OS type Linux with using the containerd driver.
  • Role-based access control (RBAC) must be enabled.
  • Minimum of 4 nodes, each node with a minimum of 8 CPU cores and 16 GB RAM. 50 GB will be allocated per node for a total of 625 GB for the entire cluster.
  • RBAC is enabled.
  • Recommended Aws Machine type: c5.4xlarge. For more information, see the AWS Instance Types.
• Installed eksctl version 0.101.0 and above.
• If the cluster will utilize autoscaling, install the Cluster Autoscaler on AWS.

• IMPORTANT NOTE
  • Organizations that intend to stop and restart their Kubernetes environment on an intentional or regular basis are recommended to use a single availability zone for their nodes. This minimizes issues such as persistent volumes in different availability zones, etc.
  • Organizations that intend to use Wallaroo Enterprise in a high availability cluster are encouraged to follow best practices including using separate availability zones for redundancy, etc.

EKSCTL Based Instructions

These commands make use of the command line tool eksctl which streamlines the process in creating Amazon Elastic Kubernetes Service clusters for our Wallaroo environment.

The following are used for the example commands below. Replace them with your specific environment settings:

• AWS Cluster Name: wallarooAWS

Create an AWS EKS Cluster

The following eksctl configuration file is an example of setting up the AWS environment for a Wallaroo cluster, including the static and adaptive nodepools. Adjust these names and settings based on your organizations requirements.

This sample YAML file can be downloaded from here:wallaroo_enterprise_aws_install.yaml

Or copied from here:

apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
  name: wallarooAWS
  region: us-east-1
  version: "1.25"

addons:
  - name: aws-ebs-csi-driver

iam:
  withOIDC: true
  serviceAccounts:
  - metadata:
      name: cluster-autoscaler
      namespace: kube-system
      labels: {aws-usage: "cluster-ops"}
    wellKnownPolicies:
      autoScaler: true
    roleName: eksctl-cluster-autoscaler-role

nodeGroups:
  - name: mainpool
    instanceType: m5.2xlarge
    desiredCapacity: 3
    containerRuntime: containerd
    amiFamily: AmazonLinux2
    availabilityZones:
      - us-east-1a
  - name: postgres
    instanceType: m5.2xlarge
    desiredCapacity: 1
    taints:
      - key: wallaroo.ai/postgres
        value: "true"
        effect: NoSchedule
    containerRuntime: containerd
    amiFamily: AmazonLinux2
    availabilityZones:
      - us-east-1a
  - name: engine-lb
    instanceType: c5.4xlarge
    minSize: 1
    maxSize: 3
    taints:
      - key: wallaroo.ai/enginelb
        value: "true"
        effect: NoSchedule
    tags:
      k8s.io/cluster-autoscaler/node-template/label/k8s.dask.org/node-purpose: engine-lb
      k8s.io/cluster-autoscaler/node-template/taint/k8s.dask.org/dedicated: "true:NoSchedule"
    iam:
      withAddonPolicies:
        autoScaler: true
    containerRuntime: containerd
    amiFamily: AmazonLinux2
    availabilityZones:
      - us-east-1a
  - name: engine
    instanceType: c5.2xlarge
    minSize: 1
    maxSize: 3
    taints:
      - key: wallaroo.ai/engine
        value: "true"
        effect: NoSchedule
    tags:
      k8s.io/cluster-autoscaler/node-template/label/k8s.dask.org/node-purpose: engine
      k8s.io/cluster-autoscaler/node-template/taint/k8s.dask.org/dedicated: "true:NoSchedule"
    iam:
      withAddonPolicies:
        autoScaler: true
    containerRuntime: containerd
    amiFamily: AmazonLinux2
    availabilityZones:
      - us-east-1a

Create the Cluster

Create the cluster with the following command, which creates the environment and sets the correct Kubernetes version.

eksctl create cluster -f wallaroo_enterprise_aws_install.yaml

During the process the Kubernetes credentials will be copied into the local environment. To verify the setup is complete, use the kubectl get nodes command to display the available nodes as in the following example:

kubectl get nodes

NAME                                           STATUS   ROLES    AGE     VERSION
ip-192-168-21-253.us-east-2.compute.internal   Ready    <none>   13m     v1.23.8-eks-9017834
ip-192-168-30-36.us-east-2.compute.internal    Ready    <none>   13m     v1.23.8-eks-9017834
ip-192-168-38-31.us-east-2.compute.internal    Ready    <none>   9m46s   v1.23.8-eks-9017834
ip-192-168-55-123.us-east-2.compute.internal   Ready    <none>   12m     v1.23.8-eks-9017834
ip-192-168-79-70.us-east-2.compute.internal    Ready    <none>   13m     v1.23.8-eks-9017834
ip-192-168-37-222.us-east-2.compute.internal   Ready    <none>   13m     v1.23.8-eks-9017834

Install Wallaroo

Organizations that use cloud services such as Google Cloud Platform (GCP), Amazon Web Services (AWS), or Microsoft Azure can install Wallaroo Enterprise through the following process. These instructions also work with Single Node Linux based installations.

Before installation, the following prerequisites must be met:

• Have a Wallaroo Enterprise license file. For more information, you can request a demonstration.
• Set up a cloud Kubernetes environment that meets the requirements. Clusters must meet the following minimum specifications:
  • Minimum number of nodes: 4
  • Minimum Number of CPU Cores: 8
  • Minimum RAM: 16 GB
  • A total of 625 GB of storage will be allocated for the entire cluster based on 5 users with up to four pipelines with five steps per pipeline, with 50 GB allocated per node, including 50 GB specifically for the Jupyter Hub service. Enterprise users who deploy additional pipelines will require an additional 50 GB of storage per lab node deployed.
  • Runtime: containerd is required.
• DNS services for integrating your Wallaroo Enterprise instance. See the DNS Integration Guide for the instructions on configuring Wallaroo Enterprise with your DNS services.

Wallaroo Enterprise can be installed either interactively or automatically through the kubectl and kots applications.

Automated Install

To automatically install Wallaroo into the namespace wallaroo, specify the administrative password and the license file during the installation as in the following format with the following variables:

• NAMESPACE: The namespace for the Wallaroo Enterprise install, typically wallaroo.
• LICENSEFILE: The location of the Wallaroo Enterprise license file.
• SHAREDPASSWORD: The password of for the Wallaroo Administrative Dashboard.

kubectl kots install wallaroo/ee -n $NAMESPACE --license-file $LICENSEFILE --shared-password $SHAREDPASSWORD

For example, the following settings translate to the following install command:

• NAMESPACE: wallaroo.
• LICENSEFILE: myWallaroolicense.yaml
• SHAREDPASSWORD: snugglebunnies

kubectl kots install wallaroo/ee -n wallaroo --license-file myWallaroolicense.yaml --shared-password wallaroo

Interactive Install

The Interactive Install process allows users to adjust the configuration settings before Wallaroo is deployed. It requires users be able to access the Wallaroo Administrative Dashboard through a browser, typically on port 8080.

• IMPORTANT NOTE: Users who install Wallaroo through another node such as in the single node installation can port use SSH tunneling to access the Wallaroo Administrative Dashboard. For example:

  ssh IP -L8800:localhost:8800
  

1. Install the Wallaroo Enterprise Edition using kots install wallaroo/ee, specifying the namespace to install Wallaroo into. For example, if wallaroo is the namespace, then the command is:

   kubectl kots install wallaroo/ee --namespace wallaroo
   

2. Wallaroo Enterprise Edition will be downloaded and installed into your Kubernetes environment in the namespace specified. When prompted, set the default password for the Wallaroo environment. When complete, Wallaroo Enterprise Edition will display the URL for the Admin Console, and how to end the Admin Console from running.

   • Deploying Admin Console
   • Creating namespace ✓
   • Waiting for datastore to be ready ✓
       Enter a new password to be used for the Admin Console: •••••••••••••
     • Waiting for Admin Console to be ready ✓
   
   • Press Ctrl+C to exit
   • Go to http://localhost:8800 to access the Admin Console
   

To relaunch the Wallaroo Administrative Dashboard and make changes or updates, use the following command:

kubectl-kots admin-console --namespace wallaroo

Configure Wallaroo

Once installed, Wallaroo will continue to run until terminated.

Change Wallaroo Administrative Dashboard Password

To change the password to the Wallaroo Administrative Dashboard:

1. From the command line, use the command:

   kubectl kots reset-password -n {namespace}
   

   For example, for default installations where the Kubernetes namespace is wallaroo, the command would be:

   kubectl kots reset-password -n wallaroo
   

   From here, enter the new password.

2. From the Wallaroo Administrative Dashboard:

   1. Login and authenticate with the current password.

   2. From the upper right hand corner, select … to access the menu and select Change password.

      

   3. Enter the current password, then update and verify with the new password.

      

Setup DNS Services

Wallaroo Enterprise requires integration into your organizations DNS services.

The DNS Integration Guide details adding the Wallaroo instance to an organizations DNS services. The following is an abbreviated guide that assumes that certificates were already generated.

1. From the Wallaroo Dashboard, select Config and set the following:

   1. Networking Configuration
      1. Ingress Mode for Wallaroo Endpoints:
         1. None: Port forwarding or other methods are used for access.
         2. Internal: For environments where only nodes within the same Kubernetes environment and no external connections are required.
         3. External: Connections from outside the Kubernetes environment is allowed.
            1. Enable external URL inference endpoints: Creates pipeline inference endpoints. For more information, see Model Endpoints Guide.
   2. DNS
      1. DNS Suffix (Mandatory): The domain name for your Wallaroo instance.
   3. TLS Certificates
      1. Use custom TLS Certs: Checked
      2. TLS Certificate: Enter your TLS Certificate (.crt file).
      3. TLS Private Key: Enter your TLS private key (.key file).
   4. Other settings as desired.

   

2. Once complete, scroll to the bottom of the Config page and select Save config.

3. A pop-up window will display The config for Wallaroo Enterprise has been updated.. Select Go to updated version to continue.

4. From the Version History page, select Deploy. Once the new deployment is finished, you will be able to access your Wallaroo services via their DNS addresses.

To verify the configuration is complete, access the Wallaroo Dashboard through the suffix domain. For example if the suffix domain is wallaroo.example.com then access https://wallaroo.example.com in a browser and verify the connection and certificates.

Setup Users

User management is handled through the Wallaroo instance Keycloak service. See the Wallaroo User Management for full guides on setting up users, identity providers, and other user configuration options. This step must be completed before using Wallaroo.

The following is an abbreviated guide on setting up new Wallaroo users.

At least one user must be created before using Wallaroo.

Accessing The Wallaroo Keycloak Dashboard

Enterprise customers may access their Wallaroo Keycloak dashboard by navigating to https://keycloak.<suffix>, depending on their choice domain suffix supplied during installation.

Obtaining Administrator Credentials

The standard Wallaroo installation creates the user admin by default and assigns them a randomly generated password. The admin user credentials are obtained which may be obtained directly from Kubernetes with the following commands, assuming the Wallaroo instance namespace is wallaroo.

• Retrieve Keycloak Admin Username

  kubectl -n wallaroo \
  get secret keycloak-admin-secret \
  -o go-template='{{.data.KEYCLOAK_ADMIN_USER | base64decode }}'
  

• Retrieve Keycloak Admin Password

  kubectl -n wallaroo \
  get secret keycloak-admin-secret \
  -o go-template='{{.data.KEYCLOAK_ADMIN_PASSWORD | base64decode }}'
  

Accessing the User Management Panel

In the Keycloak Administration Console, click Manage -> Users in the left-hand side menu. Click the View all users button to see existing users. This will be under the host name keycloak.$WALLAROO_SUFFIX. For example, if the $WALLAROO_SUFFIX is wallaroo.example.com, the Keycloak Administration Console would be keycloak.wallaroo.example.com.

Adding Users

To add a user through the Keycloak interface:

1. Click the Add user button in the top-right corner.

2. Enter the following:

   

   1. A unique username and email address.
   2. Ensure that the Email Verified checkbox is checked - Wallaroo does not perform email verification.
   3. Under Required User Actions, set Update Password so the user will update their password the next time they log in.

3. Click Save.

4. Once saved, select Credentials tab, then the Set Password section, enter the new user’s desired initial password in the Password and Password Confirmation fields.

   

5. Click Set Password. Confirm the action when prompted. This will force the user to set their own password when they log in to Wallaroo.

6. To log into the Wallaroo dashboard, log out as the Admin user and login to the Wallaroo Dashboard as a preconfigured user or via SSO.