Wallaroo Comprehensive Install Guide: OCI Marketplace Ampere VM

How to set up Wallaroo on an Ampere VM in the Oracle Cloud Infrastructure Marketplace

The following guide demonstrates setting up a new Wallaroo installation on a Ampere VM through the the Oracle Cloud Infrastructure (OCI) Marketplace.

Prerequisites

Verify the prerequisites are met before creating the new Wallaroo instance through the OCI Marketplace:

A Wallaroo license. See your Wallaroo support representative. This is stored as a YAML file.
A SSH key (Optional): Oracle instances support pre-generated SSH keys or a new SSH key is generated when the new instance is created in the step Provision Wallaroo Installation from OCI Marketplace.
A Virtual Cloud Network (VCN) with the following ports open. For more details, see Networking Scenarios: Scenario A: Public Subnet, with Step 5 detailing how to open external ports.
- 443
- 30000
- 8081
- 8443
TLS Certificates: These are used to verify the DNS settings. For more information, see Configuring DNS Servers for Kubernetes Clusters and the Wallaroo DNS Configuration Guide. The following DNS requires are required.
- Wallaroo domain: A record, NS (Name Server) record, SOA (Start Of Authority) record. For example: wallaroo.example.com.
- The IP address for the DNS settings are retrieved at the Activate the new Wallaroo instance. process.

The following steps detail:

Provision Wallaroo Installation from OCI Marketplace

The following steps provisions a new Wallaroo installation from the OCI Marketplace.

During this process the following information must be stored:

The generated or used SSH key.
The public IP address set for the new instance.

Login to the OCI Marketplace. From the OCI Marketplace listing Wallaroo AI inference platform - Starter plan (Ampere), select Get App.
Set the following required options. For other options, refer to the Oracle Cloud Infrastructure documentation Creating an Instance.
1. Name: The name of the new Wallaroo instance.
2. Create in compartment: Select the OCI compartment the instance is created in. For more details, see What is a Compartment?.
3. Image and Shape:
  1. Image: The OCI marketplace automatically selects the Wallaroo image.
  2. Shape: Select Change shape and set the following.
    1. Shape Name: VM.Standard.A1.Flex
    2. Number of ocpu: 32 ocpu
    3. Shape: 64gb E4.Flex
4. Primary VNIC information
  1. Primary Network: Select Select existing virtual cloud network as defined in the Prerequisites.
5. Add SSH Keys: Select one of the following.
  1. Select Generate a key pair for me, then select Save private key and Save public key. Save these in a secure location.
  2. Use a pre-created SSH key from the Prerequisites and either Upload public key files (.pub) or Paste public keys.
When finished, select Create.

Wait until the new instance state is Running then continue to activate the new Wallaroo instance.

Activate the Wallaroo Instance

Once the new Wallaroo instance State is Running, the new instance is activated through the following steps.

From the OCI Instances page, select the Public IP for the new Wallaroo instance.
From a browser, access the Kots Administrative Console from the instances public IP address through the following address:
1. http://instance-public-ip:30000
Login with the default password: W@11y$.
Upload your Wallaroo license file.
A preflight check verifies that all of the minimum requirements are met. This may take a few minutes. If there are any issues, Wallaroo can still be launched but may not function properly. When ready, select Continue.
The Configure Wallaroo page will be displayed which allows you to customize your Wallaroo environment. The following are the minimum required settings.
1. Networking Configuration: Set whether the Wallaroo instance is available from outside the Kubernetes cluster, or only from within it. Set the following.
  1. Ingress Mode for Wallaroo Endpoints:
    1. Select Embedded: A direct connection to the host. This option is available for embedded installs on “bare metal” or single node Linux installs.
      1. Enable external URL inference endpoints: Creates pipeline inference endpoints. For more information, see Model Endpoints Guide.
2. DNS
  1. Wallaroo Domain: The domain name of the Wallaroo instance.
3. TLS Certificates
  1. Use custom TLS Certs: Checked
  2. TLS Certificate: Enter your TLS Certificate (.crt or .pem file).
  3. TLS Private Key: Enter your TLS private key (.key or .pem file).

Edge Deployment Configuration

The following optional configurations to enable OCI Registry integration with Wallaroo for deploying models in edge and multi-cloud environments through Wallaroo. For more details, see Inference Anywhere.

To set the Edge Registry Settings through the Kots Administrative Dashboard:

Launch the Kots Administrative Dashboard using the following command, replacing the --namespace parameter with the Kubernetes namespace for the Wallaroo instance:
```
kubectl kots admin-console --namespace wallaroo
```
Open a browser at the URL detailed in the step above and authenticate using the console password set as described in the as detailed in the Wallaroo Install Guides.
From the top menu, select Config then scroll to Edge Deployment.
Enable Provide OCI registry credentials for pipelines.
Enter the following:
1. Registry URL: The address of the registry service. For example: us-west1-docker.pkg.dev.
2. email: The email address of the user account used to authenticate to the service.
3. username: The account used to authenticate to the registry service.
4. password: The password or token used to authenticate to the registry service.
5. To enable edge observability, enable Enable pipelines deployed on the edge to send data back to the OpsCenter.

Once complete, scroll to the bottom of the Config page and select Deploy.

The Wallaroo instance will complete the deployment process. Wait until the Kots Administrative Console status is Ready before proceeding.

Update Kots Administrative Console Password

To change the default Kots Administrative Console Password:

Access the Kots Administrative Console through the URL detailed above.
Authenticate with the current Kots Administrative Console password.
From the upper right corner, select the menu labeled ..., and select Change password.
Enter the current password, then enter the new password and confirm it. When finished, select Change Password.

Setup New Users

When the Wallaroo instance is finished deploying, the users are set up through the following process.

SSH into the Wallaroo instance through the following command:
```
ssh -i $PRIVATESSHKEY ubuntu@wallaroo-instance-ip
```
For example, if the private key is located in the file ~/.ssh/wallaroo-oci.pem, then the command would be:
```
ssh -i ~/.ssh/wallaroo-oci.pem ubuntu@wallaroo-instance-ip
```
Retrieve the admin user password with the following command:
```
sudo wallaroo-admin-password
```
Store the admin password.
Enter exit to terminate the ssh session.
Access the new Wallaroo instance through the public IP address: https://wallaroo-instance-ip.
Login with the username admin and the admin user password.
Select Create Users and enter the following:
1. User Email: The email address for the user. This must be in the valid email address format.
2. Assign Type: Select Admin.
3. Password: Enter the user’s password. This user password be sent to the new user.
4. Temporary or Permanent:
  1. Temporary: The user will be forced to change their login password upon their next login (Recommended).
  2. Permanent: The user will keep their password.
Create any additional users as needed. When finished, select the Wallaroo icon drop down and select Logout.

Once complete, users login in through the new Wallaroo’s public IP address at: https://wallaroo-instance-ip.

For details on deploying models and performing inferences, see the ML Operations guides.

Next Steps

Ready for more? Time to deploy models and start inferencing.

See the Wallaroo 101 for a run through of uploading, deploying, and inferencing on ML models.
For Large Language Models, see the Large Language Models Operations guides.
For edge and multi-cloud deployments, see the Run Anywhere Tutorials for instructions on deploying ML models in any cloud, any architecture, on any platform.

For access to these sample models and a demonstration on using LLMs with Wallaroo:

Contact your Wallaroo Support Representative OR
Schedule Your Wallaroo.AI Demo Today