Wallaroo MLOps API Essentials Guide

This tutorial and the assets are available as part of the Wallaroo Tutorials repository.

Wallaroo MLOps API Tutorial

The Wallaroo MLOps API allows organizations to submit requests to their Wallaroo instance to perform such actions as:

• Create a new user and invite them to the instance.
• Create workspaces and list their configuration details.
• Upload a model.
• Deploy and undeploy a pipeline.

The following examples will show how to submit queries to the Wallaroo MLOps API and the types of responses returned.

References

The following references are available for more information about Wallaroo and the Wallaroo MLOps API:

• Wallaroo Documentation Site: The Wallaroo Documentation Site
• Wallaroo MLOps API Documentation from a Wallaroo instance: A Swagger UI based documentation is available from your Wallaroo instance at https://{Wallaroo Prefix.}api.{Wallaroo Suffix}/v1/api/docs. For example, if the Wallaroo Instance suffix is example.wallaroo.ai with the prefix {lovely-rhino-5555.}, then the Wallaroo MLOps API Documentation would be available at https://lovely-rhino-5555.api.example.wallaroo.ai/v1/api/docs. Note the . is part of the prefix.
• For another example, a Wallaroo Enterprise users who do not use a prefix and has the suffix wallaroo.example.wallaroo.ai, the the Wallaroo MLOps API Documentation would be available at https://api.wallaroo.example.wallaroo.ai/v1/api/docs. For more information, see the Wallaroo Documentation Site.

IMPORTANT NOTE: The Wallaroo MLOps API is provided as an early access features. Future iterations may adjust the methods and returns to provide a better user experience. Please refer to this guide for updates.

Prerequisites

• An installed Wallaroo instance.
• The following Python libraries installed:
  • requests
  • json
  • wallaroo: The Wallaroo SDK. Included with the Wallaroo JupyterHub service by default.
  • pandas: Pandas, mainly used for Pandas DataFrame. Included with the Wallaroo JupyterHub service by default.
  • pyarrow: PyArrow for Apache Arrow support. Included with the Wallaroo JupyterHub service by default.
  • polars: Polars for DataFrame with native Apache Arrow support

OpenAPI Steps

The following demonstrates how to use each command in the Wallaroo MLOps API, and can be modified as best fits your organization’s needs.

Import Libraries

For the examples, the Python requests library will be used to make the REST HTTP(S) connections.

import wallaroo
from wallaroo.object import EntityNotFoundError
import pandas as pd
import os

import pyarrow as pa

import requests
from requests.auth import HTTPBasicAuth

import json

# used to display dataframe information without truncating
from IPython.display import display
pd.set_option('display.max_colwidth', None)

Notes About This Guide

The following guide was established with set names for workspaces, pipelines, and models. Note that some commands, such as creating a workspace, will fail if another workspace is already created with the same name. Similar, if a user is already established with the same email address as in the examples below, etc.

To reduce errors, the following variables are declared. Please change them as required to avoid issues in an established Wallaroo environment.

For wallarooPrefix = "YOUR PREFIX." and wallarooSuffix = "YOUR SUFFIX", enter the prefix and suffix for your Wallaroo instance DNS name. If the prefix instance is blank, then it can be wallarooPrefix = "". Note that the prefix includes the . for proper formatting.

## Sample Variables List

new_user = "john.hansarick@wallaroo.ai"
new_user_password = "Snugglebunnies"

example_workspace_name = "apiworkspaces"
model_name = "apimodel"
model_file_name = "./models/ccfraud.onnx"

stream_model_name = "apiteststreammodel"
stream_model_file_name = "./models/ccfraud.onnx"

empty_pipeline_name="pipelinenomodel"

model_pipeline_name="pipelinemodels"

example_copied_pipeline_name="copiedmodelpipeline"

wallarooPrefix = "YOUR PREFIX."
wallarooSuffix = "YOUR SUFFIX"

# Retrieving login data through credential file
f = open('./creds.json')
login_data = json.load(f)

Retrieve Credentials

Through Keycloak

Wallaroo comes pre-installed with a confidential OpenID Connect client. The default client is api-client, but other clients may be created and configured.

Confidential clients require its secret to be supplied when requesting a token. Administrators may obtain their API client credentials from Keycloak from the Keycloak Service URL as listed above and the prefix /auth/admin/master/console/#/realms/master/clients.

For example, if the Wallaroo DNS address is in the format https://{WALLAROO PREFIX.}{WALLAROO SUFFIX}, then the direct path to the Keycloak API client credentials would be:

https://{WALLAROO PREFIX.}keycloak.{WALLAROO SUFFIX}/auth/admin/master/console/#/realms/master/clients

If the there is no prefix, then the address would simply be:

https://keycloak.{WALLAROO SUFFIX}/auth/admin/master/console/#/realms/master/clients

Then select the client, in this case api-client, then Credentials.

By default, tokens issued for api-client are valid for up to 60 minutes. Refresh tokens are supported.

Token Types

There are two tokens used with Wallaroo API services:

• MLOps tokens: User tokens are generated with the confidential client credentials and the username/password of the Wallaroo user making the MLOps API request and requires:

  • The Wallaroo instance Keycloak address.

  • The confidential client, api-client by default.

  • The confidential client secret.

  • The Wallaroo username making the MLOps API request.

  • The Wallaroo user’s password.

    This request return includes the access_token and the refresh_token. The access_token is used to authenticate. The refresh_token can be used to create a new token without submitting the original username and password.

    A sample curl version of that request is:

    eval $(curl "https://${URL_PREFIX}keycloak.${URL_SUFFIX}/auth/realms/master/protocol/openid-connect/token" -u "${CONFIDENTIAL_CLIENT}:${CONFIDENTIAL_CLIENT_SECRET}" -d "grant_type=password&username=${USERNAME}&password=${PASSWORD}&scope=offline_access' -s | jq -r '"TOKEN=\(.access_token) REFRESH=\(.refresh_token)"')
    

    • Tokens can be refreshed via a refresh request and require:

      • The confidential client, api-client by default.

      • The confidential client secret.

      • The refresh token retrieved from the initial access token request. A curl version of that request is:

        TOKEN=$(curl "https://${URL_PREFIX}keycloak.${URL_SUFFIX}/auth/realms/master/protocol/openid-connect/token" -u "${CONFIDENTIAL_CLIENT}:${CONFIDENTIAL_CLIENT_SECRET}" -d "grant_type=refresh_token&refresh_token=${REFRESH}" -s | jq -r '.access_token')
        

• Inference Token: Tokens used as part of a Pipeline Inference URL request. These do not require a Wallaroo user credentials. Inference token request require the following:

  • The Wallaroo instance Keycloak address.

  • The confidential client, api-client by default.

  • The confidential client secret.

    A curl version of that command is:

    TOKEN=$(curl "https://${URL_PREFIX}keycloak.${URL_SUFFIX}/auth/realms/master/protocol/openid-connect/token" -u "${CONFIDENTIAL_CLIENT}:${CONFIDENTIAL_CLIENT_SECRET}" -d 'grant_type=client_credentials' -s | jq -r '.access_token')
    

The following examples demonstrate:

• Generating a MLOps API token with the confidential client, client secret, username, and password.
• Refreshing a MLOps API token with the confidential client and client secret (the username and password are not required for refreshing the token).
• Generate a Pipeline Inference URl token with the confidential client and client secret (username and password are not required).

The username and password for the user are stored in the file ./creds.json to prevent them from being displayed in a demonstration.

## Generating token with confidential client, client secret, username, password

TOKENURL=f'https://{wallarooPrefix}keycloak.{wallarooSuffix}/auth/realms/master/protocol/openid-connect/token'

USERNAME = login_data["username"]
PASSWORD = login_data["password"]
CONFIDENTIAL_CLIENT=login_data["confidentialClient"]
CONFIDENTIAL_CLIENT_SECRET=login_data["confidentialPassword"]

auth = HTTPBasicAuth(CONFIDENTIAL_CLIENT, CONFIDENTIAL_CLIENT_SECRET)
data = {
    'grant_type': 'password',
    'username': USERNAME,
    'password': PASSWORD
}
response = requests.post(TOKENURL, auth=auth, data=data, verify=True)
access_token = response.json()['access_token']
refresh_token = response.json()['refresh_token']
display(access_token)

'eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJDYkFqN19QY0xCWTFkWmJiUDZ6Q3BsbkNBYTd6US0tRHlyNy0yLXlQb25nIn0.eyJleHAiOjE2ODQzNjAxNjUsImlhdCI6MTY4NDM1NjU2NSwianRpIjoiZGQxMDFkODMtMzk5ZC00N2M2LThlZDMtNjQxMGRmNThhYmViIiwiaXNzIjoiaHR0cHM6Ly9kb2MtdGVzdC5rZXljbG9hay53YWxsYXJvb2NvbW11bml0eS5uaW5qYS9hdXRoL3JlYWxtcy9tYXN0ZXIiLCJhdWQiOlsibWFzdGVyLXJlYWxtIiwiYWNjb3VudCJdLCJzdWIiOiIwMjhjOGI0OC1jMzliLTQ1NzgtOTExMC0wYjViZGQzODI0ZGEiLCJ0eXAiOiJCZWFyZXIiLCJhenAiOiJhcGktY2xpZW50Iiwic2Vzc2lvbl9zdGF0ZSI6Ijk4MDhkZTA5LWU2NjYtNGIyNC05ZWQ4LTc2MmUxZjllODk0ZSIsImFjciI6IjEiLCJhbGxvd2VkLW9yaWdpbnMiOlsiKiJdLCJyZWFsbV9hY2Nlc3MiOnsicm9sZXMiOlsiZGVmYXVsdC1yb2xlcy1tYXN0ZXIiLCJvZmZsaW5lX2FjY2VzcyIsInVtYV9hdXRob3JpemF0aW9uIl19LCJyZXNvdXJjZV9hY2Nlc3MiOnsibWFzdGVyLXJlYWxtIjp7InJvbGVzIjpbIm1hbmFnZS11c2VycyIsInZpZXctdXNlcnMiLCJxdWVyeS1ncm91cHMiLCJxdWVyeS11c2VycyJdfSwiYWNjb3VudCI6eyJyb2xlcyI6WyJtYW5hZ2UtYWNjb3VudCIsIm1hbmFnZS1hY2NvdW50LWxpbmtzIiwidmlldy1wcm9maWxlIl19fSwic2NvcGUiOiJwcm9maWxlIGVtYWlsIiwic2lkIjoiOTgwOGRlMDktZTY2Ni00YjI0LTllZDgtNzYyZTFmOWU4OTRlIiwiZW1haWxfdmVyaWZpZWQiOmZhbHNlLCJodHRwczovL2hhc3VyYS5pby9qd3QvY2xhaW1zIjp7IngtaGFzdXJhLXVzZXItaWQiOiIwMjhjOGI0OC1jMzliLTQ1NzgtOTExMC0wYjViZGQzODI0ZGEiLCJ4LWhhc3VyYS1kZWZhdWx0LXJvbGUiOiJ1c2VyIiwieC1oYXN1cmEtYWxsb3dlZC1yb2xlcyI6WyJ1c2VyIl0sIngtaGFzdXJhLXVzZXItZ3JvdXBzIjoie30ifSwibmFtZSI6IkpvaG4gSGFuc2FyaWNrIiwicHJlZmVycmVkX3VzZXJuYW1lIjoiam9obi5odW1tZWxAd2FsbGFyb28uYWkiLCJnaXZlbl9uYW1lIjoiSm9obiIsImZhbWlseV9uYW1lIjoiSGFuc2FyaWNrIiwiZW1haWwiOiJqb2huLmh1bW1lbEB3YWxsYXJvby5haSJ9.lfesLVvWM21kbWIhQtT2ap-ruT5_qt7CVcPUt1mAS8KoksuiJIb4QxPV9FwGB1I7sPiWjXR60cR-cjNLLoTgCX9GZZbfISDR4NvqN5ZBANDzYCx64WtTZCaDPeWROClHvmmE6Mfs1mAdgC3fIxkDe6Ns5-S6wnDqW7v6-yaNo5gBywftaCFyD3lFsmpmBvcyXphtn7sUlX_W4Ku9xmaalUkLv1F8528thZAARN5Jl-_uTHNKCe5wYGiEpQkbeIZ_Rjzqnctx-onw3cVKgbS6_wATr0TZQxgR2AY459OkCJ3rcuJTTTI5PihEGKlQUX5GmDIGG8DqE3iAPJ-xCY-OBQ'

## Refresh the token

TOKENURL=f'https://{wallarooPrefix}keycloak.{wallarooSuffix}/auth/realms/master/protocol/openid-connect/token'

# Retrieving through os environmental variables 
f = open('./creds.json')
login_data = json.load(f)

CONFIDENTIAL_CLIENT=login_data["confidentialClient"]
CONFIDENTIAL_CLIENT_SECRET=login_data["confidentialPassword"]

auth = HTTPBasicAuth(CONFIDENTIAL_CLIENT, CONFIDENTIAL_CLIENT_SECRET)
data = {
    'grant_type': 'refresh_token',
    'refresh_token': refresh_token
}
response = requests.post(TOKENURL, auth=auth, data=data, verify=True)
access_token = response.json()['access_token']
refresh_token = response.json()['refresh_token']
display(access_token)

'eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJDYkFqN19QY0xCWTFkWmJiUDZ6Q3BsbkNBYTd6US0tRHlyNy0yLXlQb25nIn0.eyJleHAiOjE2ODQzNjAxNjcsImlhdCI6MTY4NDM1NjU2NywianRpIjoiZDJlNTNlNzEtYjYzMi00MzNmLThjY2UtOGIxMDI0ZjFmYzliIiwiaXNzIjoiaHR0cHM6Ly9kb2MtdGVzdC5rZXljbG9hay53YWxsYXJvb2NvbW11bml0eS5uaW5qYS9hdXRoL3JlYWxtcy9tYXN0ZXIiLCJhdWQiOlsibWFzdGVyLXJlYWxtIiwiYWNjb3VudCJdLCJzdWIiOiIwMjhjOGI0OC1jMzliLTQ1NzgtOTExMC0wYjViZGQzODI0ZGEiLCJ0eXAiOiJCZWFyZXIiLCJhenAiOiJhcGktY2xpZW50Iiwic2Vzc2lvbl9zdGF0ZSI6Ijk4MDhkZTA5LWU2NjYtNGIyNC05ZWQ4LTc2MmUxZjllODk0ZSIsImFjciI6IjEiLCJhbGxvd2VkLW9yaWdpbnMiOlsiKiJdLCJyZWFsbV9hY2Nlc3MiOnsicm9sZXMiOlsiZGVmYXVsdC1yb2xlcy1tYXN0ZXIiLCJvZmZsaW5lX2FjY2VzcyIsInVtYV9hdXRob3JpemF0aW9uIl19LCJyZXNvdXJjZV9hY2Nlc3MiOnsibWFzdGVyLXJlYWxtIjp7InJvbGVzIjpbIm1hbmFnZS11c2VycyIsInZpZXctdXNlcnMiLCJxdWVyeS1ncm91cHMiLCJxdWVyeS11c2VycyJdfSwiYWNjb3VudCI6eyJyb2xlcyI6WyJtYW5hZ2UtYWNjb3VudCIsIm1hbmFnZS1hY2NvdW50LWxpbmtzIiwidmlldy1wcm9maWxlIl19fSwic2NvcGUiOiJwcm9maWxlIGVtYWlsIiwic2lkIjoiOTgwOGRlMDktZTY2Ni00YjI0LTllZDgtNzYyZTFmOWU4OTRlIiwiZW1haWxfdmVyaWZpZWQiOmZhbHNlLCJodHRwczovL2hhc3VyYS5pby9qd3QvY2xhaW1zIjp7IngtaGFzdXJhLXVzZXItaWQiOiIwMjhjOGI0OC1jMzliLTQ1NzgtOTExMC0wYjViZGQzODI0ZGEiLCJ4LWhhc3VyYS1kZWZhdWx0LXJvbGUiOiJ1c2VyIiwieC1oYXN1cmEtYWxsb3dlZC1yb2xlcyI6WyJ1c2VyIl0sIngtaGFzdXJhLXVzZXItZ3JvdXBzIjoie30ifSwibmFtZSI6IkpvaG4gSGFuc2FyaWNrIiwicHJlZmVycmVkX3VzZXJuYW1lIjoiam9obi5odW1tZWxAd2FsbGFyb28uYWkiLCJnaXZlbl9uYW1lIjoiSm9obiIsImZhbWlseV9uYW1lIjoiSGFuc2FyaWNrIiwiZW1haWwiOiJqb2huLmh1bW1lbEB3YWxsYXJvby5haSJ9.Va5OiLkedj-mFuI7UhsxXshmaSbhXthLv-PU56f2JiQi5-wiWXXxRk3pUioavIzKbi-VjmYQfbR95VY5QSWpUuD3scPuRhHDkSeslz6390phYiFygK_PmXMviQnL2q1mwdGzwh69htOjUWLf7MGWjNmkNdzjYyBy8gfD3V2O2MCfN3onVVCqr1aA1aAQXe9y_JswhjooxAQGit1xzNicvm3IW3QhHtOrDKj7gXNuSlc5vKqe52RQYEgElltqIOV4PVe12UGthKMfvdlDIeUEpTzXVFRH8XHJCrO_YQ_W9m-Rt1_9kelBl3SksdYKOisZaGwo6lv7hhapembH0iD29Q'

## Pipeline Inference URL token - does not require Wallaroo username/password.

TOKENURL=f'https://{wallarooPrefix}keycloak.{wallarooSuffix}/auth/realms/master/protocol/openid-connect/token'

# Retrieving through os environmental variables 
f = open('./creds.json')
login_data = json.load(f)

CONFIDENTIAL_CLIENT=login_data["confidentialClient"]
CLIENT_SECRET=login_data["confidentialPassword"]

auth = HTTPBasicAuth(CONFIDENTIAL_CLIENT, CLIENT_SECRET)
data = {
    'grant_type': 'client_credentials'
}
response = requests.post(TOKENURL, auth=auth, data=data, verify=True)
inference_access_token = response.json()['access_token']
display(inference_access_token)

'eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJDYkFqN19QY0xCWTFkWmJiUDZ6Q3BsbkNBYTd6US0tRHlyNy0yLXlQb25nIn0.eyJleHAiOjE2ODQzNjAxNjgsImlhdCI6MTY4NDM1NjU2OCwianRpIjoiNjIyOTViYmUtYzVlMi00NDQ2LWFmMDctNzY5MDAwNmI2NzI3IiwiaXNzIjoiaHR0cHM6Ly9kb2MtdGVzdC5rZXljbG9hay53YWxsYXJvb2NvbW11bml0eS5uaW5qYS9hdXRoL3JlYWxtcy9tYXN0ZXIiLCJhdWQiOlsibWFzdGVyLXJlYWxtIiwiYWNjb3VudCJdLCJzdWIiOiJmNmI5ODg5NC1iZTVjLTQyZDUtYTZhNS02ZjE5ZTY1YmNiNGEiLCJ0eXAiOiJCZWFyZXIiLCJhenAiOiJhcGktY2xpZW50IiwiYWNyIjoiMSIsImFsbG93ZWQtb3JpZ2lucyI6WyIqIl0sInJlYWxtX2FjY2VzcyI6eyJyb2xlcyI6WyJkZWZhdWx0LXJvbGVzLW1hc3RlciIsIm9mZmxpbmVfYWNjZXNzIiwidW1hX2F1dGhvcml6YXRpb24iXX0sInJlc291cmNlX2FjY2VzcyI6eyJtYXN0ZXItcmVhbG0iOnsicm9sZXMiOlsiaW1wZXJzb25hdGlvbiIsIm1hbmFnZS11c2VycyIsInZpZXctdXNlcnMiLCJxdWVyeS1ncm91cHMiLCJxdWVyeS11c2VycyJdfSwiYWNjb3VudCI6eyJyb2xlcyI6WyJtYW5hZ2UtYWNjb3VudCIsIm1hbmFnZS1hY2NvdW50LWxpbmtzIiwidmlldy1wcm9maWxlIl19fSwic2NvcGUiOiJwcm9maWxlIGVtYWlsIiwiZW1haWxfdmVyaWZpZWQiOmZhbHNlLCJjbGllbnRJZCI6ImFwaS1jbGllbnQiLCJjbGllbnRIb3N0IjoiMTAuMjQ0LjEuNzQiLCJodHRwczovL2hhc3VyYS5pby9qd3QvY2xhaW1zIjp7IngtaGFzdXJhLXVzZXItaWQiOiJmNmI5ODg5NC1iZTVjLTQyZDUtYTZhNS02ZjE5ZTY1YmNiNGEiLCJ4LWhhc3VyYS1kZWZhdWx0LXJvbGUiOiJ1c2VyIiwieC1oYXN1cmEtYWxsb3dlZC1yb2xlcyI6WyJ1c2VyIl0sIngtaGFzdXJhLXVzZXItZ3JvdXBzIjoie30ifSwicHJlZmVycmVkX3VzZXJuYW1lIjoic2VydmljZS1hY2NvdW50LWFwaS1jbGllbnQiLCJjbGllbnRBZGRyZXNzIjoiMTAuMjQ0LjEuNzQifQ.fde1-NsmXqCen71sRcIarscK1j4oFGATf8jh834aAUSb_UGXEmxEnUDDGMegu7KmpbeOi2ogIGY0ndACaZqS21lvVpzWHyVdsQGXCtl1mjwgLt0kzq6U5uR8znMIV-2Babw-9eE65F9I3TdUKRlnh8J5SAPvbOj_Hv_Y3u4cNj1b_Hk_o9lAEg-m2V0ZL7UDxgnVyitbWChiP4DE3q6yBBSVoORiBXDrfUiwIpCXyVKJIO_HrowEA8bYVOhh8PcbywmVa1kZaPcMuAOzsaysE361NCvJqbikVf4KX5Ii9k7lk90v3c-9VX24bIC67HFG8TwvWVnKRBAawwXcQ2ZTIA'

Through the Wallaroo SDK

The Wallaroo SDK method Wallaroo Client wl.auth.auth_header() method provides the token with the Authorization header.

# Retrieve the token
headers = wl.auth.auth_header()
display(headers)

{'Authorization': 'Bearer abcdefg'}

Connect to Wallaroo

For this example, a connection to the Wallaroo SDK is used. This will be used to retrieve the JWT token for the MLOps API calls.

This example will store the user’s credentials into the file ./creds.json which contains the following:

{
    "username": "{Connecting User's Username}", 
    "password": "{Connecting User's Password}", 
    "email": "{Connecting User's Email Address}"
}

Replace the username, password, and email fields with the user account connecting to the Wallaroo instance. This allows a seamless connection to the Wallaroo instance and bypasses the standard browser based confirmation link. For more information, see the Wallaroo SDK Essentials Guide: Client Connection.

For wallarooPrefix = "YOUR PREFIX." and wallarooSuffix = "YOUR SUFFIX", enter the prefix and suffix for your Wallaroo instance DNS name. If the prefix instance is blank, then it can be wallarooPrefix = "". Note that the prefix includes the . for proper formatting.

# Retrieve the login credentials.
os.environ["WALLAROO_SDK_CREDENTIALS"] = './creds.json'

# Client connection from local Wallaroo instance

wallarooPrefix = "YOUR PREFIX."
wallarooSuffix = "YOUR SUFFIX"

wl = wallaroo.Client(api_endpoint=f"https://{wallarooPrefix}api.{wallarooSuffix}", 
                     auth_endpoint=f"https://{wallarooPrefix}keycloak.{wallarooSuffix}", 
                     auth_type="user_password")

API URL

The variable APIURL is used to specify the connection to the Wallaroo instance’s MLOps API URL.

APIURL=f"https://{wallarooPrefix}api.{wallarooSuffix}"

API Request Methods

This tutorial relies on the Python requests library, and the Wallaroo Wallaroo Client wl.auth.auth_header() method.

MLOps API requests are always POST. Most are submitted with the header 'Content-Type':'application/json' unless specified otherwise.