Wallaroo provides the MLOps API for changing a Wallaroo instance, such as adding workspaces, deploying pipelines, and other tasks. It also has the Pipeline Inference API for submitting inference requests to a deployed pipeline and receiving the results.

Token Types

There are two tokens used with Wallaroo API services: MLOps API tokens, and Inference Tokens.

How to Retrieve the Confidential Client and Secret

Using the confidential client for token retrieval is only required for using the API to retrieve the token.

Wallaroo comes pre-installed with a confidential OpenID Connect client. The default client is api-client, but other clients may be created and configured through the Wallaroo Keycloak service.

As aa confidential client, api-client requires its secret to be supplied when requesting a token. Administrators may obtain their API client credentials from Keycloak from the Keycloak Service URL the suffix /auth/admin/master/console/#/realms/master/clients.

For example, if the Wallaroo instance has the DNS prefix magical-rhino-5555 and the suffix wallaroo.dev, then the direct path to the Keycloak API client credentials would be:

https://magical-rhino-5555.keycloak.wallaroo.dev/auth/admin/master/console/#/realms/master/clients

See the Wallaroo DNS Guide for details on Wallaroo DNS integrations and URLs.

Once authenticated to the Wallaroo keycloak service as an administrator, select the confidential client, in this case api-client, then Credentials.

By default, tokens issued for api-client are valid for up to 60 minutes. Refresh tokens are supported.

MLOps Token

MLOps API Tokens are generated either through an API request to the Wallaroo service Keycloak service, or through the Wallaroo SDK.

Each API operation requires a valid JSON Web Token (JWT) obtained from Wallaroo’s authentication and authorization service (i.e., Keycloak).

For MLOps API Tokens, the JWT must include a valid Wallaroo user’s identity as Wallaroo access permissions are tied to specific platform users. For Pipeline Inference URL API requests, the token provided from the confidential client is sufficient.

To authenticate to the Wallaroo API, the options are either:

• Authenticate with the client secret via an API request
• Use the SDK command Wallaroo.auth.auth_header() to retrieve the HTTP header including the token used to authenticate to the API.

MLOps Token through the API

• MLOps token: MLOps API Tokens are generated with the confidential client credentials and the username/password of the Wallaroo user making the MLOps API request and requires:
  • The Wallaroo instance Keycloak address.
  • The confidential client, api-client by default.
  • The confidential client secret.
  • The Wallaroo username making the MLOps API request.
  • The Wallaroo user’s password.

This request return includes the access_token and the refresh_token. The access_token is used to authenticate. The refresh_token can be used to create a new token without submitting the original username and password.

• Tokens can be refreshed via a refresh request and require:
  • The confidential client, api-client by default.
  • The confidential client credentials.
  • The refresh token retrieved from the initial access token request.

MLOps Token through the SDK

Users connected to their Wallaroo instance via the Wallaroo SDK can use the Wallaroo.auth.auth_header() method to retrieve the appropriate header for their MLOps and Inference URL connection. The following shows making a connection to a Wallaroo instance, then retrieving the API header via the auth_header method.

MLOps Token Retrieval Examples

The following examples show how to retrieve the MLOps token through different means. MLOps tokens require the following:

• Via the Keycloak service:
  • The Wallaroo instance Keycloak address.
  • The confidential client, api-client by default.
  • The confidential client secret.
  • The Wallaroo username making the MLOps API request.
  • The Wallaroo user’s password.
• Via the Wallaroo SDK:
  • The Wallaroo username making the MLOps API request.
  • The Wallaroo user’s password.

For example, the following requests a token for the Wallaroo instance https://magical-rhino-5555.wallaroo.dev for user mary.jane@example.com with the OpenID Connect Client api-client.

• Curl
• Python Requests
• Wallaroo SDK

TOKENURL = 'https://magical-rhino-5555.keycloak.wallaroo.dev/auth/realms/master/protocol/openid-connect/token'
CONFIDENTIAL_CLIENT ='api-client'
CONFIDENTIAL_CLIENT_SECRET = 'abc123'
USERNAME = 'mary.jane@example.com'
PASSWORD = 'snugglebunnies'

eval $(curl "https://${URL_PREFIX}.keycloak.${URL_SUFFIX}/auth/realms/master/protocol/openid-connect/token" -u "${CONFIDENTIAL_CLIENT}:${CLIENT_SECRET}" -d "grant_type=password&username=${USERNAME}&password=${PASSWORD}&scope=offline_access' -s | jq -r '"TOKEN=\(.access_token) REFRESH=\(.refresh_token)"')

echo $TOKEN
  abcdefg

TOKENURL = 'https://magical-rhino-5555.keycloak.wallaroo.dev/auth/realms/master/protocol/openid-connect/token'
CONFIDENTIAL_CLIENT ='api-client'
CONFIDENTIAL_CLIENT_SECRET = 'abc123'
USERNAME = 'mary.jane@example.com'
PASSWORD = 'snugglebunnies'

auth = HTTPBasicAuth(CONFIDENTIAL_CLIENT, CONFIDENTIAL_CLIENT_SECRET)
data = {
    'grant_type': 'password',
    'username': USERNAME,
    'password': PASSWORD
}
response = requests.post(TOKENURL, auth=auth, data=data, verify=True)
access_token = response.json()['access_token']
refresh_token = response.json()['refresh_token']
display(access_token)

    abcdefg

# Retrieve the token
headers = wl.auth.auth_header()
display(headers)

{'Authorization': 'Bearer abcdefg'}

MLOps Token Refresh Examples

Tokens refreshment requires:

• Via the Keycloak API:
  • The confidential client, api-client by default.
  • The confidential client secret.
  • The refresh token retrieved from the initial access token request.
• Via the Wallaroo SDK:
  • When using the Wallaroo SDK, the token is refreshed when wl.auth.auth_header() is run.

Note that for the refresh token, the Wallaroo user’s username and password are not required, but will require the refresh token retrieved during the initial access token retrieval process.

For example, the following requests a refresh token for the Wallaroo instance https://magical-rhino-5555.wallaroo.dev for user mary.jane@example.com with the OpenID Connect Client api-client, with the refresh token already retrieved through the previous step.

• Curl
• Python
• Wallaroo SDK

TOKENURL = 'https://magical-rhino-5555.keycloak.wallaroo.dev/auth/realms/master/protocol/openid-connect/token'
CONFIDENTIAL_CLIENT ='api-client'
CLIENT_SECRET = 'abc123'

TOKEN=$(curl "${TOKENURL}" -u "${CONFIDENTIAL_CLIENT}:${CLIENT_SECRET}" -d "grant_type=refresh_token&refresh_token=${REFRESH}"    -s | jq -r '.access_token')
echo $TOKEN
  abcdefg

TOKENURL = 'https://magical-rhino-5555.keycloak.wallaroo.dev/auth/realms/master/protocol/openid-connect/token'
CONFIDENTIAL_CLIENT ='api-client'
CLIENT_SECRET = 'abc123'

auth = HTTPBasicAuth(CONFIDENTIAL_CLIENT, CONFIDENTIAL_CLIENT_SECRET)
data = {
    'grant_type': 'refresh_token',
    'refresh_token': refresh_token
}
response = requests.post(TOKENURL, auth=auth, data=data, verify=True)
access_token = response.json()['access_token']
refresh_token = response.json()['refresh_token']
display(access_token)
    abcdefg

# Retrieve the token
headers = wl.auth.auth_header()
display(headers)

{'Authorization': 'Bearer abcdefg'}

Request MLOps Operation with Token Example

With the token retrieved, a MLOps request can be performed.

The following examples shows how to make the request to retrieve a list of workspaces via the MLops API.

• Curl
• Python
• Wallaroo SDK

curl 'https://magical-rhino-5555.api.wallaroo.dev/v1/api/workspaces/list' -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' -d '{}'

## Inference through external URL

apiRequest = "https://magical-rhino-5555.api.wallaroo.ai/v1/api/workspaces/list"

# set the headers
headers= {
        'Authorization': 'Bearer ' + TOKEN,
        'Content-Type: application/json'
    }

data = {
}

# submit the request via POST
response = requests.post(apiRequest, data=data, headers=headers)

# Retrieve the token through the Wallaroo SDK
headers = wl.auth.auth_header()

apiRequest = "https://magical-rhino-5555.api.wallaroo.ai/v1/api/workspaces/list"

data = {
}

response = requests.post(apiRequest, json=data, headers=headers, verify=True).json()

Inference Tokens

Inference Tokens through the API

• Inference Token: Tokens used as part of a Pipeline Inference URL request. These do not require a Wallaroo user credentials. Inference token request require the following:
  • The Wallaroo instance Keycloak address.
  • The confidential client, api-client by default.
  • The confidential client secret.

Retrieve Pipeline Inference URL Token

Inference Token: Tokens used as part of a Pipeline Inference URL request. These do not require a Wallaroo user credentials.

Note that MLOps API tokens can be used for Pipeline URL inference requests.

Inference token request require the following:

• Via the API:
  • The Wallaroo instance Keycloak address.
  • The confidential client, api-client by default.
  • The confidential client secret.
• Via the Wallaroo SDK
  • The Wallaroo user’s username and password.

• Curl
• Python
• Wallaroo SDK

TOKENURL = 'https://magical-rhino-5555.keycloak.wallaroo.dev/auth/realms/master/protocol/openid-connect/token'
CONFIDENTIAL_CLIENT ='api-client'
CLIENT_SECRET = 'abc123'

TOKEN=$(curl "${TOKENURL}" -u "${CONFIDENTIAL_CLIENT}:${CONFIDENTIAL_CLIENT_SECRET}" -d 'grant_type=client_credentials' -s | jq -r '.access_token')

TOKENURL = 'https://magical-rhino-5555.keycloak.wallaroo.dev/auth/realms/master/protocol/openid-connect/token'
CONFIDENTIAL_CLIENT ='api-client'
CLIENT_SECRET = 'abc123'

auth = HTTPBasicAuth(CONFIDENTIAL_CLIENT, CLIENT_SECRET)
data = {
    'grant_type': 'client_credentials'
}
response = requests.post(TOKENURL, auth=auth, data=data, verify=True)
inference_access_token = response.json()['access_token']
display(inference_access_token)
    abcdefg

# Retrieve the token
headers = wl.auth.auth_header()
display(headers)

{'Authorization': 'Bearer abcdefg'}

Perform Inference via Inference URL

The following demonstrates performing an inference request via a deployed pipeline’s inference URL, stored in the variable inference_url.

• Curl
• Python
• Wallaroo SDK

dataFile="./data/data_25k.arrow"
contentType="application/vnd.apache.arrow.file"

!curl -X POST $inference_url -H "Authorization: Bearer $TOKEN" -H "Content-Type:$contentType" --data-binary @$dataFile > curl_response.df

headers = {
  "Authorization": inference_access_token,
  "Content-Type": 'application/json; format=pandas-records'
}

## Inference through external URL using dataframe

# retrieve the json data to submit
data = [
    {
        "tensor":[
            1.0678324729,
            0.2177810266,
            -1.7115145262,
            0.682285721,
            1.0138553067,
            -0.4335000013,
            0.7395859437,
            -0.2882839595,
            -0.447262688,
            0.5146124988,
            0.3791316964,
            0.5190619748,
            -0.4904593222,
            1.1656456469,
            -0.9776307444,
            -0.6322198963,
            -0.6891477694,
            0.1783317857,
            0.1397992467,
            -0.3554220649,
            0.4394217877,
            1.4588397512,
            -0.3886829615,
            0.4353492889,
            1.7420053483,
            -0.4434654615,
            -0.1515747891,
            -0.2668451725,
            -1.4549617756
        ]
    }
]

# submit the request via POST, import as pandas DataFrame
response = pd.DataFrame.from_records(
    requests.post(
        inference_url, 
        json=data, 
        headers=headers)
        .json()
    )

# Retrieve the token via the Wallaroo SDK
headers = wl.auth.auth_header()

# set Content-Type type
headers['Content-Type']='application/vnd.apache.arrow.file'

# set accept as apache arrow table
headers['Accept']="application/vnd.apache.arrow.file"

# Submit arrow file
dataFile="./data/cc_data_10k.arrow"

data = open(dataFile,'rb').read()

response = requests.post(
                    inference_url, 
                    headers=headers, 
                    data=data, 
                    verify=True
                )

# Arrow table is retrieved 
with pa.ipc.open_file(response.content) as reader:
    arrow_table = reader.read_all()