The following guides are made to help users work with Wallaroo workspaces, upload models, set up pipelines and start inferring from data from their own applications using the Wallaroo SDK, API, and other coding examples.
We recommend first time users refer to the Wallaroo Tutorials that walk new users through several different models and ways of creating pipelines.
Supported Model Versions and Libraries
The following ML Model versions and Python libraries are supported by Wallaroo. When using the Wallaroo autoconversion library or working with a local version of the Wallaroo SDK, use the following versions for maximum compatibility.
The following data types are supported for transporting data to and from Wallaroo in the following run times:
ONNX
TensorFlow
MLFlow
Float Types
Runtime
BFloat16*
Float16
Float32
Float64
ONNX
X
X
TensorFlow
X
X
X
MLFlow
X
X
X
* (Brain Float 16, represented internally as a f32)
Int Types
Runtime
Int8
Int16
Int32
Int64
ONNX
X
X
X
X
TensorFlow
X
X
X
X
MLFlow
X
X
X
X
Uint Types
Runtime
Uint8
Uint16
Uint32
Uint64
ONNX
X
X
X
X
TensorFlow
X
X
X
X
MLFlow
X
X
X
X
Other Types
Runtime
Boolean
Utf8 (String)
Complex 64
Complex 128
FixedSizeList*
ONNX
X
Tensor
X
X
X
MLFlow
X
X
X
* Fixed sized lists of any of the previously supported data types.
1 - Wallaroo API Guide
Reference Guide for the Wallaroo API
The Wallaroo API provides users the ability to commands to a Wallaroo instance via a RCP-like HTTP API interface. This allows organizations to develop applications that can administrate Wallaroo from their own custom applications.
Wallaroo API URLs
Each instance of Wallaroo provides the Wallaroo API specifications through the following URLs.
Wallaroo uses the following format for its URLs. For more information, see the DNS Integration Guide.
Each MLOps API operation requires a valid JSON Web Token (JWT) obtained from Wallaroo’s authentication and authorization service (i.e., Keycloak). Generally, the JWT must include a valid user’s identity, as Wallaroo access permissions are tied to specific platform users.
To authenticate to the Wallaroo API, the options are either to authenticate with the client secret, or to use the SDK command Wallaroo.auth.auth_header() to retrieve the HTTP header including the token used to authenticate to the API.
The following process will retrieve a token using the client secret:
Wallaroo comes pre-installed with a confidential OpenID Connect client. The default client is api-client, but other clients may be created and configured.
As it is a confidential client, api-client requires its secret to be supplied when requesting a token. Administrators may obtain their API client credentials from Keycloak from the Keycloak Service URL as listed above and the prefix /auth/admin/master/console/#/realms/master/clients.
For example, if the Wallaroo instance DNS address is https://magical-rhino-5555.wallaroo.dev, then the direct path to the Keycloak API client credentials would be:
Then select the client, in this case api-client, then Credentials.
By default, tokens issued for api-client are valid for up to 60 minutes. Refresh tokens are supported.
Retrieve MLOps API Token
To retrieve an API token for a specific user with the Client Secret, request the token from the Wallaroo instance using the client secret and provide the following:
Token Request URL: The Keycloak token retrieval URL.
OpenID Connect client name: The name of the OpenID Connect client.
OpenID Connect client Secret: The secret for the OpenID Connect client.
UserName: The username of the Wallaroo instance user, usually their email address.
Password: The password of the Wallaroo instance user.
For example, the following requests a token for the Wallaroo instance https://magical-rhino-5555.wallaroo.dev for user mary.jane@example.com with the OpenID Connect Client api-client:
The same can be done through the Python requests library:
## Inference through external URLapiRequest="https://magical-rhino-5555.api.wallaroo.ai/v1/api/workspaces/list"# set the headersheaders= {
'Authorization': 'Bearer '+TOKEN,
'Content-Type: application/json' }
data= {
}
# submit the request via POSTresponse=requests.post(apiRequest, data=data, headers=headers)
# Display the returned resultprint(response.json())
Wallaroo MLOps API Documentation from a Wallaroo instance: A Swagger UI based documentation is available from your Wallaroo instance at https://{Wallaroo Prefix}.api.{Wallaroo Suffix}/v1/api/docs. For example, if the Wallaroo Instance is YOUR SUFFIX with the prefix {lovely-rhino-5555}, then the Wallaroo MLOps API Documentation would be available at https://lovely-rhino-5555.api.example.wallaroo.ai/v1/api/docs. For another example, a Wallaroo Enterprise users who do not use a prefix and has the suffix wallaroo.example.wallaroo.ai, the the Wallaroo MLOps API Documentation would be available at https://api.wallaroo.example.wallaroo.ai/v1/api/docs. For more information, see the Wallaroo Documentation Site.
IMPORTANT NOTE: The Wallaroo MLOps API is provided as an early access features. Future iterations may adjust the methods and returns to provide a better user experience. Please refer to this guide for updates.
OpenAPI Steps
The following demonstrates how to use each command in the Wallaroo MLOps API, and can be modified as best fits your organization’s needs.
Import Libraries
For the examples, the Python requests library will be used to make the REST HTTP(S) connections.
# Requires requests and requests-toolbelt with either:# pip install requests-toolbelt# conda install -c conda-forge requests-toolbeltimportrequestsimportjsonfromrequests.authimportHTTPBasicAuth
Retrieve Credentials
Through Keycloak
Wallaroo comes pre-installed with a confidential OpenID Connect client. The default client is api-client, but other clients may be created and configured.
As it is a confidential client, api-client requires its secret to be supplied when requesting a token. Administrators may obtain their API client credentials from Keycloak from the Keycloak Service URL as listed above and the prefix /auth/admin/master/console/#/realms/master/clients.
For example, if the YOUR SUFFIX instance DNS address is https://magical-rhino-5555.example.wallaroo.ai, then the direct path to the Keycloak API client credentials would be:
Then select the client, in this case api-client, then Credentials.
By default, tokens issued for api-client are valid for up to 60 minutes. Refresh tokens are supported.
Through the Wallaroo SDK
The API token can be retrieved using the Wallaroo SDK through the wallaroo.client.mlops() command. In the following example, the token will be retrieved and stored to the variable TOKEN:
The following variables are used for the example and should be modified to fit your organization.
## VariablesURLPREFIX='YOURPREFIX'URLSUFFIX='YOURSUFFIX'SECRET="YOUR SECRET"TOKENURL=f'https://{URLPREFIX}.keycloak.{URLSUFFIX}/auth/realms/master/protocol/openid-connect/token'CLIENT="api-client"USERNAME="YOUR EMAIL"PASSWORD="YOUR PASSWORD"APIURL=f"https://{URLPREFIX}.api.{URLSUFFIX}/v1/api"newUser="NEW USER EMAIL"newPassword="NEW USER PASSWORD"
The following is an output of the TOKENURL variable to verify it matches your Wallaroo instance’s Keycloak API client credentials URL.
TOKENURL
API Example Methods
The following methods are used to retrieve the MLOPs API Token from the Wallaroo instance’s Keycloak service, and submit MLOps API requests through the Wallaroo instance’s MLOps API.
MLOps API requests are always POST, and are either submitted as 'Content-Type':'application/json' or as a multipart submission including a file.
defget_jwt_token(url, client, secret, username, password):
auth=HTTPBasicAuth(client, secret)
data= {
'grant_type': 'password',
'username': username,
'password': password }
response=requests.post(url, auth=auth, data=data, verify=True)
returnresponse.json()['access_token']
# This can either submit a plain POST request ('Content-Type':'application/json'), or with a file.defget_wallaroo_response(url, api_request, token, data, files=None, contentType='application/json', params=None):
apiUrl=f"{url}{api_request}"iffilesisNone:
# Regular POST requestheaders= {
'Authorization': 'Bearer '+token,
'Content-Type':contentType }
response=requests.post(apiUrl, json=data, headers=headers, verify=True)
elifcontentType=='application/octet-stream':
# Post request as octet-streamheaders= {
'Authorization': 'Bearer '+token,
'Content-Type':contentType }
response=requests.post(apiUrl, data=files, headers=headers, params=params)
#response = requests.post(apiUrl, data=data, headers=headers, files=files, verify=True)else:
# POST request with fileheaders= {
'Authorization': 'Bearer '+token }
response=requests.post(apiUrl, data=data, headers=headers, files=files, verify=True)
returnresponse.json()
Retrieve MLOps API Token
To retrieve an API token for a specific user with the Client Secret, request the token from the Wallaroo instance using the client secret and provide the following:
Token Request URL: The Keycloak token retrieval URL.
OpenID Connect client name: The name of the OpenID Connect client.
OpenID Connect client Secret: The secret for the OpenID Connect client.
UserName: The username of the Wallaroo instance user, usually their email address.
Password: The password of the Wallaroo instance user.
The following sample uses the variables set above to request the token, then displays it.
IMPORTANT NOTE: This command is for YOUR SUFFIX only. For more details on user management, see Wallaroo User Management.
Users can be invited through /users/invite. When using YOUR SUFFIX, this will send an invitation email to the email address listed. Note that the user must not already be a member of the Wallaroo instance, and email addresses must be unique. If the email address is already in use for another user, the request will generate an error.
Parameters
email *(REQUIRED string): The email address of the new user to invite.
password(OPTIONAL string): The assigned password of the new user to invite. If not provided, the Wallaroo instance will provide the new user a temporary password that must be changed upon initial login.
Example: In this example, a new user will be invited to the Wallaroo instance and assigned a password.
A new workspace will be created in the Wallaroo instance. Upon creating, the workspace owner will be assigned as the user making the MLOps API request.
Parameters:
workspace_name - (REQUIRED string): The name of the new workspace.
Returns:
workspace_id - (int): The ID of the new workspace.
Example: In this example, a workspace with the name testapiworkspace will be created, and the newly created workspace’s workspace_id saved as the variable exampleWorkspaceId for use in other code examples. After the request is complete, the List Workspaces command will be issued to demonstrate the new workspace has been created.
Removes the user from the given workspace. In this request, either the user’s Keycloak ID is required OR the user’s email address is required.
Parameters
workspace_id - (REQUIRED int): The id of the workspace.
user_id - (string): The Keycloak ID of the user. If email is not provided, then this parameter is REQUIRED.
email - (string): The user’s email address. If user_id is not provided, then this parameter is REQUIRED.
Returns
user_id: The user’s identification.
user_type: The user’s workspace type (owner, co-owner, etc).
Example: The following example will remove the newUser from workspace created in the Create Workspace request. Then the users for that workspace will be listed to verify newUser has been removed.
# Remove existing user from an existing workspaceTOKEN=get_jwt_token(TOKENURL, CLIENT, SECRET, USERNAME, PASSWORD)
apiRequest="/workspaces/remove_user"data= {
"email":newUser,
"workspace_id": exampleWorkspaceId}
response=get_wallaroo_response(APIURL, apiRequest, TOKEN, data)
response
# List users in a workspaceTOKEN=get_jwt_token(TOKENURL, CLIENT, SECRET, USERNAME, PASSWORD)
apiRequest="/workspaces/list_users"data= {
"workspace_id": exampleWorkspaceId}
response=get_wallaroo_response(APIURL, apiRequest, TOKEN, data)
response
Uploads a ML Model to a Wallaroo workspace via POST with Content-Type: multipart/form-data.
Parameters
name - (REQUIRED string): Name of the model
visibility - (OPTIONAL string): The visibility of the model as either public or private.
workspace_id - (REQUIRED int): The numerical id of the workspace to upload the model to.
Example: This example will upload the sample file ccfraud.onnx to the workspace created in the Create Workspace step as apitestmodel. The model name will be saved as exampleModelName for use in other examples. The id of the uploaded model will be saved as exampleModelId for use in later examples.
# upload model - uses multiform data through a Python `request`TOKEN=get_jwt_token(TOKENURL, CLIENT, SECRET, USERNAME, PASSWORD)
apiRequest="/models/upload"exampleModelName="apitestmodel"data= {
"name":exampleModelName,
"visibility":"public",
"workspace_id": exampleWorkspaceId}
files= {
"file": ('ccfraud.onnx', open('./models/ccfraud.onnx','rb'))
}
response=get_wallaroo_response(APIURL, apiRequest, TOKEN, data, files)
response
Returns a list of models added to a specific workspace.
Parameters
workspace_id - (REQUIRED int): The workspace id to list.
Example: Display the models for the workspace used in the Upload Model to Workspace step. The model id and model name will be saved as exampleModelId and exampleModelName variables for other examples.
# List models in a workspaceTOKEN=get_jwt_token(TOKENURL, CLIENT, SECRET, USERNAME, PASSWORD)
apiRequest="/models/list"data= {
"workspace_id": exampleWorkspaceId}
response=get_wallaroo_response(APIURL, apiRequest, TOKEN, data)
response
Retrieves all versions of a model based on either the name of the model or the model_pk_id.
Parameters
model_id - (REQUIRED String): The model name.
models_pk_id - (REQUIRED int): The model integer pk id.
Returns
Array(Model Details)
sha - (String): The sha hash of the model version.
models_pk_id- (int): The pk id of the model.
model_version - (String): The UUID identifier of the model version.
owner_id - (String): The Keycloak user id of the model’s owner.
model_id - (String): The name of the model.
id - (int): The integer id of the model.
file_name - (String): The filename used when uploading the model.
image_path - (String): The image path of the model.
Example: Retrieve the versions for a previously uploaded model. The variables exampleModelVersion and exampleModelSha will store the model’s version and SHA values for use in other examples.
# List models in a workspaceTOKEN=get_jwt_token(TOKENURL, CLIENT, SECRET, USERNAME, PASSWORD)
apiRequest="/models/list_versions"data= {
"model_id": exampleModelName,
"models_pk_id": exampleModelId}
response=get_wallaroo_response(APIURL, apiRequest, TOKEN, data)
response
Pipelines can be managed through the Wallaroo API. Pipelines are the vehicle used for deploying, serving, and monitoring ML models. For more information, see the Wallaroo Glossary.
Create Pipeline in a Workspace
Creates a new pipeline in the specified workspace.
Parameters
pipeline_id - (REQUIRED string): Name of the new pipeline.
workspace_id - (REQUIRED int): Numerical id of the workspace for the new pipeline.
definition - (REQUIRED string): Pipeline definitions, can be {} for none.
Example: Two pipelines are created in the workspace created in the step Create Workspace. One will be an empty pipeline without any models, the other will be created using the uploaded models in the Upload Model to Workspace step and no configuration details. The pipeline id, variant id, and variant version of each pipeline will be stored for later examples.
Deploy a an existing pipeline. Note that for any pipeline that has model steps, they must be included either in model_configs, model_ids or models.
Parameters
deploy_id (REQUIRED string): The name for the pipeline deployment.
engine_config (OPTIONAL string): Additional configuration options for the pipeline.
pipeline_version_pk_id (REQUIRED int): Pipeline version id.
model_configs (OPTIONALArray int): Ids of model configs to apply.
model_ids (OPTIONALArray int): Ids of models to apply to the pipeline. If passed in, model_configs will be created automatically.
models (OPTIONAL Array models): If the model ids are not available as a pipeline step, the models’ data can be passed to it through this method. The options below are only required if models are provided as a parameter.
name (REQUIRED string): Name of the uploaded model that is in the same workspace as the pipeline.
version (REQUIRED string): Version of the model to use.
sha (REQUIRED string): SHA value of the model.
pipeline_id (REQUIRED int): Numerical value of the pipeline to deploy.
Returns
id (int): The deployment id.
Examples: Both the empty pipeline and pipeline with model created in the step Create Pipeline in a Workspace will be deployed and their deployment information saved for later examples.
The API command /admin/get_pipeline_external_url retrieves the external inference URL for a specific pipeline in a workspace.
Parameters
workspace_id (REQUIRED integer): The workspace integer id.
pipeline_name (REQUIRED string): The name of the deployment.
In this example, a list of the workspaces will be retrieved. Based on the setup from the Internal Pipeline Deployment URL Tutorial, the workspace matching urlworkspace will have it’s workspace id stored and used for the /admin/get_pipeline_external_url request with the pipeline urlpipeline.
The External Inference URL will be stored as a variable for the next step.
The inference can now be performed through the External Inference URL. This URL will accept the same inference data file that is used with the Wallaroo SDK, or with an Internal Inference URL as used in the Internal Pipeline Inference URL Tutorial.
For this example, the externalUrl retrieved through the Get External Inference URL is used to submit a single inference request.
If the Wallaroo instance has been configured to enable Arrow support, then the file cc_data_1k.df.json will be used. This is a DataFrame object. If Arrow support has not been enabled, then the inference request is used with the Wallaroo proprietary JSON data file cc_data_1k.json.
#TOKEN=get_jwt_token(TOKENURL, CLIENT, SECRET, USERNAME, PASSWORD)# if Arrow has been enabled, set this to True. Otherwise, leave as False.arrowEnabled=True## Inference through external URL# retrieve the json data to submitifarrowEnabledisTrue:
dataFile='./data/cc_data_1k.df.json'data=json.load(open('./data/cc_data_1k.df.json','rb'))
contentType="application/json; format=pandas-records"else:
dataFile='./data/cc_data_1k.json'data=json.load(open('./data/cc_data_1k.json','rb'))
contentType="application/json"# set the headersheaders= {
'Authorization': 'Bearer '+TOKEN,
'Content-Type': contentType}
# submit the request via POSTresponse=requests.post(externalUrl, json=data, headers=headers)
# Only the first 300 characters will be displayed for brevityprintResponse=json.dumps(response.json())
print(printResponse[0:300])
IMPORTANT NOTE: These assays were run in a Wallaroo environment with canned historical data. See the Wallaroo Assay Tutorial for details on setting up this environment. This historical data is required for these examples.
Create Assay
Create a new array in a specified pipeline.
PARAMETERS
id - (OPTIONAL int): The numerical identifier for the assay.
name - (REQUIRED string): The name of the assay.
pipeline_id - (REQUIRED int): The numerical idenfifier the assay will be placed into.
pipeline_name - (REQUIRED string): The name of the pipeline
active - (REQUIRED bool): Indicates whether the assay will be active upon creation or not.
status - (REQUIRED string): The status of the assay upon creation.
iopath - (REQUIRED string): The iopath of the assay.
baseline - (REQUIRED baseline): The baseline for the assay.
Fixed - (REQUIRED AssayFixConfiguration): The fixed configuration for the assay.
pipeline - (REQUIRED string): The name of the pipeline with the baseline data.
model - (REQUIRED string): The name of the model used.
start_at - (REQUIRED string): The DateTime of the baseline start date.
end_at - (REQUIRED string): The DateTime of the baseline end date.
window (REQUIRED AssayWindow): Assay window.
pipeline - (REQUIRED string): The name of the pipeline for the assay window.
model - (REQUIRED string): The name of the model used for the assay window.
width - (REQUIRED string): The width of the assay window.
start - (OPTIONAL string): The DateTime of when to start the assay window.
interval - (OPTIONAL string): The assay window interval.
summarizer - (REQUIRED AssaySummerizer): The summarizer type for the array aka “advanced settings” in the Wallaroo Dashboard UI.
type - (REQUIRED string): Type of summarizer.
bin_mode - (REQUIRED string): The binning model type. Values can be:
run_until - (OPTIONAL string): DateTime of when to end the assay.
workspace_id - (REQUIRED integer): The workspace the assay is part of.
model_insights_url - (OPTIONAL string): URL for model insights.
RETURNS
assay_id - (integer): The id of the new assay.
As noted this example requires the Wallaroo Assay Tutorial for historical data. Before running this example, set the sample pipeline id, pipeline, name, model name, and workspace id in the code sample below. For more information on retrieving this information, see the Wallaroo Developer Guides.
# Create assayapiRequest="/assays/create"exampleAssayName="api_assay_test2"## Now get all of the assays for the pipeline in workspace 4 `housepricedrift`exampleAssayPipelineId=4exampleAssayPipelineName="housepricepipe"exampleAssayModelName="housepricemodel"exampleAssayWorkspaceId=4# iopath can be input 00 or output 0 0data= {
'name': exampleAssayName,
'pipeline_id': exampleAssayPipelineId,
'pipeline_name': exampleAssayPipelineName,
'active': True,
'status': 'active',
'iopath': "input 0 0",
'baseline': {
'Fixed': {
'pipeline': exampleAssayPipelineName,
'model': 'houseprice-model-yns',
'start_at': '2022-01-01T00:00:00-05:00',
'end_at': '2022-01-02T00:00:00-05:00' }
},
'window': {
'pipeline': exampleAssayPipelineName,
'model': exampleAssayModelName,
'width': '24 hours',
'start': None,
'interval': None },
'summarizer': {
'type': 'UnivariateContinuous',
'bin_mode': 'Quantile',
'aggregation': 'Density',
'metric': 'PSI',
'num_bins': 5,
'bin_weights': None,
'bin_width': None,
'provided_edges': None,
'add_outlier_edges': True },
'warning_threshold': 0,
'alert_threshold': 0.1,
'run_until': None,
'workspace_id': exampleAssayWorkspaceId}
response=get_wallaroo_response(APIURL, apiRequest, TOKEN, data)
example_assay_id=response['assay_id']
response
List Assays
Lists all assays in the specified pipeline.
PARAMETERS
pipeline_id - (REQUIRED int): The numerical ID of the pipeline.
RETURNS
assays - (Array assays): A list of all assays.
Example: Display a list of all assays in a workspace. This will assume we have a workspace with an existing Assay and the associated data has been upload. See the tutorial Wallaroo Assays Tutorial.
For this reason, these values are hard coded for now.
## First list all of the workspaces and the list of pipelines# List workspacesapiRequest="/workspaces/list"data= {
}
response=get_wallaroo_response(APIURL, apiRequest, TOKEN, data)
response
run_until - (OPTIONAL string): DateTime of when to end the assay.
workspace_id - (REQUIRED integer): The workspace the assay is part of.
model_insights_url - (OPTIONAL string): URL for model insights.
Returns
Assay
Example: An interactive assay will be run for Assay exampleAssayId exampleAssayName. Depending on the number of assay results and the data window, this may take some time. This returns all of the results for this assay at this time. The total number of responses will be displayed after.
assay_id - (REQUIRED integer): Numerical id for the assay.
start - (OPTIONAL string): DateTime for when the baseline starts.
end - (OPTIONAL string): DateTime for when the baseline ends.
limit - (OPTIONAL integer): Maximum number of results to return.
pipeline_id - (OPTIONAL integer): Numerical id of the pipeline the assay is in.
Returns
Assay Baseline
Example: Results for Assay 3 “example assay” will be retrieved for January 2 to January 3. For the sake of time, only the first record will be displayed.
IMPORTANT NOTE: This command is for Wallaroo Community only. For more details on user management, see Wallaroo User Management.
Users can be invited through /users/invite. When using Wallaroo Community, this will send an invitation email to the email address listed. Note that the user must not already be a member of the Wallaroo instance, and email addresses must be unique. If the email address is already in use for another user, the request will generate an error.
Parameters
email *(REQUIRED string): The email address of the new user to invite.
password(OPTIONAL string): The assigned password of the new user to invite. If not provided, the Wallaroo instance will provide the new user a temporary password that must be changed upon initial login.
Example: In this example, a new user will be invited to the Wallaroo instance and assigned a password.
Example: In this example, the workspaces for the a specific user will be displayed, then workspaces for all users will be displayed.
# List workspaces by user idapiRequest="/workspaces/list"data= {
"user_id":firstUserKeycloak}
response=get_wallaroo_response(APIURL, apiRequest, TOKEN, data)
response
A new workspace will be created in the Wallaroo instance. Upon creating, the workspace owner will be assigned as the user making the MLOps API request.
Parameters:
workspace_name - (REQUIRED string): The name of the new workspace.
Returns:
workspace_id - (int): The ID of the new workspace.
Example: In this example, a workspace with the name testapiworkspace- with a randomly generated UUID will be created, and the newly created workspace’s workspace_id saved as the variable exampleWorkspaceId for use in other code examples. After the request is complete, the List Workspaces command will be issued to demonstrate the new workspace has been created.
Existing users of the Wallaroo instance can be added to an existing workspace.
Parameters
email - (REQUIRED string): The email address of the user to add to the workspace.
workspace_id - (REQUIRED int): The id of the workspace.
Example: The following example adds the user created in Invite Users request to the workspace created in the Create Workspace request.
# Add existing user to existing workspaceapiRequest="/workspaces/add_user"data= {
"email":newUser,
"workspace_id": exampleWorkspaceId}
response=get_wallaroo_response(APIURL, apiRequest, TOKEN, data)
response
{}
List Users in a Workspace
Lists the users who are either owners or collaborators of a workspace.
Parameters
workspace_id - (REQUIRED int): The id of the workspace.
Returns
user_id: The user’s identification.
user_type: The user’s workspace type (owner, co-owner, etc).
Example: The following example will list all users part of the workspace created in the Create Workspace request.
# List users in a workspaceapiRequest="/workspaces/list_users"data= {
"workspace_id": exampleWorkspaceId}
response=get_wallaroo_response(APIURL, apiRequest, TOKEN, data)
response
Removes the user from the given workspace. In this request, either the user’s Keycloak ID is required OR the user’s email address is required.
Parameters
workspace_id - (REQUIRED int): The id of the workspace.
user_id - (string): The Keycloak ID of the user. If email is not provided, then this parameter is REQUIRED.
email - (string): The user’s email address. If user_id is not provided, then this parameter is REQUIRED.
Returns
user_id: The user’s identification.
user_type: The user’s workspace type (owner, co-owner, etc).
Example: The following example will remove the newUser from workspace created in the Create Workspace request. Then the users for that workspace will be listed to verify newUser has been removed.
# Remove existing user from an existing workspaceapiRequest="/workspaces/remove_user"data= {
"email":newUser,
"workspace_id": exampleWorkspaceId}
response=get_wallaroo_response(APIURL, apiRequest, TOKEN, data)
response
{'affected_rows': 1}
# List users in a workspaceapiRequest="/workspaces/list_users"data= {
"workspace_id": exampleWorkspaceId}
response=get_wallaroo_response(APIURL, apiRequest, TOKEN, data)
response
Uploads a ML Model to a Wallaroo workspace via POST with Content-Type: multipart/form-data.
Parameters
name - (REQUIRED string): Name of the model
visibility - (OPTIONAL string): The visibility of the model as either public or private.
workspace_id - (REQUIRED int): The numerical id of the workspace to upload the model to.
Example: This example will upload the sample file ccfraud.onnx to the workspace created in the Create Workspace step as apitestmodel. The model name will be saved as exampleModelName for use in other examples. The id of the uploaded model will be saved as exampleModelId for use in later examples.
# upload model - uses multiform data through a Python `request`apiRequest="/models/upload"exampleModelName=f"apitestmodel-{uuid.uuid4()}"data= {
"name":exampleModelName,
"visibility":"public",
"workspace_id": exampleWorkspaceId}
files= {
"file": ('ccfraud.onnx', open('./models/ccfraud.onnx','rb'))
}
response=get_wallaroo_response(APIURL, apiRequest, TOKEN, data, files)
response
Streams a potentially large ML Model to a Wallaroo workspace via POST with Content-Type: multipart/form-data.
Parameters
name - (REQUIRED string): Name of the model
filename - (REQUIRED string): Name of the file being uploaded.
visibility - (OPTIONAL string): The visibility of the model as either public or private.
workspace_id - (REQUIRED int): The numerical id of the workspace to upload the model to.
Example: This example will upload the sample file ccfraud.onnx to the workspace created in the Create Workspace step as apitestmodel.
# stream upload model - next test is adding arbitrary chunks to the streamapiRequest="/models/upload_stream"exampleModelName=f"apitestmodel-{uuid.uuid4()}"filename='streamfile.onnx'data= {
"name":exampleModelName,
"filename": 'streamfile.onnx',
"visibility":"public",
"workspace_id": exampleWorkspaceId}
contentType='application/octet-stream'file=open('./models/ccfraud.onnx','rb')
response=get_wallaroo_response(APIURL, apiRequest, TOKEN, data=None, files=file, contentType='application/octet-stream', params=data)
response
Returns a list of models added to a specific workspace.
Parameters
workspace_id - (REQUIRED int): The workspace id to list.
Example: Display the models for the workspace used in the Upload Model to Workspace step. The model id and model name will be saved as exampleModelId and exampleModelName variables for other examples.
# List models in a workspaceapiRequest="/models/list"data= {
"workspace_id": exampleWorkspaceId}
response=get_wallaroo_response(APIURL, apiRequest, TOKEN, data)
response
Returns the model details by the specific model id.
Parameters
workspace_id - (REQUIRED int): The workspace id to list.
Returns
id - (int): Numerical id of the model.
owner_id - (string): Id of the owner of the model.
workspace_id - (int): Numerical of the id the model is in.
name - (string): Name of the model.
updated_at - (DateTime): Date and time of the model’s last update.
created_at - (DateTime): Date and time of the model’s creation.
model_config - (string): Details of the model’s configuration.
Example: Retrieve the details for the model uploaded in the Upload Model to Workspace step.
# Get model details by idapiRequest="/models/get_by_id"data= {
"id": exampleModelId}
response=get_wallaroo_response(APIURL, apiRequest, TOKEN, data)
response
Retrieves all versions of a model based on either the name of the model or the model_pk_id.
Parameters
model_id - (REQUIRED String): The model name.
models_pk_id - (REQUIRED int): The model integer pk id.
Returns
Array(Model Details)
sha - (String): The sha hash of the model version.
models_pk_id- (int): The pk id of the model.
model_version - (String): The UUID identifier of the model version.
owner_id - (String): The Keycloak user id of the model’s owner.
model_id - (String): The name of the model.
id - (int): The integer id of the model.
file_name - (String): The filename used when uploading the model.
image_path - (String): The image path of the model.
Example: Retrieve the versions for a previously uploaded model. The variables exampleModelVersion and exampleModelSha will store the model’s version and SHA values for use in other examples.
# List models in a workspaceapiRequest="/models/list_versions"data= {
"model_id": exampleModelName,
"models_pk_id": exampleModelId}
response=get_wallaroo_response(APIURL, apiRequest, TOKEN, data)
response
# Stored for future examplesexampleModelVersion=response[0]['model_version']
exampleModelSha=response[0]['sha']
Get Model Configuration by Id
Returns the model’s configuration details.
Parameters
model_id - (REQUIRED int): The numerical value of the model’s id.
Example: Submit the model id for the model uploaded in the Upload Model to Workspace step to retrieve configuration details.
# Get model config by idapiRequest="/models/get_config_by_id"data= {
"model_id": exampleModelId}
response=get_wallaroo_response(APIURL, apiRequest, TOKEN, data)
response
{'model_config': None}
Get Model Details
Returns details regarding a single model, including versions.
Returns the model’s configuration details.
Parameters
model_id - (REQUIRED int): The numerical value of the model’s id.
Example: Submit the model id for the model uploaded in the Upload Model to Workspace step to retrieve configuration details.
# Get model config by idapiRequest="/models/get"data= {
"id": exampleModelId}
response=get_wallaroo_response(APIURL, apiRequest, TOKEN, data)
response
1.1.4 - Wallaroo MLOps API Essentials Guide: Pipeline Management
How to use the Wallaroo API for Pipeline Management
Pipeline Management
Pipelines can be managed through the Wallaroo API. Pipelines are the vehicle used for deploying, serving, and monitoring ML models. For more information, see the Wallaroo Glossary.
Creates a new pipeline in the specified workspace.
Parameters
pipeline_id - (REQUIRED string): Name of the new pipeline.
workspace_id - (REQUIRED int): Numerical id of the workspace for the new pipeline.
definition - (REQUIRED string): Pipeline definitions, can be {} for none.
Example: Two pipelines are created in the workspace created in the step Create Workspace. One will be an empty pipeline without any models, the other will be created using the uploaded models in the Upload Model to Workspace step and no configuration details. The pipeline id, variant id, and variant version of each pipeline will be stored for later examples.
Deploy a an existing pipeline. Note that for any pipeline that has model steps, they must be included either in model_configs, model_ids or models.
Parameters
deploy_id (REQUIRED string): The name for the pipeline deployment.
engine_config (OPTIONAL string): Additional configuration options for the pipeline.
pipeline_version_pk_id (REQUIRED int): Pipeline version id.
model_configs (OPTIONALArray int): Ids of model configs to apply.
model_ids (OPTIONALArray int): Ids of models to apply to the pipeline. If passed in, model_configs will be created automatically.
models (OPTIONAL Array models): If the model ids are not available as a pipeline step, the models’ data can be passed to it through this method. The options below are only required if models are provided as a parameter.
name (REQUIRED string): Name of the uploaded model that is in the same workspace as the pipeline.
version (REQUIRED string): Version of the model to use.
sha (REQUIRED string): SHA value of the model.
pipeline_id (REQUIRED int): Numerical value of the pipeline to deploy.
Returns
id (int): The deployment id.
Examples: Both the empty pipeline and pipeline with model created in the step Create Pipeline in a Workspace will be deployed and their deployment information saved for later examples.
The API command /admin/get_pipeline_external_url retrieves the external inference URL for a specific pipeline in a workspace.
Parameters
workspace_id (REQUIRED integer): The workspace integer id.
pipeline_name (REQUIRED string): The name of the deployment.
In this example, a list of the workspaces will be retrieved. Based on the setup from the Internal Pipeline Deployment URL Tutorial, the workspace matching urlworkspace will have it’s workspace id stored and used for the /admin/get_pipeline_external_url request with the pipeline urlpipeline.
The External Inference URL will be stored as a variable for the next step.
The inference can now be performed through the External Inference URL. This URL will accept the same inference data file that is used with the Wallaroo SDK, or with an Internal Inference URL as used in the Internal Pipeline Inference URL Tutorial.
For this example, the externalUrl retrieved through the Get External Inference URL is used to submit a single inference request.
If the Wallaroo instance has been configured to enable Arrow support, then the file cc_data_1k.df.json will be used. This is a DataFrame object. If Arrow support has not been enabled, then the inference request is used with the Wallaroo proprietary JSON data file cc_data_1k.json.
#TOKEN=get_jwt_token(TOKENURL, CLIENT, SECRET, USERNAME, PASSWORD)# if Arrow has been enabled, set this to True. Otherwise, leave as False.arrowEnabled=True## Inference through external URL# retrieve the json data to submitifarrowEnabledisTrue:
dataFile='./data/cc_data_1k.df.json'data=json.load(open('./data/cc_data_1k.df.json','rb'))
contentType="application/json; format=pandas-records"else:
dataFile='./data/cc_data_1k.json'data=json.load(open('./data/cc_data_1k.json','rb'))
contentType="application/json"# set the headersheaders= {
'Authorization': 'Bearer '+TOKEN,
'Content-Type': contentType}
# submit the request via POSTresponse=requests.post(externalUrl, json=data, headers=headers)
# Only the first 300 characters will be displayed for brevityprintResponse=json.dumps(response.json())
print(printResponse[0:300])
1.1.6 - Wallaroo MLOps API Essentials Guide: Assays Management
How to use the Wallaroo API for Assays Management
Assays
IMPORTANT NOTE: These assays were run in a Wallaroo environment with canned historical data. See the Wallaroo Assay Tutorial for details on setting up this environment. This historical data is required for these examples.
run_until - (OPTIONAL string): DateTime of when to end the assay.
workspace_id - (REQUIRED integer): The workspace the assay is part of.
model_insights_url - (OPTIONAL string): URL for model insights.
RETURNS
assay_id - (integer): The id of the new assay.
As noted this example requires the Wallaroo Assay Tutorial for historical data. Before running this example, set the sample pipeline id, pipeline, name, model name, and workspace id in the code sample below. For more information on retrieving this information, see the Wallaroo Developer Guides.
# Create assayapiRequest="/assays/create"exampleAssayName="api_assay_test2"## Now get all of the assays for the pipeline in workspace 4 `housepricedrift`exampleAssayPipelineId=4exampleAssayPipelineName="housepricepipe"exampleAssayModelName="housepricemodel"exampleAssayWorkspaceId=4# iopath can be input 00 or output 0 0data= {
'name': exampleAssayName,
'pipeline_id': exampleAssayPipelineId,
'pipeline_name': exampleAssayPipelineName,
'active': True,
'status': 'active',
'iopath': "input 0 0",
'baseline': {
'Fixed': {
'pipeline': exampleAssayPipelineName,
'model': 'houseprice-model-yns',
'start_at': '2022-01-01T00:00:00-05:00',
'end_at': '2022-01-02T00:00:00-05:00' }
},
'window': {
'pipeline': exampleAssayPipelineName,
'model': exampleAssayModelName,
'width': '24 hours',
'start': None,
'interval': None },
'summarizer': {
'type': 'UnivariateContinuous',
'bin_mode': 'Quantile',
'aggregation': 'Density',
'metric': 'PSI',
'num_bins': 5,
'bin_weights': None,
'bin_width': None,
'provided_edges': None,
'add_outlier_edges': True },
'warning_threshold': 0,
'alert_threshold': 0.1,
'run_until': None,
'workspace_id': exampleAssayWorkspaceId}
response=get_wallaroo_response(APIURL, apiRequest, TOKEN, data)
example_assay_id=response['assay_id']
response
{'assay_id': 2}
List Assays
Lists all assays in the specified pipeline.
PARAMETERS
pipeline_id - (REQUIRED int): The numerical ID of the pipeline.
RETURNS
assays - (Array assays): A list of all assays.
Example: Display a list of all assays in a workspace. This will assume we have a workspace with an existing Assay and the associated data has been upload. See the tutorial Wallaroo Assays Tutorial.
For this reason, these values are hard coded for now.
## First list all of the workspaces and the list of pipelines# List workspacesapiRequest="/workspaces/list"data= {
}
response=get_wallaroo_response(APIURL, apiRequest, TOKEN, data)
response
run_until - (OPTIONAL string): DateTime of when to end the assay.
workspace_id - (REQUIRED integer): The workspace the assay is part of.
model_insights_url - (OPTIONAL string): URL for model insights.
Returns
Assay
Example: An interactive assay will be run for Assay exampleAssayId exampleAssayName. Depending on the number of assay results and the data window, this may take some time. This returns all of the results for this assay at this time. The total number of responses will be displayed after.
assay_id - (REQUIRED integer): Numerical id for the assay.
start - (OPTIONAL string): DateTime for when the baseline starts.
end - (OPTIONAL string): DateTime for when the baseline ends.
limit - (OPTIONAL integer): Maximum number of results to return.
pipeline_id - (OPTIONAL integer): Numerical id of the pipeline the assay is in.
Returns
Assay Baseline
Example: Results for Assay 3 “example assay” will be retrieved for January 2 to January 3. For the sake of time, only the first record will be displayed.
Reference Guide for the most essential Wallaroo SDK Commands
2.1 - Wallaroo SDK Install Guides
How to install the Wallaroo SDK
The following guides demonstrate how to install the Wallaroo SDK in different environments. The Wallaroo SDK is installed by default into a Wallaroo instance for use with the JupyterHub service.
The Wallaroo SDK requires Python 3.8.6 and above and is available through the Wallaroo SDK Page.
Supported Model Versions and Libraries
The following ML Model versions and Python libraries are supported by Wallaroo. When using the Wallaroo autoconversion library or working with a local version of the Wallaroo SDK, use the following versions for maximum compatibility.
Organizations that develop machine learning models can deploy models to Wallaroo from AWS Sagemaker to a Wallaroo instance through the Wallaroo SDK. The following guide is created to assist users with installing the Wallaroo SDK and making a standard connection to a Wallaroo instance.
aloha-cnn-lstm.zip: A pre-trained open source model that uses an Aloha CNN LSTM model for classifying Domain names as being either legitimate or being used for nefarious purposes such as malware distribution.
Test Data Files:
data-1.json: 1 record
data-1k.json: 1,000 records
data-25k.json: 25,000 records
For this example, a virtual python environment will be used. This will set the necessary libraries and specific Python version required.
Prerequisites
The following is required for this tutorial:
A Wallaroo instance version 2023.1 or later.
A AWS Sagemaker domain with a Notebook Instance.
Python 3.8.6 or later installed locally.
General Steps
For our example, we will perform the following:
Install Wallaroo SDK
Set up a Python virtual environment through conda with the libraries that enable the virtual environment for use in a Jupyter Hub environment.
Install the Wallaroo SDK.
Wallaroo SDK from remote JupyterHub Demonstration (Optional): The following steps are an optional exercise to demonstrate using the Wallaroo SDK from a remote connection. The entire tutorial can be found on the Wallaroo Tutorials repository.
Connect to a remote Wallaroo instance.
Create a workspace for our work.
Upload the Aloha model.
Create a pipeline that can ingest our submitted data, submit it to the model, and export the results
Run a sample inference through our pipeline by loading a file
Retrieve the external deployment URL. This sample Wallaroo instance has been configured to create external inference URLs for pipelines. For more information, see the External Inference URL Guide.
Run a sample inference through our pipeline’s external URL and store the results in a file. This assumes that the External Inference URLs have been enabled for the target Wallaroo instance.
Undeploy the pipeline and return resources back to the Wallaroo instance’s Kubernetes environment.
Install Wallaroo SDK
Set Up Virtual Python Environment
To set up the Python virtual environment for use of the Wallaroo SDK:
From AWS Sagemaker, select the Notebook instances.
For the list of notebook instances, select Open JupyterLab for the notebook instance to be used.
From the Launcher, select Terminal.
From a terminal shell, create the Python virtual environment with conda. Replace wallaroosdk with the name of the virtual environment as required by your organization. Note that Python 3.8.6 and above is specified as a requirement for Python libraries used with the Wallaroo SDK. The following will install the latest version of Python 3.8.
conda create -n wallaroosdk python=3.8
(Optional) If the shells have not been initialized with conda, use the following to initialize it. The following examples will use the bash shell.
Initialize the bash shell with conda with the command:
conda init bash
Launch the bash shell that has been initialized for conda:
bash
Activate the new environment.
conda activate wallaroosdk
Install the ipykernel library. This allows the JupyterHub notebooks to access the Python virtual environment as a kernel, and it required for the second part of this tutorial.
conda install ipykernel
Install the new virtual environment as a python kernel.
ipython kernel install --user --name=wallaroosdk
Install the Wallaroo SDK. This process may take several minutes while the other required Python libraries are added to the virtual environment.
pip install wallaroo==2023.1.0
For organizations who will be using the Wallaroo SDK with Jupyter or similar services, the conda virtual environment has been installed, it can either be selected as a new Jupyter Notebook kernel, or the Notebook’s kernel can be set to an existing Jupyter notebook.
To use a new Notebook:
From the main menu, select File->New-Notebook.
From the Kernel selection dropbox, select the new virtual environment - in this case, wallaroosdk.
To update an existing Notebook to use the new virtual environment as a kernel:
From the main menu, select Kernel->Change Kernel.
Select the new kernel.
Sample Wallaroo Connection
With the Wallaroo Python SDK installed, remote commands and inferences can be performed through the following steps.
Open a Connection to Wallaroo
The first step is to connect to Wallaroo through the Wallaroo client.
This is accomplished using the wallaroo.Client(api_endpoint, auth_endpoint, auth_type command) command that connects to the Wallaroo instance services.
The Client method takes the following parameters:
api_endpoint (String): The URL to the Wallaroo instance API service.
auth_endpoint (String): The URL to the Wallaroo instance Keycloak service.
auth_type command (String): The authorization type. In this case, SSO.
The URLs are based on the Wallaroo Prefix and Wallaroo Suffix for the Wallaroo instance. For more information, see the DNS Integration Guide. In the example below, replace “YOUR PREFIX” and “YOUR SUFFIX” with the Wallaroo Prefix and Suffix, respectively.
Once run, the wallaroo.Client command provides a URL to grant the SDK permission to your specific Wallaroo environment. When displayed, enter the URL into a browser and confirm permissions. Depending on the configuration of the Wallaroo instance, the user will either be presented with a login request to the Wallaroo instance or be authenticated through a broker such as Google, Github, etc. To use the broker, select it from the list under the username/password login forms. For more information on Wallaroo authentication configurations, see the Wallaroo Authentication Configuration Guides.
Once authenticated, the user will verify adding the device the user is establishing the connection from. Once both steps are complete, then the connection is granted.
The connection is stored in the variable wl for use in all other Wallaroo calls.
# SSO login through keycloakwallarooPrefix="YOUR PREFIX"wallarooSuffix="YOUR SUFFIX"wl=wallaroo.Client(api_endpoint=f"https://{wallarooPrefix}.api.{wallarooSuffix}",
auth_endpoint=f"https://{wallarooPrefix}.keycloak.{wallarooSuffix}",
auth_type="sso")
Wallaroo Remote SDK Examples
The following examples can be used by an organization to test using the Wallaroo SDK from a remote location from their Wallaroo instance. These examples show how to create workspaces, deploy pipelines, and perform inferences through the SDK and API.
Create the Workspace
We will create a workspace to work in and call it the sdkworkspace, then set it as current workspace environment. We’ll also create our pipeline in advance as sdkpipeline.
IMPORTANT NOTE: For this example, the Aloha model is stored in the file alohacnnlstm.zip. When using tensor based models, the zip file must match the name of the tensor directory. For example, if the tensor directory is alohacnnlstm, then the .zip file must be named alohacnnlstm.zip.
Now we will upload our model. Note that for this example we are applying the model from a .ZIP file. The Aloha model is a protobuf file that has been defined for evaluating web pages, and we will configure it to use data in the tensorflow format.
Now that we have a model that we want to use we will create a deployment for it.
We will tell the deployment we are using a tensorflow model and give the deployment name and the configuration we want for the deployment.
To do this, we’ll create our pipeline that can ingest the data, pass the data to our Aloha model, and give us a final output. We’ll call our pipeline externalsdkpipeline, then deploy it so it’s ready to receive data. The deployment process usually takes about 45 seconds.
pipeline.add_model_step(model)
name
sdkquickpipeline
created
2022-12-20 15:56:22.974010+00:00
last_updated
2022-12-20 15:56:22.974010+00:00
deployed
(none)
tags
versions
9cb1b842-f28b-4a9b-b0c3-eeced6067582
steps
pipeline.deploy()
We can verify that the pipeline is running and list what models are associated with it.
pipeline.status()
Interferences
Infer 1 row
Now that the pipeline is deployed and our Aloha model is in place, we’ll perform a smoke test to verify the pipeline is up and running properly. We’ll use the infer_from_file command to load a single encoded URL into the inference engine and print the results back out.
The result should tell us that the tokenized URL is legitimate (0) or fraud (1). This sample data should return close to 0.
## Demonstrate via straight inferimportjsonfile=open('./data-1.json')
data=json.load(file)
result=pipeline.infer(data)
print(result)
# Demonstrate from infer_from_fileresult=pipeline.infer_from_file("./data-1.json")
result[0].data()
Batch Inference
Now that our smoke test is successful, let’s really give it some data. We have two inference files we can use:
data-1k.json: Contains 10,000 inferences
data-25k.json: Contains 25,000 inferences
We’ll pipe the data-25k.json file through the pipeline deployment URL, and place the results in a file named response.txt. We’ll also display the time this takes. Note that for larger batches of 50,000 inferences or more can be difficult to view in Juypter Hub because of its size.
When retrieving the pipeline inference URL through an external SDK connection, the External Inference URL will be returned. This URL will function provided that the Enable external URL inference endpoints is enabled. For more information, see the Wallaroo Model Endpoints Guide.
The API connection details can be retrieved through the Wallaroo client mlops() command. This will display the connection URL, bearer token, and other information. The bearer token is available for one hour before it expires.
For this example, the API connection details will be retrieved, then used to submit an inference request through the external inference URL retrieved earlier.
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 13.0M 100 10.1M 100 2886k 631k 174k 0:00:16 0:00:16 --:--:-- 2766k886k 0 330k 0:00:08 0:00:08 --:--:-- 0
Undeploy Pipeline
When finished with our tests, we will undeploy the pipeline so we have the Kubernetes resources back for other tasks. Note that if the deployment variable is unchanged pipeline.deploy() will restart the inference engine in the same configuration as before.
Installing the Wallaroo SDK into Azure ML Workspace
Organizations that use Azure ML for model training and development can deploy models to Wallaroo through the Wallaroo SDK. The following guide is created to assist users with installing the Wallaroo SDK, setting up authentication through Azure ML, and making a standard connection to a Wallaroo instance through Azure ML Workspace.
aloha-cnn-lstm.zip: A pre-trained open source model that uses an Aloha CNN LSTM model for classifying Domain names as being either legitimate or being used for nefarious purposes such as malware distribution.
Test Data Files:
data-1.json: 1 record
data-1k.json: 1,000 records
data-25k.json: 25,000 records
To use the Wallaroo SDK within Azure ML Workspace, a virtual environment will be used. This will set the necessary libraries and specific Python version required.
Prerequisites
The following is required for this tutorial:
A Wallaroo instance version 2023.1 or later.
Python 3.8.6 or later installed locally
Conda: Used for managing python virtual environments. This is automatically included in Azure ML Workspace.
An Azure ML workspace is created with a compute configured.
General Steps
For our example, we will perform the following:
Wallaroo SDK Install
Set up a Python virtual environment through conda with the libraries that enable the virtual environment for use in a Jupyter Hub environment.
Install the Wallaroo SDK.
Wallaroo SDK from remote JupyterHub Demonstration (Optional): The following steps are an optional exercise to demonstrate using the Wallaroo SDK from a remote connection. The entire tutorial can be found on the Wallaroo Tutorials repository).
Connect to a remote Wallaroo instance.
Create a workspace for our work.
Upload the Aloha model.
Create a pipeline that can ingest our submitted data, submit it to the model, and export the results
Run a sample inference through our pipeline by loading a file
Retrieve the external deployment URL. This sample Wallaroo instance has been configured to create external inference URLs for pipelines. For more information, see the External Inference URL Guide.
Run a sample inference through our pipeline’s external URL and store the results in a file. This assumes that the External Inference URLs have been enabled for the target Wallaroo instance.
Undeploy the pipeline and return resources back to the Wallaroo instance’s Kubernetes environment.
Install Wallaroo SDK
Set Up Virtual Python Environment
To set up the virtual environment in Azure ML for using the Wallaroo SDK with Azure ML Workspace:
Select Notebooks.
Create a new folder where the Jupyter Notebooks for Wallaroo will be installed.
From this repository, upload sdk-install-guides/azure-ml-sdk-install.zip, or upload the entire folder sdk-install-guides/azure-ml-sdk-install. This tutorial will assume the .zip file was uploaded.
Select Open Terminal. Navigate to the target directory.
Run unzip azure-ml-sdk-install.zip to unzip the directory, then cd into it with cd azure-ml-sdk-install.
Create the Python virtual environment with conda. Replace wallaroosdk with the name of the virtual environment as required by your organization. Note that Python 3.8.6 and above is specified as a requirement for Python libraries used with the Wallaroo SDK. The following will install the latest version of Python 3.8, which as of this time is 3.8.15.
conda create -n wallaroosdk python=3.8
Activate the new environment.
conda activate wallaroosdk
Install the ipykernel library. This allows the JupyterHub notebooks to access the Python virtual environment as a kernel.
conda install ipykernel
Install the new virtual environment as a python kernel.
ipython kernel install --user --name=wallaroosdk
Install the Wallaroo SDK. This process may take several minutes while the other required Python libraries are added to the virtual environment.
pip install wallaroo==2023.1.0
IMPORTANT NOTE
The version of the Wallaroo SDK should match the Wallaroo instance. For example, this example connects to a Wallaroo Enterprise version <code>2023.1</code> instance, so the SDK version should be <code>wallaroo==2023.1.0</code>.
Once the conda virtual environment has been installed, it can either be selected as a new Jupyter Notebook kernel, or the Notebook’s kernel can be set to an existing Jupyter notebook. If a notebook is existing, close it then reopen to select the new Wallaroo SDK environment.
To use a new Notebook:
From the left navigation panel, select +->Notebook.
From the Kernel selection dropbox on the upper right side, select the new virtual environment - in this case, wallaroosdk.
To update an existing Notebook to use the new virtual environment as a kernel:
From the main menu, select Kernel->Change Kernel.
Select the new kernel.
Sample Wallaroo Connection
With the Wallaroo Python SDK installed, remote commands and inferences can be performed through the following steps.
Open a Connection to Wallaroo
The first step is to connect to Wallaroo through the Wallaroo client.
This is accomplished using the wallaroo.Client(api_endpoint, auth_endpoint, auth_type command) command that connects to the Wallaroo instance services.
The Client method takes the following parameters:
api_endpoint (String): The URL to the Wallaroo instance API service.
auth_endpoint (String): The URL to the Wallaroo instance Keycloak service.
auth_type command (String): The authorization type. In this case, SSO.
The URLs are based on the Wallaroo Prefix and Wallaroo Suffix for the Wallaroo instance. For more information, see the DNS Integration Guide. In the example below, replace “YOUR PREFIX” and “YOUR SUFFIX” with the Wallaroo Prefix and Suffix, respectively.
Once run, the wallaroo.Client command provides a URL to grant the SDK permission to your specific Wallaroo environment. When displayed, enter the URL into a browser and confirm permissions. Depending on the configuration of the Wallaroo instance, the user will either be presented with a login request to the Wallaroo instance or be authenticated through a broker such as Google, Github, etc. To use the broker, select it from the list under the username/password login forms. For more information on Wallaroo authentication configurations, see the Wallaroo Authentication Configuration Guides.
Once authenticated, the user will verify adding the device the user is establishing the connection from. Once both steps are complete, then the connection is granted.
The connection is stored in the variable wl for use in all other Wallaroo calls.
importwallaroofromwallaroo.objectimportEntityNotFoundError# used to display dataframe information without truncatingfromIPython.displayimportdisplayimportpandasaspdpd.set_option('display.max_colwidth', None)
# SSO login through keycloakwallarooPrefix="YOURPREFIX"wallarooSuffix="YOURSUFFIX"wl=wallaroo.Client(api_endpoint=f"https://{wallarooPrefix}.api.{wallarooSuffix}",
auth_endpoint=f"https://{wallarooPrefix}.keycloak.{wallarooSuffix}",
auth_type="sso")
Arrow Support
As of the 2023.1 release, Wallaroo provides support for dataframe and Arrow for inference inputs. This tutorial allows users to adjust their experience based on whether they have enabled Arrow support in their Wallaroo instance or not.
If Arrow support has been enabled, arrowEnabled=True. If disabled or you’re not sure, set it to arrowEnabled=False
The examples below will be shown in an arrow enabled environment.
We will create a workspace to work in and call it the azuremlsdkworkspace, then set it as current workspace environment. We’ll also create our pipeline in advance as azuremlsdkpipeline.
IMPORTANT NOTE: For this example, the Aloha model is stored in the file alohacnnlstm.zip. When using tensor based models, the zip file must match the name of the tensor directory. For example, if the tensor directory is alohacnnlstm, then the .zip file must be named alohacnnlstm.zip.
Now we will upload our model. Note that for this example we are applying the model from a .ZIP file. The Aloha model is a protobuf file that has been defined for evaluating web pages, and we will configure it to use data in the tensorflow format.
Now that we have a model that we want to use we will create a deployment for it.
We will tell the deployment we are using a tensorflow model and give the deployment name and the configuration we want for the deployment.
To do this, we’ll create our pipeline that can ingest the data, pass the data to our Aloha model, and give us a final output. We’ll call our pipeline externalsdkpipeline, then deploy it so it’s ready to receive data. The deployment process usually takes about 45 seconds.
Now that the pipeline is deployed and our Aloha model is in place, we’ll perform a smoke test to verify the pipeline is up and running properly. We’ll use the infer_from_file command to load a single encoded URL into the inference engine and print the results back out.
The result should tell us that the tokenized URL is legitimate (0) or fraud (1). This sample data should return close to 0.
# Infer from fileifarrowEnabledisTrue:
result=pipeline.infer_from_file('./data/data_1.df.json')
display(result)
else:
result=pipeline.infer_from_file("./data/data_1.json")
display(result[0].data())
Batch Inference
Now that our smoke test is successful, let’s really give it some data. We have two inference files we can use:
data-1k.json: Contains 10,000 inferences
data-25k.json: Contains 25,000 inferences
We’ll pipe the data-25k.json file through the pipeline deployment URL, and place the results in a file named response.txt. We’ll also display the time this takes. Note that for larger batches of 50,000 inferences or more can be difficult to view in Juypter Hub because of its size.
When retrieving the pipeline inference URL through an external SDK connection, the External Inference URL will be returned. This URL will function provided that the Enable external URL inference endpoints is enabled. For more information, see the Wallaroo Model Endpoints Guide.
The API connection details can be retrieved through the Wallaroo client mlops() command. This will display the connection URL, bearer token, and other information. The bearer token is available for one hour before it expires.
For this example, the API connection details will be retrieved, then used to submit an inference request through the external inference URL retrieved earlier.
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 13.0M 100 10.1M 100 2886k 2322k 642k 0:00:04 0:00:04 --:--:-- 2965k
Undeploy Pipeline
When finished with our tests, we will undeploy the pipeline so we have the Kubernetes resources back for other tasks. Note that if the deployment variable is unchanged pipeline.deploy() will restart the inference engine in the same configuration as before.
Organizations that use Azure Databricks for model training and development can deploy models to Wallaroo through the Wallaroo SDK. The following guide is created to assist users with installing the Wallaroo SDK, setting up authentication through Azure Databricks, and making a standard connection to a Wallaroo instance through Azure Databricks Workspace.
ccfraud.onnx: A pretrained model from the Machine Learning Group’s demonstration on Credit Card Fraud detection.
Sample inference test data:
ccfraud_high_fraud.json: Test input file that returns a high likelihood of credit card fraud.
ccfraud_smoke_test.json: Test input file that returns a low likelihood of credit card fraud.
cc_data_1k.json: Sample input file with 1,000 records.
cc_data_10k.json: Sample input file with 10,000 records.
To use the Wallaroo SDK within Azure Databricks Workspace, a virtual environment will be used. This will set the necessary libraries and specific Python version required.
Install the Wallaroo SDK into the Azure Databricks cluster.
Install the Wallaroo Python SDK.
Connect to a remote Wallaroo instance. This instance is configured to use the standard Keycloak service.
Wallaroo SDK from Azure Databricks Workspace (Optional)
The following steps are used to demonstrate using the Wallaroo SDK in an Azure Databricks Workspace environment. The entire tutorial can be found on the Wallaroo Tutorials repository.
Create a workspace for our work.
Upload the CCFraud model.
Create a pipeline that can ingest our submitted data, submit it to the model, and export the results
Run a sample inference through our pipeline by loading a file
Undeploy the pipeline and return resources back to the Wallaroo instance’s Kubernetes environment.
Install Wallaroo SDK
Add Wallaroo SDK to Cluster
To install the Wallaroo SDK in a Azure Databricks environment:
From the Azure Databricks dashboard, select Computer, then the cluster to use.
Select Libraries.
Select Install new.
Select PyPI. In the Package field, enter the current version of the Wallaroo SDK. It is recommended to specify the version, which as of this writing is wallaroo==2023.1.0.
Select Install.
Once the Status shows Installed, it will be available in Azure Databricks notebooks and other tools that use the cluster.
Add Tutorial Files
The following instructions can be used to upload this tutorial and it’s files into Databricks. Depending on how your Azure Databricks is configured and your organizations standards, there are multiple ways of uploading files to your Azure Databricks environment. The following example is used for the tutorial and makes it easy to reference data files from within this Notebook. Adjust based on your requirements.
IMPORTANT NOTE: Importing a repo from a Git repository may not convert the included Jupyter Notebooks into the Databricks format. This method
From the Azure Databricks dashboard, select Repos.
Select where to place the repo, then select Add Repo.
Set the following:
Create repo by cloning a Git repository: Uncheck
Repository name: Set any name based on the Databricks standard (no spaces, etc).
Select Create Repo.
Select the new tutorial, then from the repo menu dropdown, select Import.
Select the files to upload. For this example, the following files are uploaded:
ccfraud.onnx: A pretrained model from the Machine Learning Group’s demonstration on Credit Card Fraud detection.
Sample inference test data:
ccfraud_high_fraud.json: Test input file that returns a high likelihood of credit card fraud.
ccfraud_smoke_test.json: Test input file that returns a low likelihood of credit card fraud.
cc_data_1k.json: Sample input file with 1,000 records.
cc_data_10k.json: Sample input file with 10,000 records.
install-wallaroo-sdk-databricks-azure-guide.ipynb: This notebook.
Select Import.
The Jupyter Notebook can be opened from this new Azure Databricks repository, and relative files it references will be accessible with the exceptions listed below.
Zip files added via the method above are automatically decompressed, so can not be used as model files. For example, tensor based models such as the Wallaroo Aloha Demo. Zip files can be uploaded using DBFS and used through the following process:
To upload model files to Azure Databricks using DBFS:
From the Azure Databricks dashboard, select Data.
Select Add->Add data.
Select DBFS.
Select Upload File and enter the following:
DBFS Target Directory (Optional): Optional step: Set the directory where the files will be uploaded.
Select the files to upload. Note that each file will be given a location and they can be access with /dbfs/PATH. For example, the file alohacnnlstm.zip uploaded to the directory aloha would be referenced with `/dbfs/FileStore/tables/aloha/alohacnnlstm.zip
Sample Wallaroo Connection
With the Wallaroo Python SDK installed, remote commands and inferences can be performed through the following steps.
Open a Connection to Wallaroo
The first step is to connect to Wallaroo through the Wallaroo client.
This is accomplished using the wallaroo.Client(api_endpoint, auth_endpoint, auth_type command) command that connects to the Wallaroo instance services.
The Client method takes the following parameters:
api_endpoint (String): The URL to the Wallaroo instance API service.
auth_endpoint (String): The URL to the Wallaroo instance Keycloak service.
auth_type command (String): The authorization type. In this case, SSO.
The URLs are based on the Wallaroo Prefix and Wallaroo Suffix for the Wallaroo instance. For more information, see the DNS Integration Guide. In the example below, replace “YOUR PREFIX” and “YOUR SUFFIX” with the Wallaroo Prefix and Suffix, respectively. In the example below, replace “YOUR PREFIX” and “YOUR SUFFIX” with the Wallaroo Prefix and Suffix, respectively.
Once run, the wallaroo.Client command provides a URL to grant the SDK permission to your specific Wallaroo environment. When displayed, enter the URL into a browser and confirm permissions.
Depending on the configuration of the Wallaroo instance, the user will either be presented with a login request to the Wallaroo instance or be authenticated through a broker such as Google, Github, etc. To use the broker, select it from the list under the username/password login forms. For more information on Wallaroo authentication configurations, see the Wallaroo Authentication Configuration Guides.
Once authenticated, the user will verify adding the device the user is establishing the connection from. Once both steps are complete, then the connection is granted.
The connection is stored in the variable wl for use in all other Wallaroo calls.
Replace YOUR PREFIX and YOUR SUFFIX with the DNS prefix and suffix for the Wallaroo instance. For more information, see the DNS Integration Guide.
# SSO login through keycloakwallarooPrefix="YOUR PREFIX"wallarooSuffix="YOUR SUFFIX"wl=wallaroo.Client(api_endpoint=f"https://{wallarooPrefix}.api.{wallarooSuffix}",
auth_endpoint=f"https://{wallarooPrefix}.keycloak.{wallarooSuffix}",
auth_type="sso")
Create the Workspace
We will create a workspace to work in and call it the databricksazuresdkworkspace, then set it as current workspace environment. We’ll also create our pipeline in advance as databricksazuresdkpipeline.
IMPORTANT NOTE: For this example, the CCFraud model is stored in the file ccfraud.onnx and is referenced from a relative link. For platforms such as Databricks, the files may need to be in a universal file format. For those, the example file location below may be:
Now that we have a model that we want to use we will create a deployment for it.
To do this, we’ll create our pipeline that can ingest the data, pass the data to our CCFraud model, and give us a final output. We’ll call our pipeline databricksazuresdkpipeline, then deploy it so it’s ready to receive data. The deployment process usually takes about 45 seconds.
Now that the pipeline is deployed and our CCfraud model is in place, we’ll perform a smoke test to verify the pipeline is up and running properly. We’ll use the infer_from_file command to load a single transaction and determine if it is flagged for fraud. If it returns correctly, a small value should be returned indicating a low likelihood that the transaction was fraudulent.
Now that our smoke test is successful, let’s really give it some data. We’ll use the cc_data_1k.json file that contains 1,000 inferences to be performed.
result=pipeline.infer_from_file("./cc_data_1k.json")
result
When finished with our tests, we will undeploy the pipeline so we have the Kubernetes resources back for other tasks. Note that if the deployment variable is unchanged pipeline.deploy() will restart the inference engine in the same configuration as before.
Installing the Wallaroo SDK into Google Vertex Workbench
Organizations that use Google Vertex for model training and development can deploy models to Wallaroo through the Wallaroo SDK. The following guide is created to assist users with installing the Wallaroo SDK, setting up authentication through Google Cloud Platform (GCP), and making a standard connection to a Wallaroo instance through Google Workbench.
aloha-cnn-lstm.zip: A pre-trained open source model that uses an Aloha CNN LSTM model for classifying Domain names as being either legitimate or being used for nefarious purposes such as malware distribution.
Test Data Files:
data-1.json: 1 record
data-1k.json: 1,000 records
data-25k.json: 25,000 records
To use the Wallaroo SDK within Google Workbench, a virtual environment will be used. This will set the necessary libraries and specific Python version required.
Prerequisites
The following is required for this tutorial:
A Wallaroo instance version 2023.1 or later.
Python 3.8.6 or later installed locally
Conda: Used for managing python virtual environments.
General Steps
For our example, we will perform the following:
Wallaroo SDK Install
Set up a Python virtual environment through conda with the libraries that enable the virtual environment for use in a Jupyter Hub environment.
Install the Wallaroo SDK.
Wallaroo SDK from remote JupyterHub Demonstration (Optional): The following steps are an optional exercise to demonstrate using the Wallaroo SDK from a remote connection. The entire tutorial can be found on the Wallaroo Tutorials repository.
Connect to a remote Wallaroo instance.
Create a workspace for our work.
Upload the Aloha model.
Create a pipeline that can ingest our submitted data, submit it to the model, and export the results
Run a sample inference through our pipeline by loading a file
Retrieve the external deployment URL. This sample Wallaroo instance has been configured to create external inference URLs for pipelines. For more information, see the External Inference URL Guide.
Run a sample inference through our pipeline’s external URL and store the results in a file. This assumes that the External Inference URLs have been enabled for the target Wallaroo instance.
Undeploy the pipeline and return resources back to the Wallaroo instance’s Kubernetes environment.
Install Wallaroo SDK
Set Up Virtual Python Environment
To set up the virtual environment in Google Workbench for using the Wallaroo SDK with Google Workbench:
Start a separate terminal by selecting File->New->Terminal.
Create the Python virtual environment with conda. Replace wallaroosdk with the name of the virtual environment as required by your organization. Note that Python 3.8.6 and above is specified as a requirement for Python libraries used with the Wallaroo SDK. The following will install the latest version of Python 3.8, which as of this time is 3.8.15.
conda create -n wallaroosdk python=3.8
Activate the new environment.
conda activate wallaroosdk
Install the ipykernel library. This allows the JupyterHub notebooks to access the Python virtual environment as a kernel.
conda install ipykernel
Install the new virtual environment as a python kernel.
ipython kernel install --user --name=wallaroosdk
Install the Wallaroo SDK. This process may take several minutes while the other required Python libraries are added to the virtual environment.
pip install wallaroo==2023.1.0
IMPORTANT NOTE
The version of the Wallaroo SDK should match the Wallaroo instance. For example, this example connects to a Wallaroo Enterprise version <code>2023.1</code> instance, so the SDK version should be <code>wallaroo==2023.1.0</code>.
Once the conda virtual environment has been installed, it can either be selected as a new Jupyter Notebook kernel, or the Notebook’s kernel can be set to an existing Jupyter notebook.
To use a new Notebook:
From the main menu, select File->New-Notebook.
From the Kernel selection dropbox, select the new virtual environment - in this case, wallaroosdk.
To update an existing Notebook to use the new virtual environment as a kernel:
From the main menu, select Kernel->Change Kernel.
Select the new kernel.
Sample Wallaroo Connection
With the Wallaroo Python SDK installed, remote commands and inferences can be performed through the following steps.
Open a Connection to Wallaroo
The first step is to connect to Wallaroo through the Wallaroo client.
This is accomplished using the wallaroo.Client(api_endpoint, auth_endpoint, auth_type command) command that connects to the Wallaroo instance services.
The Client method takes the following parameters:
api_endpoint (String): The URL to the Wallaroo instance API service.
auth_endpoint (String): The URL to the Wallaroo instance Keycloak service.
auth_type command (String): The authorization type. In this case, SSO.
The URLs are based on the Wallaroo Prefix and Wallaroo Suffix for the Wallaroo instance. For more information, see the DNS Integration Guide. In the example below, replace “YOUR PREFIX” and “YOUR SUFFIX” with the Wallaroo Prefix and Suffix, respectively.
Once run, the wallaroo.Client command provides a URL to grant the SDK permission to your specific Wallaroo environment. When displayed, enter the URL into a browser and confirm permissions. Depending on the configuration of the Wallaroo instance, the user will either be presented with a login request to the Wallaroo instance or be authenticated through a broker such as Google, Github, etc. To use the broker, select it from the list under the username/password login forms. For more information on Wallaroo authentication configurations, see the Wallaroo Authentication Configuration Guides.
Once authenticated, the user will verify adding the device the user is establishing the connection from. Once both steps are complete, then the connection is granted.
The connection is stored in the variable wl for use in all other Wallaroo calls.
importwallaroofromwallaroo.objectimportEntityNotFoundError# used to display dataframe information without truncatingfromIPython.displayimportdisplayimportpandasaspdpd.set_option('display.max_colwidth', None)
# SSO login through keycloakwallarooPrefix="YOURPREFIX"wallarooSuffix="YOURSUFFIX"wl=wallaroo.Client(api_endpoint=f"https://{wallarooPrefix}.api.{wallarooSuffix}",
auth_endpoint=f"https://{wallarooPrefix}.keycloak.{wallarooSuffix}",
auth_type="sso")
Arrow Support
As of the 2023.1 release, Wallaroo provides support for dataframe and Arrow for inference inputs. This tutorial allows users to adjust their experience based on whether they have enabled Arrow support in their Wallaroo instance or not.
If Arrow support has been enabled, arrowEnabled=True. If disabled or you’re not sure, set it to arrowEnabled=False
The examples below will be shown in an arrow enabled environment.
We will create a workspace to work in and call it the gcpsdkworkspace, then set it as current workspace environment. We’ll also create our pipeline in advance as gcpsdkpipeline.
IMPORTANT NOTE: For this example, the Aloha model is stored in the file alohacnnlstm.zip. When using tensor based models, the zip file must match the name of the tensor directory. For example, if the tensor directory is alohacnnlstm, then the .zip file must be named alohacnnlstm.zip.
Now we will upload our model. Note that for this example we are applying the model from a .ZIP file. The Aloha model is a protobuf file that has been defined for evaluating web pages, and we will configure it to use data in the tensorflow format.
Now that we have a model that we want to use we will create a deployment for it.
We will tell the deployment we are using a tensorflow model and give the deployment name and the configuration we want for the deployment.
To do this, we’ll create our pipeline that can ingest the data, pass the data to our Aloha model, and give us a final output. We’ll call our pipeline externalsdkpipeline, then deploy it so it’s ready to receive data. The deployment process usually takes about 45 seconds.
Now that the pipeline is deployed and our Aloha model is in place, we’ll perform a smoke test to verify the pipeline is up and running properly. We’ll use the infer_from_file command to load a single encoded URL into the inference engine and print the results back out.
The result should tell us that the tokenized URL is legitimate (0) or fraud (1). This sample data should return close to 0.
# Infer from fileifarrowEnabledisTrue:
result=pipeline.infer_from_file('./data/data_1.df.json')
display(result)
else:
result=pipeline.infer_from_file("./data/data_1.json")
display(result[0].data())
Batch Inference
Now that our smoke test is successful, let’s really give it some data. We have two inference files we can use:
data-1k.json: Contains 10,000 inferences
data-25k.json: Contains 25,000 inferences
We’ll pipe the data-25k.json file through the pipeline deployment URL, and place the results in a file named response.txt. We’ll also display the time this takes. Note that for larger batches of 50,000 inferences or more can be difficult to view in Juypter Hub because of its size.
When retrieving the pipeline inference URL through an external SDK connection, the External Inference URL will be returned. This URL will function provided that the Enable external URL inference endpoints is enabled. For more information, see the Wallaroo Model Endpoints Guide.
The API connection details can be retrieved through the Wallaroo client mlops() command. This will display the connection URL, bearer token, and other information. The bearer token is available for one hour before it expires.
For this example, the API connection details will be retrieved, then used to submit an inference request through the external inference URL retrieved earlier.
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 13.0M 100 10.1M 100 2886k 2322k 642k 0:00:04 0:00:04 --:--:-- 2965k
Undeploy Pipeline
When finished with our tests, we will undeploy the pipeline so we have the Kubernetes resources back for other tasks. Note that if the deployment variable is unchanged pipeline.deploy() will restart the inference engine in the same configuration as before.
pipeline.undeploy()
2.1.5 - Wallaroo SDK Standard Install Guide
How to install the Wallaroo SDK in typical environment
Organizations that develop machine learning models can deploy models to Wallaroo from their local systems to a Wallaroo instance through the Wallaroo SDK. The following guide is created to assist users with installing the Wallaroo SDK and making a standard connection to a Wallaroo instance.
aloha-cnn-lstm.zip: A pre-trained open source model that uses an Aloha CNN LSTM model for classifying Domain names as being either legitimate or being used for nefarious purposes such as malware distribution.
Test Data Files:
data-1.json: 1 record
data-1k.json: 1,000 records
data-25k.json: 25,000 records
For this example, a virtual python environment will be used. This will set the necessary libraries and specific Python version required.
Prerequisites
The following is required for this tutorial:
A Wallaroo instance version 2023.1 or later.
Python 3.8.6 or later installed locally.
Conda: Used for managing python virtual environments.
General Steps
For our example, we will perform the following:
Wallaroo SDK Install
Set up a Python virtual environment through conda with the libraries that enable the virtual environment for use in a Jupyter Hub environment.
Install the Wallaroo SDK.
Wallaroo SDK from remote JupyterHub Demonstration (Optional): The following steps are an optional exercise to demonstrate using the Wallaroo SDK from a remote connection. The entire tutorial can be found on the Wallaroo Tutorials repository.
Connect to a remote Wallaroo instance.
Create a workspace for our work.
Upload the Aloha model.
Create a pipeline that can ingest our submitted data, submit it to the model, and export the results
Run a sample inference through our pipeline by loading a file
Retrieve the external deployment URL. This sample Wallaroo instance has been configured to create external inference URLs for pipelines. For more information, see the External Inference URL Guide.
Run a sample inference through our pipeline’s external URL and store the results in a file. This assumes that the External Inference URLs have been enabled for the target Wallaroo instance.
Undeploy the pipeline and return resources back to the Wallaroo instance’s Kubernetes environment.
Install Wallaroo SDK
Set Up Virtual Python Environment
To set up the Python virtual environment for use of the Wallaroo SDK:
From a terminal shell, create the Python virtual environment with conda. Replace wallaroosdk with the name of the virtual environment as required by your organization. Note that Python 3.8.6 and above is specified as a requirement for Python libraries used with the Wallaroo SDK. The following will install the latest version of Python 3.8.
conda create -n wallaroosdk python=3.8
Activate the new environment.
conda activate wallaroosdk
(Optional) For organizations who want to use the Wallaroo SDk from within Jupyter and similar environments:
Install the ipykernel library. This allows the JupyterHub notebooks to access the Python virtual environment as a kernel, and it required for the second part of this tutorial.
conda install ipykernel
Install the new virtual environment as a python kernel.
ipython kernel install --user --name=wallaroosdk
Install the Wallaroo SDK. This process may take several minutes while the other required Python libraries are added to the virtual environment.
pip install wallaroo==2023.1.0
IMPORTANT NOTE
The version of the Wallaroo SDK should match the Wallaroo instance. For example, this example connects to a Wallaroo Enterprise version <code>2023.1</code> instance, so the SDK version should be <code>wallaroo==2023.1.0</code>.
For organizations who will be using the Wallaroo SDK with Jupyter or similar services, the conda virtual environment has been installed, it can either be selected as a new Jupyter Notebook kernel, or the Notebook’s kernel can be set to an existing Jupyter notebook.
To use a new Notebook:
From the main menu, select File->New-Notebook.
From the Kernel selection dropbox, select the new virtual environment - in this case, wallaroosdk.
To update an existing Notebook to use the new virtual environment as a kernel:
From the main menu, select Kernel->Change Kernel.
Select the new kernel.
Sample Wallaroo Connection
With the Wallaroo Python SDK installed, remote commands and inferences can be performed through the following steps.
Open a Connection to Wallaroo
The first step is to connect to Wallaroo through the Wallaroo client.
This is accomplished using the wallaroo.Client(api_endpoint, auth_endpoint, auth_type command) command that connects to the Wallaroo instance services.
The Client method takes the following parameters:
api_endpoint (String): The URL to the Wallaroo instance API service.
auth_endpoint (String): The URL to the Wallaroo instance Keycloak service.
auth_type command (String): The authorization type. In this case, SSO.
The URLs are based on the Wallaroo Prefix and Wallaroo Suffix for the Wallaroo instance. For more information, see the DNS Integration Guide. In the example below, replace “YOUR PREFIX” and “YOUR SUFFIX” with the Wallaroo Prefix and Suffix, respectively.
Once run, the wallaroo.Client command provides a URL to grant the SDK permission to your specific Wallaroo environment. When displayed, enter the URL into a browser and confirm permissions. Depending on the configuration of the Wallaroo instance, the user will either be presented with a login request to the Wallaroo instance or be authenticated through a broker such as Google, Github, etc. To use the broker, select it from the list under the username/password login forms. For more information on Wallaroo authentication configurations, see the Wallaroo Authentication Configuration Guides.
Once authenticated, the user will verify adding the device the user is establishing the connection from. Once both steps are complete, then the connection is granted.
The connection is stored in the variable wl for use in all other Wallaroo calls.
importwallaroofromwallaroo.objectimportEntityNotFoundError# used to display dataframe information without truncatingfromIPython.displayimportdisplayimportpandasaspdpd.set_option('display.max_colwidth', None)
As of the 2023.1 release, Wallaroo provides support for dataframe and Arrow for inference inputs. This tutorial allows users to adjust their experience based on whether they have enabled Arrow support in their Wallaroo instance or not.
If Arrow support has been enabled, arrowEnabled=True. If disabled or you’re not sure, set it to arrowEnabled=False
The examples below will be shown in an arrow enabled environment.
importos# Only set the below to make the OS environment ARROW_ENABLED to TRUE. Otherwise, leave as is.# os.environ["ARROW_ENABLED"]="True"if"ARROW_ENABLED"notinos.environoros.environ["ARROW_ENABLED"].casefold() =="False".casefold():
arrowEnabled=Falseelse:
arrowEnabled=Trueprint(arrowEnabled)
True
Wallaroo Remote SDK Examples
The following examples can be used by an organization to test using the Wallaroo SDK from a remote location from their Wallaroo instance. These examples show how to create workspaces, deploy pipelines, and perform inferences through the SDK and API.
Create the Workspace
We will create a workspace to work in and call it the sdkworkspace, then set it as current workspace environment. We’ll also create our pipeline in advance as sdkpipeline.
IMPORTANT NOTE: For this example, the Aloha model is stored in the file alohacnnlstm.zip. When using tensor based models, the zip file must match the name of the tensor directory. For example, if the tensor directory is alohacnnlstm, then the .zip file must be named alohacnnlstm.zip.
Now we will upload our model. Note that for this example we are applying the model from a .ZIP file. The Aloha model is a protobuf file that has been defined for evaluating web pages, and we will configure it to use data in the tensorflow format.
Now that we have a model that we want to use we will create a deployment for it.
We will tell the deployment we are using a tensorflow model and give the deployment name and the configuration we want for the deployment.
To do this, we’ll create our pipeline that can ingest the data, pass the data to our Aloha model, and give us a final output. We’ll call our pipeline externalsdkpipeline, then deploy it so it’s ready to receive data. The deployment process usually takes about 45 seconds.
Now that the pipeline is deployed and our Aloha model is in place, we’ll perform a smoke test to verify the pipeline is up and running properly. We’ll use the infer_from_file command to load a single encoded URL into the inference engine and print the results back out.
The result should tell us that the tokenized URL is legitimate (0) or fraud (1). This sample data should return close to 0.
Now that our smoke test is successful, let’s really give it some data. We have two inference files we can use:
data-1k.json: Contains 10,000 inferences
data-25k.json: Contains 25,000 inferences
We’ll pipe the data-25k.json file through the pipeline deployment URL, and place the results in a file named response.txt. We’ll also display the time this takes. Note that for larger batches of 50,000 inferences or more can be difficult to view in Juypter Hub because of its size.
When retrieving the pipeline inference URL through an external SDK connection, the External Inference URL will be returned. This URL will function provided that the Enable external URL inference endpoints is enabled. For more information, see the Wallaroo Model Endpoints Guide.
The API connection details can be retrieved through the Wallaroo client mlops() command. This will display the connection URL, bearer token, and other information. The bearer token is available for one hour before it expires.
For this example, the API connection details will be retrieved, then used to submit an inference request through the external inference URL retrieved earlier.
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 34.3M 100 16.3M 100 18.0M 1115k 1229k 0:00:15 0:00:15 --:--:-- 4153k
Undeploy Pipeline
When finished with our tests, we will undeploy the pipeline so we have the Kubernetes resources back for other tasks. Note that if the deployment variable is unchanged pipeline.deploy() will restart the inference engine in the same configuration as before.
Reference Guide for the most essential Wallaroo SDK Commands
The following commands are the most essential when working with Wallaroo.
Supported Model Versions and Libraries
The following ML Model versions and Python libraries are supported by Wallaroo. When using the Wallaroo autoconversion library or working with a local version of the Wallaroo SDK, use the following versions for maximum compatibility.
How to connect to a Wallaroo instance through the Wallaroo SDK
Connect to Wallaroo
Users connect to a Wallaroo instance with the Wallaroo Client class. This connection can be made from within the Wallaroo instance, or external from the Wallaroo instance via the Wallaroo SDK.
Once run, the wallaroo.Client command provides a URL to grant the SDK permission to your specific Wallaroo environment. When displayed, enter the URL into a browser and confirm permissions. Depending on the configuration of the Wallaroo instance, the user will either be presented with a login request to the Wallaroo instance or be authenticated through a broker such as Google, Github, etc. To use the broker, select it from the list under the username/password login forms. For more information on Wallaroo authentication configurations, see the Wallaroo Authentication Configuration Guides.
Once authenticated, the user will verify adding the device the user is establishing the connection from. Once both steps are complete, then the connection is granted.
Connect to Within the Wallaroo Instance
Users who connect to their Wallaroo instance will be authenticated with the Wallaroo Client() method.
The first step in using Wallaroo is creating a connection. To connect to your Wallaroo environment:
Import the wallaroo library:
importwallaroo
Open a connection to the Wallaroo environment with the wallaroo.Client() command and save it to a variable.
In this example, the Wallaroo connection is saved to the variable wl.
wl=wallaroo.Client()
A verification URL will be displayed. Enter it into your browser and grant access to the SDK client.
Once this is complete, you will be able to continue with your Wallaroo commands.
Connect from Outside the Wallaroo Instance
Users who have installed the Wallaroo SDK from an external location, such as their own JupyterHub service, Google Workbench, or other services can connect via Single-Sign On (SSO). This is accomplished using the wallaroo.Client(api_endpoint, auth_endpoint, auth_type command) command that connects to the Wallaroo instance services. For more information on the DNS names of Wallaroo services, see the DNS Integration Guide.
api_endpoint (String): The URL to the Wallaroo instance API service.
auth_endpoint (String): The URL to the Wallaroo instance Keycloak service.
auth_type command (String): The authorization type. In this case, SSO.
Once run, the wallaroo.Client command provides a URL to grant the SDK permission to your specific Wallaroo environment. When displayed, enter the URL into a browser and confirm permissions. This connection is stored into a variable that can be referenced later.
In this example, a connection will be made to the Wallaroo instance shadowy-unicorn-5555.wallaroo.ai through SSO authentication.
importwallaroofromwallaroo.objectimportEntityNotFoundError# SSO login through keycloakwl=wallaroo.Client(api_endpoint="https://shadowy-unicorn-5555.api.wallaroo.ai",
auth_endpoint="https://shadowy-unicorn-5555.keycloak.wallaroo.ai",
auth_type="sso")
How to create and use Wallaroo Workspaces through the Wallaroo SDK
Workspace Management
Workspaces are used to segment groups of models into separate environments. This allows different users to either manage or have access to each workspace, controlling the models and pipelines assigned to the workspace.
Workspace names are not forced to be unique. You can have 50 workspaces all named my-amazing-workspace, which can cause confusion in determining which workspace to use.
It is recommended that organizations agree on a naming convention and select the workspace to use rather than creating a new one each time.
To create a workspace, use the create_workspace("{WORKSPACE NAME}") command through an established Wallaroo connection and store the workspace settings into a new variable. Once the new workspace is created, the user who created the workspace is assigned as its owner. The following template is an example:
For example, if the connection is stored in the variable wl and the new workspace will be named imdb, then the command to store it in the new_workspace variable would be:
The command list_workspaces() displays the workspaces that are part of the current Wallaroo connection. The following details are returned as an array:
Parameter
Type
Description
Name
String
The name of the workspace. Note that workspace names are not unique.
Created At
DateTime
The date and time the workspace was created.
Users
Array[Users]
A list of all users assigned to this workspace.
Models
Integer
The number of models uploaded to the workspace.
Pipelines
Integer
The number of pipelines in the environment.
For example, for the Wallaroo connection wl the following workspaces are returned:
The current workspace can be set through set_current_workspace for the Wallaroo connection through the following call, and returns the workspace details as a JSON object:
The following example creates the workspace imdb-workspace through the Wallaroo connection stored in the variable wl, then sets it as the current workspace:
To set the current workspace from an established workspace, the easiest method is to use list_workspaces() then set the current workspace as the array value displayed. For example, from the following list_workspaces() command the 3rd workspace element demandcurve-workspace can be assigned as the current workspace:
Users are added to the workspace via their email address through the wallaroo.workspace.Workspace.add_user({email address}) command. The email address must be assigned to a current user in the Wallaroo platform before they can be assigned to the workspace.
For example, the following workspace imdb-workspace has the user steve@ex.co. We will add the user john@ex.co to this workspace:
Removing a user from a workspace is performed through the wallaroo.workspace.Workspace.remove_user({email address}) command, where the {email address} matches a user in the workspace.
In the following example, the user john@ex.co is removed from the workspace imdb-workspace.
To update the owner of workspace, or promote an existing user of a workspace to the owner of workspace, use the wallaroo.workspace.Workspace.add_owner({email address}) command. The email address must be assigned to a current user in the Wallaroo platform before they can be assigned as the owner to the workspace.
The following example shows assigning the user john@ex.co as an owner to the workspace imdb-workspace:
2.2.3 - Wallaroo SDK Essentials Guide: Model Management
How to create and manage Wallaroo Models through the Wallaroo SDK
Supported Model Versions and Libraries
The following ML Model versions and Python libraries are supported by Wallaroo. When using the Wallaroo autoconversion library or working with a local version of the Wallaroo SDK, use the following versions for maximum compatibility.
The following data types are supported for transporting data to and from Wallaroo in the following run times:
ONNX
TensorFlow
MLFlow
Float Types
Runtime
BFloat16*
Float16
Float32
Float64
ONNX
X
X
TensorFlow
X
X
X
MLFlow
X
X
X
* (Brain Float 16, represented internally as a f32)
Int Types
Runtime
Int8
Int16
Int32
Int64
ONNX
X
X
X
X
TensorFlow
X
X
X
X
MLFlow
X
X
X
X
Uint Types
Runtime
Uint8
Uint16
Uint32
Uint64
ONNX
X
X
X
X
TensorFlow
X
X
X
X
MLFlow
X
X
X
X
Other Types
Runtime
Boolean
Utf8 (String)
Complex 64
Complex 128
FixedSizeList*
ONNX
X
Tensor
X
X
X
MLFlow
X
X
X
* Fixed sized lists of any of the previously supported data types.
Model Management
Upload Models to a Workspace
Models are uploaded to the current workspace through the Wallaroo Client upload_model("{Model Name}", "{Model Path}).configure(options). In most cases, leaving the options field can be left blank. For more details, see the full SDK guide.
Models can either be uploaded in the Open Neural Network eXchange(ONNX) format, or be auto-converted and uploaded using the Wallaroo convert_model(path, source_type, conversion_arguments) method. For more information, see the tutorial series ONNX Conversion Tutorials.
Machine Learning (ML) models can be converted and uploaded into Wallaroo workspace using the Wallaroo Client convert_model(path, source_type, conversion_arguments) method. This conversion process transforms the model into an open format that can be run across different framework at compiled C-language speeds.
The three input parameters are:
path (STRING): The path to the ML model file.
source_type (ModelConversionSource): The type of ML model to be converted. As of this time Wallaroo auto-conversion supports the following source types and their associated ModelConversionSource:
sklearn: ModelConversionSource.SKLEARN
xgboost: ModelConversionSource.XGBOOST
keras: ModelConversionSource.KERAS
conversion_arguments: The arguments for the conversion based on the type of model being converted. These are:
wallaroo.ModelConversion.ConvertKerasArguments: Used for converting keras type models and takes the following parameters:
dimensions: Corresponds to the keras xtrain in the format [{Number of Rows/None}, {Number of Columns 1}, {Number of Columns 2}...]. For a standard 1-dimensional array with 100 columns this would typically be [None, 100].
wallaroo.ModelConversion.ConvertSKLearnArguments: Used for sklearn models and takes the following parameters:
name: The name of the model being converted.
comment: Any comments for the model.
number_of_columns: The number of columns the model was trained for.
Once uploaded, they will be displayed in the Wallaroo Models Dashboard as {unique-file-id}-converted.onnx:
ModelConversionInputTypes
The following data types are supported with the ModelConversionInputType parameter:
Parameter
Data Type
Float16
float16
Float32
float32
Float64
float64
Int16
int16
Int32
int32
Int64
int64
UInt8
uint8
UInt16
uint16
UInt32
uint32
UInt64
uint64
Boolean
bool
Double
double
sk-learn Example
The following example converts and uploads a Linear Regression sklearn model lm.pickle and stores it in the variable converted_model:
wl=wallaroo.Client()
workspace_name="testconversion"_=wl.set_current_workspace(get_or_create_workspace(workspace_name))
model_conversion_args=ConvertSKLearnArguments(
name="lm-test",
comment="test linear regression",
number_of_columns=NF,
input_type=ModelConversionInputType.Double)
model_conversion_type=ModelConversionSource.SKLEARN# convert the model and store it in the variable `converted_model`:converted_model=wl.convert_model('lm.pickle', model_conversion_type, model_conversion_args)
keras Example
The following example shows converting a keras model with 100 columns and uploading it to a Wallaroo instance:
The get_user_by_email({email}) command finds the user who’s email address matches the submitted {email} field. If no email address matches then the return will be null.
For example, the user steve with the email address steve@ex.co returns the following:
To remove a user’s access to the Wallaroo instance, use the Wallaroo Client deactivate_user("{User Email Address}) method, replacing the {User Email Address} with the email address of the user to deactivate.
To activate a user, use the Wallaroo Client active_user("{User Email Address}) method, replacing the {User Email Address} with the email address of the user to activate.
This feature impacts Wallaroo Community’s license count. Wallaroo Community only allows a total of 5 users per Wallaroo Community instance. Deactivated users does not count to this total - this allows organizations to add users, then activate/deactivate them as needed to stay under the total number of licensed users count.
Wallaroo Enterprise has no limits on the number of users who can be added or active in a Wallaroo instance.
In this example, the user testuser@wallaroo.ai will be deactivated then reactivated.
How to create and manage Wallaroo Pipelines through the Wallaroo SDK
Pipeline Management
Pipelines are the method of taking submitting data and processing that data through the models. Each pipeline can have one or more steps that submit the data from the previous step to the next one. Information can be submitted to a pipeline as a file, or through the pipeline’s URL.
New pipelines are created in the current workspace.
NOTICE
Pipeline names are not forced to be unique. You can have 50 pipelines all named my-pipeline, which can cause confusion in determining which workspace to use.
It is recommended that organizations agree on a naming convention and select pipeline to use rather than creating a new one each time.
To create a new pipeline, use the Wallaroo Client build_pipeline("{Pipeline Name}") command.
The following example creates a new pipeline imdb-pipeline through a Wallaroo Client connection wl:
imdb_pipeline=wl.build_pipeline("imdb-pipeline")
imdb_pipeline.status()
{'status': 'Pipeline imdb-pipeline is not deployed'}
List All Pipelines
The Wallaroo Client method list_pipelines() lists all pipelines in a Wallaroo Instance.
The following example lists all pipelines in the wl Wallaroo Client connection:
Rather than creating a new pipeline each time, an existing pipeline can be selected by using the list_pipelines() command and assigning one of the array members to a variable.
The following example sets the pipeline ccfraud-pipeline to the variable current_pipeline:
Once a pipeline has been created, or during its creation process, a pipeline step can be added. The pipeline step refers to the model that will perform an inference off of the data submitted to it. Each time a step is added, it is added to the pipeline’s models array.
A pipeline step is added through the pipeline add_model_step({Model}) command.
In the following example, two models uploaded to the workspace are added as pipeline step:
IMPORTANT NOTE
Pipeline steps can be replaced while a pipeline is deployed. This allows organizations to have pipelines deployed in a production environment and hot-swap out models for new versions without impacting performance or inferencing downtime.
The following parameters are used for replacing a pipeline step:
Parameter
Default Value
Purpose
index
null
The pipeline step to be replaced. Pipeline steps follow array numbering, where the first step is 0, etc.
model
null
The new model to be used in the pipeline step.
In the following example, a deployed pipeline will have the initial model step replaced with a new one. A status of the pipeline will be displayed after deployment and after the pipeline swap to show the model has been replaced from ccfraudoriginal to ccfraudreplacement, each with their own versions.
A Pipeline Step can be more than models - they can also be pre processing and post processing steps. For example, the Demand Curve Tutorial has both a pre and post processing steps that are added to the pipeline. The preprocessing step uses the following code:
importnumpyimportpandasimportjson# add interaction terms for the modeldefactual_preprocess(pdata):
pd=pdata.copy()
# convert boolean cust_known to 0/1pd.cust_known=numpy.where(pd.cust_known, 1, 0)
# interact UnitPrice and cust_knownpd['UnitPriceXcust_known'] =pd.UnitPrice*pd.cust_knownreturnpd.loc[:, ['UnitPrice', 'cust_known', 'UnitPriceXcust_known']]
# If the data is a json string, call this wrapper instead# Expected input:# a dictionary with fields 'colnames', 'data'# test that the code works heredefwallaroo_json(data):
obj=json.loads(data)
pdata=pandas.DataFrame(obj['query'],
columns=obj['colnames'])
pprocessed=actual_preprocess(pdata)
# return a dictionary, with the fields the model expectreturn {
'tensor_fields': ['model_input'],
'model_input': pprocessed.to_numpy().tolist()
}
It is added as a Python module by uploading it as a model:
# load the preprocess modulemodule_pre=wl.upload_model("preprocess", "./preprocess.py").configure('python')
And then added to the pipeline as a step:
# now make a pipelinedemandcurve_pipeline= (wl.build_pipeline("demand-curve-pipeline")
.add_model_step(module_pre)
.add_model_step(demand_curve_model)
.add_model_step(module_post))
Remove a Pipeline Step
To remove a step from the pipeline, use the Pipeline remove_step(index) command, where the index is the array index for the pipeline’s steps.
In the following example the pipeline imdb_pipeline will have the step with the model smodel-o removed.
Pipelines can be deployed to allocate more or fewer resources through the DeploymentConfig object that sets the pipeline’s autoscaling parameters.
Autoscaling allows the user to define how many engines a pipeline starts with, the minimum amount of engines a pipeline can be using, and the maximum amount of engines a pipeline can scale up to. The pipeline scales up and down based on the average CPU utilization across the engines in a given pipeline as the user’s workload increases and decreases.
This is performed through the wallaroo.DeploymentConfigBuilder method that returns a wallaroo.deployment_config.DeploymentConfig object. The DeploymentConfig is then applied to a Wallaroo pipeline when it is deployed.
The following parameters are used for auto-scaling:
Parameter
Default Value
Purpose
replica_count
1
Sets the initial amount of engines for the pipeline.
replica_autoscale_min_max
1
Sets the minimum number of engines and the maximum amount of engines. The maximum parameter must be set by the user.
autoscale_cpu_utilization
50
An integer representing the average CPU utilization. The default value is 50, which represents an average of 50% CPU utilization for the engines in a pipeline.
The DeploymentConfig is then built with the a build method.
The following example a DeploymentConfig will be created and saved to the variable ccfraudDeployConfig. It will set the minimum engines to 2, the maximum to 5, and use 60% of CPU utilization. This will then be applied to the deployment of the pipeline ccfraudPipeline by specifying it’s deployment_config parameter.
When a pipeline step is added or removed, the pipeline must be deployed through the pipeline deploy(). This allocates resources to the pipeline from the Kubernetes environment and make it available to submit information to perform inferences. This process typically takes 45 seconds.
Once complete, the pipeline status() command will show 'status':'Running'.
Pipeline deployments can be modified to enable auto-scaling to allow pipelines to allocate more or fewer resources based on need by setting the pipeline’s This will then be applied to the deployment of the pipelineccfraudPipelineby specifying it'sdeployment_config` optional parameter. If this optional parameter is not passed, then the deployment will defer to default values. For more information, see Manage Pipeline Deployment Configuration.
In the following example, the pipeline imdb-pipeline that contains two steps will be deployed with default deployment configuration:
If you deploy more pipelines than your environment can handle, or if you deploy more pipelines than your license allows, you may see an error like the following:
When a pipeline is not currently needed, it can be undeployed and its resources turned back to the Kubernetes environment. To undeploy a pipeline, use the pipeline undeploy() command.
In this example, the aloha_pipeline will be undeployed:
Pipeline have their own set of log files that can be retrieved and analyzed as needed with the pipeline.logs(limit=100) command. This command takes the following parameters:
Parameter
Type
Description
limit
Int
Limits how many log files to display. Defaults to 100.
Typically this only shows the last 3 commands in a Python notebook for spacing purposes.
In this example, the last 50 logs to the pipeline ccfraudpipeline are requested. Only one is shown for brevity.
Anomaly detection allows organizations to set validation parameters. A validation is added to a pipeline to test data based on a specific expression. If the expression is returned as False, this is detected as an anomaly and added to the InferenceResult object’s check_failures array and the pipeline logs.
Anomaly detection consists of the following steps:
Set a validation: Add a validation to a pipeline that, when returned False, adds an entry to the InferenceResult object’s check_failures attribute with the expression that caused the failure.
Display anomalies: Anomalies detected through a Pipeline’s validation attribute are displayed either through the InferenceResult object’s check_failures attribute, or through the pipeline’s logs.
Set A Validation
Validations are added to a pipeline through the wallaroo.pipelineadd_validation method. The following parameters are required:
The validation expression that adds the result InferenceResult object’s check_failures attribute when expression result is False.
Validation expressions take the format value Expression, with the expression being in the form of a :py:Expression:. For example, if the model housing_model is part of the pipeline steps, then a validation expression may be housing_model.outputs[0][0] < 100.0: If the output of the housing_model inference is less than 100, then the validation is True and no action is taken. Any values over 100, the validation is False which triggers adding the anomaly to the InferenceResult object’s check_failures attribute.
Note that multiple validations can be created to allow for multiple anomalies detection.
In the following example, a validation is added to the pipeline to detect housing prices that are below 100 (represented as $100 million), and trigger an anomaly for values above that level. When an inference is performed that triggers a validation failure, the results are displayed in the InferenceResult object’s check_failures attribute.
Anomalies detected through a Pipeline’s validation attribute are displayed either through the InferenceResult object’s check_failures attribute, or through the pipeline’s logs.
To display an anomaly through the InferenceResult object, display the check_failures attribute.
In the following example, the an InferenceResult where the validation failed will display the failure in the check_failures attribute:
A/B testing is a method that provides the ability to test competing ML models for performance, accuracy or other useful benchmarks. Different models are added to the same pipeline steps as follows:
Control or Champion model: The model currently used for inferences.
Challenger model(s): The model or set of models compared to the challenger model.
A/B testing splits a portion of the inference requests between the champion model and the one or more challengers through the add_random_split method. This method splits the inferences submitted to the model through a randomly weighted step.
Each model receives inputs that are approximately proportional to the weight it is assigned. For example, with two models having weights 1 and 1, each will receive roughly equal amounts of inference inputs. If the weights were changed to 1 and 2, the models would receive roughly 33% and 66% respectively instead.
When choosing the model to use, a random number between 0.0 and 1.0 is generated. The weighted inputs are mapped to that range, and the random input is then used to select the model to use. For example, for the two-models equal-weight case, a random key of 0.4 would route to the first model, 0.6 would route to the second.
Add Random Split
A random split step can be added to a pipeline through the add_random_split method.
The following parameters are used when adding a random split step to a pipeline:
Parameter
Type
Description
champion_weight
Float (Required)
The weight for the champion model.
champion_model
Wallaroo.Model (Required)
The uploaded champion model.
challenger_weight
Float (Required)
The weight of the challenger model.
challenger_model
Wallaroo.Model (Required)
The uploaded challenger model.
hash_key
String(Optional)
A key used instead of a random number for model selection. This must be between 0.0 and 1.0.
Note that multiple challenger models with different weights can be added as the random split step.
If a pipeline already had steps as detailed in Add a Step to a Pipeline, this step can be replaced with a random split with the replace_with_random_split method.
The following parameters are used when adding a random split step to a pipeline:
Parameter
Type
Description
index
Integer (Required)
The pipeline step being replaced.
champion_weight
Float (Required)
The weight for the champion model.
champion_model
Wallaroo.Model (Required)
The uploaded champion model.
**challenger_weight
Float (Required)
The weight of the challenger model.
challenger_model
Wallaroo.Model (Required)
The uploaded challenger model.
hash_key
String(Optional)
A key used instead of a random number for model selection. This must be between 0.0 and 1.0.
Note that one or more challenger models can be added for the random split step:
Wallaroo provides a method of testing the same data against two different models or sets of models at the same time through shadow deployments otherwise known as parallel deployments or A/B test. This allows data to be submitted to a pipeline with inferences running on several different sets of models. Typically this is performed on a model that is known to provide accurate results - the champion - and a model or set of models that is being tested to see if it provides more accurate or faster responses depending on the criteria known as the challenger(s). Multiple challengers can be tested against a single champion to determine which is “better” based on the organization’s criteria.
In data science, A/B tests can also be used to choose between two models in production, by measuring which model performs better in the real world. In this formulation, the control is often an existing model that is currently in production, sometimes called the champion. The treatment is a new model being considered to replace the old one. This new model is sometimes called the challenger….
Keep in mind that in machine learning, the terms experiments and trials also often refer to the process of finding a training configuration that works best for the problem at hand (this is sometimes called hyperparameter optimization).
When a shadow deployment is created, only the inference from the champion is returned in the InferenceResult Object data, while the result data for the shadow deployments is stored in the InferenceResult Object shadow_data.
Create Shadow Deployment
Create a parallel or shadow deployment for a pipeline with the pipeline.add_shadow_deploy(champion, challengers[]) method, where the champion is a Wallaroo Model object, and challengers[] is one or more Wallaroo Model objects.
Each inference request sent to the pipeline is sent to all the models. The prediction from the champion is returned by the pipeline, while the predictions from the challengers are not part of the standard output, but are kept stored in the shadow_data attribute and in the logs for later comparison.
In this example, a shadow deployment is created with the champion versus two challenger models.
For Arrow enabled Wallaroo instances the model outputs are listed by column. The output data is set by the term out, followed by the name of the model. For the default model, this is out.dense_1, while the shadow deployed models are in the format out_{model name}.variable, where {model name} is the name of the shadow deployed model.
For Arrow disabled environments, the output is from the Wallaroo InferenceResult object.
Running an inference in a pipeline that has shadow deployments enabled will have the inference run through the champion model returned in the InferenceResult Object’s data element, while the challengers is returned in the InferenceResult Object’s shadow_data element:
For Arrow enabled Wallaroo instances the shadow deploy results are part of the Pipeline.logs() method. The output data is set by the term out, followed by the name of the model. For the default model, this is out.dense_1, while the shadow deployed models are in the format out_{model name}.variable, where {model name} is the name of the shadow deployed model.
Inferences run against a pipeline that has shadow deployments (also known as a parallel deployment) will not be visible in the pipeline logs. To view the results against the challenger models of a shadow deployment, use the Pipeline.logs_shadow_deploy() method. The results will be grouped by inputs, allowing evaluation against multiple models performance based on the same data.
In this example, the Shadow Deployment logs are retrieved after an inference.
The Pipeline URL Endpoint or the Pipeline Deploy URL is used to submit data to a pipeline to use for an inference. This is done through the pipeline _deployment._url() method.
In this example, the pipeline URL endpoint for the pipeline ccfraud_pipeline will be displayed:
2.2.6 - Wallaroo SDK Essentials Guide: Tag Management
How to create and manage Wallaroo Tags through the Wallaroo SDK
Wallaroo SDK Tag Management
Tags are applied to either model versions or pipelines. This allows organizations to track different versions of models, and search for what pipelines have been used for specific purposes such as testing versus production use.
Create Tag
Tags are created with the Wallaroo client command create_tag(String tagname). This creates the tag and makes it available for use.
The tag will be saved to the variable currentTag to be used in the rest of these examples.
# Now we create our tagcurrentTag=wl.create_tag("My Great Tag")
List Tags
Tags are listed with the Wallaroo client command list_tags(), which shows all tags and what models and pipelines they have been assigned to.
Tags are used with pipelines to track different pipelines that are built or deployed with different features or functions.
Add Tag to Pipeline
Tags are added to a pipeline through the Wallaroo Tag add_to_pipeline(pipeline_id) method, where pipeline_id is the pipeline’s integer id.
For this example, we will add currentTag to testtest_pipeline, then verify it has been added through the list_tags command and list_pipelines command.
# add this tag to the pipelinecurrentTag.add_to_pipeline(tagtest_pipeline.id())
{'pipeline_pk_id': 1, 'tag_pk_id': 1}
Search Pipelines by Tag
Pipelines can be searched through the Wallaroo Client search_pipelines(search_term) method, where search_term is a string value for tags assigned to the pipelines.
In this example, the text “My Great Tag” that corresponds to currentTag will be searched for and displayed.
wl.search_pipelines('My Great Tag')
name
version
creation_time
last_updated_time
deployed
tags
steps
tagtestpipeline
5a4ff3c7-1a2d-4b0a-ad9f-78941e6f5677
2022-29-Nov 17:15:21
2022-29-Nov 17:15:21
(unknown)
My Great Tag
Remove Tag from Pipeline
Tags are removed from a pipeline with the Wallaroo Tag remove_from_pipeline(pipeline_id) command, where pipeline_id is the integer value of the pipeline’s id.
For this example, currentTag will be removed from tagtest_pipeline. This will be verified through the list_tags and search_pipelines command.
## remove from pipelinecurrentTag.remove_from_pipeline(tagtest_pipeline.id())
{'pipeline_pk_id': 1, 'tag_pk_id': 1}
Wallaroo Model Tag Management
Tags are used with models to track differences in model versions.
Assign Tag to a Model
Tags are assigned to a model through the Wallaroo Tag add_to_model(model_id) command, where model_id is the model’s numerical ID number. The tag is applied to the most current version of the model.
For this example, the currentTag will be applied to the tagtest_model. All tags will then be listed to show it has been assigned to this model.
# add tag to modelcurrentTag.add_to_model(tagtest_model.id())
{'model_id': 1, 'tag_id': 1}
Search Models by Tag
Model versions can be searched via tags using the Wallaroo Client method search_models(search_term), where search_term is a string value. All models versions containing the tag will be displayed. In this example, we will be using the text from our tag to list all models that have the text from currentTag in them.
# Search models by tagwl.search_models('My Great Tag')
name
version
file_name
image_path
last_update_time
tagtestmodel
70169e97-fb7e-4922-82ba-4f5d37e75253
ccfraud.onnx
None
2022-11-29 17:15:21.703465+00:00
Remove Tag from Model
Tags are removed from models using the Wallaroo Tag remove_from_model(model_id) command.
In this example, the currentTag will be removed from tagtest_model. A list of all tags will be shown with the list_tags command, followed by searching the models for the tag to verify it has been removed.
### remove tag from modelcurrentTag.remove_from_model(tagtest_model.id())
How to use Wallaroo for model inferencing through the Wallaroo SDK
Inferencing
Once a pipeline has been deployed, an inference can be run. This will submit data to the pipeline, where it is processed through each of the pipeline’s steps, with the output of the previous step providing the input for the new step. The final step will then output the result of all of the pipeline’s steps.
The input sent and the output received depends on whether Arrow support is enabled in the Wallaroo instance.
When Arrow support is enabled, the method pipeline infer(data, timeout, dataset, dataset_exclude, dataset_separator) performs an inference as defined by the pipeline steps and takes the following arguments:
data (REQUIRED): The data submitted to the pipeline for inference. Inputs are either sent as a pandas.DataFrame or an Apache Arrow.
timeout (OPTIONAL): A timeout in seconds before the inference throws an exception. The default is 15 second per call to accommodate large, complex models. Note that for a batch inference, this is per call - with 10 inference requests, each would have a default timeout of 15 seconds.
dataset (OPTIONAL): The datasets to be returned. By default this is set to ["*"] which returns, [“time”, “in”, “out”, “check_failures”].
dataset_exclude (OPTIONAL): Allows users to exclude parts of the dataset.
dataset_separator (OPTIONAL): Allows other types of dataset separators to be used. If set to “.”, the returned dataset will be flattened.
The following example is an inference request using a pandas DataFrame, and the returning values. Note that columns are labeled based on the inputs and outputs. This model only has one output - dense_1, which is listed in the out.dense_1 column. If the model had returned multiple outputs, they would be listed as out.output1, out.output2, etc.
When Arrow support is disabled, the method pipeline infer(data, timeout) performs an inference as defined by the pipeline steps and takes the following arguments:
data (REQUIRED): The data submitted to the pipeline for inference in the Wallaroo JSON format.
timeout (OPTIONAL): A timeout in seconds before the inference throws an exception. The default is 15 second per call to accommodate large, complex models. Note that for a batch inference, this is per call - with 10 inference requests, each would have a default timeout of 15 seconds.
The following example shows running an inference on data with a timeout of 20 seconds:
The method pipeline _deployment._url() provides a URL where information can be submitted through HTTP POST in JSON format to the pipeline to perform an inference. This is useful in providing a resource where information can be submitted to the pipeline from different sources to the same pipeline remotely.
For Arrow enabled instances of Wallaroo:
For DataFrame formatted JSON, the Content-Type is application/json; format=pandas-records.
For Arrow binary files, the Content-Type is application/vnd.apache.arrow.file.
For Arrow disabled instances of Wallaroo:
The Content-Type is application/json.
In this example, the aloha_pipeline’s deployment URL will be determined in an Arrow enabled Wallaroo instance. An inference will then be made on data submitted to the aloha_pipeline through its deployment URL via a curl HTTP POST command.
IMPORTANT NOTE: The _deployment._url() method will return an internal URL when using Python commands from within the Wallaroo instance - for example, the Wallaroo JupyterHub service. When connecting via an external connection, _deployment._url() returns an external URL. External URL connections requires the authentication be included in the HTTP request, and that Model Endpoints Guide external endpoints are enabled in the Wallaroo configuration options.
!curl -X POST http://engine-lb.aloha-test-demo-5:29502/pipelines/aloha-test-demo -H "application/json; format=pandas-records" --data @data-25k.json > curl_response.txt
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 12.9M 100 10.1M 100 2886k 539k 149k 0:00:19 0:00:19 --:--:-- 2570k
Run Inference From A File
To submit a data file directly to a pipeline, use the pipeline infer_from_file({Data File}, timeout).
Arrow Enabled infer_from_file
When Arrow support is enabled, the method pipeline infer_from_file(data, timeout, dataset, dataset_exclude, dataset_separator) performs an inference as defined by the pipeline steps and takes the following arguments:
data (REQUIRED): The name of the file submitted to the pipeline for inference. Inputs are either sent as a pandas.DataFrame or an Apache Arrow.
timeout (OPTIONAL): A timeout in seconds before the inference throws an exception. The default is 15 second per call to accommodate large, complex models. Note that for a batch inference, this is per call - with 10 inference requests, each would have a default timeout of 15 seconds.
dataset (OPTIONAL): The datasets to be returned. By default this is set to ["*"] which returns, [“time”, “in”, “out”, “check_failures”].
dataset_exclude (OPTIONAL): Allows users to exclude parts of the dataset.
dataset_separator (OPTIONAL): Allows other types of dataset separators to be used. If set to “.”, the returned dataset will be flattened.
In this example, an inference will be submitted to the ccfraud_pipeline with the file smoke_test.df.json, a DataFrame formatted JSON file.
When Arrow support is not enabled, the method pipeline `infer_from_file(filename, timeout) performs an inference as defined by the pipeline steps and takes the following arguments:
{Data File} : {Data File} is the path name to the submitted file in the Wallaroo JSON format.
timeout: A timeout in seconds before the inference throws an exception. The default is 15 second per call to accommodate large, complex models. Note that for a batch inference, this is per call - with 10 inference requests, each would have a default timeout of 15 seconds.
In this example, an inference will be submitted to the aloha_pipeline with the file data-1.json with a timeout of 20 seconds:
input_data(): Returns the data provided to the pipeline to run the inference.
In this example, an inference will be submitted to the ccfraud_pipeline with the file cc_data_1k.json, with only the first element in the array returned:
How to create and manage Wallaroo Assays through the Wallaroo SDK
Model Insights and Interactive Analysis Introduction
Wallaroo provides the ability to perform interactive analysis so organizations can explore the data from a pipeline and learn how the data is behaving. With this information and the knowledge of your particular business use case you can then choose appropriate thresholds for persistent automatic assays as desired.
IMPORTANT NOTE
Model insights operates over time and is difficult to demo in a notebook without pre-canned data. We assume you have an active pipeline that has been running and making predictions over time and show you the code you may use to analyze your pipeline.
Monitoring tasks called assays monitors a model’s predictions or the data coming into the model against an established baseline. Changes in the distribution of this data can be an indication of model drift, or of a change in the environment that the model trained for. This can provide tips on whether a model needs to be retrained or the environment data analyzed for accuracy or other needs.
Assay Details
Assays contain the following attributes:
Attribute
Default
Description
Name
The name of the assay. Assay names must be unique.
Baseline Data
Data that is known to be “typical” (typically distributed) and can be used to determine whether the distribution of new data has changed.
Schedule
Every 24 hours at 1 AM
New assays are configured to run a new analysis for every 24 hours starting at the end of the baseline period. This period can be configured through the SDK.
Group Results
Daily
Groups assay results into groups based on either Daily (the default), Weekly, or Monthly.
Metric
PSI
Population Stability Index (PSI) is an entropy-based measure of the difference between distributions. Maximum Difference of Bins measures the maximum difference between the baseline and current distributions (as estimated using the bins). Sum of the difference of bins sums up the difference of occurrences in each bin between the baseline and current distributions.
Threshold
0.1
The threshold for deciding whether the difference between distributions, as evaluated by the above metric, is large (the distributions are different) or small (the distributions are similar). The default of 0.1 is generally a good threshold when using PSI as the metric.
Number of Bins
5
Sets the number of bins that will be used to partition the baseline data for comparison against how future data falls into these bins. By default, the binning scheme is percentile (quantile) based. The binning scheme can be configured (see Bin Mode, below). Note that the total number of bins will include the set number plus the left_outlier and the right_outlier, so the total number of bins will be the total set + 2.
Bin Mode
Quantile
Set the binning scheme. Quantile binning defines the bins using percentile ranges (each bin holds the same percentage of the baseline data). Equal binning defines the bins using equally spaced data value ranges, like a histogram. Custom allows users to set the range of values for each bin, with the Left Outlier always starting at Min (below the minimum values detected from the baseline) and the Right Outlier always ending at Max (above the maximum values detected from the baseline).
Bin Weight
Equally Weighted
The bin weights can be either set to Equally Weighted (the default) where each bin is weighted equally, or Custom where the bin weights can be adjusted depending on which are considered more important for detecting model drift.
Manage Assays via the Wallaroo SDK
List Assays
Assays are listed through the client.list_assays method, and returns a List object.
Assays are built with the Wallaroo client.build_assay(assayName, pipeline, modelName, baselineStart, baselineEnd), and returns the wallaroo.assay_config.AssayBuilder. The method requires the following parameters:
Parameter
Type
Description
assayName
String
The human friendly name of the created assay.
pipeline
Wallaroo.pipeline
The pipeline the assay is assigned to.
modelName
String
The model to perform the assay on.
baselineStart
DateTime
When to start the baseline period.
baselineStart
DateTime
When to end the baseline period.
When called, this method will then pool the pipeline between the baseline start and end periods to establish what values are considered normal outputs for the specified model.
Assays by default will run a new a new analysis every 24 hours starting at the end of the baseline period, using a 24 hour observation window.
In this example, an assay will be created named example assay and stored into the variable assay_builder.
By default assays are scheduled to run every 24 hours starting immediately after the baseline period ends. This scheduled period is referred to as the assay window and has the following properties:
width: The period of data included in the analysis. By default this is 24 hours.
interval:
How often the analysis is run (every 5 minutes, every 24 hours, etc). By default this is the window width.
start: When the analysis should start. By default this is at the end of the baseline period.
These are adjusted through the assay window_builder method that includes the following methods:
add_width: Sets the width of the window.
add_interval: Sets how often the analysis is run.
In this example, the assay will be set to run an analysis every 12 hours on the previous 24 hours of data:
Interactive baselines can be run against an assay to generate a list of the values that are established in the baseline. This is done through the AssayBuilder.interactive_baseline_run() method, which returns the following:
Parameter
Type
Description
count
Integer
The number of records evaluated.
min
Float
The minimum value found
max
Float
The maximum value found
mean
Float
The mean value derived from the values evaluated.
median
Float
The median value derived from the values evaluated.
std
Float
The standard deviation from the values evaluated.
start
DateTime
The start date for the records to evaluate.
end
DateTime
The end date for the records to evaluate.
In this example, an interactive baseline will be run against a new assay, and the results displayed:
Histogram, kernel density estimate (KDE), and Empirical Cumulative Distribution (ecdf) charts can be generated from an assay to provide a visual representation of the values evaluated and where they fit within the established baseline.
These methods are part of the AssayBuilder object and are as follows:
Method
Description
baseline_histogram()
Creates a histogram chart from the assay baseline.
baseline_kde()
Creates a kernel density estimate (KDE) chart from the assay baseline.
baseline_ecdf()
Creates an Empirical Cumulative Distribution (ecdf) from the assay baseline.
In this example, each of the three different charts will be generated from an assay:
assay_builder.baseline_histogram()
assay_builder.baseline_kde()
assay_builder.baseline_ecdf()
Run Interactive Assay
Users can issue an assay to be run through an interactive assay instead of waiting for the next scheduled assay to run through the wallaroo.assay_config.interactive_run method. This is usually run through the wallaroo.client.build_assay method, which returns a wallaroo.assay_config.AssayBuilder object.
The following example creates the AssayBuilder object then runs an interactive assay.
As defined under Assay Details, bins can be adjusted by number of bins, bin mode, and bin weight.
Number of Bins
The number of bins can be changed from the default of 5 through the wallaroo.assay_config.summarizer_builder.add_num_buns method. Note that the total number of bins will include the set bins, plus the left_outlier and the right_outlier bins. So the total number of bins are the set number of bins + 2.
The following example shows how to change the number of bins to 10 in an assay, then the assay results displayed in a chart with the total bins of 12 total (10 manually set, 1 left_outlier, 1 right_outlier).
BinMode.QUANTILE (Default): Defines the bins using percentile ranges (each bin holds the same percentage of the baseline data).
BinMode.EQUAL defines the bins using equally spaced data value ranges, like a histogram.
Custom aka BinMode.PROVIDED allows users to set the range of values for each bin, with the Left Outlier always starting at Min (below the minimum values detected from the baseline) and the Right Outlier always ending at Max (above the maximum values detected from the baseline). When using BinMode.PROVIDED the edges are passed as an array value.
Bin modes are set through the wallaroo.assay_config.summarizer_builder.add_bin_mode method.
The following examples will demonstrate changing the bin mode to equal, then setting custom provided values.
None
baseline mean = 12.940910643273655
window mean = 12.969964654406132
baseline median = 12.884286880493164
window median = 12.899214744567873
bin_mode = Provided
aggregation = Density
metric = PSI
weighted = False
score = 0.0321620386600679
scores = [0.0, 0.0, 0.0014576920813015586, 3.549754401142936e-05, 0.030668849034754912, 0.0, 0.0]
index = None
Bin Weights
Bin weights can be adjusted so bins that that bins with more importance can be given more prominence in the final assay score. This is done through the wallaroo.assay_config.summarizer_builder.add_bin_weights, where the weights are assigned as array values matching the bins.
The following example has 10 bins (12 total including the left_outlier and the right_outlier bins), with weights assigned of 0 for the first six bins, 1 for the last six, and the resulting score from these weights.
None
baseline mean = 12.940910643273655
window mean = 12.956829186961135
baseline median = 12.884286880493164
window median = 12.929338455200195
bin_mode = Quantile
aggregation = Density
metric = PSI
weighted = True
score = 0.012600694309416988
scores = [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.00019654033061397393, 0.00850384373737565, 0.0015735766052488358, 0.0014437605903522511, 0.000882973045826275, 0.0]
index = None
/opt/homebrew/anaconda3/envs/arrowtests/lib/python3.8/site-packages/wallaroo/assay.py:315: UserWarning: FixedFormatter should only be used together with FixedLocator
ax.set_xticklabels(labels=edge_names, rotation=45)
Metrics
The metric score is a distance or dis-similarity measure. The larger it is the less similar the two distributions are. The following metrics are supported:
baseline mean = 12.940910643273655
window mean = 12.969964654406132
baseline median = 12.884286880493164
window median = 12.899214744567873
bin_mode = Quantile
aggregation = Density
metric = MaxDiff
weighted = False
score = 0.01548175581324751
scores = [0.0, 0.009956893934794486, 0.006648048084512165, 0.01548175581324751, 0.006648048084512165, 0.012142553579017668, 0.0]
index = 3
Aggregation Options
Bin aggregation can be done in histogram Aggregation.DENSITY style (the default) where we count the number/percentage of values that fall in each bin or Empirical Cumulative Density Function style Aggregation.CUMULATIVE where we keep a cumulative count of the values/percentages that fall in each bin.
baseline mean = 12.940910643273655
window mean = 12.969964654406132
baseline median = 12.884286880493164
window median = 12.899214744567873
bin_mode = Quantile
aggregation = Cumulative
metric = PSI
weighted = False
score = 0.04419889502762442
scores = [0.0, 0.009956893934794486, 0.0033088458502823492, 0.01879060166352986, 0.012142553579017725, 0.0, 0.0]
index = None
2.3 - Wallaroo SDK Reference Guide
Wallaroo SDK Reference Guide
2.3.1 - wallaroo.assay
classAssay(wallaroo.object.Object):
An Assay represents a record in the database. An assay contains
some high level attributes such as name, status, active, etc. as well
as the sub objects Baseline, Window and Summarizer which specify how
the Baseline is derived, how the Windows should be created and how the
analysis should be conducted.
a GraphQL client - in order to fill its missing members dynamically
an initial data blob - typically from unserialized JSON, contains at
least the data for required
members (typically the object's primary key) and optionally other data
members.
defturn_on(self):
Sets the Assay to active causing it to run and backfill any
missing analysis.
defturn_off(self):
Disables the Assay. No further analysis will be conducted until the assay
is enabled.
defset_alert_threshold(self, threshold:float):
Sets the alert threshold at the specified level. The status in the AssayAnalysis
will show if this level is exceeded however currently alerting/notifications are
not implemented.
defset_warning_threshold(self, threshold:float):
Sets the warning threshold at the specified level. The status in the AssayAnalysis
will show if this level is exceeded however currently alerting/notifications are
not implemented.
Creates a dataframe specifically for the edge information in the baseline or window.
Parameters
window_or_baseline: The dict from the assay result of either the window or baseline
classAssayAnalysis:
The AssayAnalysis class helps handle the assay analysis logs from the Plateau
logs. These logs are a json document with meta information on the assay and analysis
as well as summary information on the baseline and window and information on the comparison
between them.
Creates a simple dataframe to with the edge/bin data for a baseline.
defchart(self, show_scores=True):
Quickly create a chart showing the bins, values and scores of an assay analysis.
show_scores will also label each bin with its final weighted (if specified) score.
classAssayAnalysisList:
Helper class primarily to easily create a dataframe from a list
of AssayAnalysis objects.
