For details on adding ARM nodepools to a Wallaroo cluster, see Create ARM Nodepools for Kubernetes Clusters.

This tutorial is available on the Wallaroo Tutorials repository.

Classification Financial Services with Arm Architecture

This tutorial demonstrates how to use the Wallaroo combined with ARM processors to perform inferences with pre-trained classification financial services ML models. This demonstration assumes that:

• Wallaroo Version 2023.3 or above instance is installed.
• A nodepools with ARM architecture virtual machines are part of the Kubernetes cluster. For example, Azure supports Ampere® Altra® Arm-based processor included with the following virtual machines:
  • Dpsv5 and Dpdsv5-series
  • Epsv5 and Epdsv5-series

In this notebook we will the example model and sample data from the Machine Learning Group’s demonstration on Credit Card Fraud detection.

Tutorial Goals

For our example, we will perform the following:

• Create a workspace for our work.
• Upload the CCFraud model and set the architecture setting to ARM.
• Create a pipeline and deploy the model on ARM then perform sample inferences.

import wallaroo
from wallaroo.object import EntityNotFoundError

import pyarrow as pa

from wallaroo.framework import Framework

# used to display dataframe information without truncating
from IPython.display import display
import pandas as pd
pd.set_option('display.max_colwidth', None)
import time

wallaroo.__version__

'2023.3.0+8a3f2fb38'

Connect to the Wallaroo Instance through the User Interface

The next step is to connect to Wallaroo through the Wallaroo client. The Python library is included in the Wallaroo install and available through the Jupyter Hub interface provided with your Wallaroo environment.

This is accomplished using the wallaroo.Client() command, which provides a URL to grant the SDK permission to your specific Wallaroo environment. When displayed, enter the URL into a browser and confirm permissions. Store the connection into a variable that can be referenced later.

If logging into the Wallaroo instance through the internal JupyterHub service, use wl = wallaroo.Client(). For more information on Wallaroo Client settings, see the Client Connection guide.

# Login through local Wallaroo instance

wl = wallaroo.Client()

Create Workspace

Now we’ll use the SDK below to create our workspace , assign as our current workspace, then display all of the workspaces we have at the moment. We’ll also set up for our models and pipelines down the road, so we have one spot to change names to whatever fits your organization’s standards best.

When we create our new workspace, we’ll save it in the Python variable workspace so we can refer to it as needed.

For more information, see the Wallaroo SDK Essentials Guide: Workspace Management.

workspace_name = f'arm-classification-finserv'
pipeline_name = 'arm-classification-example'
model_name = 'ccfraudmodel'
model_file_name = './models/ccfraud.onnx'

workspace = wl.get_workspace(name=workspace_name, create_if_not_exist=True)

wl.set_current_workspace(workspace)

{'name': 'arm-classification-finservjohn', 'id': 25, 'archived': False, 'created_by': '0e5060a5-218c-47c1-9678-e83337494184', 'created_at': '2023-09-08T21:40:58.35659+00:00', 'models': [], 'pipelines': []}

Upload a model

Our workspace is created. Let’s upload our credit card fraud model to it. This is the file name ccfraud.onnx, and we’ll upload it as ccfraudmodel. The credit card fraud model is trained to detect credit card fraud based on a 0 to 1 model: The closer to 0 the less likely the transactions indicate fraud, while the closer to 1 the more likely the transactions indicate fraud.

During the model upload, we will set the model’s architecture to ARM.

For more information, see the Wallaroo SDK Essentials Guide: Model Uploads and Registrations.

from wallaroo.engine_config import Architecture

arm_ccfraud_model = (wl.upload_model(model_name, 
                                 model_file_name, 
                                 wallaroo.framework.Framework.ONNX,
                                 arch=Architecture.ARM)
                                 .configure(tensor_fields=["tensor"])
                )

Create a Pipeline

With our model uploaded, time to create our pipeline and deploy it so it can accept data and process it through our ccfraudmodel. We’ll call our pipeline ccfraudpipeline.

• NOTE: Pipeline names must be unique. If two pipelines are assigned the same name, the new pipeline is created as a new version of the pipeline.

For more information, see the Wallaroo SDK Essentials Guide: Pipeline Management

pipeline = wl.build_pipeline(pipeline_name)

Deploy Pipeline

We will now deploy the pipeline. The pipeline deployment inherits its architecture from the model, so this deployment will automatically use the nodepool with the ARM processors.

For more information, see the Wallaroo SDK Essentials Guide: Pipeline Deployment Configuration.

deployment_config = (wallaroo.deployment_config
                     .DeploymentConfigBuilder()
                     .cpus(1)
                     .memory('1Gi')
                     .build()
                     )
deployment_config

{'engine': {'cpu': 1,
  'resources': {'limits': {'cpu': 1, 'memory': '1Gi'},
   'requests': {'cpu': 1, 'memory': '1Gi'}}},
 'enginelb': {},
 'engineAux': {'images': {}},
 'node_selector': {}}

pipeline.deploy(deployment_config=deployment_config)

Waiting for deployment - this will take up to 45s ........ ok

Inference with ARM

We’ll perform sample inferences on the model deployed on ARM. The input is an Apache Arrow table.

start_time = time.time()
result = pipeline.infer_from_file('./data/cc_data_10k.arrow')
end_time = time.time()
arm_time = end_time - start_time

outputs =  result.to_pandas()
display(outputs.head(5))

With our work in the pipeline done, we’ll undeploy it to get back our resources from the Kubernetes cluster. If we keep the same settings we can redeploy the pipeline with the same configuration in the future.

pipeline.undeploy()

Waiting for undeployment - this will take up to 45s .................................... ok