Model Conversion Tutorials

• 1: Keras Convert and Upload Within Wallaroo
• 2: PyTorch to ONNX Outside Wallaroo
• 3: sklearn and XGBoost Regression Auto-Conversion Tutorial Within Wallaroo
• 4: sk-learn Regression Model to ONNX Outside Wallaroo
• 5: Statsmodel Upload to Wallaroo Tutorial
• 6: XGBoost Classification Auto-Convert Within Wallaroo
• 7: XGBoost Regression Auto-Convert Within Wallaroo
• 8: XGBoost Convert to ONNX

1 - Keras Convert and Upload Within Wallaroo

This tutorial and the assets can be downloaded as part of the Wallaroo Tutorials repository.

Introduction

Machine Learning (ML) models can be converted into a Wallaroo Model object and uploaded into Wallaroo workspace using the Wallaroo Client convert_model(path, source_type, conversion_arguments) method. This conversion process transforms the model into an open format that can be run across different frameworks at compiled C-language speeds.

The following tutorial is a brief example of how to convert a Keras or Tensor ML model to ONNX. This allows organizations that have trained Keras or Tensor models to convert them and use them with Wallaroo.

This tutorial assumes that you have a Wallaroo instance and are running this Notebook from the Wallaroo Jupyter Hub service.

This tutorial demonstrates how to:

• Convert a keras ML model and upload it into the Wallaroo engine.
• Run a sample inference on the converted model in a Wallaroo instance.

This tutorial provides the following:

• simple_sentiment_model.zip: A pre-trained keras sentiment model to be converted. This has 100 columns.

Prerequisites

• An installed Wallaroo instance.
• The following Python libraries installed:
  • os
  • wallaroo: The Wallaroo SDK. Included with the Wallaroo JupyterHub service by default.
  • pandas: Pandas, mainly used for Pandas DataFrame

Conversion Steps

To use the Wallaroo autoconverter convert_model(path, source_type, conversion_arguments) method takes 3 parameters. The paramters for keras conversions are:

• path (STRING): The path to the ML model file.
• source_type (ModelConversionSource): The type of ML model to be converted. As of this time Wallaroo auto-conversion supports the following source types and their associated ModelConversionSource:
  • sklearn: ModelConversionSource.SKLEARN
  • xgboost: ModelConversionSource.XGBOOST
  • keras: ModelConversionSource.KERAS
• conversion_arguments: The arguments for the conversion based on the type of model being converted. These are:
  • wallaroo.ModelConversion.ConvertKerasArguments: Used for converting keras type models and takes the following parameters:
    • name: The name of the model being converted.
    • comment: Any comments for the model.
    • input_type: A tensorflow Dtype called in the format ModelConversionInputType.{type}, where {type} is Float, Double, etc depending on the model.
    • dimensions: Corresponds to the keras xtrain in the format List[Union[None, int, float]].

Import Libraries

The first step is to import the libraries needed.

import wallaroo

from wallaroo.ModelConversion import ConvertKerasArguments, ModelConversionSource, ModelConversionInputType
from wallaroo.object import EntityNotFoundError
import pandas as pd

# used to display dataframe information without truncating
from IPython.display import display
pd.set_option('display.max_colwidth', None)

Configuration and Methods

The following will set the workspace, pipeline, model name, the model file name used when uploading and converting the keras model, and the sample data.

The functions get_workspace(name) will either set the current workspace to the requested name, or create it if it does not exist. The function get_pipeline(name) will either set the pipeline used to the name requested, or create it in the current workspace if it does not exist.

workspace_name = 'externalkerasautoconvertworkspace'
pipeline_name = 'externalkerasautoconvertpipeline'
model_name = 'externalsimple-sentiment-model'
model_file_name = 'simple_sentiment_model.zip'
sample_data = 'simple_sentiment_testdata.json'

def get_workspace(name):
    workspace = None
    for ws in wl.list_workspaces():
        if ws.name() == name:
            workspace= ws
    if(workspace == None):
        workspace = wl.create_workspace(name)
    return workspace

def get_pipeline(name):
    try:
        pipeline = wl.pipelines_by_name(name)[0]
    except EntityNotFoundError:
        pipeline = wl.build_pipeline(name)
    return pipeline

Connect to the Wallaroo Instance

The first step is to connect to Wallaroo through the Wallaroo client. The Python library is included in the Wallaroo install and available through the Jupyter Hub interface provided with your Wallaroo environment.

This is accomplished using the wallaroo.Client() command, which provides a URL to grant the SDK permission to your specific Wallaroo environment. When displayed, enter the URL into a browser and confirm permissions. Store the connection into a variable that can be referenced later.

If logging into the Wallaroo instance through the internal JupyterHub service, use wl = wallaroo.Client(). If logging in externally, update the wallarooPrefix and wallarooSuffix variables with the proper DNS information. For more information on Wallaroo DNS settings, see the Wallaroo DNS Integration Guide.

# Client connection from local Wallaroo instance

wl = wallaroo.Client()

Set the Workspace and Pipeline

Set or create the workspace and pipeline based on the names configured earlier.

workspace = get_workspace(workspace_name)

wl.set_current_workspace(workspace)

pipeline = get_pipeline(pipeline_name)
pipeline

Set the Model Autoconvert Parameters

Set the paramters for converting the simple-sentiment-model. This includes the shape of the model.

model_columns = 100

model_conversion_args = ConvertKerasArguments(
    name=model_name,
    comment="simple keras model",
    input_type=ModelConversionInputType.Float32,
    dimensions=(None, model_columns)
)
model_conversion_type = ModelConversionSource.KERAS

Upload and Convert the Model

Now we can upload the convert the model. Once finished, it will be stored as {unique-file-id}-converted.onnx.

# converts and uploads model.
model_wl = wl.convert_model('simple_sentiment_model.zip', model_conversion_type, model_conversion_args)
model_wl

{'name': 'externalsimple-sentiment-model', 'version': 'c378425b-b70f-465a-a15b-d9e662b15263', 'file_name': '19ec5f96-d3a6-47af-ae6f-928187735de2-converted.onnx', 'image_path': None, 'last_update_time': datetime.datetime(2023, 5, 17, 21, 13, 29, 933149, tzinfo=tzutc())}

Test Inference

With the model uploaded and converted, we can run a sample inference.

Add Pipeline Step and Deploy

We will add the model as a step into our pipeline, then deploy it.

pipeline.add_model_step(model_wl).deploy()

pipeline.status()

{'status': 'Running',
 'details': [],
 'engines': [{'ip': '10.244.3.139',
   'name': 'engine-59fb67fcc6-tns2j',
   'status': 'Running',
   'reason': None,
   'details': [],
   'pipeline_statuses': {'pipelines': [{'id': 'externalkerasautoconvertpipeline',
      'status': 'Running'}]},
   'model_statuses': {'models': [{'name': 'externalsimple-sentiment-model',
      'version': 'c378425b-b70f-465a-a15b-d9e662b15263',
      'sha': '49f7367eede690b369aef322569c5b54c4133692610a11dc29b14d4c49ea983c',
      'status': 'Running'}]}}],
 'engine_lbs': [{'ip': '10.244.4.170',
   'name': 'engine-lb-584f54c899-fnntk',
   'status': 'Running',
   'reason': None,
   'details': []}],
 'sidekicks': []}

Run a Test Inference

We can run a test inference from the simple_sentiment_testdata.json file, then display just the results.

sample_data = 'simple_sentiment_testdata.df.json'
result = pipeline.infer_from_file(sample_data)
display(result["out.dense"])

0    [0.094697624]
1       [0.991031]
2     [0.93407357]
3     [0.56030995]
4      [0.9964503]
Name: out.dense, dtype: object

Undeploy the Pipeline

With the tests complete, we will undeploy the pipeline to return the resources back to the Wallaroo instance.

pipeline.undeploy()

2 - PyTorch to ONNX Outside Wallaroo

This tutorial and the assets can be downloaded as part of the Wallaroo Tutorials repository.

How to Convert PyTorch to ONNX

The following tutorial is a brief example of how to convert a PyTorth (aka sk-learn) ML model to ONNX. This allows organizations that have trained sk-learn models to convert them and use them with Wallaroo.

This tutorial assumes that you have a Wallaroo instance and are running this Notebook from the Wallaroo Jupyter Hub service. This sample code is based on the guide Convert your PyTorch model to ONNX.

This tutorial provides the following:

• pytorchbikeshare.pt: a RandomForestRegressor PyTorch model. This model has a total of 58 inputs, and uses the class BikeShareRegressor.

Prerequisites

• An installed Wallaroo instance.
• The following Python libraries installed:
  • os
  • torch

Conversion Process

Libraries

The first step is to import our libraries we will be using. For this example, the PyTorth torch library will be imported into this kernel.

# the Pytorch libraries
# Import into this kernel

import torch
import torch.onnx 

Load the Model

To load a PyTorch model into a variable, the model’s class has to be defined. For out example we are using the BikeShareRegressor class as defined below.

class BikeShareRegressor(torch.nn.Module):
    def __init__(self):
        super(BikeShareRegressor, self).__init__()

        
        self.net = nn.Sequential(nn.Linear(input_size, l1),
                                 torch.nn.ReLU(),
                                 torch.nn.Dropout(p=dropout),
                                 nn.BatchNorm1d(l1),
                                 nn.Linear(l1, l2),
                                 torch.nn.ReLU(),
                                 torch.nn.Dropout(p=dropout),                                
                                 nn.BatchNorm1d(l2),                                                                                                   
                                 nn.Linear(l2, output_size))

    def forward(self, x):
        return self.net(x)

Now we will load the model into the variable pytorch_tobe_converted.

# load the Pytorch model
model = torch.load("./pytorch_bikesharingmodel.pt")

Convert_ONNX Inputs

Now we will define our method Convert_ONNX() which has the following inputs:

• PyTorchModel: the PyTorch we are converting.

• modelInputs: the model input or tuple for multiple inputs.

• onnxPath: The location to save the onnx file.

• opset_version: The ONNX version to export to.

• input_names: Array of the model’s input names.

• output_names: Array of the model’s output names.

• dynamic_axes: Sets variable length axes in the format, replacing the batch_size as necessary: {'modelInput' : { 0 : 'batch_size'}, 'modelOutput' : {0 : 'batch_size'}}

• export_params: Whether to store the trained parameter weight inside the model file. Defaults to True.

• do_constant_folding: Sets whether to execute constant folding for optimization. Defaults to True.

#Function to Convert to ONNX 
def Convert_ONNX(): 

    # set the model to inference mode 
    model.eval() 

    # Export the model   
    torch.onnx.export(model,         # model being run 
         dummy_input,       # model input (or a tuple for multiple inputs) 
         pypath,       # where to save the model  
         export_params=True,  # store the trained parameter weights inside the model file 
         opset_version=15,    # the ONNX version to export the model to 
         do_constant_folding=True,  # whether to execute constant folding for optimization 
         input_names = ['modelInput'],   # the model's input names 
         output_names = ['modelOutput'], # the model's output names 
         dynamic_axes = {'modelInput' : {0 : 'batch_size'}, 'modelOutput' : {0 : 'batch_size'}} # variable length axes 
    ) 
    print(" ") 
    print('Model has been converted to ONNX') 

Convert the Model

We’ll now set our variables and run our conversion. For out example, the input_size is known to be 58, and the device value we’ll derive from torch.cuda. We’ll also set the ONNX version for exporting to 15.

pypath = "pytorchbikeshare.onnx"

input_size = 58

if torch.cuda.is_available():
    device = 'cuda'
else:
    device = 'cpu'

onnx_opset_version = 15

# Set up some dummy input tensor for the model
dummy_input = torch.randn(1, input_size, requires_grad=True).to(device)

Convert_ONNX()

================ Diagnostic Run torch.onnx.export version 2.0.0 ================
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================
Model has been converted to ONNX

Conclusion

And now our conversion is complete. Please feel free to use this sample code in your own projects.

3 - sklearn and XGBoost Regression Auto-Conversion Tutorial Within Wallaroo

Auto-Conversion And Upload Tutorial

Machine Learning (ML) models can be converted into a Wallaroo and uploaded into Wallaroo workspace using the Wallaroo Client convert_model(path, source_type, conversion_arguments) method. This conversion process transforms the model into an open format that can be run across different frameworks at compiled C-language speeds.

The three input parameters are:

• path (STRING): The path to the ML model file.
• source_type (ModelConversionSource): The type of ML model to be converted. As of this time Wallaroo auto-conversion supports the following source types and their associated ModelConversionSource:
  • sklearn: ModelConversionSource.SKLEARN
  • xgboost: ModelConversionSource.XGBOOST
• conversion_arguments: The arguments for the conversion:
  • name: The name of the model being converted.
  • comment: Any comments for the model.
  • number_of_columns: The number of columns the model was trained for.
  • input_type: The ModelConversationInputType, typically Float or Double depending on the model.

The following tutorial demonstrates how to convert a sklearn Linear Model and a XGBoost Regression Model, and upload them into a Wallaroo Workspace. The following is provided for the tutorial:

• sklearn-linear-model.pickle: A sklearn linear model. An example of training the model is provided in the Jupyter Notebook sklearn-linear-model-example.ipynb. It has 25 columns.
• xgb_reg.pickle: A XGBoost regression model. An example of training the model is provided in the Jupyter Notebook xgboost-regression-model-example.ipynb. It has 25 columns.

Prerequisites

• An installed Wallaroo instance.
• The following Python libraries installed:
  • os
  • wallaroo: The Wallaroo SDK. Included with the Wallaroo JupyterHub service by default.
  • scikit-learn Version 1.1.1
  • xgboost Version 1.6.2
  • pickle

Steps

Import Libraries

Import the libraries that will be used for the auto-conversion process.

import pickle
import json

import wallaroo

from wallaroo.ModelConversion import ConvertSKLearnArguments, ConvertXGBoostArgs, ModelConversionSource, ModelConversionInputType
from wallaroo.object import EntityNotFoundError

# Verify the version of XGBoost used to generate the models

import sklearn
import sklearn.datasets

import xgboost as xgb

print(xgb.__version__)
print(sklearn.__version__)

1.6.2
1.1.2

The following code is used to either connect to an existing workspace or to create a new one. For more details on working with workspaces, see the Wallaroo Workspace Management Guide.

Connect to the Wallaroo Instance

The first step is to connect to Wallaroo through the Wallaroo client. The Python library is included in the Wallaroo install and available through the Jupyter Hub interface provided with your Wallaroo environment.

This is accomplished using the wallaroo.Client() command, which provides a URL to grant the SDK permission to your specific Wallaroo environment. When displayed, enter the URL into a browser and confirm permissions. Store the connection into a variable that can be referenced later.

If logging into the Wallaroo instance through the internal JupyterHub service, use wl = wallaroo.Client(). If logging in externally, update the wallarooPrefix and wallarooSuffix variables with the proper DNS information. For more information on Wallaroo DNS settings, see the Wallaroo DNS Integration Guide.

# Client connection from local Wallaroo instance

wl = wallaroo.Client()

Set the Workspace

We’ll connect or create the workspace testautoconversion and use it for our model testing.

workspace_name = 'testautoconversion'

def get_workspace(name):
    workspace = None
    for ws in wl.list_workspaces():
        if ws.name() == name:
            workspace= ws
    if(workspace == None):
        workspace = wl.create_workspace(name)
    return workspace

workspace = get_workspace(workspace_name)

wl.set_current_workspace(workspace)

wl.get_current_workspace()

{'name': 'testautoconversion', 'id': 11, 'archived': False, 'created_by': '028c8b48-c39b-4578-9110-0b5bdd3824da', 'created_at': '2023-05-17T21:11:40.856672+00:00', 'models': [], 'pipelines': []}

Set the Model Conversion Arguments

We’ll create two different configurations, one for each of our models:

• sklearn_model_conversion_args: Used for our sklearn model.
• xgboost_model_converstion_args: Used for our XGBoost model.

# The number of columns
NF=25

sklearn_model_conversion_args = ConvertSKLearnArguments(
    name="sklearntest",
    comment="test linear regression",
    number_of_columns=NF,
    input_type=ModelConversionInputType.Double
)
sklearn_model_conversion_type = ModelConversionSource.SKLEARN

xgboost_model_conversion_args = ConvertXGBoostArgs(
    name="xgbtestreg",
    comment="xgboost regression model test",
    number_of_columns=NF,
    input_type=ModelConversionInputType.Float32
)
xgboost_model_conversion_type = ModelConversionSource.XGBOOST

Convert the Models

The convert_model method converts the model using the arguments, and uploads it into the current workspace - in this case, testconversion. Once complete, we can run get_current_workspace to verify that the models were uploaded.

# converts and uploads the sklearn model.
wl.convert_model('sklearn-linear-model.pickle', sklearn_model_conversion_type, sklearn_model_conversion_args)

# converts and uploads the XGBoost model.
wl.convert_model('xgb_reg.pickle', xgboost_model_conversion_type, xgboost_model_conversion_args)

{'name': 'xgbtestreg', 'version': '07b1ff68-0a7b-4687-b64f-76ddc995e7c6', 'file_name': '957178f8-fcd1-496d-b985-29beb89ba46e-converted.onnx', 'image_path': None, 'last_update_time': datetime.datetime(2023, 5, 17, 21, 11, 43, 585229, tzinfo=tzutc())}

wl.get_current_workspace()

{'name': 'testautoconversion', 'id': 11, 'archived': False, 'created_by': '028c8b48-c39b-4578-9110-0b5bdd3824da', 'created_at': '2023-05-17T21:11:40.856672+00:00', 'models': [{'name': 'sklearntest', 'versions': 1, 'owner_id': '""', 'last_update_time': datetime.datetime(2023, 5, 17, 21, 11, 42, 463507, tzinfo=tzutc()), 'created_at': datetime.datetime(2023, 5, 17, 21, 11, 42, 463507, tzinfo=tzutc())}, {'name': 'xgbtestreg', 'versions': 1, 'owner_id': '""', 'last_update_time': datetime.datetime(2023, 5, 17, 21, 11, 43, 585229, tzinfo=tzutc()), 'created_at': datetime.datetime(2023, 5, 17, 21, 11, 43, 585229, tzinfo=tzutc())}], 'pipelines': []}

4 - sk-learn Regression Model to ONNX Outside Wallaroo

This tutorial and the assets can be downloaded as part of the Wallaroo Tutorials repository.

How to Convert sk-learn Regression Model to ONNX

The following tutorial is a brief example of how to convert a scikit-learn (aka sk-learn) regression ML model to the ONNX.

This tutorial assumes that you have a Wallaroo instance and are running this Notebook from the Wallaroo Jupyter Hub service.

This tutorial provides the following:

• demand_curve.pickle: a demand curve trained sk-learn model. Once this file is converted to ONNX format, it can be used as part of the Demand Curve Pipeline Tutorial.

  This model contains 3 columns: UnitPrice, cust_known, and UnitPriceXcust_known.

Prerequisites

• An installed Wallaroo instance.
• The following Python libraries installed:
  • pickle
  • skl2onnx
  • onnxmltools
  • onnx
  • warnings

Conversion Process

Libraries

The first step is to import our libraries we will be using.

# Used to load the sk-learn model
import pickle

# Used for the conversion process
import onnx, skl2onnx, onnxmltools
from skl2onnx.common.data_types import FloatTensorType
from skl2onnx.common.data_types import DoubleTensorType

# ignoring warnings for demonstration
import warnings
warnings.filterwarnings('ignore')

Next let’s define our model_to_onnx method for converting a sk-learn model to ONNX. This has the following inputs:

• model: The sk-learn model we’re converting.
• cols: The number of inputs the model expects
• input_type: Determines how to manage float values, which can either be DoubleTensorType or FloatTensorType.

# convert model to ONNX

def model_to_onnx(model, cols, *, input_type='Double'):
    input_type_lower=input_type.lower()
    # How to manage float values
    if input_type=='Double':
        tensor_type=DoubleTensorType
    elif input_type=='Float':
        tensor_type=FloatTensorType
    else:
        raise ValueError("bad input type")
    tensor_size=cols
    initial_type=[(f'{input_type_lower}_input', tensor_type([None, tensor_size]))]
    onnx_model=onnxmltools.convert_sklearn(model,initial_types=initial_type)
    return onnx_model

With our method defined, now it’s time to convert. Let’s load our sk-learn model and save it into the variable sklearn_model.

# pickle the model, so I can try the Wallaroo converter on it

sklearn_model = pickle.load(open('./demand_curve.pickle', 'rb'))

Now we’ll convert our sklearn-model into the variable onnx_model using our model_to_onnx method. Recall that our sklearn-model has 3 columns.

onnx_model_converted = model_to_onnx(sklearn_model, 3)

Now we can save our model to a onnx file.

onnx.save_model(onnx_model_converted, "demand_curve.onnx")

5 - Statsmodel Upload to Wallaroo Tutorial

This tutorial and the assets can be downloaded as part of the Wallaroo Tutorials repository.

Introduction

Organizations can deploy a Machine Learning (ML) model based on the statsmodels directly into Wallaroo through the following process. This conversion process transforms the model into an open format that can be run across different frameworks at compiled C-language speeds.

This example provides the following:

• train-statsmodel.ipynb: A sample Jupyter Notebook that trains a sample model. The model predicts how many bikes will be rented on each of the next 7 days, based on the previous 7 days’ bike rentals, temperature, and wind speed. Additional files to support this example are:
  • day.csv: Data used to train the sample statsmodel example.
  • infer.py: The inference script that is part of the statsmodel.
• convert-statsmodel-tutorial.ipynb: A sample Jupyter Notebook that demonstrates how to upload, convert, and deploy the statsmodel example into a Wallaroo instance. Additional files to support this example are:

  • bike_day_model.pkl: A statsmodel ML model trained from the train-statsmodel.ipynb Notebook.

    IMPORTANT NOTE: The statsmodel ML model is composed of two parts that are contained in the .pkl file:

    • The pickled Python runtime expects a dictionary with two keys: model and script:

      • model—the pickled model, which will be automatically loaded into the python runtime with the name ‘model’
      • script—the text of the python script to be run, in a format similar to the existing python script steps (i.e. defining a wallaroo_json method which operates on the data). In this cae, the file infer.py is the script used.

  • bike_day_eval.json: Evaluation data used to test the model’s performance.

Steps

The following steps will perform the following:

1. Upload the statsmodel ML model bike_day_model.pkl into a Wallaroo.
2. Deploy the model into a pipeline.
3. Run a test inference.
4. Undeploy the pipeline.

Import Libraries

The first step is to import the libraries that we will need.

import json
import os
import datetime

import wallaroo
from wallaroo.object import EntityNotFoundError

# used to display dataframe information without truncating
from IPython.display import display
import pandas as pd
pd.set_option('display.max_colwidth', None)

Connect to the Wallaroo Instance

The first step is to connect to Wallaroo through the Wallaroo client. The Python library is included in the Wallaroo install and available through the Jupyter Hub interface provided with your Wallaroo environment.

This is accomplished using the wallaroo.Client() command, which provides a URL to grant the SDK permission to your specific Wallaroo environment. When displayed, enter the URL into a browser and confirm permissions. Store the connection into a variable that can be referenced later.

If logging into the Wallaroo instance through the internal JupyterHub service, use wl = wallaroo.Client(). If logging in externally, update the wallarooPrefix and wallarooSuffix variables with the proper DNS information. For more information on Wallaroo DNS settings, see the Wallaroo DNS Integration Guide.

# Login through local Wallaroo instance

wl = wallaroo.Client()

Set Configurations

The following will set the workspace, model name, and pipeline that will be used for this example. If the workspace or pipeline already exist, then they will assigned for use in this example. If they do not exist, they will be created based on the names listed below.

workspace_name = 'statsmodelworkspace'
pipeline_name = 'statsmodelpipeline'
model_name = 'bikedaymodel'
model_file_name = 'bike_day_model.pkl'

Set the Workspace and Pipeline

This sample code will create or use the existing workspace bike-day-workspace as the current workspace.

def get_workspace(name):
    workspace = None
    for ws in wl.list_workspaces():
        if ws.name() == name:
            workspace= ws
    if(workspace == None):
        workspace = wl.create_workspace(name)
    return workspace

def get_pipeline(name):
    try:
        pipeline = wl.pipelines_by_name(pipeline_name)[0]
    except EntityNotFoundError:
        pipeline = wl.build_pipeline(pipeline_name)
    return pipeline

workspace = get_workspace(workspace_name)

wl.set_current_workspace(workspace)

pipeline = get_pipeline(pipeline_name)
pipeline

Upload Pickled Package Statsmodel Model

Upload the statsmodel stored into the pickled package bike_day_model.pkl. See the Notebook train-statsmodel.ipynb for more details on creating this package.

Note that this package is being specified as a python configuration.

file_name = "bike_day_model.pkl"

bike_day_model = wl.upload_model(model_name, model_file_name).configure(runtime="python")

Deploy the Pipeline

We will now add the uploaded model as a step for the pipeline, then deploy it.

pipeline.add_model_step(bike_day_model)

pipeline.deploy()

pipeline.status()

{'status': 'Running',
 'details': [],
 'engines': [{'ip': '10.244.3.141',
   'name': 'engine-c77f759f7-f7fxd',
   'status': 'Running',
   'reason': None,
   'details': [],
   'pipeline_statuses': {'pipelines': [{'id': 'statsmodelpipeline',
      'status': 'Running'}]},
   'model_statuses': {'models': [{'name': 'bikedaymodel',
      'version': '66bf61d5-d144-4f77-82f1-58dabf2bbc33',
      'sha': '09b50a8e6a5cff566598dae6fb94f5d7d35c94e278373251cd8b1fd9a000c0a7',
      'status': 'Running'}]}}],
 'engine_lbs': [{'ip': '10.244.4.172',
   'name': 'engine-lb-584f54c899-67jbk',
   'status': 'Running',
   'reason': None,
   'details': []}],
 'sidekicks': []}

Run Inference

Perform an inference from the evaluation data JSON file bike_day_eval.json.

results = pipeline.infer_from_file('bike_day_eval.json', data_format="custom-json")

display(results)

[{'forecast': [1882.3784555157672,
   2130.607915701861,
   2340.84005381799,
   2895.754978552066,
   2163.657515565616,
   1509.1792126509536,
   2431.1838923957016]}]

Undeploy the Pipeline

Undeploy the pipeline and return the resources back to the Wallaroo instance.

pipeline.undeploy()

6 - XGBoost Classification Auto-Convert Within Wallaroo

This tutorial and the assets can be downloaded as part of the Wallaroo Tutorials repository.

Introduction

The following tutorial is a brief example of how to convert a XGBoost Classification ML model with the convert_model method and upload it into your Wallaroo instance.

This tutorial assumes that you have a Wallaroo instance and are running this Notebook from the Wallaroo Jupyter Hub service.

• Convert a XGBoost Classification ML model and upload it into the Wallaroo engine.
• Run a sample inference on the converted model in a Wallaroo instance.

This tutorial provides the following:

• xgb_class.pickle: A pretrained XGBoost Classification model with 25 columns.
• xgb_class_eval.json: Test data to perform a sample inference.

Prerequisites

Wallaroo supports the following model versions:

• XGBoost: Version 1.6.0
• SKLearn: 1.1.2

Conversion Steps

To use the Wallaroo autoconverter convert_model(path, source_type, conversion_arguments) method takes 3 parameters. The parameters for XGBoost conversions are:

• path (STRING): The path to the ML model file.
• source_type (ModelConversionSource): The type of ML model to be converted. As of this time Wallaroo auto-conversion supports the following source types and their associated ModelConversionSource:
  • sklearn: ModelConversionSource.SKLEARN
  • xgboost: ModelConversionSource.XGBOOST
  • keras: ModelConversionSource.KERAS
• conversion_arguments: The arguments for the conversion based on the type of model being converted. These are:
  • wallaroo.ModelConversion.ConvertXGBoostArgs: Used for XGBoost models and takes the following parameters:
  • name: The name of the model being converted.
  • comment: Any comments for the model.
  • number_of_columns: The number of columns the model was trained for.
  • input_type: A tensorflow Dtype called in the format ModelConversionInputType.{type}, where {type} is Float, Double, etc depending on the model.

Import Libraries

The first step is to import the libraries needed.

import wallaroo

from wallaroo.ModelConversion import ConvertXGBoostArgs, ModelConversionSource, ModelConversionInputType
from wallaroo.object import EntityNotFoundError

# used to display dataframe information without truncating
from IPython.display import display
import pandas as pd
pd.set_option('display.max_colwidth', None)

Connect to the Wallaroo Instance

The first step is to connect to Wallaroo through the Wallaroo client. The Python library is included in the Wallaroo install and available through the Jupyter Hub interface provided with your Wallaroo environment.

This is accomplished using the wallaroo.Client() command, which provides a URL to grant the SDK permission to your specific Wallaroo environment. When displayed, enter the URL into a browser and confirm permissions. Store the connection into a variable that can be referenced later.

If logging into the Wallaroo instance through the internal JupyterHub service, use wl = wallaroo.Client(). If logging in externally, update the wallarooPrefix and wallarooSuffix variables with the proper DNS information. For more information on Wallaroo DNS settings, see the Wallaroo DNS Integration Guide.

# Login through local Wallaroo instance

wl = wallaroo.Client()

Configuration and Methods

The following will set the workspace, pipeline, model name, the model file name used when uploading and converting the keras model, and the sample data.

The functions get_workspace(name) will either set the current workspace to the requested name, or create it if it does not exist. The function get_pipeline(name) will either set the pipeline used to the name requested, or create it in the current workspace if it does not exist.

workspace_name = 'xgboost-classification-autoconvert-workspace'
pipeline_name = 'xgboost-classification-autoconvert-pipeline'
model_name = 'xgb-class-model'
model_file_name = 'xgb_class.pickle'

def get_workspace(name):
    workspace = None
    for ws in wl.list_workspaces():
        if ws.name() == name:
            workspace= ws
    if(workspace == None):
        workspace = wl.create_workspace(name)
    return workspace

def get_pipeline(name):
    try:
        pipeline = wl.pipelines_by_name(name)[0]
    except EntityNotFoundError:
        pipeline = wl.build_pipeline(name)
    return pipeline

Set the Workspace and Pipeline

Set or create the workspace and pipeline based on the names configured earlier.

workspace = get_workspace(workspace_name)

wl.set_current_workspace(workspace)

pipeline = get_pipeline(pipeline_name)
pipeline

Set the Model Autoconvert Parameters

Set the paramters for converting the xgb-class-model.

#the number of columns
NF = 25

model_conversion_args = ConvertXGBoostArgs(
    name=model_name,
    comment="xgboost classification model test",
    number_of_columns=NF,
    input_type=ModelConversionInputType.Float32
)
model_conversion_type = ModelConversionSource.XGBOOST

Upload and Convert the Model

Now we can upload the convert the model. Once finished, it will be stored as {unique-file-id}-converted.onnx.

# convert and upload
model_wl = wl.convert_model(model_file_name, model_conversion_type, model_conversion_args)

Test Inference

With the model uploaded and converted, we can run a sample inference.

Deploy the Pipeline

Add the uploaded and converted model_wl as a step in the pipeline, then deploy it.

pipeline.add_model_step(model_wl).deploy()

Run the Inference

Use the evaluation data to verify the process completed successfully.

sample_data = 'xgb_class_eval.df.json'
result = pipeline.infer_from_file(sample_data)
display(result)

Undeploy the Pipeline

With the tests complete, we will undeploy the pipeline to return the resources back to the Wallaroo instance.

pipeline.undeploy()

7 - XGBoost Regression Auto-Convert Within Wallaroo

This tutorial and the assets can be downloaded as part of the Wallaroo Tutorials repository.

Introduction

The following tutorial is a brief example of how to convert a XGBoost Regression ML model with the convert_model method and upload it into your Wallaroo instance.

This tutorial assumes that you have a Wallaroo instance and are running this Notebook from the Wallaroo Jupyter Hub service.

• Convert a XGBoost Regression ML model and upload it into the Wallaroo engine.
• Run a sample inference on the converted model in a Wallaroo instance.

This tutorial provides the following:

• xgb_reg.pickle: A pretrained XGBoost Regression model with 25 columns.
• xgb_regression_eval.json: Test data to perform a sample inference.

Conversion Steps

Conversion Steps

To use the Wallaroo autoconverter convert_model(path, source_type, conversion_arguments) method takes 3 parameters. The parameters for XGBoost conversions are:

• path (STRING): The path to the ML model file.
• source_type (ModelConversionSource): The type of ML model to be converted. As of this time Wallaroo auto-conversion supports the following source types and their associated ModelConversionSource:
  • sklearn: ModelConversionSource.SKLEARN
  • xgboost: ModelConversionSource.XGBOOST
  • keras: ModelConversionSource.KERAS
• conversion_arguments: The arguments for the conversion based on the type of model being converted. These are:
  • wallaroo.ModelConversion.ConvertXGBoostArgs: Used for XGBoost models and takes the following parameters:
  • name: The name of the model being converted.
  • comment: Any comments for the model.
  • number_of_columns: The number of columns the model was trained for.
  • input_type: A tensorflow Dtype called in the format ModelConversionInputType.{type}, where {type} is Float, Double, etc depending on the model.

Import Libraries

The first step is to import the libraries needed.

import wallaroo

from wallaroo.ModelConversion import ConvertXGBoostArgs, ModelConversionSource, ModelConversionInputType
from wallaroo.object import EntityNotFoundError

# used to display dataframe information without truncating
from IPython.display import display
import pandas as pd
pd.set_option('display.max_colwidth', None)

Connect to the Wallaroo Instance

The first step is to connect to Wallaroo through the Wallaroo client. The Python library is included in the Wallaroo install and available through the Jupyter Hub interface provided with your Wallaroo environment.

This is accomplished using the wallaroo.Client() command, which provides a URL to grant the SDK permission to your specific Wallaroo environment. When displayed, enter the URL into a browser and confirm permissions. Store the connection into a variable that can be referenced later.

If logging into the Wallaroo instance through the internal JupyterHub service, use wl = wallaroo.Client(). If logging in externally, update the wallarooPrefix and wallarooSuffix variables with the proper DNS information. For more information on Wallaroo DNS settings, see the Wallaroo DNS Integration Guide.

# Login through local Wallaroo instance

wl = wallaroo.Client()

Configuration and Methods

The following will set the workspace, pipeline, model name, the model file name used when uploading and converting the keras model, and the sample data.

The functions get_workspace(name) will either set the current workspace to the requested name, or create it if it does not exist. The function get_pipeline(name) will either set the pipeline used to the name requested, or create it in the current workspace if it does not exist.

workspace_name = 'xgboost-regression-autoconvert-workspace'
pipeline_name = 'xgboost-regression-autoconvert-pipeline'
model_name = 'xgb-regression-model'
model_file_name = 'xgb_reg.pickle'

def get_workspace(name):
    workspace = None
    for ws in wl.list_workspaces():
        if ws.name() == name:
            workspace= ws
    if(workspace == None):
        workspace = wl.create_workspace(name)
    return workspace

def get_pipeline(name):
    try:
        pipeline = wl.pipelines_by_name(name)[0]
    except EntityNotFoundError:
        pipeline = wl.build_pipeline(name)
    return pipeline

Set the Workspace and Pipeline

Set or create the workspace and pipeline based on the names configured earlier.

workspace = get_workspace(workspace_name)

wl.set_current_workspace(workspace)

pipeline = get_pipeline(pipeline_name)
pipeline

Set the Model Autoconvert Parameters

Set the paramters for converting the xgb-class-model.

#the number of columns
NF = 25

model_conversion_args = ConvertXGBoostArgs(
    name=model_name,
    comment="xgboost regression model test",
    number_of_columns=NF,
    input_type=ModelConversionInputType.Float32
)
model_conversion_type = ModelConversionSource.XGBOOST

Upload and Convert the Model

Now we can upload the convert the model. Once finished, it will be stored as {unique-file-id}-converted.onnx.

# convert and upload
model_wl = wl.convert_model(model_file_name, model_conversion_type, model_conversion_args)

Test Inference

With the model uploaded and converted, we can run a sample inference.

Deploy the Pipeline

Add the uploaded and converted model_wl as a step in the pipeline, then deploy it.

pipeline.add_model_step(model_wl).deploy()

pipeline.status()

{'status': 'Running',
 'details': [],
 'engines': [{'ip': '10.244.2.161',
   'name': 'engine-5578b4dccb-k4xmk',
   'status': 'Running',
   'reason': None,
   'details': [],
   'pipeline_statuses': {'pipelines': [{'id': 'xgboost-regression-autoconvert-pipeline',
      'status': 'Running'}]},
   'model_statuses': {'models': [{'name': 'xgb-regression-model',
      'version': 'da71f6f2-28f9-4e33-be56-bec9ffddd3c8',
      'sha': 'b39f4982fa58efe81b5dcc8ee40fe4ef3348d0d53aa76d74fd82332e1aac394b',
      'status': 'Running'}]}}],
 'engine_lbs': [{'ip': '10.244.4.174',
   'name': 'engine-lb-584f54c899-gqsk4',
   'status': 'Running',
   'reason': None,
   'details': []}],
 'sidekicks': []}

Run the Inference

Use the test_class_eval.json as set earlier as our sample_data and perform the inference.

sample_data = 'xgb_regression_eval.df.json'
result = pipeline.infer_from_file(sample_data)
display(result)    

Undeploy the Pipeline

With the tests complete, we will undeploy the pipeline to return the resources back to the Wallaroo instance.

pipeline.undeploy()

8 - XGBoost Convert to ONNX

This tutorial and the assets can be downloaded as part of the Wallaroo Tutorials repository.

How to Convert XGBoost to ONNX

The following tutorial is a brief example of how to convert a XGBoost ML model to the ONNX standard. This allows organizations that have trained XGBoost models to convert them and use them with Wallaroo.

This tutorial assumes that you have a Wallaroo instance and are running this Notebook from the Wallaroo Jupyter Hub service.

This tutorial provides the following:

• housing_model_xgb.pkl: A pretrained model used as part of the Notebooks in Production tutorial. This model has a total of 18 columns.

Conversion Process

Libraries

The first step is to import our libraries we will be using.

import onnx
import pickle
from onnxmltools.convert import convert_xgboost

from skl2onnx.common.data_types import FloatTensorType, DoubleTensorType

Set Variables

The following variables are required to be known before the process can be started:

• number of columns: The number of columns used by the model.
• TARGET_OPSET: Verify the TARGET_OPSET value taht will be used in the conversion process matches the current Wallaroo model uploads requirements.

# set the number of columns
ncols = 18
TARGET_OPSET = 15

Load the XGBoost Model

Next we will load our model that has been saved in the pickle format and unpickle it.

# load the xgboost model
with open("housing_model_xgb.pkl", "rb") as f:
    xgboost_model = pickle.load(f)

Conversion Inputs

The convert_xgboost method has the following format and requires the following inputs:

convert_xgboost({XGBoost Model}, 
                {XGBoost Model Type},
                [
                    ('input', 
                    {Tensor Data Type}([None, {ncols}]))
                ],
                target_opset={TARGET_OPSET})

1. XGBoost Model: The XGBoost Model to convert.
2. XGBoost Model Type: The type of XGBoost model. In this example is it a tree-based classifier.
3. Tensor Data Type: Either FloatTensorType or DoubleTensorType from the skl2onnx.common.data_types library.
4. ncols: Number of columns in the model.
5. TARGET_OPSET: The target opset which can be derived in code showed below.

Convert the Model

With all of our data in place we can now convert our XBBoost model to ONNX using the convert_xgboost method.

onnx_model_converted = convert_xgboost(xgboost_model, 'tree-based classifier',
                             [('input', FloatTensorType([None, ncols]))],
                             target_opset=TARGET_OPSET)

Save the Model

With the model converted to ONNX, we can now save it and use it in a Wallaroo pipeline.

onnx.save_model(onnx_model_converted, "housing_model_xgb.onnx")