Wallaroo SDK Upload Tutorials: SKLearn
How to upload different SKLearn models to Wallaroo.
The following tutorials cover how to upload sample SKLearn models.
Sci-kit Learn aka SKLearn.
During the model upload process, the Wallaroo instance will attempt to convert the model to a Native Wallaroo Runtime. If unsuccessful based , it will create a Wallaroo Containerized Runtime for the model. See the model deployment section for details on how to configure pipeline resources based on the model’s runtime.
SKLearn schema follows a different format than other models. To prevent inputs from being out of order, the inputs should be submitted in a single row in the order the model is trained to accept, with all of the data types being the same. For example, the following DataFrame has 4 columns, each column a float
.
| sepal length (cm) | sepal width (cm) | petal length (cm) | petal width (cm) |
---|
0 | 5.1 | 3.5 | 1.4 | 0.2 |
1 | 4.9 | 3.0 | 1.4 | 0.2 |
For submission to an SKLearn model, the data input schema will be a single array with 4 float values.
input_schema = pa.schema([
pa.field('inputs', pa.list_(pa.float64(), list_size=4))
])
When submitting as an inference, the DataFrame is converted to rows with the column data expressed as a single array. The data must be in the same order as the model expects, which is why the data is submitted as a single array rather than JSON labeled columns: this insures that the data is submitted in the exact order as the model is trained to accept.
Original DataFrame:
| sepal length (cm) | sepal width (cm) | petal length (cm) | petal width (cm) |
---|
0 | 5.1 | 3.5 | 1.4 | 0.2 |
1 | 4.9 | 3.0 | 1.4 | 0.2 |
Converted DataFrame:
| inputs |
---|
0 | [5.1, 3.5, 1.4, 0.2] |
1 | [4.9, 3.0, 1.4, 0.2] |
SKLearn Schema Outputs
Outputs for SKLearn that are meant to be predictions
or probabilities
when output by the model are labeled in the output schema for the model when uploaded to Wallaroo. For example, a model that outputs either 1 or 0 as its output would have the output schema as follows:
output_schema = pa.schema([
pa.field('predictions', pa.int32())
])
When used in Wallaroo, the inference result is contained in the out
metadata as out.predictions
.
pipeline.infer(dataframe)
| time | in.inputs | out.predictions | check_failures |
---|
0 | 2023-07-05 15:11:29.776 | [5.1, 3.5, 1.4, 0.2] | 0 | 0 |
1 | 2023-07-05 15:11:29.776 | [4.9, 3.0, 1.4, 0.2] | 0 | 0 |
1 - Wallaroo SDK Upload Tutorial: SKLearn Clustering Kmeans
How to upload a SKLearn Clustering Kmeans to Wallaroo
This tutorial can be downloaded as part of the Wallaroo Tutorials repository.
Wallaroo Model Upload via the Wallaroo SDK: SKLearn Clustering KMeans
The following tutorial demonstrates how to upload a SKLearn Clustering KMeans model to a Wallaroo instance.
Tutorial Goals
Demonstrate the following:
- Upload a SKLearn Clustering KMeans model to a Wallaroo instance.
- Create a pipeline and add the model as a pipeline step.
- Perform a sample inference.
Prerequisites
- A Wallaroo version 2023.2.1 or above instance.
References
Tutorial Steps
Import Libraries
The first step is to import the libraries we’ll be using. These are included by default in the Wallaroo instance’s JupyterHub service.
import json
import os
import pickle
import wallaroo
from wallaroo.pipeline import Pipeline
from wallaroo.deployment_config import DeploymentConfigBuilder
from wallaroo.object import EntityNotFoundError
from wallaroo.framework import Framework
import os
os.environ["MODELS_ENABLED"] = "true"
import pyarrow as pa
import numpy as np
import pandas as pd
Open a Connection to Wallaroo
The next step is connect to Wallaroo through the Wallaroo client. The Python library is included in the Wallaroo install and available through the Jupyter Hub interface provided with your Wallaroo environment.
This is accomplished using the wallaroo.Client()
command, which provides a URL to grant the SDK permission to your specific Wallaroo environment. When displayed, enter the URL into a browser and confirm permissions. Store the connection into a variable that can be referenced later.
If logging into the Wallaroo instance through the internal JupyterHub service, use wl = wallaroo.Client()
. If logging in externally, update the wallarooPrefix
and wallarooSuffix
variables with the proper DNS information. For more information on Wallaroo DNS settings, see the Wallaroo DNS Integration Guide.
Set Variables and Helper Functions
We’ll set the name of our workspace, pipeline, models and files. Workspace names must be unique across the Wallaroo workspace. For this, we’ll add in a randomly generated 4 characters to the workspace name to prevent collisions with other users’ workspaces. If running this tutorial, we recommend hard coding the workspace name so it will function in the same workspace each time it’s run.
We’ll set up some helper functions that will either use existing workspaces and pipelines, or create them if they do not already exist.
def get_workspace(name):
workspace = None
for ws in wl.list_workspaces():
if ws.name() == name:
workspace= ws
if(workspace == None):
workspace = wl.create_workspace(name)
return workspace
def get_pipeline(name):
try:
pipeline = wl.pipelines_by_name(name)[0]
except EntityNotFoundError:
pipeline = wl.build_pipeline(name)
return pipeline
import string
import random
# make a random 4 character suffix to prevent overwriting other user's workspaces
suffix= ''.join(random.choice(string.ascii_lowercase) for i in range(4))
suffix=''
workspace_name = f'sklearn-clustering-kmeans{suffix}'
pipeline_name = f'sklearn-clustering-kmeans'
model_name = 'sklearn-clustering-kmeans'
model_file_name = "models/model-auto-conversion_sklearn_kmeans.pkl"
Create Workspace and Pipeline
We will now create the Wallaroo workspace to store our model and set it as the current workspace. Future commands will default to this workspace for pipeline creation, model uploads, etc. We’ll create our Wallaroo pipeline to deploy our model.
workspace = get_workspace(workspace_name)
wl.set_current_workspace(workspace)
pipeline = get_pipeline(pipeline_name)
SKLearn models are uploaded to Wallaroo through the Wallaroo Client upload_model
method.
Upload SKLearn Model Parameters
The following parameters are required for SKLearn models. Note that while some fields are considered as optional for the upload_model
method, they are required for proper uploading of a SKLearn model to Wallaroo.
Parameter | Type | Description |
---|
name | string (Required) | The name of the model. Model names are unique per workspace. Models that are uploaded with the same name are assigned as a new version of the model. |
path | string (Required) | The path to the model file being uploaded. |
framework | string (Upload Method Optional, SKLearn model Required) | Set as the Framework.SKLEARN . |
input_schema | pyarrow.lib.Schema (Upload Method Optional, SKLearn model Required) | The input schema in Apache Arrow schema format. |
output_schema | pyarrow.lib.Schema (Upload Method Optional, SKLearn model Required) | The output schema in Apache Arrow schema format. |
convert_wait | bool (Upload Method Optional, SKLearn model Optional) (Default: True) | - True: Waits in the script for the model conversion completion.
- False: Proceeds with the script without waiting for the model conversion process to display complete.
|
Once the upload process starts, the model is containerized by the Wallaroo instance. This process may take up to 10 minutes.
input_schema = pa.schema([
pa.field('inputs', pa.list_(pa.float64(), list_size=4))
])
output_schema = pa.schema([
pa.field('predictions', pa.int32())
])
Upload Model
The model will be uploaded with the framework set as Framework.SKLEARN
.
model = wl.upload_model(model_name,
model_file_name,
framework=Framework.SKLEARN,
input_schema=input_schema,
output_schema=output_schema,
)
model
Waiting for model loading - this will take up to 10.0min.
Model is pending loading to a native runtime..
Model is attempting loading to a native runtime..incompatible
Model is pending loading to a container runtime..
Model is attempting loading to a container runtime..............successful
Ready
Name | sklearn-clustering-kmeans |
Version | 34e40a39-41a1-42b9-a13f-7d49f0c52830 |
File Name | model-auto-conversion_sklearn_kmeans.pkl |
SHA | b378a614854619dd573ec65b9b4ac73d0b397d50a048e733d96b68c5fdbec896 |
Status | ready |
Image Path | proxy.replicated.com/proxy/wallaroo/ghcr.io/wallaroolabs/mlflow-deploy:v2023.4.0-main-4005 |
Architecture | None |
Updated At | 2023-20-Oct 21:04:46 |
'flight'
Deploy Pipeline
The model is uploaded and ready for use. We’ll add it as a step in our pipeline, then deploy the pipeline. For this example we’re allocated 0.25 cpu and 4 Gi RAM to the pipeline through the pipeline’s deployment configuration.
deployment_config = DeploymentConfigBuilder() \
.cpus(0.25).memory('1Gi') \
.build()
# clear the pipeline if it was used before
pipeline.undeploy()
pipeline.clear()
pipeline.add_model_step(model)
pipeline.deploy(deployment_config=deployment_config)
Inference
SKLearn models must have all of the data as one line to prevent columns from being read out of order when submitting in JSON. The following will take in the data, convert the rows into a single inputs
for the table, then perform the inference. From the output_schema
we have defined the output as predictions
which will be displayed in our inference result output as out.predictions
.
data = pd.read_json('./data/test-sklearn-kmeans.json')
display(data)
# move the column values to a single array input
mock_dataframe = pd.DataFrame({"inputs": data[:2].values.tolist()})
display(mock_dataframe)
| sepal length (cm) | sepal width (cm) | petal length (cm) | petal width (cm) |
---|
0 | 5.1 | 3.5 | 1.4 | 0.2 |
---|
1 | 4.9 | 3.0 | 1.4 | 0.2 |
---|
| inputs |
---|
0 | [5.1, 3.5, 1.4, 0.2] |
---|
1 | [4.9, 3.0, 1.4, 0.2] |
---|
result = pipeline.infer(mock_dataframe)
display(result)
| time | in.inputs | out.predictions | check_failures |
---|
0 | 2023-10-20 21:08:04.496 | [5.1, 3.5, 1.4, 0.2] | 1 | 0 |
---|
1 | 2023-10-20 21:08:04.496 | [4.9, 3.0, 1.4, 0.2] | 1 | 0 |
---|
Undeploy Pipelines
With the tutorial complete, the pipeline is undeployed to return the resources back to the cluster.
Waiting for undeployment - this will take up to 45s ..................................... ok
name | sklearn-clustering-kmeans |
---|
created | 2023-10-20 20:47:07.107730+00:00 |
---|
last_updated | 2023-10-20 21:05:30.043676+00:00 |
---|
deployed | False |
---|
tags | |
---|
versions | 7df8f5d9-01db-40ae-bd47-2a871db46058, afbe6d0e-ecb6-47e0-9982-2eb1228f82a2, 35562729-c690-4da2-a1fd-37a760b3909f, d9bf604a-0310-4783-ae52-c6606eb3e228, f33e9409-c562-404c-a09e-a0d6d8e63d1a, a4caed26-a5d7-4d08-9c84-11be2f2c02e8 |
---|
steps | sklearn-clustering-kmeans |
---|
published | False |
---|
2 - Wallaroo SDK Upload Tutorial: SKLearn Clustering SVM
How to upload a SKLearn Clustering SVM to Wallaroo
This tutorial can be downloaded as part of the Wallaroo Tutorials repository.
Wallaroo Model Upload via the Wallaroo SDK: Sklearn Clustering SVM
The following tutorial demonstrates how to upload a SKLearn Clustering Support Vector Machine(SVM) model to a Wallaroo instance.
Tutorial Goals
Demonstrate the following:
- Upload a Sklearn Clustering SVM model to a Wallaroo instance.
- Create a pipeline and add the model as a pipeline step.
- Perform a sample inference.
Prerequisites
- A Wallaroo version 2023.2.1 or above instance.
References
Tutorial Steps
Import Libraries
The first step is to import the libraries we’ll be using. These are included by default in the Wallaroo instance’s JupyterHub service.
import json
import os
import pickle
import wallaroo
from wallaroo.pipeline import Pipeline
from wallaroo.deployment_config import DeploymentConfigBuilder
from wallaroo.object import EntityNotFoundError
from wallaroo.framework import Framework
import os
os.environ["MODELS_ENABLED"] = "true"
import pyarrow as pa
import numpy as np
import pandas as pd
Open a Connection to Wallaroo
The next step is connect to Wallaroo through the Wallaroo client. The Python library is included in the Wallaroo install and available through the Jupyter Hub interface provided with your Wallaroo environment.
This is accomplished using the wallaroo.Client()
command, which provides a URL to grant the SDK permission to your specific Wallaroo environment. When displayed, enter the URL into a browser and confirm permissions. Store the connection into a variable that can be referenced later.
If logging into the Wallaroo instance through the internal JupyterHub service, use wl = wallaroo.Client()
. If logging in externally, update the wallarooPrefix
and wallarooSuffix
variables with the proper DNS information. For more information on Wallaroo DNS settings, see the Wallaroo DNS Integration Guide.
Set Variables and Helper Functions
We’ll set the name of our workspace, pipeline, models and files. Workspace names must be unique across the Wallaroo workspace. For this, we’ll add in a randomly generated 4 characters to the workspace name to prevent collisions with other users’ workspaces. If running this tutorial, we recommend hard coding the workspace name so it will function in the same workspace each time it’s run.
We’ll set up some helper functions that will either use existing workspaces and pipelines, or create them if they do not already exist.
def get_workspace(name):
workspace = None
for ws in wl.list_workspaces():
if ws.name() == name:
workspace= ws
if(workspace == None):
workspace = wl.create_workspace(name)
return workspace
def get_pipeline(name):
try:
pipeline = wl.pipelines_by_name(name)[0]
except EntityNotFoundError:
pipeline = wl.build_pipeline(name)
return pipeline
import string
import random
# make a random 4 character suffix to prevent overwriting other user's workspaces
suffix= ''.join(random.choice(string.ascii_lowercase) for i in range(4))
suffix=''
workspace_name = f'sklearn-clustering-svm{suffix}'
pipeline_name = f'sklearn-clustering-svm'
model_name = 'sklearn-clustering-svm'
model_file_name = './models/model-auto-conversion_sklearn_svm_pipeline.pkl'
Create Workspace and Pipeline
We will now create the Wallaroo workspace to store our model and set it as the current workspace. Future commands will default to this workspace for pipeline creation, model uploads, etc. We’ll create our Wallaroo pipeline to deploy our model.
workspace = get_workspace(workspace_name)
wl.set_current_workspace(workspace)
pipeline = get_pipeline(pipeline_name)
SKLearn models are uploaded to Wallaroo through the Wallaroo Client upload_model
method.
Upload SKLearn Model Parameters
The following parameters are required for SKLearn models. Note that while some fields are considered as optional for the upload_model
method, they are required for proper uploading of a SKLearn model to Wallaroo.
Parameter | Type | Description |
---|
name | string (Required) | The name of the model. Model names are unique per workspace. Models that are uploaded with the same name are assigned as a new version of the model. |
path | string (Required) | The path to the model file being uploaded. |
framework | string (Upload Method Optional, SKLearn model Required) | Set as the Framework.SKLEARN . |
input_schema | pyarrow.lib.Schema (Upload Method Optional, SKLearn model Required) | The input schema in Apache Arrow schema format. |
output_schema | pyarrow.lib.Schema (Upload Method Optional, SKLearn model Required) | The output schema in Apache Arrow schema format. |
convert_wait | bool (Upload Method Optional, SKLearn model Optional) (Default: True) | - True: Waits in the script for the model conversion completion.
- False: Proceeds with the script without waiting for the model conversion process to display complete.
|
Once the upload process starts, the model is containerized by the Wallaroo instance. This process may take up to 10 minutes.
input_schema = pa.schema([
pa.field('inputs', pa.list_(pa.float64(), list_size=4))
])
output_schema = pa.schema([
pa.field('predictions', pa.int32())
])
Upload Model
The model will be uploaded with the framework set as Framework.SKLEARN
.
model = wl.upload_model(model_name,
model_file_name,
framework=Framework.SKLEARN,
input_schema=input_schema,
output_schema=output_schema)
model
Waiting for model loading - this will take up to 10.0min.
Model is pending loading to a native runtime..
Model is attempting loading to a native runtime..incompatible
Model is pending loading to a container runtime.
Model is attempting loading to a container runtime.........successful
Ready
Name | sklearn-clustering-svm |
Version | 9e9f6913-999d-4a19-894c-ecc372debcaf |
File Name | model-auto-conversion_sklearn_svm_pipeline.pkl |
SHA | c6eec69d96f7eeb3db034600dea6b12da1d2b832c39252ec4942d02f68f52f40 |
Status | ready |
Image Path | proxy.replicated.com/proxy/wallaroo/ghcr.io/wallaroolabs/mlflow-deploy:v2023.4.0-main-4005 |
Architecture | None |
Updated At | 2023-20-Oct 21:10:50 |
'flight'
Deploy Pipeline
The model is uploaded and ready for use. We’ll add it as a step in our pipeline, then deploy the pipeline. For this example we’re allocated 0.25 cpu and 4 Gi RAM to the pipeline through the pipeline’s deployment configuration.
deployment_config = DeploymentConfigBuilder() \
.cpus(0.25).memory('1Gi') \
.build()
# clear the pipeline if it was used before
pipeline.undeploy()
pipeline.clear()
pipeline.add_model_step(model)
pipeline.deploy(deployment_config=deployment_config)
ok
Waiting for deployment - this will take up to 45s ...........................................
*** An error occurred while deploying your pipeline.
Deployment failed. See status for details.
Status: {'status': 'Error', 'details': [], 'engines': [{'ip': '10.244.3.151', 'name': 'engine-8444db4dcb-md57f', 'status': 'Running', 'reason': None, 'details': [], 'pipeline_statuses': {'pipelines': [{'id': 'sklearn-clustering-svm', 'status': 'Running'}]}, 'model_statuses': {'models': [{'name': 'sklearn-clustering-svm', 'version': '9e9f6913-999d-4a19-894c-ecc372debcaf', 'sha': 'c6eec69d96f7eeb3db034600dea6b12da1d2b832c39252ec4942d02f68f52f40', 'status': 'Running'}]}}], 'engine_lbs': [{'ip': '10.244.2.207', 'name': 'engine-lb-584f54c899-nkj9h', 'status': 'Running', 'reason': None, 'details': []}], 'sidekicks': [{'ip': '10.244.3.150', 'name': 'engine-sidekick-sklearn-clustering-svm-71-59bc7cf755-vkq92', 'status': 'Running', 'reason': None, 'details': [], 'statuses': None}]}
Inference
SKLearn models must have all of the data as one line to prevent columns from being read out of order when submitting in JSON. The following will take in the data, convert the rows into a single inputs
for the table, then perform the inference. From the output_schema
we have defined the output as predictions
which will be displayed in our inference result output as out.predictions
.
data = pd.read_json('./data/test_cluster-svm.json')
display(data)
# move the column values to a single array input
dataframe = pd.DataFrame({"inputs": data[:2].values.tolist()})
display(dataframe)
| sepal length (cm) | sepal width (cm) | petal length (cm) | petal width (cm) |
---|
0 | 5.1 | 3.5 | 1.4 | 0.2 |
---|
1 | 4.9 | 3.0 | 1.4 | 0.2 |
---|
| inputs |
---|
0 | [5.1, 3.5, 1.4, 0.2] |
---|
1 | [4.9, 3.0, 1.4, 0.2] |
---|
pipeline.infer(dataframe)
| time | in.inputs | out.predictions | check_failures |
---|
0 | 2023-10-20 21:11:44.879 | [5.1, 3.5, 1.4, 0.2] | 0 | 0 |
---|
1 | 2023-10-20 21:11:44.879 | [4.9, 3.0, 1.4, 0.2] | 0 | 0 |
---|
Undeploy Pipelines
With the tutorial complete, the pipeline is undeployed to return the resources back to the cluster.
Waiting for undeployment - this will take up to 45s ..................................... ok
name | sklearn-clustering-svm |
---|
created | 2023-10-20 21:00:38.923149+00:00 |
---|
last_updated | 2023-10-20 21:10:52.412340+00:00 |
---|
deployed | False |
---|
tags | |
---|
versions | bf6b7315-b5c8-47fe-b088-ded36b18a3c6, 1758d8ac-6f86-4863-b194-1d7baf20fc1f, 8a5fbd0d-028a-478d-990d-4aa34a8e2b1b |
---|
steps | sklearn-clustering-svm |
---|
published | False |
---|
3 - Wallaroo SDK Upload Tutorial: SKLearn Linear Regression
How to upload a SKLearn Linear Regression model to Wallaroo
This tutorial can be downloaded as part of the Wallaroo Tutorials repository.
Wallaroo Model Upload via the Wallaroo SDK: SKLearn Linear Regression
The following tutorial demonstrates how to upload a SKLearn Linear Regression model to a Wallaroo instance.
Tutorial Goals
Demonstrate the following:
- Upload a SKLearn Linear Regression model to a Wallaroo instance.
- Create a pipeline and add the model as a pipeline step.
- Perform a sample inference.
Prerequisites
- A Wallaroo version 2023.2.1 or above instance.
References
Tutorial Steps
Import Libraries
The first step is to import the libraries we’ll be using. These are included by default in the Wallaroo instance’s JupyterHub service.
import json
import os
import pickle
import wallaroo
from wallaroo.pipeline import Pipeline
from wallaroo.deployment_config import DeploymentConfigBuilder
from wallaroo.object import EntityNotFoundError
from wallaroo.framework import Framework
import os
os.environ["MODELS_ENABLED"] = "true"
import pyarrow as pa
import numpy as np
import pandas as pd
Open a Connection to Wallaroo
The next step is connect to Wallaroo through the Wallaroo client. The Python library is included in the Wallaroo install and available through the Jupyter Hub interface provided with your Wallaroo environment.
This is accomplished using the wallaroo.Client()
command, which provides a URL to grant the SDK permission to your specific Wallaroo environment. When displayed, enter the URL into a browser and confirm permissions. Store the connection into a variable that can be referenced later.
If logging into the Wallaroo instance through the internal JupyterHub service, use wl = wallaroo.Client()
. If logging in externally, update the wallarooPrefix
and wallarooSuffix
variables with the proper DNS information. For more information on Wallaroo DNS settings, see the Wallaroo DNS Integration Guide.
Set Variables and Helper Functions
We’ll set the name of our workspace, pipeline, models and files. Workspace names must be unique across the Wallaroo workspace. For this, we’ll add in a randomly generated 4 characters to the workspace name to prevent collisions with other users’ workspaces. If running this tutorial, we recommend hard coding the workspace name so it will function in the same workspace each time it’s run.
We’ll set up some helper functions that will either use existing workspaces and pipelines, or create them if they do not already exist.
def get_workspace(name):
workspace = None
for ws in wl.list_workspaces():
if ws.name() == name:
workspace= ws
if(workspace == None):
workspace = wl.create_workspace(name)
return workspace
def get_pipeline(name):
try:
pipeline = wl.pipelines_by_name(name)[0]
except EntityNotFoundError:
pipeline = wl.build_pipeline(name)
return pipeline
import string
import random
# make a random 4 character suffix to prevent overwriting other user's workspaces
suffix= ''.join(random.choice(string.ascii_lowercase) for i in range(4))
suffix=''
workspace_name = f'sklearn-linear-regression{suffix}'
pipeline_name = f'sklearn-linear-regression'
model_name = 'sklearn-linear-regression'
model_file_name = 'models/model-auto-conversion_sklearn_linreg_diabetes.pkl'
Create Workspace and Pipeline
We will now create the Wallaroo workspace to store our model and set it as the current workspace. Future commands will default to this workspace for pipeline creation, model uploads, etc. We’ll create our Wallaroo pipeline to deploy our model.
workspace = get_workspace(workspace_name)
wl.set_current_workspace(workspace)
pipeline = get_pipeline(pipeline_name)
SKLearn models are uploaded to Wallaroo through the Wallaroo Client upload_model
method.
Upload SKLearn Model Parameters
The following parameters are required for SKLearn models. Note that while some fields are considered as optional for the upload_model
method, they are required for proper uploading of a SKLearn model to Wallaroo.
Parameter | Type | Description |
---|
name | string (Required) | The name of the model. Model names are unique per workspace. Models that are uploaded with the same name are assigned as a new version of the model. |
path | string (Required) | The path to the model file being uploaded. |
framework | string (Upload Method Optional, SKLearn model Required) | Set as the Framework.SKLEARN . |
input_schema | pyarrow.lib.Schema (Upload Method Optional, SKLearn model Required) | The input schema in Apache Arrow schema format. |
output_schema | pyarrow.lib.Schema (Upload Method Optional, SKLearn model Required) | The output schema in Apache Arrow schema format. |
convert_wait | bool (Upload Method Optional, SKLearn model Optional) (Default: True) | - True: Waits in the script for the model conversion completion.
- False: Proceeds with the script without waiting for the model conversion process to display complete.
|
Once the upload process starts, the model is containerized by the Wallaroo instance. This process may take up to 10 minutes.
input_schema = pa.schema([
pa.field('inputs', pa.list_(pa.float64(), list_size=10))
])
output_schema = pa.schema([
pa.field('predictions', pa.float32())
])
Upload Model
The model will be uploaded with the framework set as Framework.SKLEARN
.
model = wl.upload_model(model_name,
model_file_name,
framework=Framework.SKLEARN,
input_schema=input_schema,
output_schema=output_schema)
model
Waiting for model loading - this will take up to 10.0min.
Model is pending loading to a native runtime..
Model is attempting loading to a native runtime..incompatible
Model is pending loading to a container runtime.
Model is attempting loading to a container runtime...........successful
Ready
Name | sklearn-linear-regression |
Version | 44d45548-b606-4c53-a741-072a8948b26c |
File Name | model-auto-conversion_sklearn_linreg_diabetes.pkl |
SHA | 6a9085e2d65bf0379934651d2272d3c6c4e020e36030933d85df3a8d15135a45 |
Status | ready |
Image Path | proxy.replicated.com/proxy/wallaroo/ghcr.io/wallaroolabs/mlflow-deploy:v2023.4.0-main-4005 |
Architecture | None |
Updated At | 2023-20-Oct 21:12:03 |
'flight'
Deploy Pipeline
The model is uploaded and ready for use. We’ll add it as a step in our pipeline, then deploy the pipeline. For this example we’re allocated 0.25 cpu and 4 Gi RAM to the pipeline through the pipeline’s deployment configuration.
deployment_config = DeploymentConfigBuilder() \
.cpus(0.25).memory('1Gi') \
.build()
# clear the pipeline if it was used before
pipeline.undeploy()
pipeline.clear()
pipeline.add_model_step(model)
pipeline.deploy(deployment_config=deployment_config)
Inference
SKLearn models must have all of the data as one line to prevent columns from being read out of order when submitting in JSON. The following will take in the data, convert the rows into a single inputs
for the table, then perform the inference. From the output_schema
we have defined the output as predictions
which will be displayed in our inference result output as out.predictions
.
data = pd.read_json('data/test_linear_regression_data.json')
display(data)
# move the column values to a single array input
dataframe = pd.DataFrame({"inputs": data[:2].values.tolist()})
display(dataframe)
| age | sex | bmi | bp | s1 | s2 | s3 | s4 | s5 | s6 |
---|
0 | 0.038076 | 0.050680 | 0.061696 | 0.021872 | -0.044223 | -0.034821 | -0.043401 | -0.002592 | 0.019907 | -0.017646 |
---|
1 | -0.001882 | -0.044642 | -0.051474 | -0.026328 | -0.008449 | -0.019163 | 0.074412 | -0.039493 | -0.068332 | -0.092204 |
---|
| inputs |
---|
0 | [0.0380759064, 0.0506801187, 0.0616962065, 0.0... |
---|
1 | [-0.0018820165, -0.0446416365, -0.051474061200... |
---|
pipeline.infer(dataframe)
| time | in.inputs | out.predictions | check_failures |
---|
0 | 2023-10-20 21:13:02.803 | [0.0380759064, 0.0506801187, 0.0616962065, 0.0... | 206.116677 | 0 |
---|
1 | 2023-10-20 21:13:02.803 | [-0.0018820165, -0.0446416365, -0.0514740612, ... | 68.071033 | 0 |
---|
Undeploy Pipelines
With the tutorial complete, the pipeline is undeployed to return the resources back to the cluster.
Waiting for undeployment - this will take up to 45s ...................................... ok
name | sklearn-linear-regression |
---|
created | 2023-10-20 21:10:45.329449+00:00 |
---|
last_updated | 2023-10-20 21:12:08.793882+00:00 |
---|
deployed | False |
---|
tags | |
---|
versions | 96565b51-c90c-4d0d-bee8-8c1a777af534, 682a9e09-a139-4546-a469-969be071ffcb |
---|
steps | sklearn-linear-regression |
---|
published | False |
---|
4 - Wallaroo SDK Upload Tutorial: SKLearn Logistic Regression
How to upload a SKLearn Logistic Regression to Wallaroo
This tutorial can be downloaded as part of the Wallaroo Tutorials repository.
Wallaroo Model Upload via the Wallaroo SDK: SKLearn Logistic Regression
The following tutorial demonstrates how to upload a SKLearn Logistic Regression model to a Wallaroo instance.
Tutorial Goals
Demonstrate the following:
- Upload a SKLearn Logistic Regression model to a Wallaroo instance.
- Create a pipeline and add the model as a pipeline step.
- Perform a sample inference.
Prerequisites
- A Wallaroo version 2023.2.1 or above instance.
References
Tutorial Steps
Import Libraries
The first step is to import the libraries we’ll be using. These are included by default in the Wallaroo instance’s JupyterHub service.
import json
import os
import pickle
import wallaroo
from wallaroo.pipeline import Pipeline
from wallaroo.deployment_config import DeploymentConfigBuilder
from wallaroo.object import EntityNotFoundError
from wallaroo.framework import Framework
import os
os.environ["MODELS_ENABLED"] = "true"
import pyarrow as pa
import numpy as np
import pandas as pd
Open a Connection to Wallaroo
The next step is connect to Wallaroo through the Wallaroo client. The Python library is included in the Wallaroo install and available through the Jupyter Hub interface provided with your Wallaroo environment.
This is accomplished using the wallaroo.Client()
command, which provides a URL to grant the SDK permission to your specific Wallaroo environment. When displayed, enter the URL into a browser and confirm permissions. Store the connection into a variable that can be referenced later.
If logging into the Wallaroo instance through the internal JupyterHub service, use wl = wallaroo.Client()
. If logging in externally, update the wallarooPrefix
and wallarooSuffix
variables with the proper DNS information. For more information on Wallaroo DNS settings, see the Wallaroo DNS Integration Guide.
Set Variables and Helper Functions
We’ll set the name of our workspace, pipeline, models and files. Workspace names must be unique across the Wallaroo workspace. For this, we’ll add in a randomly generated 4 characters to the workspace name to prevent collisions with other users’ workspaces. If running this tutorial, we recommend hard coding the workspace name so it will function in the same workspace each time it’s run.
We’ll set up some helper functions that will either use existing workspaces and pipelines, or create them if they do not already exist.
def get_workspace(name):
workspace = None
for ws in wl.list_workspaces():
if ws.name() == name:
workspace= ws
if(workspace == None):
workspace = wl.create_workspace(name)
return workspace
def get_pipeline(name):
try:
pipeline = wl.pipelines_by_name(name)[0]
except EntityNotFoundError:
pipeline = wl.build_pipeline(name)
return pipeline
import string
import random
# make a random 4 character suffix to prevent overwriting other user's workspaces
suffix= ''.join(random.choice(string.ascii_lowercase) for i in range(4))
suffix=''
workspace_name = f'sklearn-logistic-regression{suffix}'
pipeline_name = f'sklearn-logistic-regression'
model_name = 'sklearn-logistic-regression'
model_file_name = 'models/logreg.pkl'
Create Workspace and Pipeline
We will now create the Wallaroo workspace to store our model and set it as the current workspace. Future commands will default to this workspace for pipeline creation, model uploads, etc. We’ll create our Wallaroo pipeline to deploy our model.
workspace = get_workspace(workspace_name)
wl.set_current_workspace(workspace)
pipeline = get_pipeline(pipeline_name)
SKLearn models are uploaded to Wallaroo through the Wallaroo Client upload_model
method.
Upload SKLearn Model Parameters
The following parameters are required for SKLearn models. Note that while some fields are considered as optional for the upload_model
method, they are required for proper uploading of a SKLearn model to Wallaroo.
Parameter | Type | Description |
---|
name | string (Required) | The name of the model. Model names are unique per workspace. Models that are uploaded with the same name are assigned as a new version of the model. |
path | string (Required) | The path to the model file being uploaded. |
framework | string (Upload Method Optional, SKLearn model Required) | Set as the Framework.SKLEARN . |
input_schema | pyarrow.lib.Schema (Upload Method Optional, SKLearn model Required) | The input schema in Apache Arrow schema format. |
output_schema | pyarrow.lib.Schema (Upload Method Optional, SKLearn model Required) | The output schema in Apache Arrow schema format. |
convert_wait | bool (Upload Method Optional, SKLearn model Optional) (Default: True) | - True: Waits in the script for the model conversion completion.
- False: Proceeds with the script without waiting for the model conversion process to display complete.
|
Once the upload process starts, the model is containerized by the Wallaroo instance. This process may take up to 10 minutes.
input_schema = pa.schema([
pa.field('inputs', pa.list_(pa.float64(), list_size=4))
])
output_schema = pa.schema([
pa.field('predictions', pa.int32()),
pa.field('probabilities', pa.list_(pa.float64(), list_size=3))
])
Upload Model
The model will be uploaded with the framework set as Framework.SKLEARN
.
model = wl.upload_model(model_name,
model_file_name,
framework=Framework.SKLEARN,
input_schema=input_schema,
output_schema=output_schema)
model
Waiting for model loading - this will take up to 10.0min.
Model is pending loading to a native runtime..
Model is attempting loading to a native runtime..incompatible
Model is pending loading to a container runtime.
Model is attempting loading to a container runtime...........successful
Ready
Name | sklearn-logistic-regression |
Version | 56627137-162a-4437-a417-5f7af8c4241d |
File Name | logreg.pkl |
SHA | 9302df6cc64a2c0d12daa257657f07f9db0bb2072bb3fb92396500b21358e0b9 |
Status | ready |
Image Path | proxy.replicated.com/proxy/wallaroo/ghcr.io/wallaroolabs/mlflow-deploy:v2023.4.0-main-4005 |
Architecture | None |
Updated At | 2023-20-Oct 21:11:53 |
'flight'
Deploy Pipeline
The model is uploaded and ready for use. We’ll add it as a step in our pipeline, then deploy the pipeline. For this example we’re allocated 0.25 cpu and 4 Gi RAM to the pipeline through the pipeline’s deployment configuration.
deployment_config = DeploymentConfigBuilder() \
.cpus(0.25).memory('1Gi') \
.build()
# clear the pipeline if it was used before
pipeline.undeploy()
pipeline.clear()
pipeline.add_model_step(model)
pipeline.deploy(deployment_config=deployment_config)
Inference
SKLearn models must have all of the data as one line to prevent columns from being read out of order when submitting in JSON. The following will take in the data, convert the rows into a single inputs
for the table, then perform the inference. From the output_schema
we have defined the output as predictions
which will be displayed in our inference result output as out.predictions
.
data = pd.read_json('data/test_logreg_data.json')
display(data)
# move the column values to a single array input
dataframe = pd.DataFrame({"inputs": data[:2].values.tolist()})
display(dataframe)
| sepal length (cm) | sepal width (cm) | petal length (cm) | petal width (cm) |
---|
0 | 5.1 | 3.5 | 1.4 | 0.2 |
---|
1 | 4.9 | 3.0 | 1.4 | 0.2 |
---|
| inputs |
---|
0 | [5.1, 3.5, 1.4, 0.2] |
---|
1 | [4.9, 3.0, 1.4, 0.2] |
---|
pipeline.infer(dataframe)
| time | in.inputs | out.predictions | out.probabilities | check_failures |
---|
0 | 2023-10-20 21:12:53.100 | [5.1, 3.5, 1.4, 0.2] | 0 | [0.9815821465852236, 0.018417838912958125, 1.4... | 0 |
---|
1 | 2023-10-20 21:12:53.100 | [4.9, 3.0, 1.4, 0.2] | 0 | [0.9713374799347873, 0.028662489870060148, 3.0... | 0 |
---|
Undeploy Pipelines
With the tutorial complete, the pipeline is undeployed to return the resources back to the cluster.
Waiting for undeployment - this will take up to 45s .................................... ok
name | sklearn-logistic-regression |
---|
created | 2023-10-20 21:10:37.789707+00:00 |
---|
last_updated | 2023-10-20 21:11:58.541031+00:00 |
---|
deployed | False |
---|
tags | |
---|
versions | b8170bfd-fafb-44ce-a62c-c82c6da46f4c, 7b233ca1-b1bb-4448-a948-b8fbb1481b75 |
---|
steps | sklearn-logistic-regression |
---|
published | False |
---|
5 - Wallaroo SDK Upload Tutorial: SKLearn SVM PCA
How to upload a SKLearn SVM PCA to Wallaroo
This tutorial can be downloaded as part of the Wallaroo Tutorials repository.
Wallaroo Model Upload via the Wallaroo SDK: Sklearn Clustering SVM PCA
The following tutorial demonstrates how to upload a SKLearn Clustering Support Vector Machine(SVM) Principal Component Analysis (PCA) model to a Wallaroo instance.
Tutorial Goals
Demonstrate the following:
- Upload a Sklearn Clustering SVM PCA model to a Wallaroo instance.
- Create a pipeline and add the model as a pipeline step.
- Perform a sample inference.
Prerequisites
- A Wallaroo version 2023.2.1 or above instance.
References
Tutorial Steps
Import Libraries
The first step is to import the libraries we’ll be using. These are included by default in the Wallaroo instance’s JupyterHub service.
import json
import os
import pickle
import wallaroo
from wallaroo.pipeline import Pipeline
from wallaroo.deployment_config import DeploymentConfigBuilder
from wallaroo.object import EntityNotFoundError
from wallaroo.framework import Framework
import os
os.environ["MODELS_ENABLED"] = "true"
import pyarrow as pa
import numpy as np
import pandas as pd
Open a Connection to Wallaroo
The next step is connect to Wallaroo through the Wallaroo client. The Python library is included in the Wallaroo install and available through the Jupyter Hub interface provided with your Wallaroo environment.
This is accomplished using the wallaroo.Client()
command, which provides a URL to grant the SDK permission to your specific Wallaroo environment. When displayed, enter the URL into a browser and confirm permissions. Store the connection into a variable that can be referenced later.
If logging into the Wallaroo instance through the internal JupyterHub service, use wl = wallaroo.Client()
. If logging in externally, update the wallarooPrefix
and wallarooSuffix
variables with the proper DNS information. For more information on Wallaroo DNS settings, see the Wallaroo DNS Integration Guide.
Set Variables and Helper Functions
We’ll set the name of our workspace, pipeline, models and files. Workspace names must be unique across the Wallaroo workspace. For this, we’ll add in a randomly generated 4 characters to the workspace name to prevent collisions with other users’ workspaces. If running this tutorial, we recommend hard coding the workspace name so it will function in the same workspace each time it’s run.
We’ll set up some helper functions that will either use existing workspaces and pipelines, or create them if they do not already exist.
def get_workspace(name):
workspace = None
for ws in wl.list_workspaces():
if ws.name() == name:
workspace= ws
if(workspace == None):
workspace = wl.create_workspace(name)
return workspace
def get_pipeline(name):
try:
pipeline = wl.pipelines_by_name(name)[0]
except EntityNotFoundError:
pipeline = wl.build_pipeline(name)
return pipeline
import string
import random
# make a random 4 character suffix to prevent overwriting other user's workspaces
suffix= ''.join(random.choice(string.ascii_lowercase) for i in range(4))
suffix=''
workspace_name = f'sklearn-clustering-svm-pca{suffix}'
pipeline_name = f'sklearn-clustering-svm-pca'
model_name = 'sklearn-clustering-svm-pca'
model_file_name = 'models/model-auto-conversion_sklearn_svm_pca_pipeline.pkl'
Create Workspace and Pipeline
We will now create the Wallaroo workspace to store our model and set it as the current workspace. Future commands will default to this workspace for pipeline creation, model uploads, etc. We’ll create our Wallaroo pipeline to deploy our model.
workspace = get_workspace(workspace_name)
wl.set_current_workspace(workspace)
pipeline = get_pipeline(pipeline_name)
SKLearn models are uploaded to Wallaroo through the Wallaroo Client upload_model
method.
Upload SKLearn Model Parameters
The following parameters are required for SKLearn models. Note that while some fields are considered as optional for the upload_model
method, they are required for proper uploading of a SKLearn model to Wallaroo.
Parameter | Type | Description |
---|
name | string (Required) | The name of the model. Model names are unique per workspace. Models that are uploaded with the same name are assigned as a new version of the model. |
path | string (Required) | The path to the model file being uploaded. |
framework | string (Upload Method Optional, SKLearn model Required) | Set as the Framework.SKLEARN . |
input_schema | pyarrow.lib.Schema (Upload Method Optional, SKLearn model Required) | The input schema in Apache Arrow schema format. |
output_schema | pyarrow.lib.Schema (Upload Method Optional, SKLearn model Required) | The output schema in Apache Arrow schema format. |
convert_wait | bool (Upload Method Optional, SKLearn model Optional) (Default: True) | - True: Waits in the script for the model conversion completion.
- False: Proceeds with the script without waiting for the model conversion process to display complete.
|
Once the upload process starts, the model is containerized by the Wallaroo instance. This process may take up to 10 minutes.
input_schema = pa.schema([
pa.field('inputs', pa.list_(pa.float64(), list_size=4))
])
output_schema = pa.schema([
pa.field('predictions', pa.int32())
])
Upload Model
The model will be uploaded with the framework set as Framework.SKLEARN
.
model = wl.upload_model(model_name,
model_file_name,
framework=Framework.SKLEARN,
input_schema=input_schema,
output_schema=output_schema)
model
Waiting for model loading - this will take up to 10.0min.
Model is pending loading to a native runtime..
Model is attempting loading to a native runtime..incompatible
Model is pending loading to a container runtime.
Model is attempting loading to a container runtime.............successful
Ready
Name | sklearn-clustering-svm-pca |
Version | 8568d7b1-1f50-48c8-839b-00b6d0dfb734 |
File Name | model-auto-conversion_sklearn_svm_pca_pipeline.pkl |
SHA | 524b05d22f13fa4ce5feaf07b86710b447f0c80a02601be86ee5b6bc748fe7fd |
Status | ready |
Image Path | proxy.replicated.com/proxy/wallaroo/ghcr.io/wallaroolabs/mlflow-deploy:v2023.4.0-main-4005 |
Architecture | None |
Updated At | 2023-20-Oct 21:06:26 |
'flight'
Deploy Pipeline
The model is uploaded and ready for use. We’ll add it as a step in our pipeline, then deploy the pipeline. For this example we’re allocated 0.25 cpu and 4 Gi RAM to the pipeline through the pipeline’s deployment configuration.
deployment_config = DeploymentConfigBuilder() \
.cpus(0.25).memory('1Gi') \
.build()
# clear the pipeline if it was used before
pipeline.undeploy()
pipeline.clear()
pipeline.add_model_step(model)
pipeline.deploy(deployment_config=deployment_config)
Inference
SKLearn models must have all of the data as one line to prevent columns from being read out of order when submitting in JSON. The following will take in the data, convert the rows into a single inputs
for the table, then perform the inference. From the output_schema
we have defined the output as predictions
which will be displayed in our inference result output as out.predictions
.
data = pd.read_json('data/test-sklearn-kmeans.json')
display(data)
# move the column values to a single array input
dataframe = pd.DataFrame({"inputs": data[:2].values.tolist()})
display(dataframe)
| sepal length (cm) | sepal width (cm) | petal length (cm) | petal width (cm) |
---|
0 | 5.1 | 3.5 | 1.4 | 0.2 |
---|
1 | 4.9 | 3.0 | 1.4 | 0.2 |
---|
| inputs |
---|
0 | [5.1, 3.5, 1.4, 0.2] |
---|
1 | [4.9, 3.0, 1.4, 0.2] |
---|
pipeline.infer(dataframe)
| time | in.inputs | out.predictions | check_failures |
---|
0 | 2023-10-20 21:08:35.188 | [5.1, 3.5, 1.4, 0.2] | 0 | 0 |
---|
1 | 2023-10-20 21:08:35.188 | [4.9, 3.0, 1.4, 0.2] | 0 | 0 |
---|
Undeploy Pipelines
With the tutorial complete, the pipeline is undeployed to return the resources back to the cluster.
Waiting for undeployment - this will take up to 45s ...................................... ok
name | sklearn-clustering-svm-pca |
---|
created | 2023-10-20 20:49:31.399831+00:00 |
---|
last_updated | 2023-10-20 21:06:27.900292+00:00 |
---|
deployed | False |
---|
tags | |
---|
versions | 4f174cb4-a314-4f62-9960-e1bb7dcb8782, 0b7b909b-10e1-4ae4-a2e9-6c07d020fe27, 984f5d05-2799-4d36-a0eb-deb076c1d1be, 76968e9f-774e-4730-9c84-3c4143b5e123 |
---|
steps | sklearn-clustering-svm-pca |
---|
published | False |
---|