Healthcare: Upload and Deploy
This tutorial and the assets can be downloaded as part of the Wallaroo Workshops repository.
For this workshop, let’s pretend that you work for a health care company wants detect mitochondria from microscope images. You have developed a model to predict the sale price of properties that the company has listed, based on data collected in the company’s listings database.
In this set of exercises, you will used a pre-trained model and deploy it to Wallaroo. This will require understanding the following concepts:
- Wallaroo Workspaces: Workspaces are environments were users upload models, create pipelines and other artifacts. The workspace should be considered the fundamental area where work is done. Workspaces are shared with other users to give them access to the same models, pipelines, etc.
- Wallaroo Model Upload and Registration: ML Models are uploaded to Wallaroo through the SDK or the MLOps API to a workspace. ML models include default runtimes (ONNX, Python Step, and TensorFlow) that are run directly through the Wallaroo engine, and containerized runtimes (Hugging Face, PyTorch, etc) that are run through in a container through the Wallaroo engine.
- Wallaroo Pipelines: Pipelines are used to deploy models for inferencing. Each model is a pipeline step in a pipelines, where the inputs of the previous step are fed into the next. Pipeline steps can be ML models, Python scripts, or Arbitrary Python (these contain necessary models and artifacts for running a model).
For this tutorial, we will be providing pre-trained models in ONNX format. To see how to upload and deploy your particular model, see the Wallaroo Documentation site.
This tutorial includes a helper module CVDemoUtils
, which converts images into a DataFrame format that our computer vision model expects.
Preliminary Steps
Before starting this module, ensure that the models have been downloaded.
To download the Wallaroo Computer Vision models, use the following link:
https://storage.googleapis.com/wallaroo-public-data/cv-demo-models/cv-retail-models.zip
Unzip the contents into the directory models
.
Before we start, let’s load some libraries that we will need for this notebook (note that this may not be a complete list).
- IMPORTANT NOTE: This tutorial is geared towards a Wallaroo 2023.2.1 environment.
import json
import IPython.display as display
import time
import matplotlib.pyplot as plt
from IPython.display import clear_output, display
import tifffile as tiff
import pandas as pd
import wallaroo
from wallaroo.object import EntityNotFoundError
import numpy as np
from matplotlib import pyplot as plt
import cv2
from tensorflow.keras.utils import normalize
# setting path - only needed when running this from the `with-code` folder.
import sys
sys.path.append('../')
from lib.TiffImageUtils import TiffUtils
tiff_utils = TiffUtils()
# ignoring warnings for demonstration
import warnings
warnings.filterwarnings('ignore')
import requests
Get ready to work with Wallaroo
With the libraries loaded, you can log into Wallaroo. This will provide access to your workspaces, workspaces shared with you from other users, and all other aspects of the Wallaroo environment.
Logging into Wallaroo via the cluster’s integrated JupyterLab is quite straight forward:
# Login through local Wallaroo instance
wl = wallaroo.Client()
See the documentation if you are logging into Wallaroo some other way such as from a remote location. This tutorial assumes you’re logging in through the Wallaroo JupyterHub service.
Notice that the Wallaroo client connection is stored into a variable called wl. This variable can be anything you want it to be - you can have client = wallaroo.Client()
or myWallarooClient = wallaroo.Client()
.
This variable is about to become your best friend - a lot of the commands you’ll be running will be through this variable, like wl.list_workspaces()
to get all of the workspaces available to you in your Wallaroo environment, or wl.list_models()
to show all of the models in your current workspace. We’ll go into these commands and more - just make sure you saved that Wallaroo client to a variable so you can use it for the other commands.
When we log into the Wallaroo through the SDK, the Client will provide a url to verify your authentication. Either click it, or copy and past that URL, then authenticate into your Wallaroo instance with your email address and password.
Login to Wallaroo Exercise
Time to login to your Wallaroo instance. By now you should be logged into your Wallaroo JupyterHub service and looking at this notebook.
Copy the code below and place it into the code block and run it. When prompted, select the authentication URL by either clicking it, or copying and pasting it into a browser. Log into your Wallaroo instance, and then the client will be set.
# Login through local Wallaroo instance
wl = wallaroo.Client()
# put your wallaroo Client login code here.
# Login through local Wallaroo instance
wl = wallaroo.Client()
Run Client Commands
We’re now logged into our Wallaroo instance, let’s run some commands to get used to working within the environment.
The following are going to be very useful as you work in Wallaroo.
List Workspaces
The command wallaroo.Client.list_workspaces
gives a list of all of the workspaces you have access to in the Wallaroo environment. Here’s an example of running it. For this example, our Wallaroo client is stored in the wl
variable, but this could have been named wallaroo_client
or whatever you like.
wl.list_workspaces()
Name | Created At | Users | Models | Pipelines |
---|---|---|---|---|
john.hummel@wallaroo.ai - Default Workspace | 2023-08-21 19:06:07 | [‘john.hummel@wallaroo.ai’] | 1 | 1 |
edge-publish-demojohn | 2023-08-21 20:54:35 | [‘john.hummel@wallaroo.ai’] | 1 | 1 |
biolabsworkspacegomj | 2023-08-22 15:11:11 | [‘john.hummel@wallaroo.ai’] | 1 | 1 |
biolabsworkspacedtrv | 2023-08-22 16:03:32 | [‘john.hummel@wallaroo.ai’] | 1 | 1 |
biolabsworkspacejohn | 2023-08-22 16:07:15 | [‘john.hummel@wallaroo.ai’] | 1 | 1 |
Listing the workspaces will show the following fields:
- name: The user created workspace name. Workspace names must be unique across the Wallaroo instance.
- created_at: The date and time the workspace was created.
- users: The users in the workspace. The user who created the workspace is always listed.
- models: The number of models in the workspace.
- pipelines: The number of pipelines in the workspace.
When you first login to Wallaroo, the workspace {your email address} - Default Workspace
is created. This is assigned as your current workspace once you have logged into Wallaroo. We’ll cover workspaces in a moment - for now, just remember that every time you do wallaroo.Client()
, that your default workspace is set as your current workspace, and any commands you issue will be directed to that workspace.
Notice that in that example above, you only see one default workspace - the one of the user who ran the list_workspaces
command. Other users will have their own default workspaces. You can only see workspaces you have access to.
List Users
Users are listed with the wallaroo.Client.list_users
command. This shows all of the user’s email addresses and names across the Wallaroo instance.
There’s more commands, but we’ll stop here until we’ve created our workspace and uploaded some models.
Client Commands Exercise
For this exercise, use your Wallaroo client variable and list the current workspaces and current users. You can do this with your Wallaroo client variable you saved in a previous step.
For example, if your Wallaroo client variable is wl
, then the following would list the workspaces and then the users:
# list the workspaces available to you
workspaces = wl.list_workspaces()
print(workspaces)
# list the users
users = wl.list_users()
print(users)
# empty space to get workspaces and users
display(wl.list_workspaces())
display(wl.list_users())
Name | Created At | Users | Models | Pipelines |
---|---|---|---|---|
john.hummel@wallaroo.ai - Default Workspace | 2023-09-27 15:17:23 | ['john.hummel@wallaroo.ai'] | 1 | 2 |
cv-arm-edge-demo-jch | 2023-09-27 15:17:44 | ['john.hummel@wallaroo.ai'] | 1 | 1 |
workshop-workspace-john-10 | 2023-09-27 16:29:36 | ['john.hummel@wallaroo.ai'] | 2 | 1 |
workshop-workspace-john-05 | 2023-09-27 16:50:52 | ['john.hummel@wallaroo.ai'] | 1 | 1 |
workshop-workspace-john-cv-yolo | 2023-09-28 16:06:55 | ['john.hummel@wallaroo.ai'] | 2 | 1 |
retail | 2023-09-28 19:44:06 | ['john.hummel@wallaroo.ai'] | 1 | 1 |
workshop-workspace-john-cv-medical | 2023-09-28 20:22:21 | ['john.hummel@wallaroo.ai'] | 1 | 1 |
[User({"id": "568f86b1-8b18-4db8-a40b-eca338fe371d", "email": "admin@keycloak", "username": "admin", "enabled": "True),
User({"id": "b030ff9c-41eb-49b4-afdf-2ccbecb6be5d", "email": "john.hummel@wallaroo.ai", "username": "john.hummel@wallaroo.ai", "enabled": "True)]
Workspace Creation and Management
A Wallaroo workspace is place to organize the deployment artifacts for a project, and to collaborate with other team members. For more information, see the Wallaroo 101.
When you upload a ML model to Wallaroo, you upload it to a workspace. When a pipeline is created and models assigned to it, that pipeline is in side of a workspace. Wallaroo workload orchestrations? They’re assigned to a workspace.
.
When you first login, the SDK assigns you to your default workspace. Most of the commands issued through your Wallaroo client will target that workspace.
You can see what workspace you are currently in with the wallaroo.Client.get_current_workspace()
method.
This shows you the following workspace fields:
- name: The user created workspace name. Workspace names must be unique across the Wallaroo instance.
- id: The numerical identifier of the workspace.
- archived: Whether the workspace was archived or not.
- created_by: The Keycloak ID of the user who created the workspace. This is in UUID format, and is used to identify specific users. Most of the time you’ll refer to users by their email address.
- created_at: The date and time the workspace was created.
- models: The models uploaded to the workspace and their details.
- pipelines: The number of pipelines in the workspace and their details.
Get Current Workspace Exercise
Get your current workspace with the wallaroo.Client.get_current_workspace()
. For example, if your Wallaroo client was saved to the variable wl
, this command would be:
wl.get_current_workspace()
When done,
## blank space to get your current workspace
wl.get_current_workspace()
{'name': 'john.hummel@wallaroo.ai - Default Workspace', 'id': 1, 'archived': False, 'created_by': 'b030ff9c-41eb-49b4-afdf-2ccbecb6be5d', 'created_at': '2023-09-27T15:17:23.923705+00:00', 'models': [{'name': 'v5s6', 'versions': 2, 'owner_id': '""', 'last_update_time': datetime.datetime(2023, 9, 28, 19, 23, 3, 425821, tzinfo=tzutc()), 'created_at': datetime.datetime(2023, 9, 28, 18, 55, 24, 860357, tzinfo=tzutc())}], 'pipelines': [{'name': 'cv-arm-edge', 'create_time': datetime.datetime(2023, 9, 27, 15, 17, 45, 593032, tzinfo=tzutc()), 'definition': '[]'}, {'name': 'retailimage', 'create_time': datetime.datetime(2023, 9, 28, 18, 55, 14, 954338, tzinfo=tzutc()), 'definition': '[]'}]}
Create New Workspace
Workspaces are created with the wallaroo.Client.create_workspace(name)
, where name
is the new name of the workspace. When a new workspace is created, the workspace user is assigned to the user that created it (in this case - you). For example, if the Wallaroo client is stored to the variable wl
, then the following will create the new workspace ‘sparkly-bunnies, then store the workspace information into the variable
workspace`:
workspace = wl.create_workspace('sparkly-bunnies')
Once this is created, this shows you the following workspace fields:
- name: The user created workspace name. Workspace names must be unique across the Wallaroo instance, and must be DNS compliant. So ‘my-cool-workspace’ is ok, but ‘?? workspace’ is not.
-
are ok, but_
is not. - id: The numerical identifier of the workspace.
- archived: Whether the workspace was archived or not.
- created_by: The Keycloak ID of the user who created the workspace. This is in UUID format, and is used to identify specific users. Most of the time you’ll refer to users by their email address.
- created_at: The date and time the workspace was created.
- models: The models uploaded to the workspace and their details. For a new workspace, this will be empty
[]
. - pipelines: The number of pipelines in the workspace and their details. For a new workspace, this will be empty
[]
.
Workspace names must be unique. So the following will fail:
wl.create_workspace('sparkly-bunnies')
wl.create_workspace('sparkly-bunnies')
Exception: Failed to create workspace.
Create New Workspace Exercise
Now it’s time for us to create our own workspace. To make this easy, we’ll call the workspace ‘workshop-workspace-{firstname}’. If someone else has the same first name and is in this workshop, each of you decide who should change their name for this to work. Or - if it’s easier - change the firstname
to something else like john1
.
For example, if your Wallaroo client was saved to the variable wl
, then the command to create a new workspace workshop-workspace-sample
is:
wl.create_workspace('workshop-workspace-sample')
When you’re done, list the workspaces. You did that in a previous step, so you can copy that here.
# blank space to create your workspace
print(wl.create_workspace('workshop-workspace-john-cv-medical'))
# list all the workspaces here
wl.list_workspaces()
Retrieve Workspace
In the above example, you saw this:
workspace = wl.create_workspace('sparkly-bunnies')
This creates the workspace sparkly-bunnies
, then assigns it to the variable workspace
. We can display that workspace and see what it looks like. For example:
workspace = wl.create_workspace('sparkly-bunnies')
print(workspace)
{'name': 'sparkly-bunnies', 'id': 9, 'archived': False, 'created_by': '66d3b2c4-9b22-4429-b16e-3bcdc1ac28e3', 'created_at': '2023-08-22T17:30:40.475785+00:00', 'models': [], 'pipelines': []}
If we had created a workspace earlier, and want to work with it, we will have to retrieve it. We do that with the wallaroo.Client.list_workspaces
method, which returns a List. Then we can set the workspace we want to a variable.
Here’s an example. We start with list_workspaces:
wl.list_workspaces()
Name | Created At | Users | Models | Pipelines |
---|---|---|---|---|
john.hummel@wallaroo.ai - Default Workspace | 2023-08-21 19:06:07 | [‘john.hummel@wallaroo.ai’] | 1 | 1 |
sparkly-bunnies | 2023-08-22 17:30:40 | [‘john.hummel@wallaroo.ai’] | 0 | 0 |
workshop-workspace-sample | 2023-08-22 17:38:07 | [‘john.hummel@wallaroo.ai’] | 0 | 0 |
workshop-workspace-john | 2023-08-22 17:38:13 | [‘john.hummel@wallaroo.ai’] | 0 | 0 |
In our case, we want to use sparkly-bunnies
. If we count down from the top, it’s the 2nd in the List. Since lists start at 0, it’s as position 1. For those not experienced with Lists, here’s how it looks.
position | workspace |
---|---|
0 | john.hummel@wallaroo.ai - Default Workspace |
1 | sparkly-bunnies |
2 | workshop-workspace-sample |
And so on.
To store a specific workspace from list_workspaces
, we assign it to a variable based on its position. If we want to store the workspace sparkly-bunnies
to a variable workspace
, we do it like so:
workspace = wl.list_workspaces()[1]
print(workspace)
{'name': 'sparkly-bunnies', 'id': 9, 'archived': False, 'created_by': '66d3b2c4-9b22-4429-b16e-3bcdc1ac28e3', 'created_at': '2023-08-22T17:30:40.475785+00:00', 'models': [], 'pipelines': []}
For those more familiar with lists, you can do things like filter by name like so:
def get_workspace(name, client):
workspace = None
for ws in client.list_workspaces():
if ws.name() == name:
workspace= ws
We’ll introduce helper functions like this later, but know for now if you know how to use a List, then you can retrieve a workspace.
Retrieve Workspace Exercise
Retrieve the workspace to a variable you created earlier through the following steps:
- List the workspaces.
- Determine the position of the workspace from the list - remember the list positions start at 0.
- Assign the workspace to a variable that you’ll use later.
For example, if the workspace is position 1 in the list_workspaces
list, then you would retrieve the workspace like so:
workspace = wl.list_workspaces()[1]
# blank space to retrieve your workspace
display(wl.list_workspaces())
workspace = wl.list_workspaces()[-1]
print(workspace)
Name | Created At | Users | Models | Pipelines |
---|---|---|---|---|
john.hummel@wallaroo.ai - Default Workspace | 2023-09-27 15:17:23 | ['john.hummel@wallaroo.ai'] | 1 | 2 |
cv-arm-edge-demo-jch | 2023-09-27 15:17:44 | ['john.hummel@wallaroo.ai'] | 1 | 1 |
workshop-workspace-john-10 | 2023-09-27 16:29:36 | ['john.hummel@wallaroo.ai'] | 2 | 1 |
workshop-workspace-john-05 | 2023-09-27 16:50:52 | ['john.hummel@wallaroo.ai'] | 1 | 1 |
workshop-workspace-john-cv-yolo | 2023-09-28 16:06:55 | ['john.hummel@wallaroo.ai'] | 2 | 1 |
retail | 2023-09-28 19:44:06 | ['john.hummel@wallaroo.ai'] | 1 | 1 |
workshop-workspace-john-cv-medical | 2023-09-28 20:22:21 | ['john.hummel@wallaroo.ai'] | 1 | 1 |
{'name': 'workshop-workspace-john-cv-medical', 'id': 10, 'archived': False, 'created_by': 'b030ff9c-41eb-49b4-afdf-2ccbecb6be5d', 'created_at': '2023-09-28T20:22:21.506043+00:00', 'models': [{'name': 'mitochondria-detector', 'versions': 1, 'owner_id': '""', 'last_update_time': datetime.datetime(2023, 9, 28, 20, 25, 2, 359367, tzinfo=tzutc()), 'created_at': datetime.datetime(2023, 9, 28, 20, 25, 2, 359367, tzinfo=tzutc())}], 'pipelines': [{'name': 'cv-mitochondria', 'create_time': datetime.datetime(2023, 9, 28, 20, 25, 17, 379732, tzinfo=tzutc()), 'definition': '[]'}]}
Set the Current Workspace
We mentioned earlier that when you login to Wallaroo, the SDK assigns you to the default workspace - the one named {your email address} - Default Workspace
- replacing your email address in the front.
Usually you’ll want to work in some other workspace - perhaps one that you’re a part of with other users, or one you set up yourself for test purposes. It is highly recommended that workspaces be divided by project or some specific goal where the same models are used for different purposes.
We’ve gone over how to create a workspace, and how to retrieve a workspace that was previously created. Now we’ll use that to set our current workspace with the wallaroo.Client.set_current_workspace(workspace)
.
The current workspace is where your SDK commands are routed to. When you give the upload models command - they are uploaded to your current workspace. Build a pipeline? Associated to the current workspace.
So we’re going to make sure that what we’re doing is done in the right workspace with two commands:
wallaroo.Client.get_current_workspace()
: Shows what the current workspace.wallaroo.Client.set_current_workspace(workspace)
: Sets the current workspace to the target workspace.
For example, if your workspace is saved to a variable as shown in the previous step, we can change from the default workspace and set our current workspace to the new one as follows:
# show the current workspace
print(wl.get_current_workspace())
{'name': 'john.hummel@wallaroo.ai - Default Workspace', 'id': 1, 'archived': False, 'created_by': '66d3b2c4-9b22-4429-b16e-3bcdc1ac28e3', 'created_at': '2023-08-21T19:06:07.404363+00:00', 'models': [{'name': 'm1', 'versions': 1, 'owner_id': '""', 'last_update_time': datetime.datetime(2023, 8, 21, 19, 38, 36, 672465, tzinfo=tzutc()), 'created_at': datetime.datetime(2023, 8, 21, 19, 38, 36, 672465, tzinfo=tzutc())}], 'pipelines': [{'name': 'p1', 'create_time': datetime.datetime(2023, 8, 21, 19, 38, 44, 314377, tzinfo=tzutc()), 'definition': '[]'}]}
# change to the new workspace
wl.set_current_workspace(workspace)
# show the new current workspace
print(wl.get_current_workspace())
{'name': 'workshop-workspace-john', 'id': 12, 'archived': False, 'created_by': '66d3b2c4-9b22-4429-b16e-3bcdc1ac28e3', 'created_at': '2023-08-22T17:38:13.612187+00:00', 'models': [], 'pipelines': []}
Setting your current workspace to the one you want to work in is an important step. We highly recommend that once a Wallaroo client connection is established, the next task should be setting whatever workspace is the proper one to work in as the current workspace, then proceeding with any other tasks.
Set the Current Workspace Exercise
Previously you created a workspace and retrieved it to a variable. Using the * wallaroo.Client.get_current_workspace()
and wallaroo.Client.set_current_workspace(workspace)
methods:
- Get your current workspace.
- Set your current workspace to the one created in the previous steps.
- Get your current workspace again to verify that the change was made.
For example, if your Wallaroo client was stored as the variable wl
, and your new workspace saved to the variable workspace
, you can change your current workspace to the new one with the following:
wl.set_current_workspace(workspace)
wl.get_current_workspace()
print(wl.get_current_workspace())
wl.set_current_workspace(workspace)
print(wl.get_current_workspace())
{'name': 'john.hummel@wallaroo.ai - Default Workspace', 'id': 1, 'archived': False, 'created_by': 'b030ff9c-41eb-49b4-afdf-2ccbecb6be5d', 'created_at': '2023-09-27T15:17:23.923705+00:00', 'models': [{'name': 'v5s6', 'versions': 2, 'owner_id': '""', 'last_update_time': datetime.datetime(2023, 9, 28, 19, 23, 3, 425821, tzinfo=tzutc()), 'created_at': datetime.datetime(2023, 9, 28, 18, 55, 24, 860357, tzinfo=tzutc())}], 'pipelines': [{'name': 'cv-arm-edge', 'create_time': datetime.datetime(2023, 9, 27, 15, 17, 45, 593032, tzinfo=tzutc()), 'definition': '[]'}, {'name': 'retailimage', 'create_time': datetime.datetime(2023, 9, 28, 18, 55, 14, 954338, tzinfo=tzutc()), 'definition': '[]'}]}
{'name': 'workshop-workspace-john-cv-medical', 'id': 10, 'archived': False, 'created_by': 'b030ff9c-41eb-49b4-afdf-2ccbecb6be5d', 'created_at': '2023-09-28T20:22:21.506043+00:00', 'models': [{'name': 'mitochondria-detector', 'versions': 1, 'owner_id': '""', 'last_update_time': datetime.datetime(2023, 9, 28, 20, 25, 2, 359367, tzinfo=tzutc()), 'created_at': datetime.datetime(2023, 9, 28, 20, 25, 2, 359367, tzinfo=tzutc())}], 'pipelines': [{'name': 'cv-mitochondria', 'create_time': datetime.datetime(2023, 9, 28, 20, 25, 17, 379732, tzinfo=tzutc()), 'definition': '[]'}]}
Upload a Model
Now that we have our current workspace set, it’s time to start uploading some models.
We already have some ONNX models available in the folder ./models
. All three do the same thing: predict house price values based on some values. We have those values stored in the folder ./data
.
ML Models are uploaded to the current Wallaroo workspace the wallaroo.Client.upload_model
method. Wallaroo supports different model types, as well as Arbitrary Python and containerized models.
For full details, see Wallaroo SDK Essentials Guide: Model Uploads and Registrations.
Wallaroo supports ONNX models as part of the default runtime, so these will run in Wallaroo without additional configurations.
When uploading models, the following is needed:
- The name of the model. This needs to be unique across the workspace.
- The path to the ML model file. For example,
./models/xgb_model.onnx
. - The framework of the model. These are listed through the
wallaroo.framework.Framework
list. For these examples we will be usingwallaroo.framework.Framework.ONNX
to specify we are using ONNX models. - The input_schema and output_schema. For ONNX models, we can skip this. For non-native runtime models, that has to be specified in Apache Arrow schema format. See the Wallaroo SDK Essentials Guide: Model Uploads and Registrations for full details.
Here’s an example of uploading a model to a Wallaroo workspace and assigning it the name ‘house-price-prime’, with the Wallaroo client assigned to the variable wl
, then retrieving the model version from Wallaroo once the upload it complete:
house_price_model_version = wl.upload_model('house-price-prime',
'./models/xgb_model.onnx',
framework=wallaroo.framework.Framework.ONNX)
house_price_model_version
Name | house-price-prime |
---|---|
Version | cc3ba784-ffdf-4a0f-982a-9a8ac4db8ba9 |
File Name | xgb_model.onnx |
SHA | 31e92d6ccb27b041a324a7ac22cf95d9d6cc3aa7e8263a229f7c4aec4938657c |
Status | ready |
Image Path | None |
Updated At | 2023-22-Aug 19:55:26 |
We store this new version of a model to the variable house_price_model_version
. This is used for later processes involving pipeline deployments, generating assays, and so on.
In Wallaroo, you have the model, which is based on the name parameter. Each model has one or more versions.
IMPORTANT NOTE: Models in Wallaroo are organized by name. If a model is uploaded with the same name, it will create a new version of the model with the same name. For example, the following will create a model named house-price-prime
with two versions.
# set the model from the XGB model converted to ONNX
house_price_model_version = wl.upload_model('house-price-prime',
'./models/xgb_model.onnx',
framework=wallaroo.framework.Framework.ONNX)
print(house_price_model_version)
{'name': 'house-price-prime', 'version': '83d89260-9aac-41ea-b2b4-79aae48b5a65', 'file_name': 'xgb_model.onnx', 'image_path': None, 'last_update_time': datetime.datetime(2023, 8, 22, 19, 59, 2, 26718, tzinfo=tzutc())}
# create the new model version to the model converted from an RF model
house_price_model_version = wl.upload_model('house-price-prime',
'./models/rf_model.onnx',
framework=wallaroo.framework.Framework.ONNX)
print(house_price_model_version)
{'name': 'house-price-prime', 'version': 'c86fd309-7c28-4e95-9d3e-831fefa51a12', 'file_name': 'rf_model.onnx', 'image_path': None, 'last_update_time': datetime.datetime(2023, 8, 22, 19, 59, 3, 381581, tzinfo=tzutc())}
Notice that the model is the same - house-price-prime
- but the model version changes each time we do an upload_model
. This allows you to change the model version to a totally different flavor and framework if you desire.
Upload a Model Exercise
For this exercise, upload the model /models/mobilenet.pt.onnx
and assign it a name, with the framework =wallaroo.framework.Framework.ONNX
. This model has an additional configuration option that we’ll include in the code below: batch_config="single"
, indicating that the model expects just one inference row at a time.
Here’s an example of uploading the mobilenet model.
from wallaroo.framework import Framework
wl.upload_model('mitochondria-detector',
"./models/mitochondria_epochs_15.onnx",
framework=Framework.ONNX)
from wallaroo.framework import Framework
wl.upload_model('mitochondria-detector',
"../models/mitochondria_epochs_15.onnx",
framework=Framework.ONNX)
Name | mitochondria-detector |
Version | 11a34cce-0eaa-4cdd-8739-43bd18b67a04 |
File Name | mitochondria_epochs_15.onnx |
SHA | e80fcdaf563a183b0c32c027dcb3890a64e1764d6d7dcd29524cd270dd42e7bd |
Status | ready |
Image Path | None |
Updated At | 2023-29-Sep 17:40:06 |
Retrieve Model Version
Once a model is uploaded to Wallaroo, we can list the models in a workspace with the wallaroo.workspace.models()
method. This returns a List of all of the models and how many versions are associated with that model.
Here’s an example:
workspace.models()
[{'name': 'house-price-prime', 'versions': 3, 'owner_id': '""', 'last_update_time': datetime.datetime(2023, 8, 22, 19, 59, 3, 381581, tzinfo=tzutc()), 'created_at': datetime.datetime(2023, 8, 22, 19, 55, 26, 603685, tzinfo=tzutc())}]
We can retrieve the model by specifying the position on the list. In our case, we only have one model, so we can retrieve it to a model by specifying position 0. For example:
my_model = workspace.models()[0]
my_model
Name | house-price-prime |
---|---|
# of Versions | 3 |
Owner ID | "" |
Last Updated | 2023-08-22 19:59:03.381581+00:00 |
Created At | 2023-08-22 19:55:26.603685+00:00 |
And finally - we retrieve what we really need, the model version by using the wallaroo.model.version()
method, which returns the list of versions of the model:
my_model = workspace.models()[0]
my_model.versions()
[{'name': 'house-price-prime', 'version': 'cc3ba784-ffdf-4a0f-982a-9a8ac4db8ba9', 'file_name': 'xgb_model.onnx', 'image_path': None, 'last_update_time': datetime.datetime(2023, 8, 22, 19, 55, 26, 603685, tzinfo=tzutc())},
{'name': 'house-price-prime', 'version': '83d89260-9aac-41ea-b2b4-79aae48b5a65', 'file_name': 'xgb_model.onnx', 'image_path': None, 'last_update_time': datetime.datetime(2023, 8, 22, 19, 59, 2, 26718, tzinfo=tzutc())},
{'name': 'house-price-prime', 'version': 'c86fd309-7c28-4e95-9d3e-831fefa51a12', 'file_name': 'rf_model.onnx', 'image_path': None, 'last_update_time': datetime.datetime(2023, 8, 22, 19, 59, 3, 381581, tzinfo=tzutc())}]
As you can see - the most recent version is the last element in the list, or the [-1]
position. We can now retrieve the most recent version of the model version as follows:
my_model = workspace.models()[0]
my_model_version = my_model.versions()[-1]
my_model_version
Name | house-price-prime |
---|---|
Version | c86fd309-7c28-4e95-9d3e-831fefa51a12 |
File Name | rf_model.onnx |
SHA | e22a0831aafd9917f3cc87a15ed267797f80e2afa12ad7d8810ca58f173b8cc6 |
Status | ready |
Image Path | None |
Updated At | 2023-22-Aug 19:59:03 |
The model version is an important concept - that is what is added to a pipeline for deployments and other uses.
Retrieve Model Version Exercise
This exercise will have you retrieving the model you uploaded earlier. For example, if the Wallaroo client was stored as wl
, and the workspace saved to workspace
, then the command to get the current model version would be:
my_model = workspace.models()[0]
my_model_version = my_model.versions()[-1]
my_model_version
# blank space to retrieve the model version and store it
my_model = workspace.models()[0]
my_model_version = my_model.versions()[-1]
my_model_version
Name | mitochondria-detector |
Version | 11a34cce-0eaa-4cdd-8739-43bd18b67a04 |
File Name | mitochondria_epochs_15.onnx |
SHA | e80fcdaf563a183b0c32c027dcb3890a64e1764d6d7dcd29524cd270dd42e7bd |
Status | ready |
Image Path | None |
Updated At | 2023-29-Sep 17:40:06 |
Build a Pipeline
Pipelines are the method of taking submitting data and processing that data through the models. Each pipeline can have one or more steps that submit the data from the previous step to the next one. Information can be submitted to a pipeline as a file, or through the pipeline’s URL.
Each pipeline step is a model version. These can be ML models like we uploaded earlier, or they can be Python scripts that manipulate the data into a format needed for another model in the chain.
When an inference is performed, data is submitted to the pipeline. The pipeline then submits the data to the first step, receives the output, then transmits that data to the next step. When all steps are complete, the pipeline returns the final values to the requesting client.
For this workshop, we will focus one a very simple pipeline: one ML model.
Pipeline are created in the current workspace with the wallaroo.Client.build_pipeline(name)
command, where the name is unique to the workspace. For example, to create the pipeline named houseprice-estimator
and the Wallaroo client is saved to wl
the command would be:
pipeline = wl.build_pipeline('houseprice-estimator')
pipeline
name | houseprice-estimator
created | 2023-08-22 20:39:35.853683+00:00
last_updated | 2023-08-22 20:39:35.853683+00:00
deployed | (none)
tags |
versions | f42c0457-e4f3-4370-b152-0a220347de11
steps |
Just like models, pipelines have version. Each time pipeline steps are changed, a new version is created.
See Wallaroo SDK Essentials Guide: Pipeline Management for full details.
Build a Pipeline
Build your own pipeline! Use the wallaroo.Client.build_pipeline(name)
command and create a pipeline named houseprice-pipeline
. Recall that this creates the pipeline in the current workspace, so verify that the current workspace is the one you want to create a pipeline in.
For example, if the Wallaroo client is saved to the variable wl
, the command would be:
wl.build_pipeline('cv-retail')
# blank space for you to create the pipeline
wl.build_pipeline('cv-mitochondria')
name | cv-mitochondria |
---|---|
created | 2023-09-28 20:25:17.379732+00:00 |
last_updated | 2023-09-29 17:40:15.164148+00:00 |
deployed | False |
tags | |
versions | cf4bdfb4-1eec-46f8-9af4-b16dea894de6, 507cb1eb-8034-4b5b-bc96-2427730a6407, 50ed1d2f-6dba-411c-9579-1090791b33bd, 6208c9cf-fcd6-4b20-bbfc-f6ce714596e3 |
steps | mitochondria-detector |
published | False |
Retrieve a Pipeline
Pipelines that are associated with a workspace are retrieved wallaroo.Client.list_pipelines
method - this returns a List of pipelines. From there, a variable is assigned to the pipeline in the list we want to work with.
The list_pipelines
method returns a list of pipelines and their details as follows:
wl.list_pipelines()
name | created | last_updated | deployed | tags | versions | steps |
---|---|---|---|---|---|---|
houseprice-estimator | 2023-22-Aug 20:39:35 | 2023-22-Aug 20:39:35 | (unknown) | f42c0457-e4f3-4370-b152-0a220347de11 | ||
biolabspipeline | 2023-22-Aug 16:07:20 | 2023-22-Aug 16:24:40 | False | 4c6dceb7-e692-4b8b-b615-4f7873eb020b, 59d0babe-bc1d-4dbb-959f-711c74f7b05d, ae834c0d-7a5b-4f87-9e2e-1f06f3cd25e7, 7c438222-28d8-4fca-9a70-eabee8a0fac5 | biolabsmodel | |
biolabspipeline | 2023-22-Aug 16:03:33 | 2023-22-Aug 16:03:38 | False | 4e103a7d-cd4d-464b-b182-61d4041518a8, ec2a0fd6-21d4-4843-b7c3-65b1e5be1b85, 516f3848-be98-40d7-8564-a1e48eecb7a8 | biolabsmodel | |
biolabspipelinegomj | 2023-22-Aug 15:11:12 | 2023-22-Aug 15:42:44 | False | 1dc9f89f-82aa-4a71-b21a-75dc8d5e4e51, 152d12f2-1200-46ad-ad04-60078c5aa284, 6ca59ffd-802e-4ad5-bd9a-35146b9fbda5, bdab08cc-3e99-4afc-b22d-657e33b76f29, 3c8feb0d-3124-4018-8dfa-06162156d51e | biolabsmodelgomj | |
edge-pipeline | 2023-21-Aug 20:54:37 | 2023-22-Aug 19:06:46 | False | 2be013d9-a438-453c-a013-3fd8e6218394, a02b6af5-4235-42af-92c6-5ae678b35be4, e721ccad-11d8-4874-8388-4211c4957d18, d642e766-cffb-451f-b197-e058bedbdd5f, eb586aba-4908-4bff-84e1-bdeb1fa4b7d3, 2163d718-a5ea-41e3-b69f-095efa858462 | ccfraud | |
p1 | 2023-21-Aug 19:38:44 | 2023-21-Aug 19:38:44 | (unknown) | 5f93e90a-e8d6-4e8a-8a1a-22eee80a3e13, 5f78247f-7bf9-445b-98a6-e146fb22b8e9 |
Just like with a model version, we can set a variable to the pipeline by assigning it from its position in the list. In my case, if we want to retrieve the pipeline houseprice-estimator
at position 0, we do so as follows:
this_pipeline = wl.list_pipelines()[0]
this_pipeline
Retrieve a Pipeline Exercise
For this exercise, retrieve your the pipeline you built in the previous step and store it into the variable my_pipeline
. You’ll need to use list_pipelines()
to get the list, then see where the position of the pipeline is in the list and assign it to the variable. Here’s an example if the Wallaroo client is stored in the variable wl
and our pipeline is at position 0 of the list_pipelines
list, the command would be:
my_pipeline = wl.list_pipelines()[0]
my_pipeline
# empty space to retrieve your pipeline
my_pipeline = wl.list_pipelines()[0]
my_pipeline
name | cv-mitochondria |
---|---|
created | 2023-09-28 20:25:17.379732+00:00 |
last_updated | 2023-09-29 17:40:15.164148+00:00 |
deployed | False |
tags | |
versions | cf4bdfb4-1eec-46f8-9af4-b16dea894de6, 507cb1eb-8034-4b5b-bc96-2427730a6407, 50ed1d2f-6dba-411c-9579-1090791b33bd, 6208c9cf-fcd6-4b20-bbfc-f6ce714596e3 |
steps | mitochondria-detector |
published | False |
Add Model Step
Models are added to a pipeline as pipeline steps. There are different kinds of pipeline steps that can host one or more models.
For this workshop, we will use the method wallaroo.pipeline.add_model_step(model_version)
. This adds a single step to a Pipeline. Pipeline steps start at 0 and increment from there. We can see the steps in our pipeline with the wallaroo.pipeline.steps()
method.
# add modelA as a pipeline steps
pipeline.add_model_step(modelA)
# display the steps
pipeline.steps()
[{'ModelInference': {'models': [{'name': 'house-price-prime', 'version': 'c86fd309-7c28-4e95-9d3e-831fefa51a12', 'sha': 'e22a0831aafd9917f3cc87a15ed267797f80e2afa12ad7d8810ca58f173b8cc6'}]}}]
For now, there’s three commands you should know:
wallaroo.pipelineadd_model_step(model_version)
: Add a step to a pipeline from a model version.wallaroo.pipeline.steps()
: Display the current steps in the pipeline.wallaroo.pipeline.clear()
: Clear all pipeline steps.
Pipeline steps are not saved in the Wallaroo instance until the pipeline has been deployed - more on that shortly. So you can add steps, clear them, add new ones - they all stay in your local script until you issue the command to deploy the pipeline. More on that later.
The second thing to watch out for it every time add_model_step
is performed on a pipeline, another step is created. For example, if we have a pipeline with two models, modelA
and modelB
, then the following creates two steps in the same pipelines:
# clear the steps
pipeline.clear()
# add modelA then modelB as pipeline steps
pipeline.add_model_step(modelA)
pipeline.add_model_step(modelB)
# display the steps
pipeline.steps()
[{'ModelInference': {'models': [{'name': 'house-price-prime', 'version': 'c86fd309-7c28-4e95-9d3e-831fefa51a12', 'sha': 'e22a0831aafd9917f3cc87a15ed267797f80e2afa12ad7d8810ca58f173b8cc6'}]}},
{'ModelInference': {'models': [{'name': 'house-price-gbr', 'version': '248a6eab-b159-4821-830b-22cc137a1ace', 'sha': 'ed6065a79d841f7e96307bb20d5ef22840f15da0b587efb51425c7ad60589d6a'}]}}]
This means the data from the inference is fed first to modelA
, and that output is fed into modelB
. The problem is if the input data from modelA
doesn’t match what modelB
expects, the process will fail.
Because of this, check your pipeline steps before you deploy a pipeline, clear them if you need to.
Add Pipeline Step Exercise
We have our model version uploaded from the previous steps, and we have our pipeline. Time to put them together and create a pipeline step with our model version.
Just for practice, do the following:
- Clear the pipeline steps.
- Add the sample model uploaded earlier. In our examples, that was
my_model_version
. - Show the current pipeline steps.
Here’s an example with the Wallaroo client stored to wl
, with the pipeline my_pipeline
and my_model_version
:
my_pipeline.clear()
my_pipeline.add_model_step(my_model_version)
my_pipeline.steps()
my_pipeline.clear()
my_pipeline.add_model_step(my_model_version)
my_pipeline.steps()
[{'ModelInference': {'models': [{'name': 'mitochondria-detector', 'version': '11a34cce-0eaa-4cdd-8739-43bd18b67a04', 'sha': 'e80fcdaf563a183b0c32c027dcb3890a64e1764d6d7dcd29524cd270dd42e7bd'}]}}]
Deploy a Pipeline
Now we reach what we’ve been aiming for: deploying a pipeline.
By now, you’ve seen how workspaces contain the models, pipelines, and other artifacts. You’ve uploaded a model and retrieved the latest version of the model. You’ve built a pipeline and added the model version as a pipeline step.
Now we will deploy the pipeline. Deploying a pipeline allocated resources from the cluster to that pipeline for it’s use. The amount of resources has a default value of 4 CPUs, but for this workshop we’ll be adjusting that to just 0.5 cpus and 1 GB RAM per pipeline.
- IMPORTANT NOTE: Please stick to these resource configurations when using a Wallaroo instance with other users. Otherwise, pipelines might not deploy if the available resources are used up.
To deploy a pipeline, we do two things:
- Create a deployment configuration: This is an optional step, but we will make it mandatory for this workshop to allow other users to work in the same Wallaroo instance without running out of resources.
- Deploy the pipeline with the deployment configuration: This is the active step that saves the pipeline steps, and allocates system resources to the pipeline for performing inferences.
Deployment configurations are made with the wallaroo.DeploymentConfigBuilder()
class, and then we assign the resource settings from there. This is saved to a variable so we can apply it to our pipeline deployment.
Here’s an example of setting up a deployment with just 0.5 cpu and 1Gi RAM:
deploy_config = wallaroo.DeploymentConfigBuilder().replica_count(1).cpus(0.5).memory("1Gi").build()
Notice the replica_count(1)
configuration - this tells Wallaroo to only spin up one replica for this pipeline. In a production environment, we could spin multiple replicas either manually or automatically as more resources are needed to improve performance.
Now we deploy the pipeline with our deployment configuration with the wallaroo.pipeline.deploy(deploy_configuration)
method. If our pipeline variable is my_pipeline
, then we would deploy it as follows:
deploy_config = wallaroo.DeploymentConfigBuilder().replica_count(1).cpus(0.5).memory("1Gi").build()
my_pipeline.deploy(deployment_config=deploy_config)
We can check the status of the pipeline deployment with the wallaroo.pipeline.status()
method:
my_pipeline.status()
{'status': 'Running',
'details': [],
'engines': [{'ip': '10.244.3.83',
'name': 'engine-6d4fccf5cb-dmwfl',
'status': 'Running',
'reason': None,
'details': [],
'pipeline_statuses': {'pipelines': [{'id': 'houseprice-estimator',
'status': 'Running'}]},
'model_statuses': {'models': [{'name': 'house-price-prime',
'version': 'c86fd309-7c28-4e95-9d3e-831fefa51a12',
'sha': 'e22a0831aafd9917f3cc87a15ed267797f80e2afa12ad7d8810ca58f173b8cc6',
'status': 'Running'}]}}],
'engine_lbs': [{'ip': '10.244.4.100',
'name': 'engine-lb-584f54c899-sswnw',
'status': 'Running',
'reason': None,
'details': []}],
'sidekicks': []}
Deploy a Pipeline Exercise
This exercise will have you deploy your pipeline with the deployment settings we listed above. For example, if your pipeline was called my_pipeline
, then your deployment will look like this:
deploy_config = wallaroo.DeploymentConfigBuilder().replica_count(1).cpus(0.5).memory("1Gi").build()
my_pipeline.deploy(deployment_config=deploy_config)
deploy_config = wallaroo.DeploymentConfigBuilder().replica_count(1).cpus(0.5).memory("1Gi").build()
my_pipeline.deploy(deployment_config=deploy_config)
my_pipeline.status()
{'status': 'Running',
'details': [],
'engines': [{'ip': '10.244.3.236',
'name': 'engine-669fcff449-w5j62',
'status': 'Running',
'reason': None,
'details': [],
'pipeline_statuses': {'pipelines': [{'id': 'cv-mitochondria',
'status': 'Running'}]},
'model_statuses': {'models': [{'name': 'mitochondria-detector',
'version': '11a34cce-0eaa-4cdd-8739-43bd18b67a04',
'sha': 'e80fcdaf563a183b0c32c027dcb3890a64e1764d6d7dcd29524cd270dd42e7bd',
'status': 'Running'}]}}],
'engine_lbs': [{'ip': '10.244.4.12',
'name': 'engine-lb-584f54c899-d4z4h',
'status': 'Running',
'reason': None,
'details': []}],
'sidekicks': []}
Pipeline Inference with DataFrames
Wallaroo deployed pipelines accept three types of data:
- pandas DataFrames
- Apache Arrow
We do this with one of two commands on a deployed pipeline.
wallaroo.pipeline.infer(input)
: Submits either JSON, a DataFrame, or Apache Arrow to the pipeline for inferences.wallaroo.pipeline.infer_from_file(path)
: Submits either a JSON, a DataFrame in pandas Record format, or an Apache Arrow binary file inferences.
We’ll start with a single DataFrame stored in the file: ./data/singleton.df.json
, which contains input data as a tensor
:
[
{
"tensor": [
4.0,
3.0,
3710.0,
20000.0,
2.0,
0.0,
2.0,
5.0,
10.0,
2760.0,
950.0,
47.6696014404,
-122.2610015869,
3970.0,
20000.0,
79.0,
0.0,
0.0
]
}
]
This translates into a pandas DataFrame, which is submitted with the pipeline.infer
method.
df = pd.DataFrame([
{
"tensor": [
4.0,
3.0,
3710.0,
20000.0,
2.0,
0.0,
2.0,
5.0,
10.0,
2760.0,
950.0,
47.6696014404,
-122.2610015869,
3970.0,
20000.0,
79.0,
0.0,
0.0
]
}
])
pipeline.infer(df)
When we use infer
, Wallaroo determines whether the object is a pandas DataFrame or Apache arrow, and handles the inferences accordingly.
The data received through the SDK is always of the same type submitted: Submit a DataFrame, get a DataFrame with the data back. Submit an Arrow table file, get an Arrow table back. Here’s an example of submitting our sample DataFrame through a pipeline saved to the variable pipeline
:
result = pipeline.infer(df)
display(result)
time | in.tensor | out.variable | check_failures | |
---|---|---|---|---|
0 | 2023-08-23 15:02:41.452 | [4.0, 3.0, 3710.0, 20000.0, 2.0, 0.0, 2.0, 5.0, 10.0, 2760.0, 950.0, 47.6696014404, -122.2610015869, 3970.0, 20000.0, 79.0, 0.0, 0.0] | [1514079.4] | 0 |
Let’s break down each of these fields:
- Index (unnamed): This field doesn’t have a label, but in the example above this is the
index
. We only have one submission, so we have one result. If we had 20 inputs, we’d have 20 inference results, and each result would be aligned with each row we sent as an input. - time: The date and time the inference request.
- in.{variable}: Every input to the inference request is listed as
in.{variable_name}
. For example, if our inputs werehouse_size_in_square_feet
andyear_house_built
, then the inputs would be listed asin.house_size_in_square_feet
andin.year_house_built
. - out.{variable}: Every output to the inference is listed as
out.{variable_name}
. Our sample model outputs just one output:variable
, but others such might outputestimated_house_price
,initial_offer_price
, and in the inference result those would be listed asout.estimated_house_price
andout.initial_offer_price
. - check_failures: Indicates if any validation checks failed. This is covered in later sessions.
There’s additional data that an inference has that is retrieved by requesting it. For full details, see the Wallaroo SDK Essentials Guide: Inference Management.
Pipeline Inference with Files Exercise
The computer vision models accepts tensor inputs. We have converted the file ./patches/ms-01-atl-3-22-23_9-50/images/image_0_21.tif"
to a DataFrame. See the helper file TiffImageUtils.py
for the loadImageAndConvertTiff
that converts TIFF images to DataFrames. This DataFrame of the image is stored to ./data/image_0_21.tif.df.json
. We can perform a sample inferece using the wallaroo.pipeline.infer_from_file
method like so:
my_pipeline.infer_from_file('./data/image_0_21.tif.df.json')
my_pipeline.infer_from_file('../data/image_0_21.tif.df.json')
time | in.tensor | out.conv2d_37 | check_failures | |
---|---|---|---|---|
0 | 2023-09-29 18:13:48.458 | [0.0572924651, 0.0555159546, 0.0621778691, 0.0... | [0.073827654, 0.04537511, 0.022077948, 0.02752... | 0 |
deploy_url = my_pipeline._deployment._url()
headers = wl.auth.auth_header()
headers['Content-Type']='application/json; format=pandas-records'
headers['Accept']='application/json; format=pandas-records'
dataFile = '../data/image_0_21.tif.df.json'
# test_df = pd.read_json('../data/image_0_21.tif.df.json')
# test_df.to_json('../data/image_0_21.tif.df.json', orient="records")
# display(test_df)
print(f'''
!curl -X POST {deploy_url} \\
-H "Authorization:{headers['Authorization']}" \\
-H "Content-Type:{headers['Content-Type']}" \\
-H "Accept:{headers['Accept']}" \\
--data @{dataFile} > curl_response.df
''')
!curl -X POST https://doc-test.api.wallarooexample.ai/v1/api/pipelines/infer/cv-mitochondria-25/cv-mitochondria \
-H "Authorization:Bearer eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJURV9fTFgzRTZHZVdkRlBXaXVwaXN0OU0ySHR3dFVZbzk2NkJCUG1CUjJvIn0.eyJleHAiOjE2OTYwMTEyNjYsImlhdCI6MTY5NjAxMTIwNiwiYXV0aF90aW1lIjoxNjk2MDA5MDA4LCJqdGkiOiI5MGQwNWZjNC1iYTc3LTQ5ZjItYWY4Yi03ZDQ3ZDQyZDFiZGUiLCJpc3MiOiJodHRwczovL2RvYy10ZXN0LmtleWNsb2FrLndhbGxhcm9vY29tbXVuaXR5Lm5pbmphL2F1dGgvcmVhbG1zL21hc3RlciIsImF1ZCI6WyJtYXN0ZXItcmVhbG0iLCJhY2NvdW50Il0sInN1YiI6ImIwMzBmZjljLTQxZWItNDliNC1hZmRmLTJjY2JlY2I2YmU1ZCIsInR5cCI6IkJlYXJlciIsImF6cCI6InNkay1jbGllbnQiLCJzZXNzaW9uX3N0YXRlIjoiMTBlYTU4MGEtYTQyOS00YTkwLThmMzUtNTkwMmZmNGViMzQzIiwiYWNyIjoiMSIsInJlYWxtX2FjY2VzcyI6eyJyb2xlcyI6WyJkZWZhdWx0LXJvbGVzLW1hc3RlciIsIm9mZmxpbmVfYWNjZXNzIiwidW1hX2F1dGhvcml6YXRpb24iXX0sInJlc291cmNlX2FjY2VzcyI6eyJtYXN0ZXItcmVhbG0iOnsicm9sZXMiOlsibWFuYWdlLXVzZXJzIiwidmlldy11c2VycyIsInF1ZXJ5LWdyb3VwcyIsInF1ZXJ5LXVzZXJzIl19LCJhY2NvdW50Ijp7InJvbGVzIjpbIm1hbmFnZS1hY2NvdW50IiwibWFuYWdlLWFjY291bnQtbGlua3MiLCJ2aWV3LXByb2ZpbGUiXX19LCJzY29wZSI6InByb2ZpbGUgZW1haWwiLCJzaWQiOiIxMGVhNTgwYS1hNDI5LTRhOTAtOGYzNS01OTAyZmY0ZWIzNDMiLCJlbWFpbF92ZXJpZmllZCI6ZmFsc2UsImh0dHBzOi8vaGFzdXJhLmlvL2p3dC9jbGFpbXMiOnsieC1oYXN1cmEtdXNlci1pZCI6ImIwMzBmZjljLTQxZWItNDliNC1hZmRmLTJjY2JlY2I2YmU1ZCIsIngtaGFzdXJhLWRlZmF1bHQtcm9sZSI6InVzZXIiLCJ4LWhhc3VyYS1hbGxvd2VkLXJvbGVzIjpbInVzZXIiXSwieC1oYXN1cmEtdXNlci1ncm91cHMiOiJ7fSJ9LCJuYW1lIjoiSm9obiBIYW5zYXJpY2siLCJwcmVmZXJyZWRfdXNlcm5hbWUiOiJqb2huLmh1bW1lbEB3YWxsYXJvby5haSIsImdpdmVuX25hbWUiOiJKb2huIiwiZmFtaWx5X25hbWUiOiJIYW5zYXJpY2siLCJlbWFpbCI6ImpvaG4uaHVtbWVsQHdhbGxhcm9vLmFpIn0.K9oClU6gK6hWf1P__HYKr2GWn08T08h16Y2nA9fccp6MDKZzOb2RW5tdBJ9AtzkmMeChBG_TG-BLLsnZRwjpi_0V7vphMgxkp88p1QhbxIdgzQA5enGDDR_183DC0JC9ZGiRTTk3jCp9qoIgygN67959gONhGanH19S1SsOdNWzNltXcdE8lzclwutyKeR4bXMplgKXZP1VxGuM1SBN0SfmC1MPCR50wn8vI_bko011OyCJhd48OUUFuRvdjFjLqcD6J1Wqtc8mYvkphYBwuc2jmXwpW3vyIEQKs7NMATHXFH925awloE27XCT8sKLUvRGFN_MxVAuOPuiL9hJrU3w" \
-H "Content-Type:application/json; format=pandas-records" \
-H "Accept:application/json; format=pandas-records" \
--data @../data/image_0_21.tif.df.json > curl_response.df
!curl -X POST https://doc-test.api.wallarooexample.ai/v1/api/pipelines/infer/cv-mitochondria-25/cv-mitochondria \
-H "Authorization:Bearer eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJURV9fTFgzRTZHZVdkRlBXaXVwaXN0OU0ySHR3dFVZbzk2NkJCUG1CUjJvIn0.eyJleHAiOjE2OTYwMTEyNjYsImlhdCI6MTY5NjAxMTIwNiwiYXV0aF90aW1lIjoxNjk2MDA5MDA4LCJqdGkiOiI5MGQwNWZjNC1iYTc3LTQ5ZjItYWY4Yi03ZDQ3ZDQyZDFiZGUiLCJpc3MiOiJodHRwczovL2RvYy10ZXN0LmtleWNsb2FrLndhbGxhcm9vY29tbXVuaXR5Lm5pbmphL2F1dGgvcmVhbG1zL21hc3RlciIsImF1ZCI6WyJtYXN0ZXItcmVhbG0iLCJhY2NvdW50Il0sInN1YiI6ImIwMzBmZjljLTQxZWItNDliNC1hZmRmLTJjY2JlY2I2YmU1ZCIsInR5cCI6IkJlYXJlciIsImF6cCI6InNkay1jbGllbnQiLCJzZXNzaW9uX3N0YXRlIjoiMTBlYTU4MGEtYTQyOS00YTkwLThmMzUtNTkwMmZmNGViMzQzIiwiYWNyIjoiMSIsInJlYWxtX2FjY2VzcyI6eyJyb2xlcyI6WyJkZWZhdWx0LXJvbGVzLW1hc3RlciIsIm9mZmxpbmVfYWNjZXNzIiwidW1hX2F1dGhvcml6YXRpb24iXX0sInJlc291cmNlX2FjY2VzcyI6eyJtYXN0ZXItcmVhbG0iOnsicm9sZXMiOlsibWFuYWdlLXVzZXJzIiwidmlldy11c2VycyIsInF1ZXJ5LWdyb3VwcyIsInF1ZXJ5LXVzZXJzIl19LCJhY2NvdW50Ijp7InJvbGVzIjpbIm1hbmFnZS1hY2NvdW50IiwibWFuYWdlLWFjY291bnQtbGlua3MiLCJ2aWV3LXByb2ZpbGUiXX19LCJzY29wZSI6InByb2ZpbGUgZW1haWwiLCJzaWQiOiIxMGVhNTgwYS1hNDI5LTRhOTAtOGYzNS01OTAyZmY0ZWIzNDMiLCJlbWFpbF92ZXJpZmllZCI6ZmFsc2UsImh0dHBzOi8vaGFzdXJhLmlvL2p3dC9jbGFpbXMiOnsieC1oYXN1cmEtdXNlci1pZCI6ImIwMzBmZjljLTQxZWItNDliNC1hZmRmLTJjY2JlY2I2YmU1ZCIsIngtaGFzdXJhLWRlZmF1bHQtcm9sZSI6InVzZXIiLCJ4LWhhc3VyYS1hbGxvd2VkLXJvbGVzIjpbInVzZXIiXSwieC1oYXN1cmEtdXNlci1ncm91cHMiOiJ7fSJ9LCJuYW1lIjoiSm9obiBIYW5zYXJpY2siLCJwcmVmZXJyZWRfdXNlcm5hbWUiOiJqb2huLmh1bW1lbEB3YWxsYXJvby5haSIsImdpdmVuX25hbWUiOiJKb2huIiwiZmFtaWx5X25hbWUiOiJIYW5zYXJpY2siLCJlbWFpbCI6ImpvaG4uaHVtbWVsQHdhbGxhcm9vLmFpIn0.K9oClU6gK6hWf1P__HYKr2GWn08T08h16Y2nA9fccp6MDKZzOb2RW5tdBJ9AtzkmMeChBG_TG-BLLsnZRwjpi_0V7vphMgxkp88p1QhbxIdgzQA5enGDDR_183DC0JC9ZGiRTTk3jCp9qoIgygN67959gONhGanH19S1SsOdNWzNltXcdE8lzclwutyKeR4bXMplgKXZP1VxGuM1SBN0SfmC1MPCR50wn8vI_bko011OyCJhd48OUUFuRvdjFjLqcD6J1Wqtc8mYvkphYBwuc2jmXwpW3vyIEQKs7NMATHXFH925awloE27XCT8sKLUvRGFN_MxVAuOPuiL9hJrU3w" \
-H "Content-Type:application/json; format=pandas-records" \
-H "Accept:application/json; format=pandas-records" \
--data @../data/image_0_21.tif.df.json > curl_response.df
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 2338k 100 1513k 100 824k 1049k 571k 0:00:01 0:00:01 --:--:-- 1624k--:--:-- --:--:-- --:--:-- 917k
Undeploying Your Pipeline
You should always undeploy your pipelines when you are done with them, or don’t need them for a while. This releases the resources that the pipeline is using for other processes to use. You can always redeploy the pipeline when you need it again. As a reminder, here are the commands to deploy and undeploy a pipeline:
# when the pipeline is deployed, it's ready to receive data and infer
pipeline.deploy()
# "turn off" the pipeline and releaase its resources
pipeline.undeploy()
If you are continuing on to the next notebook now, you can leave the pipeline deployed to keep working; but if you are taking a break, then you should undeploy.
## blank space to undeploy the pipeline, if needed
my_pipeline.undeploy()
name | cv-mitochondria |
---|---|
created | 2023-09-28 20:25:17.379732+00:00 |
last_updated | 2023-09-29 18:08:12.573990+00:00 |
deployed | False |
tags | |
versions | 4b6dab7d-b3ff-4f14-8425-7d9b6de76637, 66e72bc0-a3e3-4872-bc20-19b992c194b4, cf4bdfb4-1eec-46f8-9af4-b16dea894de6, 507cb1eb-8034-4b5b-bc96-2427730a6407, 50ed1d2f-6dba-411c-9579-1090791b33bd, 6208c9cf-fcd6-4b20-bbfc-f6ce714596e3 |
steps | mitochondria-detector |
published | False |
Congratulations!
You have now
- Created a workspace and set it as the current workspace.
- Uploaded an ONNX model.
- Created a Wallaroo pipeline, and set the most recent version of the uploaded model as a pipeline step.
- Successfully send data to your pipeline for inference through the SDK and through an API call.