This can be downloaded as part of the Wallaroo Tutorials repository.
This tutorial demonstrates deploying multiple pipelines using Wallaroo ML Workload Orchestrations. For this use case, we look at a situation where multiple models are deployed, each which inference data from different regions.
The tutorial will demonstrate the following:
./models/gbr_model.onnx
./models/rf_model.onnx
./models/xgb_model.onnx
./data/zonea.df.json
./data/zoneb.df.json
./data/zonec.df.json
Running
.For this tutorial, we’ll create a workspace, upload our sample model and deploy a pipeline. We’ll perform some quick sample inferences to verify that everything it working.
Here we’ll import the various libraries we’ll use for the tutorial.
import wallaroo
from wallaroo.object import EntityNotFoundError, RequiredAttributeMissing
# to display dataframe tables
from IPython.display import display
# used to display dataframe information without truncating
import pandas as pd
pd.set_option('display.max_colwidth', None)
pd.set_option('display.max_columns', None)
The first step is to connect to Wallaroo through the Wallaroo client. The Python library is included in the Wallaroo install and available through the Jupyter Hub interface provided with your Wallaroo environment.
This is accomplished using the wallaroo.Client()
command, which provides a URL to grant the SDK permission to your specific Wallaroo environment. When displayed, enter the URL into a browser and confirm permissions. Store the connection into a variable that can be referenced later.
If logging into the Wallaroo instance through the internal JupyterHub service, use wl = wallaroo.Client()
. For more information on Wallaroo Client settings, see the Client Connection guide.
# Login through local Wallaroo instance
wl = wallaroo.Client()
# Setting variables for later steps
workspace_name = f'multiple_pipeline_deployment_tutorial'
model_name_a = 'house-price-model-a'
model_name_b = 'house-price-model-b'
model_name_c = 'house-price-model-c'
model_file_name_a = './models/gbr_model.onnx'
model_file_name_b = './models/rf_model.onnx'
model_file_name_c = './models/xgb_model.onnx'
pipeline_name_a = 'house-price-zone-a'
pipeline_name_b = 'house-price-zone-b'
pipeline_name_c = 'house-price-zone-c'
We’ll now create our workspace and pipeline for the tutorial. If this tutorial has been run previously, then this will retrieve the existing ones with the assumption they’re for us with this tutorial.
We’ll set the retrieved workspace as the current workspace in the SDK, so all commands will default to that workspace.
workspace = wl.get_workspace(name=workspace_name, create_if_not_exist=True)
wl.set_current_workspace(workspace)
{'name': 'multiple_pipeline_deployment_tutorial', 'id': 9, 'archived': False, 'created_by': 'c6fd6e38-db8c-4788-b6b1-dc882fd60e14', 'created_at': '2024-09-17T15:22:50.837949+00:00', 'models': [], 'pipelines': []}
We’ll upload our model into our sample workspace, then add it as a pipeline step before deploying the pipeline to it’s ready to accept inference requests.
# Upload the models
housing_model_a = (wl.upload_model(model_name_a,
model_file_name_a,
framework=wallaroo.framework.Framework.ONNX)
.configure(tensor_fields=["tensor"])
)
housing_model_b = (wl.upload_model(model_name_b,
model_file_name_b,
framework=wallaroo.framework.Framework.ONNX)
.configure(tensor_fields=["tensor"])
)
housing_model_c = (wl.upload_model(model_name_c,
model_file_name_c,
framework=wallaroo.framework.Framework.ONNX)
.configure(tensor_fields=["tensor"])
)
We will now create the pipelines, and add the the models as pipeline steps.
pipeline_a = wl.build_pipeline(pipeline_name_a)
pipeline_b = wl.build_pipeline(pipeline_name_b)
pipeline_c = wl.build_pipeline(pipeline_name_c)
pipeline_a.add_model_step(housing_model_a)
pipeline_b.add_model_step(housing_model_b)
pipeline_c.add_model_step(housing_model_c)
name | house-price-zone-c |
---|---|
created | 2024-09-17 15:22:52.701839+00:00 |
last_updated | 2024-09-17 15:22:52.701839+00:00 |
deployed | (none) |
workspace_id | 9 |
workspace_name | multiple_pipeline_deployment_tutorial |
arch | None |
accel | None |
tags | |
versions | d853c867-4b93-4327-80b1-95c81e0244bc |
steps | |
published | False |
The following code simulates how the ML Orchestration will behave:
wait_for_status
parameter, which skips waiting for the pipeline to complete deployment before moving to the next step.# retrieve the pipelines
pipeline_a = wl.get_pipeline(pipeline_name_a)
pipeline_b = wl.get_pipeline(pipeline_name_b)
pipeline_c = wl.get_pipeline(pipeline_name_c)
# deploy the pipelines without waiting for the pipeline status
deploy_config = wallaroo.DeploymentConfigBuilder().replica_count(1).cpus(0.5).memory("1Gi").build()
pipeline_a.deploy(deployment_config=deploy_config, wait_for_status=False)
pipeline_b.deploy(deployment_config=deploy_config, wait_for_status=False)
pipeline_c.deploy(deployment_config=deploy_config, wait_for_status=False)
Deployment initiated for house-price-zone-a. Please check pipeline status.
Deployment initiated for house-price-zone-b. Please check pipeline status.
Deployment initiated for house-price-zone-c. Please check pipeline status.
# verify the pipeline is deployed
import time
while pipeline_a.status()['status'] != 'Running':
time.sleep(15)
print("Waiting for deployment.")
pipeline_a.status()
Waiting for deployment.
{‘status’: ‘Running’,
‘details’: [],
’engines’: [{‘ip’: ‘10.28.2.12’,
’name’: ’engine-5cf9547bf-bbnvb’,
‘status’: ‘Running’,
‘reason’: None,
‘details’: [],
‘pipeline_statuses’: {‘pipelines’: [{‘id’: ‘house-price-zone-a’,
‘status’: ‘Running’,
‘version’: ‘30e46049-6488-4ed5-b3fd-34764471e0b0’}]},
‘model_statuses’: {‘models’: [{‘model_version_id’: 1,
’name’: ‘house-price-model-a’,
‘sha’: ’ed6065a79d841f7e96307bb20d5ef22840f15da0b587efb51425c7ad60589d6a’,
‘status’: ‘Running’,
‘version’: ‘ade98e8e-1006-4931-8c7c-6b9e6efbc06b’}]}}],
’engine_lbs’: [{‘ip’: ‘10.28.2.11’,
’name’: ’engine-lb-6b59985857-57kl7’,
‘status’: ‘Running’,
‘reason’: None,
‘details’: []}],
‘sidekicks’: []}
# perform inference on the deployed pipelines
display(pipeline_a.infer_from_file('./data/zonea.df.json').head(20))
display(pipeline_b.infer_from_file('./data/zoneb.df.json').head(20))
display(pipeline_c.infer_from_file('./data/zonec.df.json').head(20))
time | in.tensor | out.variable | anomaly.count | |
---|---|---|---|---|
0 | 2024-09-17 15:43:49.624 | [4.0, 2.5, 2900.0, 5505.0, 2.0, 0.0, 0.0, 3.0, 8.0, 2900.0, 0.0, 47.6063, -122.02, 2970.0, 5251.0, 12.0, 0.0, 0.0] | [704901.9] | 0 |
1 | 2024-09-17 15:43:49.624 | [2.0, 2.5, 2170.0, 6361.0, 1.0, 0.0, 2.0, 3.0, 8.0, 2170.0, 0.0, 47.7109, -122.017, 2310.0, 7419.0, 6.0, 0.0, 0.0] | [695994.44] | 0 |
2 | 2024-09-17 15:43:49.624 | [3.0, 2.5, 1300.0, 812.0, 2.0, 0.0, 0.0, 3.0, 8.0, 880.0, 420.0, 47.5893, -122.317, 1300.0, 824.0, 6.0, 0.0, 0.0] | [416164.8] | 0 |
3 | 2024-09-17 15:43:49.624 | [4.0, 2.5, 2500.0, 8540.0, 2.0, 0.0, 0.0, 3.0, 9.0, 2500.0, 0.0, 47.5759, -121.994, 2560.0, 8475.0, 24.0, 0.0, 0.0] | [655277.2] | 0 |
4 | 2024-09-17 15:43:49.624 | [3.0, 1.75, 2200.0, 11520.0, 1.0, 0.0, 0.0, 4.0, 7.0, 2200.0, 0.0, 47.7659, -122.341, 1690.0, 8038.0, 62.0, 0.0, 0.0] | [426854.66] | 0 |
5 | 2024-09-17 15:43:49.624 | [3.0, 2.0, 2140.0, 4923.0, 1.0, 0.0, 0.0, 4.0, 8.0, 1070.0, 1070.0, 47.6902, -122.339, 1470.0, 4923.0, 86.0, 0.0, 0.0] | [632556.06] | 0 |
6 | 2024-09-17 15:43:49.624 | [4.0, 3.5, 3590.0, 5334.0, 2.0, 0.0, 2.0, 3.0, 9.0, 3140.0, 450.0, 47.6763, -122.267, 2100.0, 6250.0, 9.0, 0.0, 0.0] | [1100465.3] | 0 |
7 | 2024-09-17 15:43:49.624 | [3.0, 2.0, 1280.0, 960.0, 2.0, 0.0, 0.0, 3.0, 9.0, 1040.0, 240.0, 47.602, -122.311, 1280.0, 1173.0, 0.0, 0.0, 0.0] | [528278.06] | 0 |
8 | 2024-09-17 15:43:49.624 | [4.0, 2.5, 2820.0, 15000.0, 2.0, 0.0, 0.0, 4.0, 9.0, 2820.0, 0.0, 47.7255, -122.101, 2440.0, 15000.0, 29.0, 0.0, 0.0] | [659439.94] | 0 |
9 | 2024-09-17 15:43:49.624 | [3.0, 2.25, 1790.0, 11393.0, 1.0, 0.0, 0.0, 3.0, 8.0, 1790.0, 0.0, 47.6297, -122.099, 2290.0, 11894.0, 36.0, 0.0, 0.0] | [534331.44] | 0 |
10 | 2024-09-17 15:43:49.624 | [3.0, 1.5, 1010.0, 7683.0, 1.5, 0.0, 0.0, 5.0, 7.0, 1010.0, 0.0, 47.72, -122.318, 1550.0, 7271.0, 61.0, 0.0, 0.0] | [377187.2] | 0 |
11 | 2024-09-17 15:43:49.624 | [3.0, 2.0, 1270.0, 1323.0, 3.0, 0.0, 0.0, 3.0, 8.0, 1270.0, 0.0, 47.6934, -122.342, 1330.0, 1323.0, 8.0, 0.0, 0.0] | [403964.3] | 0 |
12 | 2024-09-17 15:43:49.624 | [4.0, 1.75, 2070.0, 9120.0, 1.0, 0.0, 0.0, 4.0, 7.0, 1250.0, 820.0, 47.6045, -122.123, 1650.0, 8400.0, 57.0, 0.0, 0.0] | [528991.3] | 0 |
13 | 2024-09-17 15:43:49.624 | [4.0, 1.0, 1620.0, 4080.0, 1.5, 0.0, 0.0, 3.0, 7.0, 1620.0, 0.0, 47.6696, -122.324, 1760.0, 4080.0, 91.0, 0.0, 0.0] | [612201.75] | 0 |
14 | 2024-09-17 15:43:49.624 | [4.0, 3.25, 3990.0, 9786.0, 2.0, 0.0, 0.0, 3.0, 9.0, 3990.0, 0.0, 47.6784, -122.026, 3920.0, 8200.0, 10.0, 0.0, 0.0] | [893874.7] | 0 |
15 | 2024-09-17 15:43:49.624 | [4.0, 2.0, 1780.0, 19843.0, 1.0, 0.0, 0.0, 3.0, 7.0, 1780.0, 0.0, 47.4414, -122.154, 2210.0, 13500.0, 52.0, 0.0, 0.0] | [318054.94] | 0 |
16 | 2024-09-17 15:43:49.624 | [4.0, 2.5, 2130.0, 6003.0, 2.0, 0.0, 0.0, 3.0, 8.0, 2130.0, 0.0, 47.4518, -122.12, 1940.0, 4529.0, 11.0, 0.0, 0.0] | [357816.7] | 0 |
17 | 2024-09-17 15:43:49.624 | [3.0, 1.75, 1660.0, 10440.0, 1.0, 0.0, 0.0, 3.0, 7.0, 1040.0, 620.0, 47.4448, -121.77, 1240.0, 10380.0, 36.0, 0.0, 0.0] | [294034.63] | 0 |
18 | 2024-09-17 15:43:49.624 | [3.0, 2.5, 2110.0, 4118.0, 2.0, 0.0, 0.0, 3.0, 8.0, 2110.0, 0.0, 47.3878, -122.153, 2110.0, 4044.0, 25.0, 0.0, 0.0] | [323254.28] | 0 |
19 | 2024-09-17 15:43:49.624 | [4.0, 2.25, 2200.0, 11250.0, 1.5, 0.0, 0.0, 5.0, 7.0, 1300.0, 900.0, 47.6845, -122.201, 2320.0, 10814.0, 94.0, 0.0, 0.0] | [770916.6] | 0 |
time | in.tensor | out.variable | anomaly.count | |
---|---|---|---|---|
0 | 2024-09-17 15:43:49.697 | [2.0, 1.0, 810.0, 4080.0, 1.0, 0.0, 0.0, 4.0, 6.0, 810.0, 0.0, 47.5337, -122.379, 1400.0, 4080.0, 73.0, 0.0, 0.0] | [375012.0] | 0 |
1 | 2024-09-17 15:43:49.697 | [3.0, 1.75, 1240.0, 10956.0, 1.0, 0.0, 0.0, 3.0, 6.0, 1240.0, 0.0, 47.3705, -122.15, 1240.0, 8137.0, 27.0, 0.0, 0.0] | [243063.13] | 0 |
2 | 2024-09-17 15:43:49.697 | [4.0, 2.5, 2550.0, 7555.0, 2.0, 0.0, 0.0, 3.0, 8.0, 2550.0, 0.0, 47.2614, -122.29, 2550.0, 6800.0, 13.0, 0.0, 0.0] | [353912.0] | 0 |
3 | 2024-09-17 15:43:49.697 | [2.0, 1.0, 770.0, 6731.0, 1.0, 0.0, 0.0, 4.0, 6.0, 770.0, 0.0, 47.7505, -122.312, 1120.0, 9212.0, 72.0, 0.0, 0.0] | [340764.53] | 0 |
4 | 2024-09-17 15:43:49.697 | [4.0, 3.5, 3450.0, 7832.0, 2.0, 0.0, 0.0, 3.0, 10.0, 3450.0, 0.0, 47.5637, -122.123, 3220.0, 8567.0, 7.0, 0.0, 0.0] | [921695.4] | 0 |
5 | 2024-09-17 15:43:49.697 | [3.0, 2.5, 1660.0, 2890.0, 2.0, 0.0, 0.0, 3.0, 7.0, 1660.0, 0.0, 47.5434, -122.293, 1540.0, 2890.0, 14.0, 0.0, 0.0] | [544392.06] | 0 |
6 | 2024-09-17 15:43:49.697 | [4.0, 2.0, 1450.0, 5456.0, 1.0, 0.0, 0.0, 5.0, 7.0, 1450.0, 0.0, 47.5442, -122.297, 980.0, 6100.0, 63.0, 0.0, 0.0] | [451058.3] | 0 |
7 | 2024-09-17 15:43:49.697 | [2.0, 1.0, 1170.0, 6543.0, 1.0, 0.0, 0.0, 3.0, 7.0, 1170.0, 0.0, 47.537, -122.385, 1550.0, 7225.0, 101.0, 0.0, 0.0] | [434534.22] | 0 |
8 | 2024-09-17 15:43:49.697 | [4.0, 2.5, 2750.0, 11830.0, 2.0, 0.0, 0.0, 3.0, 9.0, 2750.0, 0.0, 47.4698, -122.121, 2310.0, 11830.0, 0.0, 0.0, 0.0] | [515844.28] | 0 |
9 | 2024-09-17 15:43:49.697 | [3.0, 2.0, 2060.0, 2900.0, 1.5, 0.0, 0.0, 5.0, 8.0, 1330.0, 730.0, 47.5897, -122.292, 1910.0, 3900.0, 84.0, 0.0, 0.0] | [630865.5] | 0 |
10 | 2024-09-17 15:43:49.697 | [3.0, 1.75, 2350.0, 20820.0, 1.0, 0.0, 0.0, 4.0, 8.0, 1800.0, 550.0, 47.6095, -122.059, 2040.0, 10800.0, 36.0, 0.0, 0.0] | [700294.25] | 0 |
11 | 2024-09-17 15:43:49.697 | [3.0, 1.0, 1160.0, 3700.0, 1.5, 0.0, 0.0, 3.0, 7.0, 1160.0, 0.0, 47.5651, -122.359, 1340.0, 3750.0, 105.0, 0.0, 0.0] | [435628.72] | 0 |
12 | 2024-09-17 15:43:49.697 | [3.0, 2.0, 2070.0, 9600.0, 1.0, 0.0, 1.0, 3.0, 7.0, 1590.0, 480.0, 47.616, -122.239, 3000.0, 16215.0, 68.0, 0.0, 0.0] | [636559.56] | 0 |
13 | 2024-09-17 15:43:49.697 | [4.0, 2.5, 2490.0, 5812.0, 2.0, 0.0, 0.0, 3.0, 8.0, 2490.0, 0.0, 47.3875, -122.155, 2690.0, 6012.0, 14.0, 0.0, 0.0] | [352272.53] | 0 |
14 | 2024-09-17 15:43:49.697 | [2.0, 1.0, 980.0, 3800.0, 1.0, 0.0, 0.0, 3.0, 7.0, 980.0, 0.0, 47.6903, -122.34, 1520.0, 5010.0, 89.0, 0.0, 0.0] | [448627.8] | 0 |
15 | 2024-09-17 15:43:49.697 | [2.0, 1.0, 1130.0, 2640.0, 1.0, 0.0, 0.0, 4.0, 8.0, 1130.0, 0.0, 47.6438, -122.357, 1680.0, 3200.0, 87.0, 0.0, 0.0] | [449699.88] | 0 |
16 | 2024-09-17 15:43:49.697 | [3.0, 0.75, 1240.0, 4000.0, 1.0, 0.0, 0.0, 4.0, 7.0, 1240.0, 0.0, 47.6239, -122.297, 1460.0, 4000.0, 47.0, 0.0, 0.0] | [448627.8] | 0 |
17 | 2024-09-17 15:43:49.697 | [3.0, 2.5, 1400.0, 2036.0, 2.0, 0.0, 0.0, 3.0, 7.0, 1400.0, 0.0, 47.5516, -122.382, 1500.0, 2036.0, 11.0, 0.0, 0.0] | [435628.72] | 0 |
18 | 2024-09-17 15:43:49.697 | [3.0, 2.5, 1530.0, 8500.0, 1.0, 0.0, 0.0, 5.0, 7.0, 1030.0, 500.0, 47.3592, -122.046, 1850.0, 8140.0, 19.0, 0.0, 0.0] | [281411.13] | 0 |
19 | 2024-09-17 15:43:49.697 | [3.0, 2.5, 1830.0, 8133.0, 1.0, 0.0, 0.0, 3.0, 8.0, 1390.0, 440.0, 47.7478, -122.247, 2310.0, 11522.0, 19.0, 0.0, 0.0] | [437177.97] | 0 |
time | in.tensor | out.variable | anomaly.count | |
---|---|---|---|---|
0 | 2024-09-17 15:43:49.768 | [2.0, 1.0, 1650.0, 9780.0, 1.0, 0.0, 0.0, 3.0, 7.0, 950.0, 700.0, 47.6549, -122.394, 1650.0, 5458.0, 72.0, 0.0, 0.0] | [572618.25] | 0 |
1 | 2024-09-17 15:43:49.768 | [3.0, 1.75, 1700.0, 7532.0, 1.0, 0.0, 0.0, 3.0, 7.0, 1700.0, 0.0, 47.355, -122.176, 1690.0, 7405.0, 27.0, 0.0, 0.0] | [250294.31] | 0 |
2 | 2024-09-17 15:43:49.768 | [3.0, 2.25, 2206.0, 82031.0, 1.0, 0.0, 2.0, 3.0, 6.0, 866.0, 1340.0, 47.6302, -122.069, 2590.0, 53024.0, 31.0, 0.0, 0.0] | [525186.9] | 0 |
3 | 2024-09-17 15:43:49.768 | [2.0, 1.0, 750.0, 4000.0, 1.0, 0.0, 0.0, 4.0, 6.0, 750.0, 0.0, 47.5547, -122.272, 1120.0, 5038.0, 89.0, 0.0, 0.0] | [334268.3] | 0 |
4 | 2024-09-17 15:43:49.768 | [3.0, 1.0, 1100.0, 5400.0, 1.5, 0.0, 0.0, 3.0, 7.0, 1100.0, 0.0, 47.6604, -122.396, 1770.0, 4400.0, 106.0, 0.0, 0.0] | [546357.56] | 0 |
5 | 2024-09-17 15:43:49.768 | [4.0, 2.5, 2340.0, 8990.0, 2.0, 0.0, 0.0, 3.0, 8.0, 2340.0, 0.0, 47.3781, -122.03, 2980.0, 6718.0, 11.0, 0.0, 0.0] | [393116.88] | 0 |
6 | 2024-09-17 15:43:49.768 | [4.0, 2.5, 2120.0, 3220.0, 2.0, 0.0, 0.0, 3.0, 9.0, 2120.0, 0.0, 47.6662, -122.083, 2120.0, 3547.0, 9.0, 0.0, 0.0] | [647530.56] | 0 |
7 | 2024-09-17 15:43:49.768 | [3.0, 2.5, 2520.0, 53143.0, 1.5, 0.0, 0.0, 3.0, 7.0, 2520.0, 0.0, 47.743, -121.925, 2020.0, 56628.0, 26.0, 0.0, 0.0] | [409507.84] | 0 |
8 | 2024-09-17 15:43:49.768 | [5.0, 2.5, 2960.0, 8968.0, 1.0, 0.0, 0.0, 4.0, 8.0, 1640.0, 1320.0, 47.6233, -122.102, 1890.0, 9077.0, 49.0, 0.0, 0.0] | [632067.8] | 0 |
9 | 2024-09-17 15:43:49.768 | [4.0, 1.0, 1510.0, 7200.0, 1.5, 0.0, 0.0, 4.0, 7.0, 1510.0, 0.0, 47.761, -122.307, 1950.0, 10656.0, 60.0, 0.0, 0.0] | [366419.13] | 0 |
10 | 2024-09-17 15:43:49.768 | [4.0, 1.75, 2350.0, 18800.0, 1.0, 0.0, 2.0, 3.0, 8.0, 2350.0, 0.0, 47.5904, -122.177, 3050.0, 14640.0, 55.0, 0.0, 0.0] | [767840.9] | 0 |
11 | 2024-09-17 15:43:49.768 | [4.0, 2.5, 1980.0, 6566.0, 2.0, 0.0, 0.0, 3.0, 8.0, 1980.0, 0.0, 47.3809, -122.097, 2590.0, 6999.0, 11.0, 0.0, 0.0] | [316403.6] | 0 |
12 | 2024-09-17 15:43:49.768 | [3.0, 2.5, 2170.0, 3200.0, 1.5, 0.0, 0.0, 5.0, 7.0, 1280.0, 890.0, 47.6543, -122.347, 1180.0, 1224.0, 91.0, 0.0, 0.0] | [649745.7] | 0 |
13 | 2024-09-17 15:43:49.768 | [3.0, 3.5, 3940.0, 11632.0, 2.0, 0.0, 0.0, 3.0, 10.0, 3940.0, 0.0, 47.438, -122.344, 2015.0, 11632.0, 0.0, 0.0, 0.0] | [769068.6] | 0 |
14 | 2024-09-17 15:43:49.768 | [3.0, 2.5, 1390.0, 2815.0, 2.0, 0.0, 0.0, 3.0, 8.0, 1390.0, 0.0, 47.566, -122.366, 1390.0, 3700.0, 16.0, 0.0, 0.0] | [352616.03] | 0 |
15 | 2024-09-17 15:43:49.768 | [3.0, 2.0, 1400.0, 7384.0, 1.0, 0.0, 0.0, 3.0, 7.0, 1150.0, 250.0, 47.4655, -122.174, 1820.0, 7992.0, 35.0, 0.0, 0.0] | [247729.9] | 0 |
16 | 2024-09-17 15:43:49.768 | [3.0, 2.5, 2780.0, 32880.0, 1.0, 0.0, 0.0, 3.0, 9.0, 2780.0, 0.0, 47.4798, -121.727, 2780.0, 40091.0, 21.0, 0.0, 0.0] | [606714.5] | 0 |
17 | 2024-09-17 15:43:49.768 | [3.0, 2.5, 1250.0, 1150.0, 2.0, 0.0, 0.0, 3.0, 8.0, 1080.0, 170.0, 47.5582, -122.363, 1250.0, 1150.0, 6.0, 0.0, 0.0] | [336369.63] | 0 |
18 | 2024-09-17 15:43:49.768 | [3.0, 1.75, 2360.0, 8290.0, 1.0, 0.0, 0.0, 4.0, 7.0, 1180.0, 1180.0, 47.6738, -122.281, 1880.0, 7670.0, 65.0, 0.0, 0.0] | [718904.06] | 0 |
19 | 2024-09-17 15:43:49.768 | [4.0, 2.5, 3070.0, 6923.0, 2.0, 0.0, 0.0, 3.0, 9.0, 3070.0, 0.0, 47.669, -122.172, 2190.0, 9218.0, 5.0, 0.0, 0.0] | [838463.4] | 0 |
## undeploy the pipeline
pipeline_a.undeploy()
pipeline_b.undeploy()
pipeline_c.undeploy()
Waiting for undeployment - this will take up to 45s .................................... ok
Waiting for undeployment - this will take up to 45s .................................... ok
Waiting for undeployment - this will take up to 45s .................................... ok
name | house-price-zone-c |
---|---|
created | 2024-09-17 15:22:52.701839+00:00 |
last_updated | 2024-09-17 15:43:32.988837+00:00 |
deployed | False |
workspace_id | 9 |
workspace_name | multiple_pipeline_deployment_tutorial |
arch | x86 |
accel | none |
tags | |
versions | ef891151-8238-4ee3-8bab-9b2d071db490, 69650cf4-31d1-4290-ba63-cd5c18cf3829, d853c867-4b93-4327-80b1-95c81e0244bc |
steps | house-price-model-c |
published | False |
With the pipeline deployed and our connections set, we will now generate our ML Workload Orchestration. See the Wallaroo ML Workload Orchestrations guide for full details.
Orchestrations are uploaded to the Wallaroo instance as a ZIP file with the following requirements:
Parameter | Type | Description |
---|---|---|
User Code | (Required) Python script as .py files | If main.py exists, then that will be used as the task entrypoint. Otherwise, the first main.py found in any subdirectory will be used as the entrypoint. |
Python Library Requirements | (Optional) requirements.txt file in the requirements file format. A standard Python requirements.txt for any dependencies to be provided in the task environment. The Wallaroo SDK will already be present and should not be included in the requirements.txt. Multiple requirements.txt files are not allowed. | |
Other artifacts | Other artifacts such as files, data, or code to support the orchestration. |
For our example, our orchestration will:
This sample script is stored in remote_inference/main.py
with an empty requirements.txt
file, and packaged into the orchestration as ./remote_inference/multiple_pieplines.zip
. We’ll display the steps in uploading the orchestration to the Wallaroo instance.
Orchestrations are uploaded with the Wallaroo client upload_orchestration(path)
method with the following parameters.
Parameter | Type | Description |
---|---|---|
path | string (Required) | The path to the ZIP file to be uploaded. |
Once uploaded, the deployment will be prepared and any requirements will be downloaded and installed.
For this example, the orchestration ./remote_inference/remote_inference.zip
will be uploaded and saved to the variable orchestration
.
We will loop until the uploaded orchestration’s status
displays ready
.
orchestration = wl.upload_orchestration(path="./remote_inference/multiple_pipelines.zip")
while orchestration.status() != 'ready' and orchestration.status() != 'error':
print(orchestration.status())
time.sleep(5)
pending_packaging
pending_packaging
packaging
packaging
packaging
packaging
packaging
packaging
packaging
packaging
packaging
Once an Orchestration has the status ready
, it can be run as a task. Tasks have the following run options.
Type | SDK Call | How triggered |
---|---|---|
Once | orchestration.run_once(name, json_args, timeout) | Task runs once and exits. |
Scheduled | orchestration.run_scheduled(name, schedule, timeout, json_args) | User provides schedule. Task runs exits whenever schedule dictates. |
We’ll do both a Run Once task and generate our Run Once Task from our orchestration.
Tasks are generated and run once with the Orchestration run_once(name, json_args, timeout)
method. Any arguments for the orchestration are passed in as a Dict
. If there are no arguments, then an empty set {}
is passed.
# Example: run once
import datetime
task_start = datetime.datetime.now()
task = orchestration.run_once(name="multiple_pipelines_demo",
json_args={})
The list of tasks in the Wallaroo instance is retrieves through the Wallaroo Client list_tasks()
method. This returns an array list of the following.
Parameter | Type | Description |
---|---|---|
id | string | The UUID identifier for the task. |
last run status | string | The last reported status the task. Values are:
|
type | string | The type of the task. Values are:
|
created at | DateTime | The date and time the task was started. |
updated at | DateTime | The date and time the task was updated. |
For this example, the status of the previously created task will be generated, then looped until it has reached status started
.
while task.status() != "started":
display(task.status())
time.sleep(5)
task.status()
'pending'
‘pending’
‘started’
We can view the inferences from our logs and verify that new entries were added from our task. We can do that with the task last_runs()
method to see the list of task runs executed, then show the log from the last completed task run.
# wait for the deployment and inference to complete
time.sleep(45)
task.last_runs()[0]
Field | Value |
---|---|
Task | 7924c132-c7fa-472c-bd51-c348ce20f02b |
Pod ID | 624d1a4b-5584-4604-b478-ba33fbdeb0f6 |
Status | running |
Created At | 2024-17-Sep 18:02:36 |
Updated At | 2024-17-Sep 18:02:36 |
task.last_runs()[0].logs()
2024-17-Sep 18:04:37 Getting the workspace multiple_pipeline_deployment_tutorial
2024-17-Sep 18:04:37 Getting the pipelines
2024-17-Sep 18:04:37 Deployment initiated for house-price-zone-a. Please check pipeline status.
2024-17-Sep 18:04:37 Deployment initiated for house-price-zone-b. Please check pipeline status.
2024-17-Sep 18:04:37 Deployment initiated for house-price-zone-c. Please check pipeline status.
2024-17-Sep 18:04:37 time ... anomaly.count
2024-17-Sep 18:04:37 0 2024-09-17 18:02:45.072 ... 0
2024-17-Sep 18:04:37 1 2024-09-17 18:02:45.072 ... 0
2024-17-Sep 18:04:37 2 2024-09-17 18:02:45.072 ... 0
2024-17-Sep 18:04:37 4 2024-09-17 18:02:45.072 ... 0
2024-17-Sep 18:04:37 3 2024-09-17 18:02:45.072 ... 0
2024-17-Sep 18:04:37 5 2024-09-17 18:02:45.072 ... 0
2024-17-Sep 18:04:37 7 2024-09-17 18:02:45.072 ... 0
2024-17-Sep 18:04:37 6 2024-09-17 18:02:45.072 ... 0
2024-17-Sep 18:04:37 8 2024-09-17 18:02:45.072 ... 0
2024-17-Sep 18:04:37 9 2024-09-17 18:02:45.072 ... 0
2024-17-Sep 18:04:37 10 2024-09-17 18:02:45.072 ... 0
2024-17-Sep 18:04:37 11 2024-09-17 18:02:45.072 ... 0
2024-17-Sep 18:04:37 12 2024-09-17 18:02:45.072 ... 0
2024-17-Sep 18:04:37 13 2024-09-17 18:02:45.072 ... 0
2024-17-Sep 18:04:37 14 2024-09-17 18:02:45.072 ... 0
2024-17-Sep 18:04:37 16 2024-09-17 18:02:45.072 ... 0
2024-17-Sep 18:04:37 15 2024-09-17 18:02:45.072 ... 0
2024-17-Sep 18:04:37 17 2024-09-17 18:02:45.072 ... 0
2024-17-Sep 18:04:37 19 2024-09-17 18:02:45.072 ... 0
2024-17-Sep 18:04:37 18 2024-09-17 18:02:45.072 ... 0
2024-17-Sep 18:04:37
2024-17-Sep 18:04:37 time ... anomaly.count
2024-17-Sep 18:04:37 [20 rows x 4 columns]
2024-17-Sep 18:04:37 0 2024-09-17 18:02:45.129 ... 0
2024-17-Sep 18:04:37 2 2024-09-17 18:02:45.129 ... 0
2024-17-Sep 18:04:37 1 2024-09-17 18:02:45.129 ... 0
2024-17-Sep 18:04:37 3 2024-09-17 18:02:45.129 ... 0
2024-17-Sep 18:04:37 4 2024-09-17 18:02:45.129 ... 0
2024-17-Sep 18:04:37 6 2024-09-17 18:02:45.129 ... 0
2024-17-Sep 18:04:37 5 2024-09-17 18:02:45.129 ... 0
2024-17-Sep 18:04:37 7 2024-09-17 18:02:45.129 ... 0
2024-17-Sep 18:04:37 8 2024-09-17 18:02:45.129 ... 0
2024-17-Sep 18:04:37 10 2024-09-17 18:02:45.129 ... 0
2024-17-Sep 18:04:37 9 2024-09-17 18:02:45.129 ... 0
2024-17-Sep 18:04:37 11 2024-09-17 18:02:45.129 ... 0
2024-17-Sep 18:04:37 13 2024-09-17 18:02:45.129 ... 0
2024-17-Sep 18:04:37 12 2024-09-17 18:02:45.129 ... 0
2024-17-Sep 18:04:37 14 2024-09-17 18:02:45.129 ... 0
2024-17-Sep 18:04:37 16 2024-09-17 18:02:45.129 ... 0
2024-17-Sep 18:04:37 15 2024-09-17 18:02:45.129 ... 0
2024-17-Sep 18:04:37 17 2024-09-17 18:02:45.129 ... 0
2024-17-Sep 18:04:37 19 2024-09-17 18:02:45.129 ... 0
2024-17-Sep 18:04:37 18 2024-09-17 18:02:45.129 ... 0
2024-17-Sep 18:04:37 [20 rows x 4 columns]
2024-17-Sep 18:04:37
2024-17-Sep 18:04:37 time ... anomaly.count
2024-17-Sep 18:04:37 0 2024-09-17 18:02:45.216 ... 0
2024-17-Sep 18:04:37 1 2024-09-17 18:02:45.216 ... 0
2024-17-Sep 18:04:37 2 2024-09-17 18:02:45.216 ... 0
2024-17-Sep 18:04:37 3 2024-09-17 18:02:45.216 ... 0
2024-17-Sep 18:04:37 4 2024-09-17 18:02:45.216 ... 0
2024-17-Sep 18:04:37 5 2024-09-17 18:02:45.216 ... 0
2024-17-Sep 18:04:37 6 2024-09-17 18:02:45.216 ... 0
2024-17-Sep 18:04:37 7 2024-09-17 18:02:45.216 ... 0
2024-17-Sep 18:04:37 9 2024-09-17 18:02:45.216 ... 0
2024-17-Sep 18:04:37 10 2024-09-17 18:02:45.216 ... 0
2024-17-Sep 18:04:37 8 2024-09-17 18:02:45.216 ... 0
2024-17-Sep 18:04:37 11 2024-09-17 18:02:45.216 ... 0
2024-17-Sep 18:04:37 12 2024-09-17 18:02:45.216 ... 0
2024-17-Sep 18:04:37 13 2024-09-17 18:02:45.216 ... 0
2024-17-Sep 18:04:37 14 2024-09-17 18:02:45.216 ... 0
2024-17-Sep 18:04:37 15 2024-09-17 18:02:45.216 ... 0
2024-17-Sep 18:04:37 16 2024-09-17 18:02:45.216 ... 0
2024-17-Sep 18:04:37 17 2024-09-17 18:02:45.216 ... 0
2024-17-Sep 18:04:37 18 2024-09-17 18:02:45.216 ... 0
2024-17-Sep 18:04:37 19 2024-09-17 18:02:45.216 ... 0
2024-17-Sep 18:04:37
2024-17-Sep 18:04:37 [20 rows x 4 columns]
This completes the tutorial and example. The sample code and orchestration is available for organizations to use for deploying multiple pipelines and other tasks.