ML Models are deployed through pipelines by being added as pipeline steps. Updated models are deployed via in line model updates, which “hot swaps” the existing model with the new one.
In Line Model Updates via the Wallaroo SDK
In Line Model Update
The model deployed in a pipeline replaced with the pipeline method replace_with_model_step(index, model).
IMPORTANT NOTE
Pipeline steps can be replaced while a pipeline is deployed. This allows organizations to have pipelines deployed in a production environment and hot-swap out models for new versions without impacting performance or inferencing downtime.The following parameters are used for replacing a pipeline step:
| Parameter | Default Value | Purpose |
|---|---|---|
index | null | The pipeline step to be replaced. Pipeline steps follow array numbering, where the first step is 0, etc. |
model | null | The new model to be used in the pipeline step. |
In Line Model Update Example
In the following example, a deployed pipeline will have the initial model step replaced with a new one. A status of the pipeline is displayed after deployment and after the pipeline swap to show the model has been replaced from ccfraudoriginal to ccfraudreplacement, each with their own versions.
# current deployed pipeline
pipeline.status()
{'status': 'Running',
'details': [],
'engines': [{'ip': '10.244.2.145',
'name': 'engine-75bfd7dc9d-7p9qk',
'status': 'Running',
'reason': None,
'details': [],
'pipeline_statuses': {'pipelines': [{'id': 'hotswappipeline',
'status': 'Running'}]},
'model_statuses': {'models': [{'name': 'ccfraudoriginal',
'version': '3a03dc94-716e-46bb-84c8-91bc99ceb2c3',
'sha': 'bc85ce596945f876256f41515c7501c399fd97ebcb9ab3dd41bf03f8937b4507',
'status': 'Running'}]}}],
'engine_lbs': [{'ip': '10.244.2.144',
'name': 'engine-lb-55dcdff64c-vf74s',
'status': 'Running',
'reason': None,
'details': []}],
'sidekicks': []}
# replace the pipeline step
pipeline.replace_with_model_step(0, replacement_model)
# deploy the new model version - note that the pipeline is not undeployed first
pipeline.deploy()
# the new model is deployed
pipeline.status()
{'status': 'Running',
'details': [],
'engines': [{'ip': '10.244.2.153',
'name': 'engine-96486c95d-zfchr',
'status': 'Running',
'reason': None,
'details': [],
'pipeline_statuses': {'pipelines': [{'id': 'hotswappipeline',
'status': 'Running'}]},
'model_statuses': {'models': [{'name': 'ccfraudreplacement',
'version': '714efd19-5c83-42a8-aece-24b4ba530925',
'sha': 'bc85ce596945f876256f41515c7501c399fd97ebcb9ab3dd41bf03f8937b4507',
'status': 'Running'}]}}],
'engine_lbs': [{'ip': '10.244.2.154',
'name': 'engine-lb-55dcdff64c-9np9k',
'status': 'Running',
'reason': None,
'details': []}],
'sidekicks': []}