Wallaroo Model Management

How to manage your Wallaroo models

Models are the Machine Learning (ML) models that are uploaded to your Wallaroo workspace and used to solve problems based on data submitted to them in a pipeline.

Supported Models and Libraries

The following ML Model versions and Python libraries are supported by Wallaroo. When using the Wallaroo autoconversion library or working with a local version of the Wallaroo SDK, use the following versions for maximum compatibility.

Library Supported Version
Python 3.8.6 and above
onnx 1.12.0
tensorflow 2.9.1
keras 2.9.0
pytorch Latest stable version. When converting from PyTorch to onnx, verify that the onnx version matches the version above.
sk-learn aka scikit-learn 1.1.2
XGBoost 1.6.2
MLFlow 1.30.0

How to Upload Models to a Workspace

Upload Models to a Workspace

Models are uploaded to the current workspace through the Wallaroo Client upload_model("{Model Name}", "{Model Path}).configure(options). In most cases, leaving the options field can be left blank. For more details, see the full SDK guide.

Models can either be uploaded in the Open Neural Network eXchange(ONNX) format, or be auto-converted and uploaded using the Wallaroo convert_model(path, source_type, conversion_arguments) method. For more information, see the tutorial series ONNX Conversion Tutorials.

Wallaroo can directly import Open Neural Network Exchange models into the Wallaroo engine. Other ML Models can be imported with the Auto-Convert Models methods.

The following models are supported natively by Wallaroo:

Wallaroo Version ONNX Version ONNX IR Version ONNX OPset Version ONNX ML Opset Version Tensorflow Version
2022.4 (December 2022) 1.12.1 8 17 3 2.9.1
After April 2022 until release 2022.4 (December 2022) 1.10.* 7 15 2 2.4
Before April 2022 1.6.* 7 13 2 2.4

For the most recent release of Wallaroo September 2022, the following native runtimes are supported:

Using different versions or settings outside of these specifications may result in inference issues and other unexpected behavior.

The following example shows how to upload two models to the imdb-workspace workspace:

wl.get_current_workspace()

{'name': 'imdb-workspace', 'id': 8, 'archived': False, 'created_by': '45e6b641-fe57-4fb2-83d2-2c2bd201efe8', 'created_at': '2022-03-30T21:13:21.87287+00:00', 'models': [], 'pipelines': []}

embedder = wl.upload_model('embedder-o', './embedder.onnx').configure()
smodel = wl.upload_model('smodel-o', './sentiment_model.onnx').configure()

{'name': 'imdb-workspace', 'id': 9, 'archived': False, 'created_by': '45e6b641-fe57-4fb2-83d2-2c2bd201efe8', 'created_at': '2022-03-30T21:14:37.733171+00:00', 'models': [{'name': 'embedder-o', 'version': '28ecb706-473e-4f24-9eae-bfa71b897108', 'file_name': 'embedder.onnx', 'last_update_time': datetime.datetime(2022, 3, 30, 21, 14, 37, 815243, tzinfo=tzutc())}, {'name': 'smodel-o', 'version': '5d2782e1-fb88-430f-b6eb-c0a0eb46beb9', 'file_name': 'sentiment_model.onnx', 'last_update_time': datetime.datetime(2022, 3, 30, 21, 14, 38, 77973, tzinfo=tzutc())}], 'pipelines': []}

Auto-Convert Models

Machine Learning (ML) models can be converted and uploaded into Wallaroo workspace using the Wallaroo Client convert_model(path, source_type, conversion_arguments) method. This conversion process transforms the model into an open format that can be run across different framework at compiled C-language speeds.

The three input parameters are:

  • path (STRING): The path to the ML model file.
  • source_type (ModelConversionSource): The type of ML model to be converted. As of this time Wallaroo auto-conversion supports the following source types and their associated ModelConversionSource:
    • sklearn: ModelConversionSource.SKLEARN
    • xgboost: ModelConversionSource.XGBOOST
    • keras: ModelConversionSource.KERAS
  • conversion_arguments: The arguments for the conversion based on the type of model being converted. These are:
    • wallaroo.ModelConversion.ConvertKerasArguments: Used for converting keras type models and takes the following parameters:
      • name: The name of the model being converted.
      • comment: Any comments for the model.
      • input_type: A tensorflow Dtype called in the format ModelConversionInputType.{type}. See ModelConversionTypes for more details.
      • dimensions: Corresponds to the keras xtrain in the format [{Number of Rows/None}, {Number of Columns 1}, {Number of Columns 2}...]. For a standard 1-dimensional array with 100 columns this would typically be [None, 100].
    • wallaroo.ModelConversion.ConvertSKLearnArguments: Used for sklearn models and takes the following parameters:
      • name: The name of the model being converted.
      • comment: Any comments for the model.
      • number_of_columns: The number of columns the model was trained for.
      • input_type: A tensorflow Dtype called in the format ModelConversionInputType.{type}. See ModelConversionTypes for more details.
    • wallaroo.ModelConversion.ConvertXGBoostArgs: Used for XGBoost models and takes the following parameters:
      • name: The name of the model being converted.
      • comment: Any comments for the model.
      • number_of_columns: The number of columns the model was trained for.
      • input_type: A tensorflow Dtype called in the format ModelConversionInputType.{type}. See ModelConversionTypes for more details.

Once uploaded, they will be displayed in the Wallaroo Models Dashboard as {unique-file-id}-converted.onnx:

Converted Model

ModelConversionInputTypes

The following data types are supported with the ModelConversionInputType parameter:

Parameter Data Type
Float16 float16
Float32 float32
Float64 float64
Int16 int16
Int32 int32
Int64 int64
UInt8 uint8
UInt16 uint16
UInt32 uint32
UInt64 uint64
Boolean bool
Double double

sk-learn Example

The following example converts and uploads a Linear Regression sklearn model lm.pickle and stores it in the variable converted_model:

wl = wallaroo.Client()

workspace_name = "testconversion"
_ = wl.set_current_workspace(get_or_create_workspace(workspace_name))

model_conversion_args = ConvertSKLearnArguments(
    name="lm-test",
    comment="test linear regression",
    number_of_columns=NF,
    input_type=ModelConversionInputType.Double
)

model_conversion_type = ModelConversionSource.SKLEARN

# convert the model and store it in the variable `converted_model`:

converted_model = wl.convert_model('lm.pickle', model_conversion_type, model_conversion_args)

keras Example

The following example shows converting a keras model with 100 columns and uploading it to a Wallaroo instance:

model_columns = 100

model_conversion_args = ConvertKerasArguments(
    name=model_name,
    comment="simple keras model",
    input_type=ModelConversionInputType.Float32,
    dimensions=(None, model_columns)
)
model_conversion_type = ModelConversionSource.KERAS

model_wl = wl.convert_model('simple_sentiment_model.zip', model_conversion_type, model_conversion_args)
model_wl
{'name': 'simple-sentiment-model', 'version': 'c76870f8-e16b-4534-bb17-e18a3e3806d5', 'file_name': '14d9ab8d-47f4-4557-82a7-6b26cb67ab05-converted.onnx', 'last_update_time': datetime.datetime(2022, 7, 7, 16, 41, 22, 528430, tzinfo=tzutc())}

How to View Uploaded Models

Models uploaded to the current workspace can be seen through the following process:

  1. From the Wallaroo Dashboard, select the workspace to set as the current workspace from the navigation panel above. The number of models for the workspace will be displayed.
  2. Select View Models. A list of the models in the workspace will be displayed.
  3. To view details on the model, select the model name from the list.

Model Details

From the Model Details page the following is displayed:

  • The name of the model.
  • The unique ID of the model represented as a UUID.
  • The file name of the model
  • The version history of the model.

Wallaroo Model Tag Management

How to manage tags and models.