Wallaroo Model Management

Models are the Machine Learning (ML) models that are uploaded to your Wallaroo workspace and used to solve problems based on data submitted to them in a pipeline.

Model Naming Requirements

Model names map onto Kubernetes objects, and must be DNS compliant. The strings for model names must be ASCII alpha-numeric characters or dash (-) only. . and _ are not allowed.

Supported Models and Libraries

The following ML Model versions and Python libraries are supported by Wallaroo. When using the Wallaroo autoconversion library or working with a local version of the Wallaroo SDK, use the following versions for maximum compatibility.

Supported Data Types

The following data types are supported for transporting data to and from Wallaroo in the following run times:

• ONNX
• TensorFlow
• MLFlow

Data Type Conditions

The following conditions apply to data types used in inference requests.

• None or Null data types are not submitted. All fields must have submitted values that match their data type. For example, if the schema expects a float value, then some value of type float must be submitted and can not be None or Null. If a schema expects a string value, then some value of type string must be submitted, etc.
• datetime data types must be converted to string.
• ONNX models support multiple inputs only of the same data type.

• Float Types
• Int Types
• Uint Types
• Other Types

How to Upload Models to a Workspace

Upload Models to a Workspace

Models are uploaded to the current workspace through the Wallaroo Client upload_model("{Model Name}", "{Model Path}).configure(options). In most cases, leaving the options field can be left blank. For more details, see the SDK reference guide.

Models can either be uploaded in the Open Neural Network eXchange(ONNX) format, or be auto-converted and uploaded using the Wallaroo convert_model(path, source_type, conversion_arguments) method. For more information, see the tutorial series ONNX Conversion Tutorials.

Wallaroo can directly import Open Neural Network Exchange models into the Wallaroo engine. Other ML Models can be imported with the Auto-Convert Models methods.

The following models are supported natively by Wallaroo:

For the most recent release of Wallaroo 2023.2, the following native runtimes are supported:

• The model is version ONNX 1.12.0
• Tensorflow version is Tensorflow 2.4
• If converting another ML Model to ONNX (PyTorch, XGBoost, etc) using the onnxconverter-common library, the supported DEFAULT_OPSET_NUMBER is 17.

Using different versions or settings outside of these specifications may result in inference issues and other unexpected behavior.

The following example shows how to upload two models to the imdb-workspace workspace:

wl.get_current_workspace()

{'name': 'imdb-workspace', 'id': 8, 'archived': False, 'created_by': '45e6b641-fe57-4fb2-83d2-2c2bd201efe8', 'created_at': '2022-03-30T21:13:21.87287+00:00', 'models': [], 'pipelines': []}

embedder = wl.upload_model('embedder-o', './embedder.onnx').configure()
smodel = wl.upload_model('smodel-o', './sentiment_model.onnx').configure()

{'name': 'imdb-workspace', 'id': 9, 'archived': False, 'created_by': '45e6b641-fe57-4fb2-83d2-2c2bd201efe8', 'created_at': '2022-03-30T21:14:37.733171+00:00', 'models': [{'name': 'embedder-o', 'version': '28ecb706-473e-4f24-9eae-bfa71b897108', 'file_name': 'embedder.onnx', 'last_update_time': datetime.datetime(2022, 3, 30, 21, 14, 37, 815243, tzinfo=tzutc())}, {'name': 'smodel-o', 'version': '5d2782e1-fb88-430f-b6eb-c0a0eb46beb9', 'file_name': 'sentiment_model.onnx', 'last_update_time': datetime.datetime(2022, 3, 30, 21, 14, 38, 77973, tzinfo=tzutc())}], 'pipelines': []}

Model Configurations

The Model configure provides methods of configuring the model depending on its type and other considerations. The configure method takes the following parameters.

Auto-Convert Models

Machine Learning (ML) models can be converted and uploaded into Wallaroo workspace using the Wallaroo Client convert_model(path, source_type, conversion_arguments) method. This conversion process transforms the model into an open format that can be run across different framework at compiled C-language speeds.

The three input parameters are:

• path (STRING): The path to the ML model file.
• source_type (ModelConversionSource): The type of ML model to be converted. As of this time Wallaroo auto-conversion supports the following source types and their associated ModelConversionSource:
  • sklearn: ModelConversionSource.SKLEARN
  • xgboost: ModelConversionSource.XGBOOST
  • keras: ModelConversionSource.KERAS
• conversion_arguments: The arguments for the conversion based on the type of model being converted. These are:
  • wallaroo.ModelConversion.ConvertKerasArguments: Used for converting keras type models and takes the following parameters:
    • name: The name of the model being converted.
    • comment: Any comments for the model.
    • input_type: A tensorflow Dtype called in the format ModelConversionInputType.{type}. See ModelConversionTypes for more details.
    • dimensions: Corresponds to the keras xtrain in the format [{Number of Rows/None}, {Number of Columns 1}, {Number of Columns 2}...]. For a standard 1-dimensional array with 100 columns this would typically be [None, 100].
  • wallaroo.ModelConversion.ConvertSKLearnArguments: Used for sklearn models and takes the following parameters:
    • name: The name of the model being converted.
    • comment: Any comments for the model.
    • number_of_columns: The number of columns the model was trained for.
    • input_type: A tensorflow Dtype called in the format ModelConversionInputType.{type}. See ModelConversionTypes for more details.
  • wallaroo.ModelConversion.ConvertXGBoostArgs: Used for XGBoost models and takes the following parameters:
    • name: The name of the model being converted.
    • comment: Any comments for the model.
    • number_of_columns: The number of columns the model was trained for.
    • input_type: A tensorflow Dtype called in the format ModelConversionInputType.{type}. See ModelConversionTypes for more details.

Once uploaded, they will be displayed in the Wallaroo Models Dashboard as {unique-file-id}-converted.onnx:

ModelConversionInputTypes

The following data types are supported with the ModelConversionInputType parameter:

sk-learn Example

The following example converts and uploads a Linear Regression sklearn model lm.pickle and stores it in the variable converted_model:

wl = wallaroo.Client()

workspace_name = "testconversion"
_ = wl.set_current_workspace(get_or_create_workspace(workspace_name))

model_conversion_args = ConvertSKLearnArguments(
    name="lm-test",
    comment="test linear regression",
    number_of_columns=NF,
    input_type=ModelConversionInputType.Double
)

model_conversion_type = ModelConversionSource.SKLEARN

# convert the model and store it in the variable `converted_model`:

converted_model = wl.convert_model('lm.pickle', model_conversion_type, model_conversion_args)

keras Example

The following example shows converting a keras model with 100 columns and uploading it to a Wallaroo instance:

model_columns = 100

model_conversion_args = ConvertKerasArguments(
    name=model_name,
    comment="simple keras model",
    input_type=ModelConversionInputType.Float32,
    dimensions=(None, model_columns)
)
model_conversion_type = ModelConversionSource.KERAS

model_wl = wl.convert_model('simple_sentiment_model.zip', model_conversion_type, model_conversion_args)
model_wl

{'name': 'simple-sentiment-model', 'version': 'c76870f8-e16b-4534-bb17-e18a3e3806d5', 'file_name': '14d9ab8d-47f4-4557-82a7-6b26cb67ab05-converted.onnx', 'last_update_time': datetime.datetime(2022, 7, 7, 16, 41, 22, 528430, tzinfo=tzutc())}

How to View Uploaded Models

Models uploaded to the current workspace can be seen through the following process:

1. From the Wallaroo Dashboard, select the workspace to set as the current workspace from the navigation panel above. The number of models for the workspace will be displayed.
2. Select View Models. A list of the models in the workspace will be displayed.
3. To view details on the model, select the model name from the list.

Model Details

From the Model Details page the following is displayed:

• The name of the model.
• The unique ID of the model represented as a UUID.
• The file name of the model
• The version history of the model.