Supported Model Versions and Libraries

The following ML Model versions and Python libraries are supported by Wallaroo. When using the Wallaroo autoconversion library or working with a local version of the Wallaroo SDK, use the following versions for maximum compatibility.

Supported Data Types

The following data types are supported for transporting data to and from Wallaroo in the following run times:

• ONNX
• TensorFlow
• MLFlow

Float Types

• * (Brain Float 16, represented internally as a f32)

Int Types

Uint Types

Other Types

• * Fixed sized lists of any of the previously supported data types.

Model Management

Upload Models to a Workspace

Models are uploaded to the current workspace through the Wallaroo Client upload_model("{Model Name}", "{Model Path}).configure(options). In most cases, leaving the options field can be left blank. For more details, see the full SDK guide.

Models can either be uploaded in the Open Neural Network eXchange(ONNX) format, or be auto-converted and uploaded using the Wallaroo convert_model(path, source_type, conversion_arguments) method. For more information, see the tutorial series ONNX Conversion Tutorials.

Wallaroo can directly import Open Neural Network Exchange models into the Wallaroo engine. Other ML Models can be imported with the Auto-Convert Models methods.

The following models are supported natively by Wallaroo:

For the most recent release of Wallaroo September 2022, the following native runtimes are supported:

• The model is version ONNX 1.10.0
• Tensorflow version is Tensorflow 2.4
• If converting another ML Model to ONNX (PyTorch, XGBoost, etc) using the onnxconverter-common library, the supported DEFAULT_OPSET_NUMBER is 15.

Using different versions or settings outside of these specifications may result in inference issues and other unexpected behavior.

The following example shows how to upload two models to the imdb-workspace workspace:

wl.get_current_workspace()

{'name': 'imdb-workspace', 'id': 8, 'archived': False, 'created_by': '45e6b641-fe57-4fb2-83d2-2c2bd201efe8', 'created_at': '2022-03-30T21:13:21.87287+00:00', 'models': [], 'pipelines': []}

embedder = wl.upload_model('embedder-o', './embedder.onnx').configure()
smodel = wl.upload_model('smodel-o', './sentiment_model.onnx').configure()

{'name': 'imdb-workspace', 'id': 9, 'archived': False, 'created_by': '45e6b641-fe57-4fb2-83d2-2c2bd201efe8', 'created_at': '2022-03-30T21:14:37.733171+00:00', 'models': [{'name': 'embedder-o', 'version': '28ecb706-473e-4f24-9eae-bfa71b897108', 'file_name': 'embedder.onnx', 'last_update_time': datetime.datetime(2022, 3, 30, 21, 14, 37, 815243, tzinfo=tzutc())}, {'name': 'smodel-o', 'version': '5d2782e1-fb88-430f-b6eb-c0a0eb46beb9', 'file_name': 'sentiment_model.onnx', 'last_update_time': datetime.datetime(2022, 3, 30, 21, 14, 38, 77973, tzinfo=tzutc())}], 'pipelines': []}

Auto-Convert Models

Machine Learning (ML) models can be converted and uploaded into Wallaroo workspace using the Wallaroo Client convert_model(path, source_type, conversion_arguments) method. This conversion process transforms the model into an open format that can be run across different framework at compiled C-language speeds.

The three input parameters are:

• path (STRING): The path to the ML model file.
• source_type (ModelConversionSource): The type of ML model to be converted. As of this time Wallaroo auto-conversion supports the following source types and their associated ModelConversionSource:
  • sklearn: ModelConversionSource.SKLEARN
  • xgboost: ModelConversionSource.XGBOOST
  • keras: ModelConversionSource.KERAS
• conversion_arguments: The arguments for the conversion based on the type of model being converted. These are:
  • wallaroo.ModelConversion.ConvertKerasArguments: Used for converting keras type models and takes the following parameters:
    • name: The name of the model being converted.
    • comment: Any comments for the model.
    • input_type: A tensorflow Dtype called in the format ModelConversionInputType.{type}. See ModelConversionTypes for more details.
    • dimensions: Corresponds to the keras xtrain in the format [{Number of Rows/None}, {Number of Columns 1}, {Number of Columns 2}...]. For a standard 1-dimensional array with 100 columns this would typically be [None, 100].
  • wallaroo.ModelConversion.ConvertSKLearnArguments: Used for sklearn models and takes the following parameters:
    • name: The name of the model being converted.
    • comment: Any comments for the model.
    • number_of_columns: The number of columns the model was trained for.
    • input_type: A tensorflow Dtype called in the format ModelConversionInputType.{type}. See ModelConversionTypes for more details.
  • wallaroo.ModelConversion.ConvertXGBoostArgs: Used for XGBoost models and takes the following parameters:
    • name: The name of the model being converted.
    • comment: Any comments for the model.
    • number_of_columns: The number of columns the model was trained for.
    • input_type: A tensorflow Dtype called in the format ModelConversionInputType.{type}. See ModelConversionTypes for more details.

Once uploaded, they will be displayed in the Wallaroo Models Dashboard as {unique-file-id}-converted.onnx:

ModelConversionInputTypes

The following data types are supported with the ModelConversionInputType parameter:

sk-learn Example

The following example converts and uploads a Linear Regression sklearn model lm.pickle and stores it in the variable converted_model:

wl = wallaroo.Client()

workspace_name = "testconversion"
_ = wl.set_current_workspace(get_or_create_workspace(workspace_name))

model_conversion_args = ConvertSKLearnArguments(
    name="lm-test",
    comment="test linear regression",
    number_of_columns=NF,
    input_type=ModelConversionInputType.Double
)

model_conversion_type = ModelConversionSource.SKLEARN

# convert the model and store it in the variable `converted_model`:

converted_model = wl.convert_model('lm.pickle', model_conversion_type, model_conversion_args)

keras Example

The following example shows converting a keras model with 100 columns and uploading it to a Wallaroo instance:

model_columns = 100

model_conversion_args = ConvertKerasArguments(
    name=model_name,
    comment="simple keras model",
    input_type=ModelConversionInputType.Float32,
    dimensions=(None, model_columns)
)
model_conversion_type = ModelConversionSource.KERAS

model_wl = wl.convert_model('simple_sentiment_model.zip', model_conversion_type, model_conversion_args)
model_wl

{'name': 'simple-sentiment-model', 'version': 'c76870f8-e16b-4534-bb17-e18a3e3806d5', 'file_name': '14d9ab8d-47f4-4557-82a7-6b26cb67ab05-converted.onnx', 'last_update_time': datetime.datetime(2022, 7, 7, 16, 41, 22, 528430, tzinfo=tzutc())}