PyTorch

During the model upload process, Wallaroo optimizes models by converting them to the Wallaroo Native Runtime, if possible, or running the model directly in the Wallaroo Containerized Runtime. See the Model Deploy for details on how to configure pipeline resources based on the model’s runtime.

• IMPORTANT CONFIGURATION NOTE: For PyTorch input schemas, the floats must be pyarrow.float32() for the PyTorch model to be converted to the Native Wallaroo Runtime during the upload process.

PyTorch Input and Output Schemas

PyTorch input and output schemas have additional requirements depending on whether the PyTorch model is single input/output or multiple input/output. This refers to the number of columns:

• Single Input/Output: Has one input and one output column.
• Multiple Input/Output: Has more than one input or more than one output column.

The column names for the model can be anything. For example:

• Model Input Fields:
  • length
  • width
  • intensity
  • etc

When creating the input and output schemas for uploading a PyTorch model in Wallaroo, the field names must match the following requirements. For example, for multi-column PyTorch models, the input would be:

• Data Schema Input Fields:
  • input_1
  • input_2
  • input_3
  • input_...

For single input/output PyTorch model, the field names must be input and output. For example, if the input field is a List of Floats of size 10, and the output field is a list of floats of list size one, the input and output schemas are:

input_schema = pa.schema([
    pa.field('input', pa.list_(pa.float32(), list_size=10))
])

output_schema = pa.schema([
    pa.field('output', pa.list_(pa.float32(), list_size=1))
])

For multi input/output PyTorch models, the data schemas for each input and output field must be named input_1, input_2... and output_1, output_2, etc. These must be in the same order that the PyTorch model is trained to accept them.

For example, a multi input/output PyTorch model that takes the following inputs and outputs:

• Inputs
  • input_1: List of Floats of length 10.
  • input_2: List of Floats of length 5.
• Outputs
  • output_1: List of Floats of length 3.
  • output_2: List of Floats of length 2.

The following input and output schemas would be used.

input_schema = pa.schema([
    pa.field('input_1', pa.list_(pa.float32(), list_size=10)),
    pa.field('input_2', pa.list_(pa.float32(), list_size=5))
])
output_schema = pa.schema([
    pa.field('output_1', pa.list_(pa.float32(), list_size=3)),
    pa.field('output_2', pa.list_(pa.float32(), list_size=2))
])