Wallaroo MLOps API Essentials Guide: Model Management

How to use the Wallaroo API for Model Management

Model Naming Requirements

Model names map onto Kubernetes objects, and must be DNS compliant. The strings for model names must be ASCII alpha-numeric characters or dash (-) only. . and _ are not allowed.

Models

Upload Model to Workspace

ML Models are uploaded to Wallaroo through the following endpoint:

Models uploaded through this method that are not native runtimes are containerized within the Wallaroo instance then run by the Wallaroo engine. See Wallaroo MLOps API Essentials Guide: Pipeline Management for details on pipeline configurations and deployments.

For these models, the following inputs are required.

Endpoint:
- /v1/api/models/upload_and_convert
Headers:
- Content-Type: multipart/form-data
Parameters
- name (String Required): The model name.
- visibility (String Required): Either public or private.
- workspace_id (String Required): The numerical ID of the workspace to upload the model to.
- conversion (String Required): The conversion parameters that include the following:
  - framework (String Required): The framework of the model being uploaded. See the list of supported models for more details.
  - python_version (String Required): The version of Python required for model.
  - requirements (String Required): Required libraries. Can be [] if the requirements are default Wallaroo JupyterHub libraries.
  - input_schema (String Optional): The input schema from the Apache Arrow pyarrow.lib.Schema format, encoded with base64.b64encode. Only required for non-native runtime models.
  - output_schema (String Optional): The output schema from the Apache Arrow pyarrow.lib.Schema format, encoded with base64.b64encode. Only required for non-native runtime models.

Upload Native Runtime Model Example

ONNX are always native runtimes. The following example shows uploading an ONNX model to a Wallaroo instance using the requests library. Note that the input_schema and output_schema encoded details are not required.

 authorization header
headers = {'Authorization': 'Bearer abcdefg'}

apiRequest = f"{APIURL}/v1/api/models/upload_and_convert"

framework='onnx'

model_name = f"{suffix}ccfraud"

data = {
    "name": model_name,
    "visibility": "public",
    "workspace_id": workspaceId,
    "conversion": {
        "framework": framework,
        "python_version": "3.8",
        "requirements": []
    }
}

files = {
    "metadata": (None, json.dumps(data), "application/json"),
    'file': (model_name, open('./ccfraud.onnx', 'rb'), "application/octet-stream")
    }

response = requests.post(apiRequest, files=files, headers=headers).json()

'Sample model name: apimodel'
‘Sample model file: ./models/ccfraud.onnx’
{‘insert_models’: {‘returning’: [{‘models’: [{‘id’: 4}]}]}}

Upload Converted Model Examples

The following example shows uploading a Hugging Face model to a Wallaroo instance using the requests library. Note that the input_schema and output_schema encoded details are required.

input_schema = pa.schema([
    pa.field('inputs', pa.string()), # required
    pa.field('candidate_labels', pa.list_(pa.string(), list_size=2)), # required
    pa.field('hypothesis_template', pa.string()), # optional
    pa.field('multi_label', pa.bool_()), # optional
])

output_schema = pa.schema([
    pa.field('sequence', pa.string()),
    pa.field('scores', pa.list_(pa.float64(), list_size=2)), # same as number of candidate labels, list_size can be skipped by may result in slightly worse performance
    pa.field('labels', pa.list_(pa.string(), list_size=2)), # same as number of candidate labels, list_size can be skipped by may result in slightly worse performance
])

encoded_input_schema = base64.b64encode(
                bytes(input_schema.serialize())
            ).decode("utf8")

encoded_output_schema = base64.b64encode(
                bytes(output_schema.serialize())
            ).decode("utf8")

metadata = {
    "name": model_name,
    "visibility": "private",
    "workspace_id": workspace_id,
    "conversion": {
        "framework": framework,
        "python_version": "3.8",
        "requirements": []
    },
    "input_schema": encoded_input_schema,
    "output_schema": encoded_output_schema,
}

headers = wl.auth.auth_header()

files = {
    'metadata': (None, json.dumps(metadata), "application/json"),
    'file': (model_name, open(model_path,'rb'),'application/octet-stream')
}

response = requests.post('https://{APIURL}/v1/api/models/upload_and_convert', 
                         headers=headers, 
                         files=files).json()

Stream Upload Model to Workspace

Streams a potentially large ML Model to a Wallaroo workspace via POST with Content-Type: multipart/form-data.

Parameters
- name - (REQUIRED string): Name of the model. Must only include alphanumeric characters.
- filename - (REQUIRED string): Name of the file being uploaded.
- visibility - (OPTIONAL string): The visibility of the model as either public or private.
- workspace_id - (REQUIRED int): The numerical id of the workspace to upload the model to.

Example: This example will upload the sample file ccfraud.onnx to the workspace created in the Create Workspace step as apitestmodel.

# stream upload model - next test is adding arbitrary chunks to the stream

# Retrieve the token 
headers = wl.auth.auth_header()

# Set the contentType
headers['contentType']='application/octet-stream'

api_request = f"{APIURL}/v1/api/models/upload_stream"

# Model name and file to use
display(f"Sample stream model name: {stream_model_name}")
display(f"Sample model file: {stream_model_file_name}")

data = {
    "name":stream_model_name,
    "filename": stream_model_file_name,
    "visibility":"public",
    "workspace_id": example_workspace_id
}

files = {
    'file': (stream_model_name, open(stream_model_file_name, 'rb'))
    }

response = requests.post(apiRequest, files=files, data=data, headers=headers).json()
response

'Sample stream model name: apiteststreammodel'
‘Sample model file: ./models/ccfraud.onnx’
{‘insert_models’: {‘returning’: [{‘models’: [{‘id’: 5}]}]}}

List Models in Workspace

Returns a list of models added to a specific workspace.

Parameters
- workspace_id - (REQUIRED int): The workspace id to list.

Example: Display the models for the workspace used in the Upload Model to Workspace step. The model id and model name will be saved as example_model_id and exampleModelName variables for other examples.

# List models in a workspace
# Retrieve the token 
headers = wl.auth.auth_header()

api_request = f"{APIURL}/v1/api/models/list"

data = {
  "workspace_id": example_workspace_id
}

response = requests.post(api_request, json=data, headers=headers, verify=True).json()
response

{'models': [{'id': 5,
   'name': 'apiteststreammodel',
   'owner_id': '""',
   'created_at': '2023-05-17T20:51:53.077997+00:00',
   'updated_at': '2023-05-17T20:51:53.077997+00:00'},
  {'id': 4,
   'name': 'apimodel',
   'owner_id': '""',
   'created_at': '2023-05-17T20:51:51.092416+00:00',
   'updated_at': '2023-05-17T20:51:51.092416+00:00'}]}

model = next(model for model in response["models"] if model["name"] == "apimodel")
example_model_id = model['id']

Get Model Details By ID

Returns the model details by the specific model id.

Parameters
- workspace_id - (REQUIRED int): The workspace id to list.
Returns
- id - (int): Numerical id of the model.
- owner_id - (string): Id of the owner of the model.
- workspace_id - (int): Numerical of the id the model is in.
- name - (string): Name of the model.
- updated_at - (DateTime): Date and time of the model’s last update.
- created_at - (DateTime): Date and time of the model’s creation.
- model_config - (string): Details of the model’s configuration.

Example: Retrieve the details for the model uploaded in the Upload Model to Workspace step.

# Get model details by id
# Retrieve the token 
headers = wl.auth.auth_header()

api_request = f"{APIURL}/v1/api/models/get_by_id"

data = {
  "id": example_model_id
}

response = requests.post(api_request, json=data, headers=headers, verify=True).json()
response

{'id': 4,
 'owner_id': '""',
 'workspace_id': 7,
 'name': 'apimodel',
 'updated_at': '2023-05-17T20:51:51.092416+00:00',
 'created_at': '2023-05-17T20:51:51.092416+00:00',
 'model_config': None}

Get Model Versions

Retrieves all versions of a model based on either the name of the model or the model_pk_id.

Parameters
- model_id - (REQUIRED String): The model name.
- models_pk_id - (REQUIRED int): The model integer pk id.
Returns
- Array(Model Details)
  - sha - (String): The sha hash of the model version.
  - models_pk_id- (int): The pk id of the model.
  - model_version - (String): The UUID identifier of the model version.
  - owner_id - (String): The Keycloak user id of the model’s owner.
  - model_id - (String): The name of the model.
  - id - (int): The integer id of the model.
  - file_name - (String): The filename used when uploading the model.
  - image_path - (String): The image path of the model.

Example: Retrieve the versions for a previously uploaded model. The variables example_model_version and example_model_sha will store the model’s version and SHA values for use in other examples.

## List model versions

# Retrieve the token 
headers = wl.auth.auth_header()
api_request = f"{APIURL}/v1/api/models/list_versions"

data = {
  "model_id": model_name,
  "models_pk_id": example_model_id
}

response = requests.post(api_request, json=data, headers=headers, verify=True).json()
response

[{'sha': 'bc85ce596945f876256f41515c7501c399fd97ebcb9ab3dd41bf03f8937b4507',
  'models_pk_id': 4,
  'model_version': '0b989008-5f1d-453e-8085-98d97be1b722',
  'owner_id': '""',
  'model_id': 'apimodel',
  'id': 4,
  'file_name': 'apimodel',
  'image_path': None,
  'status': 'ready'}]

# Stored for future examples

example_model_version = response[-1]['model_version']
example_model_sha = response[-1]['sha']

Get Model Configuration by Id

Returns the model’s configuration details.

Parameters
- model_id - (REQUIRED int): The numerical value of the model’s id.

Example: Submit the model id for the model uploaded in the Upload Model to Workspace step to retrieve configuration details.

## Get model config by id

# Retrieve the token 
headers = wl.auth.auth_header()
api_request = f"{APIURL}/v1/api/models/get_config_by_id"

data = {
  "model_id": example_model_id
}

response = requests.post(api_request, json=data, headers=headers, verify=True).json()
response

{'model_config': None}

Get Model Details

Returns details regarding a single model, including versions.

Returns the model’s configuration details.

Parameters
- model_id - (REQUIRED int): The numerical value of the model’s id.

Example: Submit the model id for the model uploaded in the Upload Model to Workspace step to retrieve configuration details.

# Get model config by id
# Retrieve the token 
headers = wl.auth.auth_header()
api_request = f"{APIURL}/v1/api/models/get"

data = {
  "id": example_model_id
}

response = requests.post(api_request, json=data, headers=headers, verify=True).json()
response

{'id': 4,
 'name': 'apimodel',
 'owner_id': '""',
 'created_at': '2023-05-17T20:51:51.092416+00:00',
 'updated_at': '2023-05-17T20:51:51.092416+00:00',
 'models': [{'sha': 'bc85ce596945f876256f41515c7501c399fd97ebcb9ab3dd41bf03f8937b4507',
   'models_pk_id': 4,
   'model_version': '0b989008-5f1d-453e-8085-98d97be1b722',
   'owner_id': '""',
   'model_id': 'apimodel',
   'id': 4,
   'file_name': 'apimodel',
   'image_path': None}]}