Wallaroo 101

How to Use Wallaroo from Start to Finish

This demonstration is stored as a Jupyter Notebook with sample data and the pre-trained model. It can deployed into the Jupyter Hub service that is part of your Wallaroo instance.

The Wallaroo 101 tutorial can be downloaded as part of the Wallaroo Tutorials repository.

For other tutorials with sample models and data, see the Wallaroo Tutorials page.

The following video demonstrates this tutorial:

Introduction

This tutorial is created to help you get started with Wallaroo right away. We’ll start with a brief explanation of how Wallaroo works, then provide an example with a credit card fraud detection model so you can see it working.

This guide assumes that you’ve installed Wallaroo in your cloud Kubernetes cluster. This can be either:

  • Amazon Web Services (AWS)
  • Microsoft Azure
  • Google Cloud Platform

For instructions on setting up your cloud Kubernetes environment, check out the Wallaroo Install Guides for your particular cloud provider.

User Interface Introduction

Wallaroo comes with a user interface that allows you to set up workspaces, collaborate with other users, instantly spin up a jupyter environment to access the Wallaroo SDK to deploy and manage your models. Once your models are running inside Wallaroo, you can also use the Wallaroo user interface to manage and monitor your models in real time. Full details can be found in the Wallaroo Operations Guide.

Once you’ve logged into your Wallaroo instance, you’ll be presented with the Wallaroo Dashboard which allows you to do the following:

Wallaroo Dashboard
  • A Change Current Workspace and Workspace Management: Select the workspace to work in. Each workspace has its own Models and Pipelines. For more information, see Workspace Management.
  • B Pipeline Management: View information on this workspace’s pipelines. For more information, see Pipeline Management.
  • C Model Management: View information on this workspace’s models. For more information, see Model Management.
  • D User Management: Invite users to your Wallaroo instance, or to your current workspace. For more information, see either User Management or Workspace Management.
  • E Access Jupyter Hub: Access the Jupyter Hub to run Jupyter Notebooks and shell access to your Wallaroo environment. For more information, see either the Quick Start Guides for sample Jupyter Notebooks, data and models to learn how to use Wallaroo, or the Wallaroo Developer Guides for developers.
  • F View Collaborators: Displays a list of users who have been granted to this workspace.

SDK Introduction

The Wallaroo SDK lets you quickly get your models working with your data and getting results. The typical flow follows these steps:

  • Connect: Connect to your Wallaroo Instance.
  • Create or Connect to a Workspace: Create a new workspace that will contain your models and pipelines, or connect to an existing one.
  • Upload or Use Existing Models: Upload your models to your workspace, or use ones that have already been uploaded.
  • Create or Use Existing Pipelines: Create or use an existing pipeline. This is where you’ll set the steps that will ingest your data, submit it through each successive model, then return a result.
  • Deploy Your Pipeline: Deploying a pipeline allocates resources from your Kubernetes environment for your models.
  • Run an Inference: This is where it all comes together. Submit data through your pipeline either as a file or to your pipeline’s deployment url, and get results.
  • Undeploy Your Pipeline: This returns the Kubernetes resources your pipeline used back to the Kubernetes environment.

For a more detailed rundown of the Wallaroo SDK, see the Wallaroo SDK Essentials Guide.

Introduction to Workspaces

A Wallaroo Workspace allows you to manage a set of models and pipelines for a given project or team that you are part of. You can assign users to a workspace as either an owner or collaborator.

When working within the Wallaroo SDK, the first thing you’ll do after connecting is either create a workspace or set an existing workspace as your current workspace. From that point on, all models uploaded and pipelines created or used will be in the context of the current workspace.

Introduction to Models

A Wallaroo model is a trained Machine Learning (ML) model that is uploaded to your current workspace. All models uploaded to Wallaroo are converted to ONNX to run in Wallaroo.

Introduction to Pipelines

A Wallaroo pipeline is where the deployment of models is managed. A pipeline contains a series of steps - sequential sets of models which take in the data from the preceding step, process it through the model, then return a result. Some models can be simple, such as the cc_fraud example listed below where the pipeline has only one step:

  • Step 0: Take in data
  • Step 1: Submit data to the model ccfraud-model.
  • Step Final: Return a result

Some models can be more complex with a whole series of models that process the data one after the other - and those results can be submitted to still other pipelines. You can make pipelines as simple or complex as long as it meets your needs.

Once a step is created you can add additional steps, remove a step, or swap one out until everything is running perfectly.

Credit Card Fraud Detection Example

With all of that introduction out of the way, let’s proceed to our Credit Card Detection Model.

This example will demonstrate how to use Wallaroo to deploy a model that detects credit card fraud on live credit card transactions. Using this example, you’ll learn how to:

  • Start the Wallaroo client.
  • Create a workspace.
  • Upload the credit card fraud detection model to the workspace.
  • Create a new pipeline and set it to our credit card fraud detection model.
  • Run a smoke test with a credit card transaction example to verify the pipeline and model is working properly.
  • Run a batch of inferences and display the results.
  • Undeploy the pipeline to get back the resources from our Kubernetes cluster.

This example and sample data comes from the Machine Learning Group’s demonstration on Credit Card Fraud detection. This data was used to train the model, and the sample data is used to perform an inference on the trained model.

Open a Connection to Wallaroo

The first step is to connect to Wallaroo through the Wallaroo client. The Python library is included in the Wallaroo install and available through the Jupyter Hub interface provided with your Wallaroo environment.

This is accomplished using the wallaroo.Client() command, which provides a URL to grant the SDK permission to your specific Wallaroo environment. When displayed, enter the URL into a browser and confirm permissions. Store the connection into a variable that can be referenced later.

import wallaroo
wl = wallaroo.Client()

Create a New Workspace

Next we’re going to create a new workspace called ccfraud_workspace for our model, then set it as our current workspace context.

new_workspace = wl.create_workspace("ccfraud-workspace")
wl.set_current_workspace(new_workspace)
    {
    'name': 'ccfraud-workspace',
    'id': 17,
    'archived': False,
    'created_by': 'fad23e2b-d326-48da-ba23-18ee30d26030',
    'created_at': '2022-04-05T19: 27: 31.004672+00: 00',
    'models': [],
    'pipelines': []
}

Just to make sure, let’s list our current workspace. If everything is going right, it will show us we’re in the ccfraud-workspace.

wl.get_current_workspace()
    {
    "name": "ccfraud-workspace",
    "id": 17,
    "archived": False,
    "created_by": "fad23e2b-d326-48da-ba23-18ee30d26030",
    "created_at": "2022-04-05T19:27:31.004672+00:00",
    "models": [],
    "pipelines": []
}

Upload a model

Our workspace is created. Let’s upload our credit card fraud model to it. This is the file name ccfraud.onnx, and we’ll upload it as ccfraud-model. The credit card fraud model is trained to detect credit card fraud based on a 0 to 1 model: The closer to 0 the less likely the transactions indicate fraud, while the closer to 1 the more likely the transactions indicate fraud.

Since we’re already in our default workspace ccfraud-workspace, it’ll be uploaded right to there. Once that’s done uploading, we’ll list out all of the models currently deployed so we can see it included.

ccfraud_model = wl.upload_model("ccfraud-model", "./ccfraud.onnx").configure()

We can verify that our model was uploaded by listing the models uploaded to our Wallaroo instance with the list_models() command. NOte that since we uploaded this model before, we now have different versions of it we can use for our testing.

wl.list_models()
    [
        {
            "name": "ccfraud-model",
            "version": "d145cd72-8c0c-433c-9966-5572a589f743",
            "file_name": "ccfraud.onnx",
            "last_update_time": datetime.datetime(2022,
                                            4,
                                            5,
                                            19,
                                            27,
                                            31,
                                            134114,
                                    tzinfo=tzutc())"
        },
        {
            "name": "ccfraud-model",
            "version": "3378c4cd-4238-4be4-ab62-82227ac65748",
            "file_name": "ccfraud.onnx",
            "last_update_time": datetime.datetime(2022,
                                    4,
                                    5,
                                    19,
                                    22,
                                    52,
                                    614059,
                                    tzinfo=tzutc())
        }
    ]

Create a Pipeline

With our model uploaded, time to create our pipeline and deploy it so it can accept data and process it through our ccfraud-model. We’ll call our pipeline ccfraud-pipeline.

ccfraud_pipeline = wl.build_pipeline('ccfraud-pipeline')

Now our pipeline is created. Let’s add a single step to it - in this case, our ccfraud-model that we uploaded to our workspace.

ccfraud_pipeline.add_model_step(ccfraud_model)
    {'name': 'ccfraud-pipeline', 'create_time': datetime.datetime(2022, 4, 5, 19, 27, 31, 934255, tzinfo=tzutc()), 'definition': "[{'ModelInference': {'models': [{'name': 'ccfraud-model', 'version': 'd145cd72-8c0c-433c-9966-5572a589f743', 'sha': 'bc85ce596945f876256f41515c7501c399fd97ebcb9ab3dd41bf03f8937b4507'}]}}]"}

And now we can deploy our pipeline and assign resources to it. This typically takes about 45 seconds once the command is issued.

ccfraud_pipeline.deploy()
    Waiting for deployment - this will take up to 45s ...... ok

    {
        'name': 'ccfraud-pipeline', 
        'create_time': datetime.datetime(2022, 
                                            4, 
                                            5, 
                                            19, 
                                            27, 
                                            31, 
                                            934255, 
                                            tzinfo=tzutc()), 
        'definition': "
        [
            {'ModelInference': 
                {'models': 
                    [
                        {'name': 'ccfraud-model', 
                            'version': 'd145cd72-8c0c-433c-9966-5572a589f743', 
                            'sha': 'bc85ce596945f876256f41515c7501c399fd97ebcb9ab3dd41bf03f8937b4507'
                        }
                    ]
                }
            }
        ]
    "}

We can see our new pipeline with the status() command.

ccfraud_pipeline.status()
    {'status': 'Running',
     'details': None,
     'engines': [{'ip': '10.52.0.250',
       'name': 'engine-6685bc89dc-gx8rp',
       'status': 'Running',
       'reason': None,
       'pipeline_statuses': {'pipelines': [{'id': 'ccfraud-pipeline',
          'status': 'Running'}]},
       'model_statuses': {'models': [{'name': 'ccfraud-model',
          'version': 'd145cd72-8c0c-433c-9966-5572a589f743',
          'sha': 'bc85ce596945f876256f41515c7501c399fd97ebcb9ab3dd41bf03f8937b4507',
          'status': 'Running'}]}}],
     'engine_lbs': [{'ip': '10.52.0.251',
       'name': 'engine-lb-85846c64f8-ptzq4',
       'status': 'Running',
       'reason': None}]}

With our pipeline deployed, let’s run a smoke test to make sure it’s working right. We’ll run an inference through our pipeline from the file smoke_test.json and see the results.

ccfraud_pipeline.infer_from_file('./smoke_test.json')
    Waiting for inference response - this will take up to 45s .... ok

    [InferenceResult({'check_failures': [],
      'elapsed': 175233,
      'model_name': 'ccfraud-model',
      'model_version': 'd145cd72-8c0c-433c-9966-5572a589f743',
      'original_data': {'tensor': [[1.0678324729342086,
                                    0.21778102664937624,
                                    -1.7115145261843976,
                                    0.6822857209662413,
                                    1.0138553066742804,
                                    -0.43350000129006655,
                                    0.7395859436561657,
                                    -0.28828395953577357,
                                    -0.44726268795990787,
                                    0.5146124987725894,
                                    0.3791316964287545,
                                    0.5190619748123175,
                                    -0.4904593221655364,
                                    1.1656456468728569,
                                    -0.9776307444180006,
                                    -0.6322198962519854,
                                    -0.6891477694494687,
                                    0.17833178574255615,
                                    0.1397992467197424,
                                    -0.35542206494183326,
                                    0.4394217876939808,
                                    1.4588397511627804,
                                    -0.3886829614721505,
                                    0.4353492889350186,
                                    1.7420053483337177,
                                    -0.4434654615252943,
                                    -0.15157478906219238,
                                    -0.26684517248765616,
                                    -1.454961775612449]]},
      'outputs': [{'Float': {'data': [0.001497417688369751],
                             'dim': [1, 1],
                             'v': 1}}],
      'pipeline_name': 'ccfraud-pipeline',
      'time': 1649186867217})]

Looks good! Time to run the real test on some real data. Run another inference this time from the file high_fraud.json and let’s see the results:

ccfraud_pipeline.infer_from_file('./high_fraud.json')
    [InferenceResult({'check_failures': [],
      'elapsed': 132117,
      'model_name': 'ccfraud-model',
      'model_version': 'd145cd72-8c0c-433c-9966-5572a589f743',
      'original_data': {'tensor': [[1.0678324729342086,
                                    18.155556397512136,
                                    -1.658955105843852,
                                    5.2111788045436445,
                                    2.345247064454334,
                                    10.467083577773014,
                                    5.0925820522419745,
                                    12.82951536371218,
                                    4.953677046849403,
                                    2.3934736228338225,
                                    23.912131817957253,
                                    1.7599568310350209,
                                    0.8561037518143335,
                                    1.1656456468728569,
                                    0.5395988813934498,
                                    0.7784221343010385,
                                    6.75806107274245,
                                    3.927411847659908,
                                    12.462178276650056,
                                    12.307538216518656,
                                    13.787951906620115,
                                    1.4588397511627804,
                                    3.681834686805714,
                                    1.753914366037974,
                                    8.484355003656184,
                                    14.6454097666836,
                                    26.852377436250144,
                                    2.716529237720336,
                                    3.061195706890285]]},
      'outputs': [{'Float': {'data': [0.9811990261077881], 'dim': [1, 1], 'v': 1}}],
      'pipeline_name': 'ccfraud-pipeline',
      'time': 1649186867233})]

With our work in the pipeline done, we’ll undeploy it to free up computational resources. If we keep the same settings we can redeploy the pipeline with the same configuration in the future.

When using the free community edition of Wallaroo, users will have a limited number of active concurrent pipelines per instance. As a best practice, make sure you stop your pipelines when you complete your experiments.

ccfraud_pipeline.undeploy()
    {
        'name': 'ccfraud-pipeline',
        'create_time': datetime.datetime(2022,
                        4,
                        5,
                        19,
                        27,
                        31,
                        934255,
                    tzinfo=tzutc()),
        'definition': "[
            {'ModelInference': 
                {'models': 
                    [
                        {
                            'name': 'ccfraud-model',
                            'version': 'd145cd72-8c0c-433c-9966-5572a589f743',
                            'sha': 'bc85ce596945f876256f41515c7501c399fd97ebcb9ab3dd41bf03f8937b4507'
                        }
                    ]
                }
            }
        ]"
    }

And there we have it! Feel free to use this as a template for other models and pipelines that you want to deploy and manage with Wallaroo!