Computer Vision Tutorials

This tutorial and the assets can be downloaded as part of the Wallaroo Tutorials repository.

Step 00: Introduction and Setup

This tutorial demonstrates how to use the Wallaroo to detect objects in images through the following models:

• rnn mobilenet: A single stage object detector that performs fast inferences. Mobilenet is typically good at identifying objects at a distance.
• resnet50: A dual stage object detector with slower inferencing but but is able to detect objects that are closer to each other.

This tutorial series will demonstrate the following:

• How to deploy a Wallaroo pipeline with trained rnn mobilenet model and perform sample inferences to detect objects in pictures, then display those objects.
• How to deploy a Wallaroo pipeline with a trained resnet50 model and perform sample inferences to detect objects in pictures, then display those objects.
• Use the Wallaroo feature shadow deploy to have both models perform inferences, then select the inference result with the higher confidence and show the objects detected.

This tutorial assumes that users have installed the Wallaroo SDK or are running these tutorials from within their Wallaroo instance’s JupyterHub service.

This demonstration should be run within a Wallaroo JupyterHub instance for best results.

Prerequisites

The included OpenCV class is included in this demonstration as CVDemoUtils.py, and requires the following dependencies:

• ffmpeg
• libsm
• libxext

Internal JupyterHub Service

To install these dependencies in the Wallaroo JupyterHub service, use the following commands from a terminal shell via the following procedure:

1. Launch the JupyterHub Service within the Wallaroo install.

2. Select File->New->Terminal.

3. Enter the following:

   sudo apt-get update
   

   sudo apt-get install ffmpeg libsm6 libxext6  -y
   

External SDK Users

For users using the Wallaroo SDK to connect with a remote Wallaroo instance, the following commands will install the required dependancies:

For Linux users, this can be installed with:

sudo apt-get update
sudo apt-get install ffmpeg libsm6 libxext6  -y

MacOS users can prepare their environments using a package manager such as Brew with the following:

brew install ffmpeg libsm libxext

Libraries and Dependencies

1. This repository may use large file sizes for the models. If necessary, install Git Large File Storage (LFS) or use the Wallaroo Tutorials Releases to download a .zip file of the most recent computer vision tutorial that includes the models.
2. Import the following Python libraries into your environment:
   1. torch
   2. wallaroo
   3. torchvision
   4. opencv-python
   5. onnx
   6. onnxruntime
   7. imutils
   8. pytz
   9. ipywidgets

These can be installed by running the command below in the Wallaroo JupyterHub service. Note the use of pip install torch --no-cache-dir for low memory environments.

!pip install torchvision
!pip install torch --no-cache-dir
!pip install opencv-python
!pip install onnx
!pip install onnxruntime
!pip install imutils
!pip install pytz
!pip install ipywidgets

The rest of the tutorials will rely on these libraries and applications, so finish their installation before running the tutorials in this series.

Models for Wallaroo Computer Vision Tutorials

In order for the wallaroo tutorial notebooks to run properly, models must be downloaded to the models directory. As they are too large for Git to contain without extra steps, this tutorial and all models are available as a separate download. These are downloaded via the following procedure:

1. Go to https://github.com/WallarooLabs/Wallaroo_Tutorials/releases.
2. Select the most recent release.
3. Download the file computer_vision.zip.

This contains the entire tutorial, plus the model files. The most current version of this link is there:

https://github.com/WallarooLabs/Wallaroo_Tutorials/releases/download/1.27-2022.4-cv4/computer_vision.zip