The following tutorial is available on the Wallaroo Github Repository.
This notebook is used in conjunction with the Wallaroo Inference Server Free Edition for Hugging Face Summarizer. This provides a free license for performing inferences through the Computer Vision YoloV8n model.
To run this tutorial in the Wallaroo JupyterHub Service, import the tensorflow-cpu
library by executing the following command in the terminal shell:
pip install tensorflow-cpu==2.13.1 --user
Then proceed with the tutorial. This only applies to running this tutorial in Wallaroo’s JupyterHub service, and does not affect model upload and packaging in Wallaroo.
Note that GPU inference server require a VM with Nvidia GPU CUDA support.
The Resnet Model takes the following inputs.
Field | Type | Description |
---|---|---|
tensor | Float | Tensor in the shape (n, 3, 480, 640) float. This is the normalized pixel values of the 640x480 color image. |
Field | Type | Description |
---|---|---|
output0 | Variable length List[Float] | A flattened numpy array of detected objects. When reshaped into a (1, 84, 8400) returns where the bounding boxes for each detected object are elements [0:3] representing (x_coordinate, y_coordinate, width, height), the classes and scores are in elements [4:] . |
The following HTTPS API endpoints are available for Wallaroo Inference Server.
/pipelines
pipelines
with the following fields.Running
indicates the pipeline is available for inferences.The following demonstrates using curl
to retrieve the Pipelines endpoint. Replace the HOSTNAME with the address of your Wallaroo Inference Server.
!curl HOSTNAME:8080/pipelines
{"pipelines":[{"id":"yolo-v8","status":"Running"}]}
/models
models
with the following fields.sha
hash of the model.Running
indicates the models is available for inferences.The following demonstrates using curl
to retrieve the Models endpoint. Replace the HOSTNAME with the address of your Wallaroo Inference Server.
!curl HOSTNAME:8080/models
{"models":[{"name":"yolo-v8","sha":"3ed5cd199e0e6e419bd3d474cf74f2e378aacbf586e40f24d1f8c89c2c476a08","status":"Running","version":"af82c216-4590-41ad-8579-48b7eccc7144"}]}
The following endpoints are available from the Wallaroo Server for Computer Vision Yolov8n deployment.
/pipelines/hf-summarizer-standard
Content-Type: application/vnd.apache.arrow.file
: For Apache Arrow tables.Content-Type: application/json; format=pandas-records
: For pandas DataFrame in record format.application/json; format=pandas-records
OR Apache Arrow table in application/vnd.apache.arrow.file
with the shape (n, 3, 640, 640)
then flattened, with the tensor values in the field images
.The following code is used to create a DataFrame from a 640x640 image.
import cv2
import torch
import numpy as np
import pandas as pd
# load the image from disk, convert to BGR, resize to specified width, height, convert the image back to RGB
# convert the image to a float tensor and returns it. Also return the original resized image for drawing bounding boxes in BGR
def imageResize(image, 640, 640):
#self.print("Image Mode:"+image.mode)
im_pillow = np.array(image)
image = cv2.cvtColor(im_pillow, cv2.COLOR_BGR2RGB) #scott
image = cv2.flip(im_pillow, 1)
image = cv2.flip(image, 1)
#image = cv2.imread(im_pillow)
#image = cv2.cvtColor(image, cv2.COLOR_RGB2BGR)
#image = cv2.cvtColor(im_pillow, cv2.COLOR_GRAY2BGR)
self.debug("Resizing to w:"+str(width) + " height:"+str(height))
image = cv2.resize(image, (width, height))
# convert the image from BGR to RGB channel ordering and change the
# image from channels last to channels first ordering
#image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
image = image.transpose((2, 0, 1))
# add the batch dimension, scale the raw pixel intensities to the
# range [0, 1], and convert the image to a floating point tensor
image = np.expand_dims(image, axis=0)
image = image / 255.0
tensor = torch.FloatTensor(image)
tensor.flatten()
npArray = tensor.cpu().numpy()
dictData = {"images":[npArray]}
dataframedata = pd.DataFrame(dictData)
Content-Type: application/json; format=pandas-records
: pandas DataFrame in record format.time (Integer): The time since UNIX epoch.
in: The original input.
out: The outputs of the inference result separated by data type.
output0: The float outputs for the inference. This list is flattened, and when reshaped into (1,84,8400)
with each row correlating to a detected object. The elements break down as follows:
For more details for breaking down the Yolo8n inference results into objects, see the CVDemoUtils.py
module with the Computer Vision Yolov8n Deployment in Wallaroo
check_failures (List[Integer]): Whether any validation checks were triggered. For more information, see Wallaroo SDK Essentials Guide: Pipeline Management: Anomaly Testing.
metadata: Additional data for the inference.
The Wallaroo Inference Server accepts pandas DataFrame or Apache Arrow tables as inference inputs. The sample file ./data/dogbike.df.json
was converted from the file ./data/dogbike.png
as an example using the helper module CVDemoUtils
and WallarooUtils
are used to transform a sample image into a pandas DataFrame. This DataFrame is then submitted to the Yolov8n model deployed in Wallaroo.
The following code segment demonstrates converting the image to a DataFrame.
from CVDemoUtils import CVDemo
from WallarooUtils import Util
cvDemo = CVDemo()
util = Util()
width, height = 640, 640
tensor1, resizedImage1 = cvDemo.loadImageAndResize('./data/dogbike.png', width, height)
tensor1.flatten()
# add the tensor to a DataFrame and save the DataFrame in pandas record format
df = util.convert_data(tensor1,'images')
df.to_json("dogbike.df.json", orient = 'records')
The following code segment demonstrates performing an inference through the Wallaroo Inference Server with the Yolov8n model deployed. Replace HOSTNAME
with the hostname or IP address of your Wallaroo Inference Server instance.
!curl -X POST HOSTNAME:8080/pipelines/yolo-v8 \
-H "Content-Type: application/json; format=pandas-records" \
-d @./data/dogbike.df.json > edge-results.df.json
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 28.7M 100 13.6M 100 15.0M 12.7M 14.1M 0:00:03 0:00:01 0:00:02 18.9M01 0:00:01 --:--:-- 27.0M