Prerequisites
Have a Microsoft Azure account and access to the Microsoft Azure marketplace.
Installation Procedure
To install a Wallaroo Inference Server from the Azure Marketplace, complete the following steps. The installation steps are broken up by each Azure installation section.
The settings below are the minimum viable options for setting up a Wallaroo Inference Server for Azure Marketplace on an Azure Virtual Machine (VM) in the least amount of steps and time. See full Microsoft Azure VM settings, see the Virtual machines in Azure documentation.
Overview
From the Overview page, select the following:
- Plan: Select which Wallaroo Inference Server Free Edition plan. These include:
- Wallaroo.AI HF Summarizer Standard - x64
- Wallaroo.AI HF Summarizer Standard - GPU x64
- Wallaroo.AI ResNet50 Inference Server - x64
- Wallaroo.AI ResNet50 Inference Server - Arm64
- Wallaroo.AI Yolo v8 Inference Server - x64
- Wallaroo.AI Yolo v8 Inference Server - GPU
- Wallaroo.AI Faster R-CNN - x64
- Wallaroo.AI Faster R-CNN - GPU
- Wallaroo.AI Llama Inference Server - GPU
For most installations, select Start with a pre-set configuration. To fully customize, select Create. The steps below assume that Start with a pre-set configuration was selected.
Note that GPU inference server require a VM with Nvidia GPU cuda
support.
Pre-set Configuration
Most installations of the Wallaroo Inference Server Free Edition performs on minimum hardware. Use te settings Dev/Test with Compute optimized (F-Series) for a low cost option.
Select Continue to create a VM.
Basic
The Basics tab sets the name, region, and other settings.
The following options are the minimum recommended options. Modify as per your organization’s requirements.
- Project Details: Specify how the resource and costs are managed.
- Subscription: Select the Azure subscription that manages the deployed resources.
- Resource group: Select the resource group that will manage the resource.
- Instance Details: Specify the VM resources used.
- Virtual Machine Name: Set the name of the virtual machine.
- Region: Set the region.
- Availability Options: Set to No infrastructure redundancy required.
- Security type: Set to Standard.
- Image: Do not change - this is auto-selected for the appropriate image for the Wallaroo Inference Server Free Edition.
- VM architecture: Leave as x64.
- Size: Select the best size for your organization. A recommended minimum size is
Standard_F4s_v2
which provides 4 vcpus with 8 GiB RAM.
- Administrator account: Set the administrative options for this VM.
- Authentication type: Select the best option for your organization. The default option SSH public key is sufficient.
- Username: Set a username. The default is
azureuser
. - SSH public key source: Select whether to use an existing key pair or generate a new one. NOTE: If generated, the SSH key will be available for downloading and authenticating via
ssh
. Store this key in a secure location.
Disks
Keep the default settings and select Next: Networking.
Networking
Keep the default settings and select Next: Management.
Management
If Basic->Availability Options is set to No infrastructure redundancy required, set Enable Automanage to Disabled.
Select Next: Monitoring.
Monitoring
Keep the default settings and select Next: Advanced.
Advanced
Keep the default settings and select Next: Tags.
Tags
Keep the default settings and select Next: Review + create.
Review + create
Verify the details, and select Create.
The process will take 5-10 minutes to complete the process.
Post Installation Steps
The following steps are optional, but recommended to make accessing the new Wallaroo Inference Server.
Host Name Addition
To specify a host name for the Wallaroo Inference Server:
- From the Microsoft Azure dashboard, access the Virtual machines dashboard. Select the virtual machine hosting the Wallaroo Inference Server.
- From the Overview page, select DNS Name.
- From the Settings->Configuration screen, set the DNS name label (optional) field. By default the host name of the virtual machine will be the label plus the region. For example, if the label is
sample-hf-wallaroo
and the region iswestus2
, then the host name will besample-hf-wallaroo.westus2.cloudapp.azure.com
.- If the host name is not unique, an error will be displayed.
- Select Save to store the settings.
The Wallaroo Inference Server will be available from the specified host name.
Supplemental Material
- Wallaroo Inference Server Free Edition Tutorials for tutorials on performing inferences and other requests.