Inference on Any Hardware

How to deploy and perform inferences on any hardware with Wallaroo

Wallaroo supports deploying models on any hardware and infrastructure, including:

  • CPU hardware: Deploy and infer on either x86 or ARM architectures.
  • GPUs: Leverage GPUs to increase model performance.
  • AI Accelerators: Deploy models with hardware AI acceleration libraries to optimize performance.

Inference on ARM Architecture

How to deploy ML models with ARM processors and infrastructure.

Inference with GPUs

How to use package models to run on GPUs

Inference with Acceleration Libraries

How to use package models to run with hardware accelerators