We are pleased to announce the following product improvements in our 2024.4 release:

• Python Version 3.10 Support: Wallaroo Enterprise and the Wallaroo SDK support Python 3.10. This provides support for newer Python 3.10 enhanced libraries for Custom Model, inference automation, and other options.
  • Supported Model Versions and Libraries
  • Python Models
  • Custom Model aka Bring Your Own Predict (BYOP) Models
  • Inference Automation
  • Wallaroo SDK Install Guides
• LLM validation and monitoring with Harmful Language Listeners (Beta): LLM monitoring and validation can now support Harmful Language detection. These listeners can deployed in a multi-step inference pipeline with a LLM to provide instant harmful language scoring with reasoning on LLM inferences.
  • LLM Validation Listeners: Harmful Language Listener
  • LLM Harmful Language Listener Tutorial
• Infrastructure and installation updates for Wallaroo Community Edition (CE): Wallaroo Community Edition (CE) is available for installation in GitHub Codespaces and Oracle Cloud Infrastructure (OCI) Marketplace starting at 8 Cpus, allowing more people to try the free version of Wallaroo on lower compute footprint.
  • Wallaroo Community GitHub Codespaces Install Guide
  • Wallaroo Community Oracle Cloud Infrastructure (OCI) Marketplace Setup
• Replicas Reporting in Wallaroo Dashboard: The Wallaroo Dashboard is updated to display the number of replicas deployed as part of the deployment configuration and model use.
  • Retrieve Inference Metrics and Logs Via the Wallaroo Dashboard
• IBM Power Integration: Wallaroo supports IBM Power 10 for model deployment and inference in the cloud, on-prem and at the edge.
  • Inference on IBM Power10 Architecture
  • Llamacpp Deploy on IBM Power10 Tutorial
  • Run Anywhere: Deploy and Publish Computer Vision Model Resnet50 for IBM Power10
  • Run Anywhere: Deploy and Publish Computer Vision Model Resnet50 with Post Processing for IBM Power10