Inference via the Wallaroo SDK
How to perform inferences on deployed LLMs via the Wallaroo SDK.
LLMs
How to perform inferences for pipelines with Large Language Models models deployed in Wallaroo.
Inference via OpenAI Compatibility Deployments
How to perform inferences on deployed pipelines with OpenAI Compatibility enabled.