Applications

Guides on deploying and inferencing on application based models such as OpenAI and other managed inference sources.

Deploy LLMs with OpenAI Compatibility

Wallaroo provides OpenAI compatibility for improved interactive user experiences with LLM-based applications while taking advantage of Wallaroo’s ability to maximize throughput and optimizing latency. AI developers can seamlessly migrate their applications from OpenAI endpoints to Wallaroo endpoints endpoints to Wallaroo on-prem endpoints, in connected and air-gapped environments, without losing any functionality..
For access to sample models and a demonstration on using LLMs with Wallaroo:

Managed LLM Inference Endpoints (MaaS) in Wallaroo