Large Language Models Operations

Wallaroo supports the deployment of Large Language Models (LLM), used by various industries to provide answers and support to end users. These models range in size from a few hundred megabytes to over hundreds of gigabytes, using CPU and often GPU resources for performative model operations.

For access to these sample models and a demonstration on using LLMs with Wallaroo:

Wallaroo LLM Inference Platform