# Performing completions

pipeline.openai_chat_completion(messages=[{"role": "user", "content": "good morning"}]).choices[0].message.content

'Hello! How can I assist you today?'

pipeline.openai_completion(prompt="tell me about wallaroo.AI", max_tokens=200).choices[0].text

'\nWallaroo.ai is a cloud-based platform that provides a suite of artificial intelligence (AI) and machine learning (ML) tools for data scientists, developers, and business users. The platform is designed to simplify the process of building, deploying, and managing AI and ML models, making it easier for organizations to leverage the power of AI to drive business outcomes.\nKey Features of Wallaroo.ai:\n1. Model Development: Wallaroo.ai provides a range of tools and libraries for building, training, and deploying AI and ML models, including support for popular frameworks like TensorFlow, PyTorch, and scikit-learn.\n2. Model Serving: The platform offers a scalable and secure model serving layer that allows users to deploy and manage AI and ML models in production environments.\n3. Data Integration: Wallaroo.ai provides seamless integration with various data sources, including relational databases, NoSQL databases, cloud storage, and streaming data sources.\n4. Model Monitoring: The platform offers real-time monitoring and analytics capabilities'

# Now with streaming

for chunk in pipeline.openai_chat_completion(messages=[{"role": "user", "content": "this is a short story about love"}], max_tokens=100, stream=True):
    print(chunk.choices[0].delta.content, end="", flush=True)

I'm happy to help you with your short story about love. What kind of love are you writing about? Is it romantic love, familial love, or something else?

# Now with streaming

for chunk in pipeline.openai_completion(prompt="tell me about wallaroo.AI", max_tokens=200, stream=True):
    print(chunk.choices[0].text, end="", flush=True)

Wallaroo.ai is an AI platform that enables developers to build, deploy, and manage AI-powered applications with ease. Here's a brief overview of what it offers:
Key Features of Wallaroo.ai:
1. **Model Serving**: Wallaroo.ai provides a scalable and secure model serving platform that allows developers to deploy and manage AI models in production environments.
2. **Model Management**: The platform offers a centralized model management system that enables developers to manage multiple models, track performance, and monitor metrics.
3. **Auto-Scaling**: Wallaroo.ai's auto-scaling feature ensures that AI models can handle 맞 variable workloads, ensuring high performance and availability.
4. **Security**: The platform provides robust security features, including encryption, access control, and auditing, to protect sensitive data and models.
5. **Integration**: Wallaroo.ai supports integration with popular AI frameworks, such as TensorFlow, PyTorch, and scikit-learn, making it easy to deploy and manage AI models.
6

token = wl.auth.auth_header()['Authorization'].split()[1]
token

'abc123'

# Streaming: Completion
!curl -X POST \
  -H "Authorization: Bearer abc123" \
  -H "Content-Type: application/json" \
  -d '{"model": "whatever", "prompt": "tell me a short story", "max_tokens": 100, "stream": true, "stream_options": {"include_usage": true}}' \
  https://qaic-poc.pov.wallaroo.io/v1/api/pipelines/infer/llamaqaic-openai-32/llamaqaic-openai/openai/v1/completions

data: {"id":"cmpl-5a1adc32e65849f2aee5edf2e37fdb7a","created":1750125678,"model":"llama-31-8b.zip","choices":[{"text":" about","index":0,"logprobs":null,"finish_reason":null,"stop_reason":null}],"usage":{"prompt_tokens":6,"completion_tokens":1,"total_tokens":7,"ttft":0.091563446,"tps":10.921388869527693}}

data: {"id":"cmpl-5a1adc32e65849f2aee5edf2e37fdb7a","created":1750125678,"model":"llama-31-8b.zip","choices":[{"text":" a","index":0,"logprobs":null,"finish_reason":null,"stop_reason":null}],"usage":{"prompt_tokens":6,"completion_tokens":2,"total_tokens":8,"ttft":0.091563446,"tps":16.38253480899562}}

data: {"id":"cmpl-5a1adc32e65849f2aee5edf2e37fdb7a","created":1750125678,"model":"llama-31-8b.zip","choices":[{"text":" character","index":0,"logprobs":null,"finish_reason":null,"stop_reason":null}],"usage":{"prompt_tokens":6,"completion_tokens":3,"total_tokens":9,"ttft":0.091563446,"tps":16.485729285058817}}

...

data: {"id":"cmpl-5a1adc32e65849f2aee5edf2e37fdb7a","created":1750125678,"model":"llama-31-8b.zip","choices":[{"text":" mere","index":0,"logprobs":null,"finish_reason":null,"stop_reason":null}],"usage":{"prompt_tokens":6,"completion_tokens":99,"total_tokens":105,"ttft":0.091563446,"tps":10.019649983937533}}

data: {"id":"cmpl-5a1adc32e65849f2aee5edf2e37fdb7a","created":1750125678,"model":"llama-31-8b.zip","choices":[{"text":" touch","index":0,"logprobs":null,"finish_reason":"length","stop_reason":null}],"usage":{"prompt_tokens":6,"completion_tokens":100,"total_tokens":106,"ttft":0.091563446,"tps":10.020860629999026}}

data: {"id":"cmpl-5a1adc32e65849f2aee5edf2e37fdb7a","created":1750125678,"model":"llama-31-8b.zip","choices":[],"usage":{"prompt_tokens":6,"completion_tokens":100,"total_tokens":106,"ttft":0.091563446,"tps":10.020269063389119}}

data: [DONE]

# Streaming: Chat completion
!curl -X POST \
  -H "Authorization: Bearer abc123" \
  -H "Content-Type: application/json" \
  -d '{"model": "whatever", "messages": [{"role": "user", "content": "tell me a story"}], "max_tokens": 100, "stream": true, "stream_options": {"include_usage": true}}' \
  https://qaic-poc.pov.wallaroo.io/v1/api/pipelines/infer/llamaqaic-openai-32/llamaqaic-openai/openai/v1/chat/completions

data: {"id":"chat-02fbfe3ae2b54133a28b2deffd3aaab6","object":"chat.completion.chunk","created":1750125657,"model":"llama-31-8b.zip","choices":[{"index":0,"finish_reason":null,"message":null,"delta":{"role":"assistant"}}],"usage":{"prompt_tokens":39,"completion_tokens":0,"total_tokens":39,"ttft":0.093523807,"tps":0.0}}

data: {"id":"chat-02fbfe3ae2b54133a28b2deffd3aaab6","object":"chat.completion.chunk","created":1750125657,"model":"llama-31-8b.zip","choices":[{"index":0,"finish_reason":null,"message":null,"delta":{"role":null,"content":"Once"}}],"usage":{"prompt_tokens":39,"completion_tokens":1,"total_tokens":40,"ttft":0.093523807,"tps":10.679028938385025}}

data: {"id":"chat-02fbfe3ae2b54133a28b2deffd3aaab6","object":"chat.completion.chunk","created":1750125657,"model":"llama-31-8b.zip","choices":[{"index":0,"finish_reason":null,"message":null,"delta":{"role":null,"content":" upon"}}],"usage":{"prompt_tokens":39,"completion_tokens":2,"total_tokens":41,"ttft":0.093523807,"tps":15.893348258670727}}

...

data: {"id":"chat-02fbfe3ae2b54133a28b2deffd3aaab6","object":"chat.completion.chunk","created":1750125657,"model":"llama-31-8b.zip","choices":[{"index":0,"finish_reason":"length","message":null,"delta":{"role":null,"content":" pointing"}}],"usage":{"prompt_tokens":39,"completion_tokens":100,"total_tokens":139,"ttft":0.093523807,"tps":10.511513285045494}}

data: {"id":"chat-02fbfe3ae2b54133a28b2deffd3aaab6","object":"chat.completion.chunk","created":1750125657,"model":"llama-31-8b.zip","choices":[],"usage":{"prompt_tokens":39,"completion_tokens":100,"total_tokens":139,"ttft":0.093523807,"tps":10.511361686804662}}

data: [DONE]

from openai import OpenAI
client = OpenAI(
    base_url='https://qaic-poc.pov.wallaroo.io/v1/api/pipelines/infer/llamaqaic-openai-32/llamaqaic-openai/openai/v1',
    api_key=token
)

for chunk in client.chat.completions.create(model="dummy", messages=[{"role": "user", "content": "this is a short story about love"}], max_tokens=100, stream=True):
    print(chunk.choices[0].delta.content, end="", flush=True)

I'd love to hear it. Please go ahead and share the short story about love. I'll be happy to listen and respond.

for chunk in client.completions.create(model="dummy", prompt="tell me about wallaroo.AI", max_tokens=100, stream=True):
    print(chunk.choices[0].text, end="", flush=True)

Introducing wallaroo.AI
Wallايي буду Towards a Optimization Approach
For AI-Driven Sports Optimization and Trading
Background: One-click strategy models, platform agnostic, similarity testing
Theory: Extreme Value Theory ( EVT ) , GARCH , Kalman Filter algorithm
Impact: speeding through complex sett FL startup focusing implementation wallaroo.Readingmy Business model
wallaroo.ai is an AI-driven platform that aims to revolutionize the way we approach sports optimization and trading. The platform leverages

pipeline.publish(deployment_config=deployment_config)

Waiting for pipeline publish... It may take up to 600 sec.
....................................................................................... Published.