Installation

This guide takes you from an empty environment to a running chatbot in a few minutes: install Kaval.AI, configure an LLM provider, and run your first workflow — a small typed state machine that takes an input and returns a structured result.

Install Kaval.AI

Kaval.AI targets Python 3.12+. Install it into a virtual environment. The example below uses OpenAI, so install the openai extra.

uv venv
uv pip install "kavalai[openai]"

With pip

python -m venv .venv
source .venv/bin/activate
pip install "kavalai[openai]"

The bare kavalai package is provider-agnostic; extras pull in only what you need:

  • kavalai[openai] — the OpenAI client (used below).

  • kavalai[gemini] / kavalai[ollama] — the Google Gemini / Ollama clients.

  • kavalai[tools] — the Crawl4AI web-scraping tool.

  • kavalai[notebooks] — Jupyter support for running the tutorial notebooks.

Installing from source

git clone https://github.com/Kaval-AI/kaval.ai.git
cd kaval.ai
uv pip install -e ".[openai]"

Configure a provider

LLM nodes call a provider, so it needs an API key. Export it in your shell (or a .env file)…

export OPENAI_API_KEY="sk-..."

…or set it from Python, as the example below does.

Your first workflow: a chatbot

A workflow is built from typed data models and nodes. The example below is a one-node chatbot: it validates the incoming Message, asks the model to reply, and returns a structured Reply — both the prose answer and a list of suggested quick-reply choices. InMemoryDataStorage gives it memory, so each turn can use the conversation so far.

import asyncio
import os

from pydantic import BaseModel
from kavalai.workflow import WorkflowBuilder, InMemoryDataStorage

# Paste your OpenAI key here, or export OPENAI_API_KEY in your shell / .env.
os.environ["OPENAI_API_KEY"] = ""


class Message(BaseModel):
    message: str


class Reply(BaseModel):
    agent_response: str
    choices: list[str]


# A short example or two helps the model fill the choices in the right shape;
# the structured output schema enforces the format.
prompt = """You are a friendly, concise chatbot. Reply to the user in
agent_response, and suggest up to 3 short quick-reply choices the user might
tap next. Use earlier messages in the conversation for context.

Example:
user: hi
agent_response: Hey there! What can I help you with today?
choices: ["What can you do?", "Tell me a joke", "Give me an idea"]
"""

workflow = (
    WorkflowBuilder("Chatbot", llm_model="openai/gpt-5.4-mini")
    .data_model("input", Message)
    .data_model("output", Reply)
    .start("reply")
    .llm(
        "reply",
        prompt=prompt,
        inputs={"message": "input"},
        output="output",
        next="end",
    )
    .end()
    .build_engine(storage=InMemoryDataStorage())
)


async def main():
    state = await workflow.run({"message": "Hi, what can you do?"})
    print(state.output_data["agent_response"])
    print("choices:", state.output_data["choices"])


asyncio.run(main())

Every run returns a WorkflowState: the status, the ordered trace of visited nodes, the full context data, the final output_data, and an aggregate token_usage. It is JSON-serialisable and checkpointed after every node, so runs can be reloaded and inspected later.

Try it with no install or API key

Want to experiment without installing anything or signing up for a provider? Kaval.AI can run a small open model entirely in your browser over WebGPU. See Running in the browser.

Where to next