Installation¶

Kaval.AI needs Python 3.12 or newer. Install it into a virtual environment, configure one provider, and you are ready for the Quickstart.

Install¶

With uv (recommended):

uv venv
uv pip install "kavalai[common]"

With pip:

python -m venv .venv
source .venv/bin/activate
pip install "kavalai[common]"

Choosing an extra¶

The bare kavalai package is deliberately small and provider-agnostic: it is restricted to libraries that also work under Pyodide, so the core can run in a browser. Everything else is grouped into four extras, and common installs all four; it is the normal install.

The four are divided by weight, not by provider. crawl4ai (with Playwright, patchright and litellm), fastembed (with onnxruntime) and scikit-learn (with SciPy) account for most of a full install; the provider SDKs and MCP are small, so they share runtime, and replacing one provider with another remains a change of one string rather than a reinstall. A service that runs a workflow against a hosted model needs only runtime:

pip install "kavalai[runtime]"

A module that needs a package from an extra that is not installed names the extra when it fails to import:

ImportError: The agent server requires the optional 'fastapi' package.
Install it with: pip install "kavalai[runtime]"

Embedding on a GPU¶

fastembed runs the local embedding models on the CPU. fastembed-gpu is the same library under the same import name, built against onnxruntime-gpu, so the two cannot be installed together — the GPU extra replaces the CPU package instead of adding to it:

uv pip uninstall fastembed
uv pip install "kavalai[common,gpu]"

Nothing else changes. FastEmbed defaults to cuda=Device.AUTO and selects the CUDA execution provider when one is available, so a model name like fastembed/BAAI/bge-small-en-v1.5 keeps working and runs on the GPU. To be explicit instead, register the provider with cuda=True (and device_ids=[0] to pick a card): FastEmbedClient passes both straight through to FastEmbed.

Check the card before taking the newest wheel. The current onnxruntime-gpu on PyPI is a CUDA 13 build, and CUDA 13 dropped compute capability below 7.5 — on a Pascal or Volta card it fails to load libcublasLt.so.13 and quietly falls back to the CPU. Such a card needs the CUDA 12 build and a cuDNN 9 still carrying its kernels:

uv pip install "onnxruntime-gpu==1.22.0"
uv pip install nvidia-cublas-cu12 nvidia-cuda-nvrtc-cu12 \
    nvidia-cuda-runtime-cu12 nvidia-cufft-cu12 nvidia-curand-cu12 \
    "nvidia-cudnn-cu12==9.1.0.70"

Those wheels install under site-packages/nvidia/*/lib, which the dynamic loader does not search by default, so put them on the path when running:

export LD_LIBRARY_PATH=$(python -c "import glob, os, site; \
    print(':'.join(sorted({os.path.dirname(f) for f in \
    glob.glob(site.getsitepackages()[0] + '/nvidia/*/lib/*.so*')})))")

FastEmbed logs which providers it ended up with; if the list is ['CPUExecutionProvider'] the GPU was not picked up, and the embeddings are still correct but computed on the CPU.

From source¶

git clone https://github.com/Kaval-AI/kaval.ai.git
cd kaval.ai
uv pip install -e ".[common]"

The repository also carries the runnable notebooks behind every tutorial, under notebooks/.

Configure a provider¶

A model call needs a credential, read from the environment. Pick whichever you have:

export OPENAI_API_KEY="sk-..."          # openai/…
export GEMINI_API_KEY="..."             # gemini/…
export ANTHROPIC_API_KEY="..."          # anthropic/…
# Ollama runs locally and needs no key; set OLLAMA_HOST if it is not
# on http://localhost:11434

In development, keep these in a .env file and load it explicitly — nothing in the library reads .env on your behalf:

import dotenv

dotenv.load_dotenv()

Every variable Kaval.AI understands is listed in Configuration.

Check it works¶

import asyncio

from kavalai import make_client


async def main():
    client = make_client("openai/gpt-5.6-luna")
    print(await client.prompt("Say hello in Estonian."))


asyncio.run(main())

Tere!

Optional pieces¶

A database. Nothing above needs one. Persistence — sessions, runs, chat history, statistics — arrives when you hand an AgentService to a workflow. In-memory SQLite works for development; PostgreSQL is the production target, with pgvector if you use RAG. See Observability & storage and Deployment.

The backoffice UI. A separate service for configuring and monitoring agents. docker compose up postgres_db backoffice-migrations backoffice brings it up locally. See Using the Backoffice UI.

No install at all. Kaval.AI can run a small open model entirely in your browser over WebGPU — no Python, no API key. See Running in the browser.

Skills for your coding agent. kavalai-skills install copies Kaval.AI’s agent skills into your project, so an assistant writing workflows against this framework works from how it actually behaves rather than from another framework’s habits. See Agent skills.

Where to next¶

• Quickstart — a model call, structured output and a workflow, in five minutes.

• Core concepts — the vocabulary, if LLMs are new to you.