LitAI
LLM router + minimal agent framework in one. Call any LLM API with OpenAI format. Unified billing, tools, retries, fallback, logging. Build agents and AI apps in pure Python. Full control. Zero magic.
Install / Use
/learn @Lightning-AI/LitAIREADME
</div>
LitAI is an LLM router (OpenAI format) and minimal agent framework. Chat with any model (ChatGPT, Anthropic, etc) in one line with retries, fallbacks, unified billing, and logging. Build agents with tool use in clean, testable Python - no magic, no flaky APIs, no heavy frameworks.
Pay for any model with your Lightning AI credits without additional fees or subscriptions.
<div align='center'> <pre> ✅ Use any AI model (OpenAI, etc.) ✅ Unified billing dashboard ✅ 20+ public models ✅ Bring your model API keys ✅ No subscription ✅ Tool use ✅ Auto retries and fallback ✅ No MLOps glue code ✅ Start instantly </pre> </div> <div align='center'>
Quick Start
Install LitAI via pip (more options):
pip install litai
Get your API key here and chat with any AI model:
from litai import LLM
# 20+ models... "google/gemini-2.5-pro", "lightning-ai/gpt-oss-120b"
llm = LLM(model="openai/gpt-5", api_key="<LIT_API_KEY>")
answer = llm.chat("who are you?")
print(answer)
# I'm an AI by OpenAI
Or use your own OpenAI compatible client
<br/>Why LitAI?
Juggling model APIs is a mess - flaky endpoints, retries, fallbacks, billing, logging, picking the right model every time. Agent frameworks promise to help, but they’re hard to learn, full of magic, hard to control, and break down fast in real-world systems. Even simple things like tool calls or prompt formatting get rewritten behind the scenes. Teams end up rebuilding it all in raw Python just to get something they can trust.
WIth LitAI there's nothing to learn - if you know Python you already know LitAI. You get both an LLM router and a minimal agent framework in one. Just write normal Python, call any model, and sprinkle in .chat(), .if_(), or .classify() wherever the model should step in. It gives you lightweight, minimal building blocks you’d end up building yourself: model calls, retries, fallbacks, tool use, memory, streaming - all in clean, testable code. No wrappers, no magic - just code that works the way you expect.
LitAI vs other agent frameworks
<br/>Examples
If you know Python, you already know LitAI 🤯 - just sprinkle a few "smart" agent decisions.
Agent
Here's a simple agent that tells you the latest news
import re, requests
from litai import LLM
llm = LLM(model="openai/gpt-5-mini", api_key="<LIGHTNING_API_KEY>")
website_url = "https://text.npr.org/"
website_text = re.sub(r'<[^>]+>', ' ', requests.get(website_url).text)
response = llm.chat(f"Based on this, what is the latest: {website_text}")
print(response)
Agentic if statement
We believe the best way to build agents is with normal Python programs and simple “agentic if statements.” This keeps 90% of the logic deterministic, and the model only steps in when needed. No complex abstractions, no framework magic - just code you can trust and debug.
from litai import LLM
llm = LLM()
product_review = "This TV is terrible."
response = llm.chat(f"Is this review good or bad? Reply only with 'good' or 'bad': {product_review}").strip().lower()
if response == "good":
print("good review")
else:
print("bad review")
Shortcuts
Agentic workflows mostly come down to agentic-if statements or classification decisions. While you can use llm.chat yourself to do it,
we provide 2 simple shortcuts
from litai import LLM
llm = LLM()
# shortcut for agentic if statement (can do this yourself with llm.chat if needed)
product_review = "This TV is terrible."
if llm.if_(product_review, "is this a positive review?"):
print("good review")
else:
print("bad review")
# shortcut for agentic classification (can do this yourself with llm.chat if needed)
sentiment = llm.classify("This movie was awful.", ["positive", "negative"])
print("Sentiment:", sentiment)
Tools (docs)
Tools allow models to get real-world data or take actions. In LitAI, there is no magic with tool use, agents can decide to call tools (auto_call_tools=True), or you can manually call a tool with llm.call_tool(...) for full control. Zero magic, just plain Python.
from litai import LLM, tool
@tool
def get_weather(location: str):
return f"The weather in {location} is sunny"
llm = LLM(model="openai/gpt-5")
# OPTION A: automatic tool call
result = llm.chat("What's the weather in Tokyo?", tools=[get_weather], auto_call_tools=True)
# The weather in Tokyo is sunny
# OPTION B: manually call tools for more control
chosen_tool = llm.chat("What's the weather in Tokyo?", tools=[get_weather])
result = llm.call_tool(chosen_tool, tools=[get_weather])
# The weather in London is sunny
Choose automatic or manual tool calling based on production needs. auto_call_tools=True is great for quick demos, but can obscure when and why a tool runs which can lead to surprises in production. llm.call_tool(...) gives you full control to decide when tools execute, making it easier to log, debug, test, and audit. This clarity is critical for reliability, safety, and trust in real-world systems.
Key features
Track usage and spending in your Lightning AI dashboard. Model calls are paid for with Lightning AI credits.
<div align='center'> <pre> ✅ No subscription ✅ 15 free credits (~37M tokens) ✅ Pay as you go for more credits </pre> </div> <div align='center'> <img alt="Lightning" src="https://github.com/user-attachments/assets/b1e7049c-c7b0-42f3-a43c-c1e156929f50" width="800px" style="max-width: 100%;"> </div> <br/>✅ Use over 20+ models (ChatGPT, Claude, etc...)
✅ Monitor all usage in one place
✅ Async support
✅ Auto retries on failure
✅ Auto model switch on failure
✅ Switch models
✅ Multi-turn conversation logs
✅ Streaming
✅ Bring your own model (connect your API keys, coming soon...)
✅ Chat logs (coming soon...)
Advanced features
Auto fallbacks and retries (docs)
Model APIs can flake or can have outages. LitAI automatically retries in case of failures. After multiple failures it can automatically fallback to other models in case the provider is down.
from litai import LLM
llm = LLM(
model="openai/gpt-5",
fallback_models=["google/gemini-2.5-flash", "anthropic/claude-3-5-sonnet-20240620"],
max_retries=4,
)
print(llm.chat("What is a fun fact about space?"))
OpenAI compatible
For those who already have their own SDK to call LLMs (like the OpenAI sdk), you can still use LitAI via the https://lightning.ai/api/v1 endpoint,
which will track usage, billing, etc...
from openai import OpenAI
client = OpenAI(
base_url="https://lightning.ai/api/v1",
api_key="LIGHTNING_API_KEY",
)
completion = client.chat.completions.create(
model="openai/gpt-5-mini",
messages=[
{
"role": "user",
"content": "What is a fun fact about space?"
}
]
)
print(completion.choices[0].message.content)
Organize billing for API calls at organization, teamspace and user level.
Example
from openai import OpenAI
client = OpenAI(
base_url="https://lightning.ai/api/v1",
api_key="LIGHTNING_API_KEY/organization/teamspace",
)
completion = client.chat.completions.create(
model="openai/gpt-5-mini",
messages=[
{
"role": "user",
"content": "What is a fun fact about space?"
}
]
)
print(completion.choices[0].message.content)
Read all the formats here
</details> <details> <summary>Tools</summary> <br/>Models can only reply with text, but tool calling lets them get real-world data or act, like checking calendars or sending messages, which allows AI apps to actually do things, not just talk. There are 2 ways to create tools in LitAI.
@tool: Turn any function into a tool with `litai.too
