Lilium AI

Not just a chatbot, but your AI Co-pilot for total computer automation. liliumai.cn

Lilium AI is a powerful personal AI agent framework designed for autonomous execution. It connects Large Language Models (LLMs) like OpenAI and Ollama to browser automation, system-level shell control, deep file integration, and omni-channel communication, enabling AI to perform complex end-to-end tasks just like a human.

✨ Key Features

🌐 Browser Automation: Built-in browser driver for autonomous navigation, searching, clicking, and data extraction.
🤖 Autonomous Sub-Agents: Spawn background sub-agents to handle complex tasks asynchronously without interrupting your main chat.
💻 Total Computer Control: Equipped with Shell execution, Filesystem I/O, and Cron scheduling for true automation.
🔌 Omni-Channel: The only framework supporting WeChat, Feishu, DingTalk, Telegram, WhatsApp, Discord, and more.
🧠 Memory & Knowledge: Long-term memory (SQLite/LanceDB) and RAG-based knowledge base for personalized workflows.
🧠 Multi-LLM Support: Seamlessly switch between OpenAI, Claude, Ollama (local models), and more.
🧩 Minimalist Extension: Add new skills in minutes by dropping JS files into the plugins/ folder.
🗣️ Voice-to-Action: Automatic voice transcription (Whisper) to control your computer via voice commands.

🚀 Core Scenarios (What can it do?)

Automated Workflows: "Every morning at 9 AM, log into my GitHub, summarize all new issues, and send the report to my WeChat."
Web Intelligence: "Monitor this page for price drops; notify me via Slack as soon as it's below $1000."
System Ops: "Analyze the server logs in the current directory, find all 500 errors from the last hour, and generate a root-cause summary."
Cross-Platform Assistant: "Take all voice messages I send you on Telegram, transcribe them into text, and save the summary to a local Markdown file."

💡 Why Choose Lilium AI?

🌟 For Everyone: Your All-in-One Digital Assistant

Simple as Chatting: No coding required. Just tell it, "Compare the price of these headphones across major sites," and it will browse the web for you.
24/7 Personal Agent: Let it monitor web pages or summarize documents while you sleep, and report back via WhatsApp or Telegram.
Privacy First: Your chats, files, and account info stay on your own device—safe and secure.

🛠️ For Developers: Ultimate Extensibility

Node.js Ecosystem: Built on Node.js, leveraging millions of npm packages for easy extension.
Powerful Toolbox: Integrated Puppeteer, SQLite, LanceDB, Whisper, and RAG capabilities.
Flexible Plugin System: Define new AI skills simply by dropping a JS file into the plugins/ folder.

🚀 Getting Started

Prerequisites

Node.js: v18 or higher
pnpm: Recommended package manager (npm install -g pnpm)
Python 3: (Optional) For macOS permission checking script (check-permissions).

Installation

Clone the repository

git clone https://github.com/beidald/liliumai.git
cd liliumai

Install dependencies
```
pnpm install
```
Configure the project Copy the example configuration file to create your local config.
```
cp config.example.json config.json
```
Edit config.json Open config.json and fill in your details:
- LLM: Add your OpenAI API Key or Ollama base URL.
- Channels: Enable desired channels (e.g., wechat, telegram) and add tokens.
- Admin: Critical! Change the default password in the admin section.
Security Note: Never commit your config.json to version control. The project is configured to ignore it by default.
Check Permissions (macOS only) If running on macOS and using Wechaty/Puppeteer, you may need to grant accessibility permissions.
```
pnpm check-permissions
```

Running the Bot

Start the agent in development mode:

pnpm start

Or build and run for production:

pnpm build
node dist/index.js

⚠️ Disclaimer

Terms of Service: Automated interactions with some platforms (e.g., WeChat, WhatsApp) may violate their Terms of Service. Users are responsible for ensuring compliance with platform rules.
Account Risk: Using this software with personal accounts (especially on WeChat via Puppet) carries a risk of account restriction or banning. It is highly recommended to use a dedicated test account.
Data Privacy: While this project is designed to be "Local First" and privacy-focused, users are responsible for securing their own server environments and configuration files.

🗺️ Roadmap

[ ] Enhanced Web Dashboard: Richer data visualization and easier configuration management.
[ ] Plugin Ecosystem: A marketplace or easier discovery mechanism for community plugins.
[ ] Multi-modal Support: Ability to process images (Vision) and generate audio (TTS).
[ ] RAG Improvements: Better knowledge base retrieval strategies and document parsing.
[ ] Mobile App: A companion mobile app for managing the bot on the go.

⚙️ Configuration Guide

The config.json file is the heart of Lilium AI. Here are key sections:

Admin Authentication

Secure your Web Interface and API endpoints.

"admin": {
  "email": "admin@example.com",
  "password": "YOUR_STRONG_PASSWORD"
}

LLM Providers

Configure multiple providers with priorities.

"llm": [
  {
    "provider": "openai",
    "model": "gpt-4",
    "apiKey": "sk-...",
    "priority": 1
  },
  {
    "provider": "ollama",
    "model": "llama3",
    "baseUrl": "http://localhost:11434/v1",
    "priority": 2
  }
]

Voice Transcription

Enable automatic voice-to-text for audio messages.

"transcription": {
  "provider": "local",  // "local" or "groq"
  "model": "base",      // "tiny", "base", "small", "medium", "large"
  "apiKey": ""          // Required for cloud providers
}

Note: Local models will be automatically downloaded on first use from HuggingFace (with domestic mirror support).

Channels

Enable channels as needed. For example, to enable the Web UI:

"channels": {
  "web": {
    "enabled": true,
    "port": 3000
  }
}

🖥️ Web Interface

Once running, access the Web UI at: http://localhost:3000

Lilium Web Interface

Chat: Interact with Lilium directly from your browser.
Stop Generation: Click the "Stop" button to immediately halt the AI's current output.
Admin: Some features require the Admin credentials configured in config.json.

🧩 Skills Showcase

Lilium AI's true power lies in skill composition. You can give complex natural language instructions:

Browser Automation: "Log into GitHub and star the specified repository."
System Analysis: "Analyze the log files in the current directory, identify the root cause of 500 errors, and generate a summary."
Autonomous Tasks: "Spawn a sub-agent to continuously monitor price changes on a competitor's website and report to me every hour."
Smart Scheduling: "Every morning at 9 AM, summarize all my unread Feishu messages and send them to Telegram."
Multi-step Collaboration: "Search for the latest research on DeepSeek, download the PDF, extract the key findings, and save them to the local knowledge base."

🤝 Join the Evolution (Contributing)

Lilium AI is an open project, and we welcome developers to join us in building the most powerful personal AI agent!

Develop New Skills: Want the AI to support more tools? You can easily extend its capabilities by writing simple TypeScript/JavaScript classes (Check TOOLS.md).
Build Plugins: Our plugin system allows you to add features directly to the plugins/ directory without touching the core code (Check PLUGINS.md).
Improve Channels: Help us integrate more communication platforms like Slack, Discord, or deeper Feishu/DingTalk integrations.
Submit Issues/PRs: Whether it's a bug fix or a feature proposal, we value every contribution.

Check out our CONTRIBUTING.md for more details. Let's define the future of AI automation together!

📬 Contact

Email: beidaliudong@pku.org.cn

📝 License

MIT License.

Liliumai

Install / Use

README