M8M - AI-Powered Workflow Automation Platform

A powerful, open-source workflow automation platform built with modern web technologies. Create, deploy, and manage AI-driven workflows with an intuitive visual interface and robust execution engine.

Overview

M8M is a comprehensive workflow automation solution that enables you to build sophisticated AI workflows without writing code. Connect various services, APIs, and AI models to create powerful automation pipelines. Whether you're processing data, integrating services, or building complex AI applications, M8M provides the infrastructure you need.

Key Features

Workflow Management

• Visual Workflow Builder - Create workflows with an intuitive drag-and-drop interface
• Version Control - Track changes and rollback to previous workflow versions
• Workflow Templates - Start quickly with pre-built workflow templates
• Real-time Validation - Instant feedback on workflow configuration

Execution Engine

• Reliable Processing - Kafka-based message queue ensures no execution is lost
• Parallel Execution - Run multiple workflow instances simultaneously
• Error Handling - Automatic retries and comprehensive error tracking
• Execution History - Complete audit trail of all workflow runs
• Live Monitoring - Real-time execution status and logs

Credentials Management

• Secure Storage - Encrypted credential storage for API keys and secrets
• Multiple Auth Types - Support for OAuth, API keys, and custom authentication
• Credential Sharing - Share credentials across workflows and teams
• Access Control - Fine-grained permissions for credential access

Messaging Integrations

• Multi-Channel Support - Connect to Slack, Discord, and more
• AI Integrations - Built-in support for Gemini and other AI services
• Custom Webhooks - Send and receive data from any HTTP endpoint
• Event-Driven - Trigger workflows from external events

Advanced Capabilities

• State Management - Jotai-powered state handling for complex workflows
• Type Safety - Full TypeScript support across the stack
• API-First - tRPC ensures type-safe API communication
• Scalable Architecture - Turborepo monorepo for optimal development

Tech Stack

Frontend

• Next.js 14+ - React framework with App Router
• React - UI component library
• Jotai - Atomic state management
• TailwindCSS - Utility-first CSS framework
• Lucide Icons - Beautiful, consistent icons
• Zod - TypeScript-first schema validation

Backend

• tRPC - End-to-end typesafe APIs
• Prisma ORM - Type-safe database client
• BetterAuth - Complete authentication solution
• Kafka - Distributed message queue for workflow execution
• Node.js - Runtime environment

Infrastructure

• Turborepo - High-performance build system
• PostgreSQL - Primary database
• Docker - Containerization for local development
• TypeScript - Type safety across the entire stack

Architecture

m8m/
├── apps/
│   ├── web/                    # Next.js frontend application
│   │   ├── src/
│   │   │   ├── app/           # Next.js App Router pages
│   │   │   ├── components/    # Reusable React components
│   │   │   ├── features/      # Feature-specific modules
│   │   │   │   ├── executions/    # Execution management
│   │   │   │   ├── workflows/     # Workflow builder & management
│   │   │   │   ├── credentials/   # Credential management
│   │   │   │   └── messaging/     # Message channel configs
│   │   │   ├── config/        # App configuration
│   │   │   └── server/        # tRPC server configuration
│   │   └── package.json
│   └── worker/                 # Background worker process
│       ├── src/
│       │   ├── queue/         # Kafka consumer/producer
│       │   ├── executor/      # Workflow execution engine
│       │   └── handlers/      # Message handlers
│       └── package.json
├── packages/
│   ├── database/              # Prisma schema & migrations
│   │   ├── prisma/
│   │   │   ├── schema.prisma
│   │   │   └── migrations/
│   │   └── src/
│   ├── typescript-config/     # Shared TypeScript configs
│   └── eslint-config/         # Shared ESLint configs
├── docker-compose.yml         # Local development services
├── turbo.json                 # Turborepo configuration
└── package.json               # Root package.json

Getting Started

Prerequisites

• Node.js 18+ or Bun
• Docker and Docker Compose
• PostgreSQL (via Docker or external)
• Kafka (via Docker or external)

Installation

1. Clone the repository

git clone https://github.com/rohitdevsol/m8m.git
cd m8m

2. Install dependencies

bun install
# or
npm install

3. Set up environment variables

cp .env.example .env

Edit the .env file with your configuration:

# Database
DATABASE_URL="postgresql://user:password@localhost:5432/m8m"

# NextAuth
BETTERAUTH_SECRET="your-secret-key"
BETTERAUTH_URL="http://localhost:3000"

# Kafka
KAFKA_BROKERS="localhost:9092"
KAFKA_CLIENT_ID="m8m-worker"

# Optional: External Services
SLACK_CLIENT_ID=""
SLACK_CLIENT_SECRET=""
DISCORD_CLIENT_ID=""
DISCORD_CLIENT_SECRET=""
GEMINI_API_KEY=""

4. Start infrastructure services

docker-compose up -d

This will start:

• PostgreSQL database
• Kafka message broker
• Zookeeper (for Kafka)

5. Run database migrations

cd packages/database
bun run db:migrate:dev
# or
npm run db:migrate:dev

6. Seed the database (optional)

bun run db:seed
# or
npm run db:seed

7. Start the development servers

# From the root directory
bun run dev
# or
npm run dev

This will start:

• Web app on http://localhost:3000
• Worker process for background jobs

Building for Production

# Build all packages and apps
bun run build

# Start production servers
bun run start

Development

Project Structure

The project follows a modular, feature-based architecture:

• apps/web - Main Next.js application with tRPC server
• apps/worker - Background worker for processing workflow executions
• packages/database - Shared Prisma schema and database utilities
• packages/typescript-config - Shared TypeScript configurations
• packages/eslint-config - Shared ESLint rules

Key Technologies

tRPC

Type-safe API routes without code generation. All API endpoints are defined in apps/web/src/server/api/routers/ and are fully type-safe on both client and server.

// Server-side router
export const workflowRouter = createTRPCRouter({
  getAll: publicProcedure.query(async ({ ctx }) => {
    return ctx.db.workflow.findMany();
  }),
});

// Client-side usage (fully typed!)
const { data: workflows } = api.workflow.getAll.useQuery();

Jotai

Atomic state management for React. Used for managing complex UI state and workflow builder state.

import { atom, useAtom } from "jotai";

const workflowAtom = atom(null);
const [workflow, setWorkflow] = useAtom(workflowAtom);

Kafka

Message queue for reliable workflow execution. The worker process consumes messages from Kafka topics and executes workflows.

// Producer (in web app)
await producer.send({
  topic: "workflow-executions",
  messages: [{ value: JSON.stringify(execution) }],
});

// Consumer (in worker)
await consumer.subscribe({ topic: "workflow-executions" });

Prisma

Type-safe database access with auto-generated types.

const execution = await prisma.execution.create({
  data: {
    workflowId: workflow.id,
    status: "RUNNING",
    startTime: new Date(),
  },
});

Available Scripts

From the root directory:

# Development
bun run dev              # Start all apps in development mode
bun run dev:web          # Start only the web app
bun run dev:worker       # Start only the worker

# Building
bun run build            # Build all apps and packages
bun run build:web        # Build only the web app
bun run build:worker     # Build only the worker

# Database
bun run db:migrate:dev   # Create and apply migrations
bun run db:migrate:deploy # Apply migrations in production
bun run db:seed          # Seed the database
bun run db:studio        # Open Prisma Studio

# Code Quality
bun run lint             # Lint all packages
bun run format           # Format code with Prettier
bun run type-check       # Run TypeScript type checking

Features Deep Dive

Workflow Builder

Create workflows using a visual node-based editor:

• Drag and drop nodes
• Connect nodes to define execution flow
• Configure node parameters
• Test workflows in real-time

Execution Management

Monitor and manage workflow executions:

• View execution history
• Inspect detailed logs
• Retry failed executions
• Cancel running executions

Credential System

Securely manage API credentials:

• Create and store credentials
• Use credentials in workflow nodes
• Share credentials with team members
• Rotate credentials without updating workflows

Messaging Channels

Connect to external services:

• Slack: Send messages, create channels, post updates(Working on it...)
• Discord: Send webhooks, manage servers
• Gemini AI: Integrate Google's AI capabilities
• Custom Webhooks: Connect to any HTTP endpoint

Configuration

Environment Variables

| Variable | Description | Required | | ----------------------- | ---------------------------- | -------- | | DATABASE_URL | PostgreSQL connection string | Yes | | BETTERAUTH_SECRET | Secret for NextAuth.js | Yes | | BETTERAUTH_URL | Base URL of the application | Yes | | KAFKA_BROKERS | Kafka broker addresses | Yes | | KAFKA_CLIENT_ID | Kafka client identifier | Yes | | SLACK_CLIENT_ID | Slack OAuth clie