---

name: backend-dev-guidelines description: Comprehensive backend development guide for Langfuse's Next.js 14/tRPC/Express/TypeScript monorepo. Use when creating tRPC routers, public API endpoints, BullMQ queue processors, services, or working with tRPC procedures, Next.js API routes, Prisma database access, ClickHouse analytics queries, Redis queues, OpenTelemetry instrumentation, Zod v4 validation, env.mjs configuration, tenant isolation patterns, or async patterns. Covers layered architecture (tRPC procedures → services, queue processors → services), dual database system (PostgreSQL + ClickHouse), projectId filtering for multi-tenant isolation, traceException error handling, observability patterns, and testing strategies (Jest for web, vitest for worker).

Backend Development Guidelines

Purpose

Establish consistency and best practices across Langfuse's backend packages (web, worker, packages/shared) using Next.js 14, tRPC, BullMQ, and TypeScript patterns.

When to Use This Skill

Automatically activates when working on:

• Creating or modifying tRPC routers and procedures
• Creating or modifying public API endpoints (REST)
• Creating or modifying BullMQ queue consumers and producers
• Building services with business logic
• Authenticating API requests
• Accessing resources based on entitlements
• Implementing middleware (tRPC, NextAuth, public API)
• Database operations with Prisma (PostgreSQL) or ClickHouse
• Observability with OpenTelemetry, DataDog, logger, and traceException
• Input validation with Zod v4
• Environment configuration from env variables
• Backend testing and refactoring

---

Quick Start

UI: New tRPC Feature Checklist (Web)

• [ ] Router: Define in features/[feature]/server/*Router.ts
• [ ] Procedures: Use appropriate procedure type (protected, public)
• [ ] Authentication: Use JWT authorization via middlewares.
• [ ] Entitlement check: Access resources based on resource and role
• [ ] Validation: Zod v4 schema for input
• [ ] Service: Business logic in service file
• [ ] Error handling: Use traceException wrapper
• [ ] Tests: Unit + integration tests in __tests__/
• [ ] Config: Access via env.mjs

SDKs: New Public API Endpoint Checklist (Web)

• [ ] Route file: Create in pages/api/public/
• [ ] Wrapper: Use withMiddlewares + createAuthedProjectAPIRoute
• [ ] Types: Define in features/public-api/types/
• [ ] Authentication: Authorization via basic auth
• [ ] Validation: Zod schemas for query/body/response
• [ ] Versioning: Versioning in API path and Zod schemas for query/body/response
• [ ] Tests: Add end-to-end test in __tests__/async/

New Queue Processor Checklist (Worker)

• [ ] Processor: Create in worker/src/queues/
• [ ] Queue types: Create queue types in packages/shared/src/server/queues
• [ ] Service: Business logic in features/ or worker/src/features/
• [ ] Error handling: Distinguish between errors which should fail queue processing and errors which should result in a succeeded event.
• [ ] Queue registration: Add to WorkerManager in app.ts
• [ ] Tests: Add vitest tests in worker

---

Architecture Overview

Layered Architecture

# Web Package (Next.js 14)

┌─ tRPC API ──────────────────┐   ┌── Public REST API ──────────┐
│                             │   │                             │
│  HTTP Request               │   │  HTTP Request               │
│      ↓                      │   │      ↓                      │
│  tRPC Procedure             │   │  withMiddlewares +          │
│  (protectedProjectProcedure)│   │  createAuthedProjectAPIRoute│
│      ↓                      │   │      ↓                      │
│  Service (business logic)   │   │  Service (business logic)   │
│      ↓                      │   │      ↓                      │
│  Prisma / ClickHouse        │   │  Prisma / ClickHouse        │
│                             │   │                             │
└─────────────────────────────┘   └─────────────────────────────┘
                 ↓
            [optional]: Publish to Redis BullMQ queue
                 ↓
┌─ Worker Package (Express) ──────────────────────────────────┐
│                                                             │
│  BullMQ Queue Job                                           │
│      ↓                                                      │
│  Queue Processor (handles job)                              │
│      ↓                                                      │
│  Service (business logic)                                   │
│      ↓                                                      │
│  Prisma / ClickHouse                                        │
│                                                             │
└─────────────────────────────────────────────────────────────┘

Key Principles:

• Web: tRPC procedures for UI OR public API routes for SDKs → Services → Database
• Worker: Queue processors → Services → Database
• packages/shared: Shared code for Web and Worker

See architecture-overview.md for complete details.

---

Directory Structure

Web Package (/web/)

web/src/
├── features/                # Feature-organized code
│   ├── [feature-name]/
│   │   ├── server/          # Backend logic
│   │   │   ├── *Router.ts   # tRPC router
│   │   │   └── service.ts   # Business logic
│   │   ├── components/      # React components
│   │   └── types/           # Feature types
├── server/
│   ├── api/
│   │   ├── routers/         # tRPC routers
│   │   ├── trpc.ts          # tRPC setup & middleware
│   │   └── root.ts          # Main router
│   ├── auth.ts              # NextAuth.js config
│   └── db.ts                # Database client
├── pages/
│   ├── api/
│   │   ├── public/          # Public REST APIs
│   │   └── trpc/            # tRPC endpoint
│   └── [routes].tsx         # Next.js pages
├── __tests__/               # Jest tests
│   └── async/               # Integration tests
├── instrumentation.ts       # OpenTelemetry (FIRST IMPORT)
└── env.mjs                  # Environment config

Worker Package (/worker/)

worker/src/
├── queues/                  # BullMQ processors
│   ├── evalQueue.ts
│   ├── ingestionQueue.ts
│   └── workerManager.ts
├── features/                # Business logic
│   └── [feature]/
│       └── service.ts
├── instrumentation.ts       # OpenTelemetry (FIRST IMPORT)
├── app.ts                   # Express setup + queue registration
├── env.ts                   # Environment config
└── index.ts                 # Server start

Shared Package (/packages/shared/)

shared/src/
├── server/                  # Server utilities
│   ├── auth/                # Authentication helpers
│   ├── clickhouse/          # ClickHouse client & schema
│   ├── instrumentation/     # OpenTelemetry helpers
│   ├── llm/                 # LLM integration utilities
│   ├── redis/               # Redis queues & cache
│   ├── repositories/        # Data repositories
│   ├── services/            # Shared services
│   ├── utils/               # Server utilities
│   ├── logger.ts
│   └── queues.ts
├── encryption/              # Encryption utilities
├── features/                # Feature-specific code
├── tableDefinitions/        # Table schemas
├── utils/                   # Shared utilities
├── constants.ts
├── db.ts                    # Prisma client
├── env.ts                   # Environment config
└── index.ts                 # Main exports

Import Paths (package.json exports):

The shared package exposes specific import paths for different use cases:

| Import Path | Maps To | Use For | | ------------------------------------------ | --------------------------------- | --------------------------------------------------------------- | | @langfuse/shared | dist/src/index.js | General types, schemas, utilities, constants | | @langfuse/shared/src/db | dist/src/db.js | Prisma client and database types | | @langfuse/shared/src/server | dist/src/server/index.js | Server-side utilities (queues, auth, services, instrumentation) | | @langfuse/shared/src/server/auth/apiKeys | dist/src/server/auth/apiKeys.js | API key management utilities | | @langfuse/shared/encryption | dist/src/encryption/index.js | Encryption and signature utilities |

Usage Examples:

// General imports - types, schemas, constants, interfaces
import {
  CloudConfigSchema,
  StringNoHTML,
  AnnotationQueueObjectType,
  type APIScoreV2,
  type ColumnDefinition,
  Role,
} from "@langfuse/shared";

// Database - Prisma client and types
import { prisma, Prisma, JobExecutionStatus } from "@langfuse/shared/src/db";
import { type DB as Database } from "@langfuse/shared";

// Server utilities - queues, services, auth, instrumentation
import {
  logger,
  instrumentAsync,
  traceException,
  redis,
  getTracesTable,
  StorageService,
  sendMembershipInvitationEmail,
  invalidateApiKeysForProject,
  recordIncrement,
  recordHistogram,
} from "@langfuse/shared/src/server";

// API key management (specific path)
import { createAndAddApiKeysToDb } from "@langfuse/shared/src/server/auth/apiKeys";

// Encryption utilities
import { encrypt, decrypt, sign, verify } from "@langfuse/shared/encryption";

What Goes Where:

The shared package provides types, utilities, and server code used by both web and worker packages. It has 5 export paths that control frontend vs backend access:

| Import Path | Usage | What's Included | | ------------------------------------------ | --------------------- | ---------------------------------------------------------------------------------- | | @langfuse/shared | ✅ Frontend + Backend | Prisma types, Zod schemas, constants, table definitions, domain models, utilities | | @langfuse/shared/src/db | 🔒 Backend only | Prisma client instance | | @langfuse/shared/src/server | 🔒 Backend only | Services, repositories, queues, auth, ClickHouse, LLM integration, instrumentation | | @langfuse/shared/src/server/auth/apiKeys | 🔒 Backend only | API key management (separated to avoid circular deps) | | @langfuse/shared/encryption | 🔒 Backend only | Database field encryption/decryption |

Naming Conventions:

• tRPC Routers: camelCaseRouter.ts - datasetRouter.ts
• Services: service.ts in feature directory
• Queue Processors: camelCaseQueue.ts - evalQueue.ts
• Public APIs: kebab-case.ts - dataset-items.ts

---

Core Principles

1. tRPC Procedures Delegate to Services

// ❌ NEVER: Business logic in procedures
export const traceRouter = createTRPCRouter({
  byId: protectedProjectProcedure
    .input(z.object({ traceId: z.string() }))
    .query(async ({ input, ctx }) => {
      // 200 lines of logic here
    }),
});

// ✅ ALWAYS: Delegate to service
export const traceRouter = createTRPCRouter({
  byId: protectedProjectProcedure
    .input(z.object({ traceId: z.string() }))
    .query(async ({ input, ctx }) => {
      return await getTraceById(input.traceId);
    }),
});

2. Access Config via env.mjs, NEVER process.env

// ❌ NEVER (except in env.mjs itself)
const dbUrl = process.env.DATABASE_URL;

// ✅ ALWAYS
import {