gollm - Go Large Language Model

<div align="center"> <img src="img/gopherrobot4s.jpg" alt="Gophers building a robot by Renee French"> </div>

gollm is a Go package designed to help you build your own AI golems. Just as the mystical golem of legend was brought to life with sacred words, gollm empowers you to breathe life into your AI creations using the power of Large Language Models (LLMs). This package simplifies and streamlines interactions with various LLM providers, offering a unified, flexible, and powerful interface for AI engineers and developers to craft their own digital servants.

<div align="center"> <a href="https://www.youtube.com/watch?v=dDN2tUcgqII"> <img src="https://img.youtube.com/vi/dDN2tUcgqII/0.jpg" alt="Ed Zynda's Video"> </a> </div>

Documentation

Table of Contents

• Key Features
• Real-World Applications
• Installation
• Quick Start
  • Basic Usage
• Quick Reference
  • LLM Creation and Configuration
  • Prompt Creation
  • Generate Response
  • Chain of Thought
  • Prompt Optimization
  • Model Comparison
• Advanced Usage
  • Prompt Engineering
  • Pre-built Functions (Chain of Thought)
  • Working with Examples
  • Prompt Templates
  • Structured Output (JSON Output Validation)
  • Prompt Optimizer
  • Model Comparison
  • Memory Retention
  • Structured Messages and Caching
• Best Practices
• Examples and Tutorials
• Project Status
• Philosophy
• Contributing
• License

Key Features

• Unified API for Multiple LLM Providers: Interact seamlessly with various providers, including OpenAI, Anthropic, Groq, Ollama, and OpenRouter. Easily switch between models like GPT-4, Claude, and Llama-3.1.
• Easy Provider and Model Switching: Configure preferred providers and models with simple options.
• Flexible Configuration Options: Customize using environment variables, code-based configuration, or configuration files.
• Advanced Prompt Engineering: Craft sophisticated instructions to guide your AI's responses effectively.
• Prompt Optimizer: Automatically refine and improve your prompts for better results, with support for custom metrics and different rating systems.
• Memory Retention: Maintain context across multiple interactions for more coherent conversations.
• Structured Output and Validation: Ensure outputs are consistent and reliable with JSON schema generation and validation.
• Provider Comparison Tools: Test performance across different LLM providers and models for the same task.
• High-Level AI Functions: Use pre-built functions like ChainOfThought for complex reasoning tasks.
• Robust Error Handling and Retries: Built-in retry mechanisms to handle API rate limits and transient errors.
• Extensible Architecture: Easily expand support for new LLM providers and features.

Supported Providers

gollm works with a variety of LLM providers:

• OpenAI: GPT-4o, GPT-4, GPT-3.5 Turbo
• Anthropic: Claude 3 (Opus, Sonnet, Haiku), Claude 2.1
• Groq: Llama-3, Mixtral models with high-speed inference
• Ollama: Local models support (Llama-3, Mistral, etc.)
• Mistral: Mistral Large, Mistral Medium
• OpenRouter: Access to multiple providers through a single API with:
  • Model fallback capabilities
  • Auto-routing between models
  • Prompt caching
  • Reasoning tokens
  • Provider routing preferences

Real-World Applications

gollm can handle a wide range of AI-powered tasks, including:

• Content Creation Workflows: Generate research summaries, article ideas, and refined paragraphs.
• Complex Reasoning Tasks: Use the ChainOfThought function to analyze complex problems step-by-step.
• Structured Data Generation: Create and validate complex data structures with customizable JSON schemas.
• Model Performance Analysis: Compare different models' performance for specific tasks.
• Prompt Optimization: Automatically improve prompts for various tasks.
• Mixture of Agents: Combine responses from multiple LLM providers.

Installation

go get github.com/teilomillet/gollm

Quick Start

Basic Usage

package main

import (
    "context"
    "fmt"
    "log"
    "os"
    "time"

    "github.com/teilomillet/gollm"
)

func main() {
    // Load API key from environment variable
    apiKey := os.Getenv("OPENAI_API_KEY")
    if apiKey == "" {
        log.Fatalf("OPENAI_API_KEY environment variable is not set")
    }

    // Create a new LLM instance with custom configuration
    llm, err := gollm.NewLLM(
        gollm.SetProvider("openai"),
        gollm.SetModel("gpt-4o-mini"),
        gollm.SetAPIKey(apiKey),
        gollm.SetMaxTokens(200),
        gollm.SetMaxRetries(3),
        gollm.SetRetryDelay(time.Second*2),
        gollm.SetLogLevel(gollm.LogLevelInfo),
    )
    if err != nil {
        log.Fatalf("Failed to create LLM: %v", err)
    }

    ctx := context.Background()

    // Create a basic prompt
    prompt := gollm.NewPrompt("Explain the concept of 'recursion' in programming.")

    // Generate a response
    response, err := llm.Generate(ctx, prompt)
    if err != nil {
        log.Fatalf("Failed to generate text: %v", err)
    }
    fmt.Printf("Response:\n%s\n", response)
}

Quick Reference

Here's a quick reference guide for the most commonly used functions and options in the gollm package:

LLM Creation and Configuration

// Using OpenAI
llm, err := gollm.NewLLM(
    gollm.SetProvider("openai"),
    gollm.SetModel("gpt-4"),
    gollm.SetAPIKey("your-api-key"),
    gollm.SetMaxTokens(100),
    gollm.SetTemperature(0.7),
    gollm.SetMemory(4096),
)

// Using OpenRouter (with model fallback)
llm, err := gollm.NewLLM(
    gollm.SetProvider("openrouter"),
    gollm.SetModel("anthropic/claude-3-5-sonnet"),
    gollm.SetAPIKey("your-openrouter-api-key"),
    gollm.SetMaxTokens(500),
)
// Enable fallback models if primary model is unavailable
llm.SetOption("fallback_models", []string{"openai/gpt-4o", "mistral/mistral-large"})
// Or use OpenRouter's auto-routing capability
// llm.SetOption("auto_route", true)

Prompt Creation

prompt := gollm.NewPrompt("Your prompt text here",
    gollm.WithContext("Additional context"),
    gollm.WithDirectives("Be concise", "Use examples"),
    gollm.WithOutput("Expected output format"),
    gollm.WithMaxLength(300),
)

Generate Response

response, err := llm.Generate(ctx, prompt)

Chain of Thought

response, err := tools.ChainOfThought(ctx, llm, "Your question here")

Prompt Optimization

optimizer := optimizer.NewPromptOptimizer(llm, initialPrompt, taskDescription,
    optimizer.WithCustomMetrics(/* custom metrics */),
    optimizer.WithRatingSystem("numerical"),
    optimizer.WithThreshold(0.8),
)
optimizedPrompt, err := optimizer.OptimizePrompt(ctx)

Model Comparison

results, err := tools.CompareModels(ctx, promptText, validateFunc, configs...)

Advanced Usage

The gollm package offers a range of advanced features to enhance your AI applications:

Prompt Engineering

Create sophisticated prompts to guide the AI's responses:

prompt := gollm.NewPrompt(
    "Explain the concept of recursion in programming",
    gollm.WithDirectives(
        "Be concise and clear", 
        "Include code examples in multiple languages",
        "Provide a practical example.",
    ),
    gollm.WithContext("This is for a beginner programmer who is just starting to learn."),
    gollm.WithOutput("Structure your response with sections: Definition, Example, Pitfalls, Best Practices."),
    gollm.WithMaxLength(1000),
)

Provider-Specific Features

OpenRouter Special Features

OpenRouter provides access to multiple LLM providers with additional capabilities:

// Create OpenRouter client
llm, err := gollm.NewLLM(
    gollm.SetProvider("openrouter"),
    gollm.SetModel("anthropic/claude-3-5-sonnet"),
    gollm.SetAPIKey(apiKey),
)

// Enable model fallbacks (tried in order if primary model fails)
llm.SetOption("fallback_models", []string{"openai/gpt-4o", "mistral/mistral-large"})

// Use auto-routing (automatically select best model)
llm.SetOption("auto_route", true)

// Enable prompt caching (improves performance and reduces costs)
llm.SetOption("enable_prompt_caching", true)

// Enable reasoning tokens (step-by-step thinking)
llm.SetOption("enable_reasoning", true)

// Specify provider routing preferences
llm.SetOption("provider_preferences", map[string]interface{}{
    "openai": map[string]interface{}{
        "weight": 1.0,
    },
})

Pre-built Functions (Chain of Thought)

Use the ChainOfThought function for step-by-step reasoning:

question := "What is the result of 15 * 7 + 22?"
response, err := tools.ChainOfThought(ctx, llm, question)
if err != nil {
    log.Fatalf("Failed to perform chain of thought: %v", err)
}
fmt.Printf("Chain of Thought:\n%s\n", response)

Working with Examples

Load examples directly from files:

examples, err := utils.ReadExamplesFromFile("examples.txt")
if err != nil {
    log.Fatalf("Failed to read examples: %v", err)
}

prompt := gollm.NewPrompt("Generate a similar example:",
    gollm.WithExamples(examples...),
)

response, err := llm.Generate(ctx, prompt)
if err != nil {
    log.Fatalf("Failed to gen