paperpipe

The problem: You're implementing a paper. You need the exact equations, want to verify your code matches the math, and your coding agent keeps hallucinating details. Reading PDFs is slow; copy-pasting LaTeX is tedious.

The solution: paperpipe maintains a local paper database with PDFs, LaTeX source (when available), extracted equations, and coding-oriented summaries. It integrates with coding agents (Claude Code, Codex, Gemini CLI) so they can ground their responses in actual paper content.

Typical workflow

# 1. Add papers you're implementing (multiple at once, mixed sources OK)
papi add 2303.08813 1706.03762 "Attention Is All You Need"

# Or one at a time
papi add 2303.08813                    # LoRA paper
papi add https://arxiv.org/abs/1706.03762  # URL
papi add "Attention Is All You Need"   # Search by title

# 2. Check what equations you need to implement
papi show lora --level eq             # prints equations to stdout

# 3. Verify your code matches the paper
#    (or let your coding agent do this via the /papi skill)
papi show lora --level tex            # exact LaTeX definitions

# 4. Ask cross-paper questions (requires RAG backend)
papi ask "How does LoRA differ from full fine-tuning in terms of parameter count?"

# 5. Keep implementation notes
papi notes lora                       # opens notes.md in $EDITOR

Installation

# Basic (uv recommended)
uv tool install paperpipe

# With features
uv tool install paperpipe --with "paperpipe[llm]"      # better summaries via LLMs
uv tool install paperpipe --with "paperpipe[paperqa]"  # RAG via PaperQA2
uv tool install paperpipe --with "paperpipe[leann]"    # local RAG via LEANN
uv tool install paperpipe --with "paperpipe[figures]"  # figure extraction from LaTeX/PDF
uv tool install paperpipe --with "paperpipe[mcp]"      # MCP server integrations (Python 3.11+)
uv tool install paperpipe --with "paperpipe[all]"      # everything

<details markdown="1"> <summary>Alternative: pip install</summary>

pip install paperpipe
pip install 'paperpipe[llm]'
pip install 'paperpipe[paperqa]'  # PaperQA2 + multimodal PDF parsing
pip install 'paperpipe[leann]'
pip install 'paperpipe[figures]'        # figure extraction from LaTeX/PDF
pip install 'paperpipe[mcp]'
pip install 'paperpipe[all]'

</details> <details markdown="1"> <summary>From source</summary>

git clone https://github.com/hummat/paperpipe && cd paperpipe
pip install -e ".[all]"

</details>

What paperpipe stores

~/.paperpipe/                         # override with PAPER_DB_PATH
├── index.json
├── .pqa_papers/                      # staged PDFs for RAG (created on first `papi ask`)
├── .pqa_index/                       # PaperQA2 index cache
├── .leann/                           # LEANN index cache
├── papers/
│   └── lora/
│       ├── paper.pdf                 # for RAG backends
│       ├── source.tex                # full LaTeX (if available from arXiv)
│       ├── equations.md              # extracted equations with context
│       ├── summary.md                # coding-oriented summary
│       ├── tldr.md                   # one-paragraph TL;DR
│       ├── meta.json                 # metadata + tags
│       ├── notes.md                  # your implementation notes
│       └── figures/                  # extracted figures (if available)
│           ├── figure1.png
│           └── figure2.pdf

Why this structure matters:

• equations.md — Key equations with variable definitions. Use for code verification.
• source.tex — Original LaTeX. Use when you need exact notation or the equation extraction missed something.
• summary.md — High-level overview focused on implementation (not literature review). Use for understanding the approach.
• tldr.md — Quick 2-3 sentence overview of the paper's contribution.
• figures/ — Architecture diagrams, network structures, and result plots extracted from LaTeX source or PDF.
• .pqa_papers/ — Staged PDFs only (no markdown) so RAG backends don't index generated content.

Core commands

| Command | Purpose | |---------|---------| | papi add <id-or-url-or-title>... | Add one or more papers (downloads PDF + LaTeX, generates summary/equations/TL;DR) | | papi add --pdf file.pdf | Add a local PDF or URL | | papi add --from-file list.json | Import papers from a JSON list or text file | | papi list | List papers (filter with --tag) | | papi search "query" | Search across titles, tags, summaries, equations (--rg for grep-style, -p paper1,paper2 to limit scope) | | papi index --backend search | Build/update ranked search index (search.db) | | papi show <paper> --level eq | Print equations (best for agent sessions) | | papi show <paper> --level tex | Print LaTeX source | | papi show <paper> --level summary | Print summary | | papi show <paper> --level tldr | Print TL;DR | | papi export <papers...> --to ./dir | Export context files into a repo (--level summary\|equations\|full) | | papi notes <paper> | Open/print implementation notes | | papi regenerate <papers...> | Regenerate summary/equations/tags/TL;DR | | papi remove <papers...> | Remove papers | | papi ask "question" | Cross-paper RAG query (requires PaperQA2 or LEANN) | | papi index | Build/update the retrieval index | | papi tags | List all tags (--audit to find duplicates, --merge OLD NEW, --delete TAG) | | papi path | Print database location | | papi docs | Print agent integration snippet (for CLAUDE.md/AGENTS.md) | | papi rebuild-index | Rebuild index.json from on-disk paper directories (recovery) |

Run papi --help or papi <command> --help for full options.

Import/Export

Share your paper collection with others or back it up.

Export:

# Export full list to JSON
papi list --json > my_papers.json

# Export specific tag
papi list --tag "computer-vision" --json > cv_papers.json

Import:

# Import from JSON (preserves custom names and tags)
papi add --from-file my_papers.json

# Import from text file (one arXiv ID per line)
papi add --from-file paper_ids.txt --tags "imported"

# Import from BibTeX file (requires bibtexparser)
papi add --from-file papers.bib
# or install with BibTeX support:
# uv tool install paperpipe --with "paperpipe[bibtex]"

Title Search:

# Add papers by title (auto-selects if high confidence match)
papi add "Attention Is All You Need"
papi add "NeRF: Representing Scenes as Neural Radiance Fields"

Semantic Scholar Support:

# Add papers from Semantic Scholar
papi add https://www.semanticscholar.org/paper/...
papi add 0123456789abcdef0123456789abcdef01234567  # S2 paper ID

Multiple papers at once (mixed sources OK):

papi add 2303.08813 1706.03762 "Attention Is All You Need"
papi add 2303.08813 https://www.semanticscholar.org/paper/... "NeRF"

Exact text search (fast, no LLM required):

papi search --rg "AdamW"                # case-insensitive, literal string (default)
papi search --rg --case-sensitive "NeRF"  # match exact case
papi search --rg --regex "AdamW|Lion"   # regex or OR mode (opt-in)

Ranked search (BM25 via SQLite FTS5, no LLM required):

papi index --backend search --search-rebuild    # builds <paper_db>/search.db
papi search "surface reconstruction"             # uses FTS if available (default)
papi search --no-fts "surface reconstruction"    # force in-memory scan (disables FTS, uses fuzzy matching)
papi search --no-fts --exact "exact phrase"      # force scan with exact matching only

Hybrid ranked+exact search:

papi search --hybrid "surface reconstruction"
papi search --hybrid --show-grep-hits "surface reconstruction"

Limit search to specific papers:

papi search "attention" -p attention-is-all-you-need
papi search "loss" -p paper1,paper2,paper3

What are FTS and BM25?

• FTS = Full-Text Search. Here it means SQLite’s FTS5 extension, which builds an inverted index so searches don’t have to rescan every file on every query.
• BM25 = Okapi BM25, a standard relevance-ranking function used by many search engines. It ranks results based on term frequency, inverse document frequency, and document length normalization.

References (external):

https://sqlite.org/fts5.html
https://en.wikipedia.org/wiki/Okapi_BM25

<details markdown="1"> <summary>Glossary (RAG, embeddings, MCP, LiteLLM)</summary>

• RAG = retrieval‑augmented generation: retrieve relevant paper passages first, then generate an answer grounded in those passages.
• Embedding model = turns text into vectors for semantic search; changing it usually requires rebuilding an index.
• LiteLLM model id = the model string you pass to LiteLLM (provider/model routing), e.g. gpt-4o, gemini/..., ollama/....
• MCP = Model Context Protocol: lets tools/agents call into paperpipe’s retrieval helpers (e.g. “retrieve chunks”) without copying PDFs into the chat.
• Staging dir (.pqa_papers/) = PDF-only mirror used so RAG backends don’t index generated Markdown.

</details> <details markdown="1"> <summary>Config: default search mode</summary>

Set a default for papi search (CLI flags still win):

export PAPERPIPE_SEARCH_MODE=auto   # auto|fts|scan|hybrid

Or in config.toml:

[search]
mode = "auto" # auto|fts|scan|hybrid

</details>

Agent integration

paperpipe is designed to work with coding agents. Install the skill and MCP servers:

papi install                          # installs skill + MCP for detected CLIs
# or be specific:
papi install skill --claude --codex --gemini
papi install mcp --claude --codex --gemini

After installation, your agent can:

• Use /papi to get paper context (skill)
• Call MCP tools like retrieve_chunks for RAG retrieval
• Verify code against paper equations

Custom skills

| Skill | Description | |--------|-------------| | /papi | Route questions to the cheapest papi command | | /papi-init | Add/update PaperPipe integration in your project's AGENTS.md/CLAUDE.md | | /verify-with-paper | Verify code against paper equations | | /ground-with-paper | Ground responses in paper excerpts | | /compare-papers | Compare multiple papers for a decision | | /curate-paper-note | Create a project note from paper excerpts |

For a ready-to-paste snippet for your repo's agent instructions, run papi docs or see AGENT_INTEGRATION.md.

What the agent sees

When you (or your agent) run papi show <paper> --level eq, you get structured output like:

## Equation 1: LoRA Update
$$h = W_0 x + \Delta W x = W_0 x + BA x$$
where:
- $W_0 \in \mathbb{R}^{d \times k}$: pretrained weight matrix (frozen)
- $B \in \mathbb{R}^{d \times r}$, $A \in \mathbb{R}^{r \times k}$: low-rank matrices
- $r \ll \min(d, k)$: the rank (typically 1-64)

This is what makes verification possible — the agent can compare your code symbol-by-symbol.

<details markdown="1"> <summary>MCP server setup (manual)</summary>

MCP servers

paperpipe provides MCP servers for retrieval-only workflows:

• PaperQA2 retrieval: raw chunks + citations (via paperqa_mcp)
• LEANN search: fast semantic search over papers (via leann_mcp)

MCP servers are configured automatically when you run papi install mcp. The install command creates the appropriate configuration files for your agent (Claude Code, Codex CLI, or Gemini CLI).

Installation:

# Install MCP servers for all supported agents (user scope)
papi install mcp

# Install for specific agents
papi install mcp --claude
papi install mcp --codex
papi install mcp --gemini

# Install repo-local MCP configs (Claude + Gemini) and Codex globally
papi install mcp --repo

# Customize embedding model
papi install mcp --embeddin