Rule 1.6-Compliant AI Workflow for Legal Practice

Use consumer AI applications for legal work without violating your duty of confidentiality.

This repository provides an open-source, self-hosted workflow that lets lawyers use consumer-grade AI tools (Claude, ChatGPT, etc.) to draft and edit confidential legal documents — while staying compliant with ABA Model Rule 1.6 and its state equivalents.

Two deployment versions are included: a Maximum Security version that runs entirely on local hardware, and a Standard version that uses cloud APIs to orchestrate local anonymization. Both achieve the same result: real client data never reaches third-party AI services.

NEW: Fine-Tuned LDA Model

We fine-tuned Qwen3.5-35B-A3B specifically for legal document anonymization, drafting, and agentic reasoning tasks. The fine-tuned model scores 30/30 on our benchmark (perfect across all tests), outperforming the base model on complex multi-entity anonymization.

• Model: Reytian/qwen3.5-legal-q5_k_m-gguf (GGUF, Q5_K_M, ~24GB)
• Training: 4-bit LoRA via Unsloth, 117 examples (59 LDA + 23 drafting + 26 reasoning + 9 instruction)
• Runs on: Mac Mini M4 Pro (32GB+)
• Details: See finetune/README.md and benchmark/

Two Inference Backends: MLX (Recommended) and Ollama

The model can run through two backends on Apple Silicon Macs. Both are included in this repository.

| Backend | Speed | Latency (10KB doc) | JSON Reliability | Best For | |---------|-------|--------------------|--------------------|----------| | MLX (recommended) | ~30.5 t/s | ~16s | 68/70 (97%) | Mac users who want maximum performance | | Ollama | ~15.8 t/s | ~68s | 39/70 (56%) | Cross-platform, broader hardware support |

We recommend MLX for Mac users. It is 2x faster, has 4.8x lower latency, and produces more reliable structured output (JSON). MLX uses Apple's native Metal framework, which is optimized for Apple Silicon's unified memory architecture.

Ollama remains a solid choice for Linux/Windows users or for environments where MLX is unavailable. Both backends use the same fine-tuned model weights — the performance difference comes from the inference runtime, not the model itself.

See benchmark/benchmark_ollama_vs_mlx.py for the full comparison script and finetune/README.md for setup instructions for both backends.

---

Table of Contents

• The Problem: Rule 1.6 and Consumer AI
• How This Workflow Solves It
• Version 1: Maximum Security (Local-First)
• Version 2: Standard (API-Orchestrated)
• Comparison: This Workflow vs. Harvey AI
• What's Included
• Quick Start
• How the Anonymizer Works
• Batch Processing Pipeline
• Client Memory System
• Fine-Tuning Your Own Model
• Supported File Types
• Ethics & Compliance
• License

---

The Problem: Rule 1.6 and Consumer AI

What Rule 1.6 Requires

ABA Model Rule 1.6(a) states that a lawyer "shall not reveal information relating to the representation of a client unless the client gives informed consent." Rule 1.6(c) further requires lawyers to "make reasonable efforts to prevent the inadvertent or unauthorized disclosure of, or unauthorized access to, information relating to the representation of a client."

This is not limited to courtroom secrets. "Information relating to the representation" is interpreted broadly — it covers all information learned during the attorney-client relationship, including names, addresses, financial details, deal terms, and business strategies, regardless of the source.

What ABA Formal Opinion 512 Says About AI

In July 2024, the ABA Standing Committee on Ethics and Professional Responsibility issued Formal Opinion 512, the first comprehensive ethics guidance on lawyers' use of generative AI. The Opinion addresses six areas — competence (Rule 1.1), confidentiality (Rule 1.6), communication (Rule 1.4), candor (Rules 3.1/3.3), supervision (Rules 5.1/5.3), and fees — and draws on earlier opinions regarding cloud computing and outsourcing.

On confidentiality specifically, the Opinion states:

Before lawyers enter information related to client representation into a [generative AI] tool, they must assess the potential that the information entered into the tool will be "disclosed to or accessed by" other individuals inside and outside the firm.

The Opinion analogizes AI tools to cloud computing services, confirming that lawyers must:

1. Investigate the reliability, security measures, and data-handling policies of any AI tool
2. Ensure the tool is configured to protect confidentiality and security
3. Confirm that confidentiality obligations are enforceable (e.g., contractual)
4. Monitor for breaches or changes in the provider's practices

Why Consumer AI Tools Are Problematic

When you paste a client contract into ChatGPT, Claude, or any consumer AI application, the text is transmitted to the provider's servers. Even with enterprise data retention policies, this raises serious Rule 1.6 concerns:

• Data transmission: Client PII leaves your control and enters a third party's infrastructure
• Training risk: Consumer-tier products may use inputs for model training (check the ToS carefully)
• Breach exposure: You are now dependent on the provider's security for your ethical obligation
• Audit gap: You cannot verify what happens to the data after transmission
• Informed consent: Obtaining client consent for every AI interaction is impractical at scale

Most lawyers respond to this by either (a) not using AI at all (losing competitive advantage) or (b) using AI anyway and hoping for the best (risking disciplinary action). Neither is a good answer.

The Solution: Anonymize Before Transmission

The key insight is simple: if the AI never sees real client data, there is no Rule 1.6 issue.

By automatically anonymizing documents before they reach any cloud AI service — replacing real names, addresses, and identifiers with generic placeholders — and then restoring the original data locally after editing, lawyers can leverage AI assistance while maintaining full compliance. The cloud AI edits {COMPANY_1} and {PERSON_1}, not "Acme Corp" and "John Smith."

This approach satisfies every requirement of Formal Opinion 512:

• Confidential information is never entered into the AI tool
• The anonymization/deanonymization happens entirely on hardware you control
• No investigation of the AI provider's data policies is necessary (they never receive protected data)
• No client consent is required for the AI interaction (no confidential information is disclosed)

---

How This Workflow Solves It

User sends document + instructions
    |
Local Agent classifies the request
    |-- Contains client data? -> Anonymize locally -> Cloud AI edits -> Deanonymize locally -> User
    |-- Fill client info?     -> Load client memory (LOCAL only) -> Populate template -> User
    |-- Non-sensitive task?   -> Delegate to cloud AI directly -> User
    +-- Simple question?      -> Answer locally -> User

The pipeline for sensitive documents:

Original Document (with real client data)
    | [YOUR MACHINE: Fine-tuned local LLM scans and identifies all sensitive entities]
Anonymized Document ({COMPANY_1}, {PERSON_1}, etc.) + mapping.json
    | [CLOUD AI: Sees only placeholders -- edits, drafts, revises as instructed]
Edited Anonymized Document (new clauses added, placeholders preserved)
    | [YOUR MACHINE: Deterministic deanonymization restores real data]
Final Document (real client data restored, AI edits applied)

mapping.json (the Rosetta Stone between real data and placeholders) never leaves your machine.

---

Version 1: Maximum Security (Local-First)

For lawyers who want zero cloud dependency for sensitive operations.

In this version, a fine-tuned local LLM runs on your own hardware (e.g., a Mac Mini with Apple Silicon). It handles:

• Request classification and routing
• Document anonymization (2-pass entity extraction)
• Client memory management (loading/populating client details)
• Template filling from per-client YAML files

Only the anonymized document is sent to a consumer AI app (Claude Code, ChatGPT, etc.) for the actual editing work. The local LLM never needs an internet connection for any sensitive operation.

Architecture

                        YOUR MACHINE (Mac Mini / MacBook)
+------------------------------------------------------------------+
|                                                                  |
|  User --> Agent (local MLX or Ollama, fine-tuned Qwen3.5-Legal)  |
|              |                                                   |
|              +--> LDA Anonymizer (local LLM) --> anonymized.txt  |
|              |                                   + mapping.json  |
|              |                                        |          |
|              |         +------------------------------+------+   |
|              |         |  CLOUD (consumer AI app)             |   |
|              |         |  Sees ONLY: {COMPANY_1}, {PERSON_1}  |   |
|              |         |  Returns: edited anonymized text      |   |
|              |         +------------------------------+------+   |
|