Hekmatica

Hekmat means wisdom and philosophy in Persian.

Hekmatica is a deep research and conversational agent designed to answer questions on various topics by leveraging web search, external tools (like price lookups), and large language models (LLMs).

Overview

This agent uses a structured workflow orchestrated by LangGraph to:

1. Understand and potentially clarify the user's question.
2. Break down the question into searchable subqueries.
3. Plan which tools (web search, price lookup) to use.
4. Gather information using the selected tools.
5. Filter and rank the gathered information for relevance.
6. Synthesize a comprehensive answer based on the relevant information.
7. Critique its own answer and potentially refine it by performing additional searches.

The agent leverages BAML (Boundary Markup Language) to define and manage interactions with LLMs for tasks like planning, ranking, and generation.

Architecture

• Orchestration: LangGraph (agent.py) manages the flow of execution through different states and nodes.
• LLM Interaction: BAML (baml_src/) defines the prompts, functions, and data structures for interacting with LLMs reliably.
• Tools: Custom Python functions (tools.py) provide capabilities like web search and cryptocurrency price lookups.
• State Management: A Pydantic model (AgentState in agent.py) holds the data passed between steps in the LangGraph workflow.

Components

agent.py

• Defines the AgentState class to track the agent's progress.
• Implements the core agent logic using a langgraph.StateGraph.
• Contains node functions for each step of the workflow:
  • clarify_node: Checks if the question needs clarification.
  • ask_user_node: Prompts the user for clarification (interactive).
  • generate_subqueries_node: Breaks the question into subqueries (using BAML).
  • plan_node: Plans tool usage for subqueries (using BAML).
  • gather_info_node: Executes the plan using tools from tools.py.
  • filter_results_node: Ranks and filters search results (using BAML).
  • answer_node: Generates the final answer (using BAML).
  • critique_node: Critiques the generated answer (using BAML).
  • additional_search_node: Performs follow-up searches based on critique.
• Includes the DeepResearchAgent class to encapsulate the graph and execution logic.
• Provides a main block to run the agent from the command line.

tools.py

• web_search(query, max_results): Performs a general web search using DuckDuckGo and returns a list of results (content and link).
• get_current_price(coin_name): Fetches the current price of a specific item (initially implemented for cryptocurrencies using CoinGecko API) in USD. This demonstrates how specialized lookup tools can be added. Supports common crypto names and symbols (e.g., "bitcoin", "BTC", "ethereum", "ETH").

baml_src/ (BAML Definitions)

This directory contains the BAML files that define the structure and logic for interacting with LLMs:

• clients.baml: Configures the LLM clients (e.g., API keys, model names).
• generators.baml: May contain reusable BAML code snippets or configurations.
• clarify_question.baml: Defines the LLM function to analyze the user's question and ask for clarification if needed.
• generate_subqueries.baml: Defines the LLM function to generate relevant search subqueries.
• plan_steps.baml: Defines the LLM function to create a step-by-step plan involving tool usage.
• rank_results.baml: Defines the LLM function to rank search results based on relevance to the query.
• answer_question.baml: Defines the LLM function to synthesize a final, cited answer from the gathered context.
• critique_answer.baml: Defines the LLM function to evaluate the generated answer for quality and completeness.

Workflow

The agent follows these steps, managed by LangGraph:

1. Clarify: Analyze the input question. If ambiguous, generate a clarifying question.
2. Ask User (Conditional): If clarification is needed, prompt the user and wait for input.
3. Generate Subqueries: Break down the (potentially clarified) question into smaller, searchable queries.
4. Plan: Determine which tool (WebSearch or PriceLookup) to use for each subquery.
5. Gather Info: Execute the plan, calling the appropriate tools (web_search, get_current_price).
6. Filter Results: Use an LLM to rank the gathered information (search results, prices) and select the most relevant items.
7. Generate Answer: Synthesize a comprehensive answer based on the filtered, relevant information, including citations/sources where available.
8. Critique: Evaluate the generated answer.
9. Refine (Conditional): If the critique identifies missing information and the attempt limit hasn't been reached, perform an additional web search for the missing details and loop back to generate an improved answer.
10. End: Return the final answer.

Setup

1. Clone the repository:

   git clone git@github.com:kargarisaac/crypto_deep_research_agent.git
   cd crypto_deep_research_agent
   

2. Install dependencies: This project uses Poetry for dependency management.

   poetry install
   

   (If you don't have Poetry, install it first: https://python-poetry.org/docs/#installation)
3. Configure BAML:
   • Ensure your BAML clients.baml (or equivalent configuration in baml_src/) is set up with the necessary LLM API keys (e.g., Gemini, OpenAI, Anthropic). Refer to BAML documentation for configuration details.
   • Generate the BAML client code if you haven't already or if you modify BAML files:

     baml-cli init
     

4. Environment Variables: Ensure any required API keys (e.g., for LLMs via BAML) are available as environment variables in your shell session or a .env file.

Usage

Run the agent script directly from your terminal, providing your question:

python agent.py --question "<your question>"

The script will execute with the provided question (or a default general question if none is provided). It will prompt you for input if clarification is needed and then print the final answer generated by the agent.

You can also modify the default user_question within the if __name__ == "__main__": block in agent.py.

Development & Cursor Integration (Optional)

The following instructions are for setting up MCP (Model Context Protocol) servers for interacting with BAML and LangGraph documentation within the Cursor IDE during development. This is not required to simply run the agent.

Activate MCP servers for BAML/LangGraph Documentations

• Run the BAML MCP server:

  poetry run mcpdoc --urls BAMLDocs:https://docs.boundaryml.com/llms.txt \
    --transport sse \
    --port 8083 \
    --host localhost \
    --allowed-domains 'https://docs.boundaryml.com'
  

• Run the LangGraph MCP server:

  poetry run mcpdoc --urls LangGraphDocs:https://langchain-ai.github.io/langgraph/llms.txt \
    --transport sse \
    --port 8084 \
    --host localhost \
    --allowed-domains 'https://langchain-ai.github.io/'
  

• Inspect the server and test:

  npx @modelcontextprotocol/inspector
  

Set MCP Servers on Cursor

• Add the following to your Cursor mcp.json file (usually found via Cmd/Ctrl + Shift + P -> Open User Settings (JSON) -> Search for mcp.json):

  {
    "mcpServers": {
      "baml-docs": {
        "url": "http://localhost:8083/sse"
      },
      "langgraph-docs": {
        "url": "http://localhost:8084/sse"
      }
    }
  }
  

• Ensure the servers are running in your terminal and enabled in Cursor (check the AI settings / MCP Server status).

Cursor Rules

Use these rules in Cursor's agent prompt when asking questions about BAML or LangGraph development in this project:

# BAML
When asked about BAML, use the "baml-docs" MCP server:
- Call the list_doc_sources tool to find available documentation sources.
- Call the fetch_docs tool to open the BAML docs index (baml_llms.txt).
- Examine the list of documentation topics for relevance.
- Call fetch_docs on the relevant documentation URL(s) to get details.
- Use the retrieved documentation content to answer the question.

# LangGraph
When asked about LangGraph, use the "langgraph-docs" MCP server:
- Call the list_doc_sources tool to find available documentation sources.
- Call the fetch_docs tool to open the LangGraph docs index (https://langchain-ai.github.io/langgraph/llms.txt).
- Examine the list of documentation topics for relevance.
- Call fetch_docs on the relevant documentation URL(s) to get details.
- Use the retrieved documentation content to answer the question.

TODO

• [ ] Add template for output and check in the critique
• [ ] Explore browser use.
• [ ] Expand Toolset: Integrate more tools (calculator, specific APIs like arXiv, etc.).
• [ ] Generalize PriceLookup: Modify the tool/mapping to handle more item types (stocks, products) or create a system for adding new lookups.
• [ ] Enhance Planning: Improve tool selection logic in plan_steps.baml for a larger toolset.
• [ ] Dynamic Planning: Allow the execution plan to adapt based on intermediate results.
• [ ] Multi-step Reasoning: Enable more complex sequences of tool use and information synthesis.
• [ ] Robust Tool Error Handling: Improve error handling in gather_info_node (retries, alternatives).
• [ ] LLM Output Validation: Add validation/retries for BAML function calls.
• [ ] Contradiction Detection: Add logic to identify and handle conflicting information from sources.
• [ ] Nuanced Critique: Make critique_answer.baml check for accuracy, s