Symbex

Find the Python code for specified symbols

Read Symbex: search Python code for functions and classes, then pipe them into a LLM for background on this project.

Installation

Install this tool using pip:

pip install symbex

Or using Homebrew:

brew install simonw/llm/symbex

Usage

symbex can search for names of functions and classes that occur at the top level of a Python file.

To search every .py file in your current directory and all subdirectories, run like this:

symbex my_function

You can search for more than one symbol at a time:

symbex my_function MyClass

Wildcards are supported - to search for every test_ function run this (note the single quotes to avoid the shell interpreting the * as a wildcard):

symbex 'test_*'

To search for methods within classes, use class.method notation:

symbex Entry.get_absolute_url

Wildcards are supported here as well:

symbex 'Entry.*'
symbex '*.get_absolute_url'
symbex '*.get_*'

Or to view every method of every class:

symbex '*.*'

To search within a specific file, pass that file using the -f option. You can pass this more than once to search multiple files.

symbex MyClass -f my_file.py

To search within a specific directory and all of its subdirectories, use the -d/--directory option:

symbex Database -d ~/projects/datasette

If you know that you want to inspect one or more modules that can be imported by Python, you can use the -m/--module name option. This example shows the signatures for every symbol available in the asyncio package:

symbex -m asyncio -s --imports

You can search the directory containing the Python standard library using --stdlib. This can be useful for quickly looking up the source code for specific Python library functions:

symbex --stdlib -in to_thread

-in is explained below. If you provide --stdlib without any -d or -f options then --silent will be turned on automatically, since the standard library otherwise produces a number of different warnings.

The output starts like this:

# from asyncio.threads import to_thread
async def to_thread(func, /, *args, **kwargs):
    """Asynchronously run function *func* in a separate thread.
    # ...

You can exclude files in specified directories using the -x/--exclude option:

symbex Database -d ~/projects/datasette -x ~/projects/datasette/tests

If symbex encounters any Python code that it cannot parse, it will print a warning message and continue searching:

# Syntax error in path/badcode.py: expected ':' (<unknown>, line 1)

Pass --silent to suppress these warnings:

symbex MyClass --silent

Filters

In addition to searching for symbols, you can apply filters to the results.

The following filters are available:

--function - only functions
--class - only classes
--async - only async def functions
--unasync - only non-async functions
--documented - functions/classes that have a docstring
--undocumented - functions/classes that do not have a docstring
--public - functions/classes that are public - don't have a _name prefix (or are __*__ methods)
--private - functions/classes that are private - have a _name prefix and are not __*__
--dunder - functions matching __*__ - this should usually be used with *.* to find all dunder methods
--typed - functions that have at least one type annotation
--untyped - functions that have no type annotations
--partially-typed - functions that have some type annotations but not all
--fully-typed - functions that have type annotations for every argument and the return value
--no-init - Exclude __init__(self) methods. This is useful when combined with --fully-typed '*.*' to avoid returning __init__(self) methods that would otherwise be classified as fully typed, since __init__ doesn't need argument or return type annotations.

For example, to see the signatures of every async def function in your project that doesn't have any type annotations:

symbex -s --async --untyped

For class methods instead of functions, you can combine filters with a symbol search argument of *.*.

This example shows the full source code of every class method in the Python standard library that has type annotations for all of the arguments and the return value:

symbex --fully-typed --no-init '*.*' --stdlib

To find all public functions and methods that lack documentation, just showing the signature of each one:

symbex '*' '*.*' --public --undocumented --signatures

Example output

In a fresh checkout of Datasette I ran this command:

symbex MessagesDebugView get_long_description

Here's the output of the command:

# File: setup.py Line: 5
def get_long_description():
    with open(
        os.path.join(os.path.dirname(os.path.abspath(__file__)), "README.md"),
        encoding="utf8",
    ) as fp:
        return fp.read()

# File: datasette/views/special.py Line: 60
class PatternPortfolioView(View):
    async def get(self, request, datasette):
        await datasette.ensure_permissions(request.actor, ["view-instance"])
        return Response.html(
            await datasette.render_template(
                "patterns.html",
                request=request,
                view_name="patterns",
            )
        )

Just the signatures

The -s/--signatures option will list just the signatures of the functions and classes, for example:

symbex -s -f symbex/lib.py

# File: symbex/lib.py Line: 107
def function_definition(function_node: AST):

# File: symbex/lib.py Line: 13
def find_symbol_nodes(code: str, filename: str, symbols: Iterable[str]) -> List[Tuple[(AST, Optional[str])]]:

# File: symbex/lib.py Line: 175
def class_definition(class_def):

# File: symbex/lib.py Line: 209
def annotation_definition(annotation: AST) -> str:

# File: symbex/lib.py Line: 227
def read_file(path):

# File: symbex/lib.py Line: 253
class TypeSummary:

# File: symbex/lib.py Line: 258
def type_summary(node: AST) -> Optional[TypeSummary]:

# File: symbex/lib.py Line: 304
def quoted_string(s):

# File: symbex/lib.py Line: 315
def import_line_for_function(function_name: str, filepath: str, possible_root_dirs: List[str]) -> str:

# File: symbex/lib.py Line: 37
def code_for_node(code: str, node: AST, class_name: str, signatures: bool, docstrings: bool) -> Tuple[(str, int)]:

# File: symbex/lib.py Line: 71
def add_docstring(definition: str, node: AST, docstrings: bool, is_method: bool) -> str:

# File: symbex/lib.py Line: 82
def match(name: str, symbols: Iterable[str]) -> bool:

This can be combined with other options, or you can run symbex -s to see every symbol in the current directory and its subdirectories.

To include estimated import paths, such as # from symbex.lib import match, use --imports. These will be calculated relative to the directory you specified, or you can pass one or more --sys-path options to request that imports are calculated relative to those directories as if they were on sys.path:

~/dev/symbex/symbex match --imports -s --sys-path ~/dev/symbex

Example output:

# File: symbex/lib.py Line: 82
# from symbex.lib import match
def match(name: str, symbols: Iterable[str]) -> bool:

To suppress the # File: ... comments, use --no-file or -n.

So to both show import paths and suppress File comments, use -in as a shortcut:

symbex -in match

Output:

# from symbex.lib import match
def match(name: str, symbols: Iterable[str]) -> bool:

To include docstrings in those signatures, use --docstrings:

symbex match --docstrings -f symbex/lib.py

Example output:

# File: symbex/lib.py Line: 82
def match(name: str, symbols: Iterable[str]) -> bool:
    "Returns True if name matches any of the symbols, resolving wildcards"

Counting symbols

If you just want to count the number of functions and classes that match your filters, use the --count option. Here's how to count your classes:

symbex --class --count

Or to count every async test function:

symbex --async 'test_*' --count

Symbex

Install / Use

README

Symbex

Installation

Usage

Filters

Example output

Just the signatures

Counting symbols

Struct