Transfunctions

<details> <summary>ⓘ</summary>

</details>

This library is designed to solve one of the most important problems in Python programming — splitting code into two categories: sync and async. It reduces code duplication by using templates.

Table of contents

• Quick start
• The problem
• Code generation
• Markers
• Superfunctions
• Typing

Quick start

Install it:

pip install transfunctions

And use:

from asyncio import run
from transfunctions import (
    transfunction,
    sync_context,
    async_context,
    generator_context,
)

@transfunction
def template():
    print('so, ', end='')
    with sync_context:
        print("it's just usual function!")
    with async_context:
        print("it's an async function!")
    with generator_context:
        print("it's a generator function!")
        yield

function = template.get_usual_function()
function()
#> so, it's just usual function!

async_function = template.get_async_function()
run(async_function())
#> so, it's an async function!

generator_function = template.get_generator_function()
list(generator_function())
#> so, it's a generator function!

As you can see, in this case, 3 different functions were created based on the template, including both common parts and unique ones for a specific type of function.

You can also quickly try out this and other packages without having to install using instld.

The problem

Since the asyncio module appeared in Python more than 10 years ago, many well-known libraries have gained asynchronous counterparts. A lot of the code in the Python ecosystem has been duplicated, and you probably know many such examples.

The reason for this problem is that the Python community has chosen a syntax-based approach to asynchrony. There are new keywords in the language, such as async and await. Their use makes the code so-called "multicolored": functions become “red” or “blue”, and depending on the color, the rules for calling them are different. You can only call blue functions from red ones, but not vice versa.

I must say that implementing asynchronous calls using a special syntax is not the only solution. There are languages like Go where the runtime can determine under the hood where a function should be asynchronous and where not, and choose how to execute them. A programmer does not need to manually "colorize" their functions there. Personally, I think that choosing a different path is the mistake of the Python community, but that's not what we're discussing here.

The solution offered by this library is based on templating. You can take a certain function as a template and generate several others based on it: regular, asynchronous, or generator. This allows you to avoid duplicating code where it was previously impossible. And all this without major changes in Python syntax or in the internal structure of the interpreter. We are essentially hiding the syntax differences. Combined with the idea of context-aware functions, this makes for an even more powerful tool: superfunctions. This allows you to create a single function object that can be handled as you like: as a regular function, as an asynchronous function, or as a generator. The function will behave the way you use it. Thus, this library solves the problem of code duplication caused by the syntactic approach to marking asynchronous execution sections.

Code generation

This library is based on the idea of generating code at the AST level.

Several derivatives can be generated from a single template function. Let's take a simple template function as an example:

@transfunction
def template():
    print('something')

Executing this code will actually return to us not a function, but a special object that can produce functions:

print(template)
#> <transfunctions.transformer.FunctionTransformer object at 0x105368fa0>

To get a function from this object, you need to call the get_usual_function method from it:

function = template.get_usual_function()
function()
#> something

Nothing unusual so far, right? We just defined the function and got it. But! You can also get an async function from this object:

from asyncio import run

async_function = template.get_async_function()
run(async_function())
#> something

That's more interesting. In fact, we transferred all the contents from the original function to the generated async function. The content itself has not changed in any way, that is, we got a function that would look something like this:

async def template():
    print('something')

But the true power of templating is revealed when we get the opportunity to generate partially different functions. Some parts of the template will be reused in all generated versions, while others will be used only in those that relate to a specific type of function. Let's look again at the template example from the "quick start" section:

@transfunction
def template():
    print('so, ', end='')
    with sync_context:
        print("it's just usual function!")
    with async_context:
        print("it's an async function!")
    with generator_context:
        print("it's a generator function!")
        yield

The get_usual_function method will return a function that will contain a common part (the first print) and a part highlighted using the context manager as related to ordinary functions. It will look something like this:

def template():
    print('so, ', end='')
    print("it's just usual function!")

The get_async_function method will return an async function that looks like this:

async def template():
    print('so, ', end='')
    print("it's an async function!")

Finally, the method get_generator_function will return a generator function that looks like this:

def template():
    print('so, ', end='')
    print("it's a generator function!")
    yield

All generated functions:

• Inherit access to global variables and closures that the original template function had.
• Can be either ordinary standalone functions or bound methods. In the latter case, they will be linked to the same object.

There is only one known limitation: you cannot use any third-party decorators on the template using the decorator syntax, because in some situations this can lead to ambiguous behavior. If you still really need to use a third-party decorator, just generate any of the functions from the template, and then apply your decorator to the result of the generation.

Markers

Objects that we call "markers" are used to mark up specific blocks inside the template function. In the section above, we have already seen how 3 context managers work: sync_context, async_context, and generator_context; all of them are markers. When generating a function with a type corresponding to each of these context managers, the contents of this context manager remain in the generated function, and the others with their contents are cut out.

There is another marker that is used to point to the place where you want to use the await keyword, it is called await_it. In the generated code, this will be converted into an await statement. From the template function, which looks like this:

from asyncio import sleep

@transfunction
def template():
    with async_context:
        await_it(sleep(5))

... when calling the get_async_function method, you will get such an async function:

async def template():
    await sleep(5)

None of the markers need to be imported in order for the generated code to be functional: they are destroyed during the code generation. However, you can do this if your linter or syntax checker in your IDE requires it:

from transfunctions import (
    sync_context,
    async_context,
    generator_context,
    await_it,
)

Make sure that the generated functions do not include keywords that are not related to this type of function. For example, you cannot generate a regular function using the `get_usual_funct