.. image:: https://github.com/Maratyszcza/PeachPy/blob/master/logo/peachpy.png :alt: PeachPy logo :align: center

=========================================================================== Portable Efficient Assembly Code-generator in Higher-level Python (PeachPy)

.. image:: https://img.shields.io/badge/License-BSD%202--Clause%20%22Simplified%22%20License-blue.svg :alt: PeachPy License: Simplified BSD :target: https://github.com/Maratyszcza/PeachPy/blob/master/LICENSE.rst

.. image:: https://travis-ci.org/Maratyszcza/PeachPy.svg?branch=master :alt: Travis-CI Build Status :target: https://travis-ci.org/Maratyszcza/PeachPy/

.. image:: https://ci.appveyor.com/api/projects/status/p64ew9in189bu2pl?svg=true :alt: AppVeyor Build Status :target: https://ci.appveyor.com/project/MaratDukhan/peachpy

PeachPy is a Python framework for writing high-performance assembly kernels.

PeachPy aims to simplify writing optimized assembly kernels while preserving all optimization opportunities of traditional assembly. Some PeachPy features:

• Universal assembly syntax for Windows, Unix, and Golang assembly.

  • PeachPy can directly generate ELF, MS COFF and Mach-O object files and assembly listings for Golang toolchain

• Automatic adaption of function to different calling conventions and ABIs.

  • Functions for different platforms can be generated from the same assembly source
  • Supports Microsoft x64 ABI, System V x86-64 ABI (Linux, OS X, and FreeBSD), Linux x32 ABI, Native Client x86-64 SFI ABI, Golang AMD64 ABI, Golang AMD64p32 ABI

• Automatic register allocation.

  • PeachPy is flexible and lets mix auto-allocated and hardcoded registers in the same code.

• Automation of routine tasks in assembly programming:

  • Function prolog and epilog and generated by PeachPy
  • De-duplication of data constants (e.g. Constant.float32x4(1.0))
  • Analysis of ISA extensions used in a function

• Supports x86-64 instructions up to AVX-512 and SHA

  • Including 3dnow!+, XOP, FMA3, FMA4, TBM and BMI2.
  • Excluding x87 FPU and most system instructions.
  • Rigorously tested with auto-generated tests <https://github.com/Maratyszcza/PeachPy/tree/master/test/x86_64/encoding>_ to produce the same opcodes as binutils.

• Auto-generation of metadata files

  • Makefile with module dependencies (-MMD and -MF options)
  • C header for the generated functions
  • Function metadata in JSON format

• Python-based metaprogramming and code-generation.

• Multiplexing of multiple instruction streams (helpful for software pipelining).

• Compatible with Python 2 and Python 3, CPython and PyPy.

Online Demo

You can try online demo on PeachPy.IO <http://www.peachpy.io>_

Installation

PeachPy is actively developed, and thus there are presently no stable releases of 0.2 branch. We recommend that you use the master version:

.. code-block:: bash

pip install --upgrade git+https://github.com/Maratyszcza/PeachPy

Installation for development

---

If you plan to modify PeachPy, we recommend the following installation procedure:

.. code-block:: bash

git clone https://github.com/Maratyszcza/PeachPy.git cd PeachPy python setup.py develop

Using PeachPy as a command-line tool

.. code-block:: python

These two lines are not needed for PeachPy, but will help you get autocompletion in good code editors

from peachpy import * from peachpy.x86_64 import *

Lets write a function float DotProduct(const float* x, const float* y)

If you want maximum cross-platform compatibility, arguments must have names

x = Argument(ptr(const_float_), name="x")

If name is not specified, it is auto-detected

y = Argument(ptr(const_float_))

Everything inside the with statement is function body

with Function("DotProduct", (x, y), float_, # Enable instructions up to SSE4.2 # PeachPy will report error if you accidentally use a newer instruction target=uarch.default + isa.sse4_2):

# Request two 64-bit general-purpose registers. No need to specify exact names.
reg_x, reg_y = GeneralPurposeRegister64(), GeneralPurposeRegister64()

# This is a cross-platform way to load arguments. PeachPy will map it to something proper later.
LOAD.ARGUMENT(reg_x, x)
LOAD.ARGUMENT(reg_y, y)

# Also request a virtual 128-bit SIMD register...
xmm_x = XMMRegister()
# ...and fill it with data
MOVAPS(xmm_x, [reg_x])
# It is fine to mix virtual and physical (xmm0-xmm15) registers in the same code
MOVAPS(xmm2, [reg_y])

# Execute dot product instruction, put result into xmm_x
DPPS(xmm_x, xmm2, 0xF1)

# This is a cross-platform way to return results. PeachPy will take care of ABI specifics.
RETURN(xmm_x)

Now you can compile this code into a binary object file that you can link into a program...

.. code-block:: bash

Use MS-COFF format with Microsoft ABI for Windows

python -m peachpy.x86_64 -mabi=ms -mimage-format=ms-coff -o example.obj example.py

Use Mach-O format with SysV ABI for OS X

python -m peachpy.x86_64 -mabi=sysv -mimage-format=mach-o -o example.o example.py

Use ELF format with SysV ABI for Linux x86-64

python -m peachpy.x86_64 -mabi=sysv -mimage-format=elf -o example.o example.py

Use ELF format with x32 ABI for Linux x32 (x86-64 with 32-bit pointer)

python -m peachpy.x86_64 -mabi=x32 -mimage-format=elf -o example.o example.py

Use ELF format with Native Client x86-64 ABI for Chromium x86-64

python -m peachpy.x86_64 -mabi=nacl -mimage-format=elf -o example.o example.py

What else? You can convert the program to Plan 9 assembly for use with Go programming language:

.. code-block:: bash

Use Go ABI (asm version) with -S flag to generate assembly for Go x86-64 targets

python -m peachpy.x86_64 -mabi=goasm -S -o example_amd64.s example.py

Use Go-p32 ABI (asm version) with -S flag to generate assembly for Go x86-64 targets with 32-bit pointers

python -m peachpy.x86_64 -mabi=goasm-p32 -S -o example_amd64p32.s example.py

If Plan 9 assembly is too restrictive for your use-case, generate .syso objects which can be linked into Go programs <https://github.com/golang/go/wiki/GcToolchainTricks#use-syso-file-to-embed-arbitrary-self-contained-c-code>_:

.. code-block:: bash

Use Go ABI (syso version) to generate .syso objects for Go x86-64 targets

Image format can be any (ELF/Mach-O/MS-COFF)

python -m peachpy.x86_64 -mabi=gosyso -mimage-format=elf -o example_amd64.syso example.py

Use Go-p32 ABI (syso version) to generate .syso objects for Go x86-64 targets with 32-bit pointers

Image format can be any (ELF/Mach-O/MS-COFF)

python -m peachpy.x86_64 -mabi=gosyso-p32 -mimage-format=elf -o example_amd64p32.syso example.py

See examples <https://github.com/Maratyszcza/PeachPy/tree/master/examples>_ for real-world scenarios of using PeachPy with make, nmake and go generate tools.

Using PeachPy as a Python module

When command-line tool does not provide sufficient flexibility, Python scripts can import PeachPy objects from peachpy and peachpy.x86_64 modules and do arbitrary manipulations on output images, program structure, instructions, and bytecodes.

PeachPy as Inline Assembler for Python

---

PeachPy links assembly and Python: it represents assembly instructions and syntax as Python classes, functions, and objects. But it also works the other way around: PeachPy can represent your assembly functions as callable Python functions!

.. code-block:: python

from peachpy import * from peachpy.x86_64 import *

x = Argument(int32_t) y = Argument(int32_t)

with Function("Add", (x, y), int32_t) as asm_function: reg_x = GeneralPurposeRegister32() reg_y = GeneralPurposeRegister32()

  LOAD.ARGUMENT(reg_x, x)
  LOAD.ARGUMENT(reg_y, y)

  ADD(reg_x, reg_y)

  RETURN(reg_x)

python_function = asm_function.finalize(abi.detect()).encode().load()

print(python_function(2, 2)) # -> prints "4"

PeachPy as Instruction Encoder

---

PeachPy can be used to explore instruction length, opcodes, and alternative encodings:

.. code-block:: python

from peachpy.x86_64 import *

ADD(eax, 5).encode() # -> bytearray(b'\x83\xc0\x05')

MOVAPS(xmm0, xmm1).encode_options() # -> [bytearray(b'\x0f(\xc1'), bytearray(b'\x0f)\xc8')]

VPSLLVD(ymm0, ymm1, [rsi + 8]).encode_length_options() # -> {6: bytearray(b'\xc4\xe2uGF\x08'), # 7: bytearray(b'\xc4\xe2uGD&\x08'), # 9: bytearray(b'\xc4\xe2uG\x86\x08\x00\x00\x00')}

Tutorials

• Writing Go assembly functions with PeachPy <https://blog.gopheracademy.com/advent-2016/peachpy/>_ by Damian Gryski <https://github.com/dgryski>_

• Adventures in JIT compilation (Part 4) <http://eli.thegreenplace.net/2017/adventures-in-jit-compilation-part-4-in-python/>_ by Eli Bendersky <https://github.com/eliben>_

Users

• NNPACK <https://github.com/Maratyszcza/NNPACK>_ -- an acceleration layer for convolutional networks on multi-core CPUs.

• ChaCha20 <https://git.schwanenlied.me/yawning/chacha20>_ -- Go implementation of ChaCha20 cryptographic cipher.

• AEZ <https://git.schwanenlied.me/yawning/aez>_ -- Go implementation of AEZ authenticated-encryption scheme.

• bp128 <https://github.com/robskie/bp128>_ -- Go implementation of SIMD-BP128 integer encoding and decoding.

• go-marvin32 <https://github.com/dgryski/go-marvin32>_ -- Go implementation of Microsoft's Marvin32 hash function.

• go-highway <https://github.com/dgryski/go-highway>_ -- Go implementation of Google's Highway hash function.

• `go-metro <https://github.com/