PyTCP

The TCP/IP stack written in Python

<br>

<br>

PyTCP is a fully functional TCP/IP stack written in Python. It supports TCP stream-based transport with reliable packet delivery based on a sliding window mechanism and basic congestion control. It also supports IPv6/ICMPv6 protocols with SLAAC address configuration. It operates as a user space program attached to the Linux TAP interface. It has implemented simple routing and can send and receive traffic over a local network and the Internet.

Version 2.7, unlike its predecessors, contains the PyTCP stack code in the form of a library so that it can be easily imported and used by external code. This should make the user experience smoother and eventually provide the full ability to replace the standard Linux stack calls (e.g., socket library) with the PyTCP calls in any 3rd party application.

This project initially started as a purely educational effort aimed at improving my Python skills and refreshing my network knowledge as part of the preparation for the Network Engineer role at Facebook. Since then, it has become more like a 'pet project' which I dedicate some of my time on a somewhat irregular basis. However, a couple of updates are usually added to it every month or two.

I welcome any contributions and help from anyone interested in network programming. Any input is appreciated. Also, remember that some stack features may be implemented only partially (as needed for stack operation). They may be implemented in a sub-optimal fashion or not 100% RFC-compliant way (due to lack of time), or they may contain bug(s) that I still need to fix.

Please feel free to check my two other related projects:

• RusTCP - Attempt to rewrite some of PyTCP funcionality in Rust and use it to create IPv6/SRv6 lab router.
• SeaTCP - Attempt to create low latency stack using C and Assembly languages.

---

Principle of operation and the test setup

The PyTCP stack depends on the Linux TAP interface. The TAP interface is a virtual interface that, on the network end, can be 'plugged' into existing virtual network infrastructure via either Linux bridge or Open vSwitch. On the internal end, the TAP interface can be used like any other NIC by programatically sending and receiving packets to/from it.

If you wish to test the PyTCP stack in your local network, I'd suggest creating the following network setup that will allow you to connect both the Linux kernel (essentially your Linux OS) and the PyTCP stack to your local network at the same time.

<INTERNET> <---> [ROUTER] <---> (eth0)-[Linux bridge]-(br0) <---> [Linux TCP/IP stack]
                                            |
                                            |--(tap7) <---> [PyTCP TCP/IP stack]

After the example program (either client or service) starts the stack, it can comunicate with it via simplified BSD Sockets like API interface. There is also the possibility of sending packets directly by calling one of the _*_phtx() methods from PacketHandler class.

---

Clonning PyTCP from the GitHub repository

In most cases, PyTCP should be cloned directly from the GitHub repository, as this type of installation provides full development and testing environment.

git clone http://github.com/ccie18643/PyTCP

After cloning, we can run one of the included examples:

• Go to the stack root directory (it is called 'PyTCP').
• Run the sudo make bridge command to create the 'br0' bridge if needed.
• Run the sudo make tap command to create the tap7 interface and assign it to the 'br0' bridge.
• Run the make command to create the proper virtual environment for devlopment and testing.
• Run . venv/bin/activate command to activate the virtual environment.
• Execute any example, e.g., example/run_stack.py.
• Hit Ctrl-C to stop it.

To fine-tune various stack operational parameters, please edit the pytcp/config.py file accordingly.

---

Installing PyTCP from the PyPi repository

PyTCP can also be installed as a regular module from the PyPi repository.

python -m pip install PyTCP

After installation, please ensure the TAP interface is operational and added to the bridge.

sudo ip tuntap add name tap7 mode tap
sudo ip link set dev tap7 up
sudo brctl addbr br0
sudo brctl addif br0 tap7

The PyTCP stack can be imported and started using the following code. It starts the stack subsystems and autoconfigures both the IPv4 and IPv6 protocol addresses using DHCPv4 and IPv6 SLAAC, respectively.

from pytcp import TcpIpStack
stack = TcpIpStack(interface="tap7")
stack.start()

The stack subsystems run in their own threads. After starting, the stack gives control back to the user code and can be stopped using the following call.

stack.stop()

---

Features

Already implemented:

• Stack - Fast Packet Parser using 'zero copy' approach.
• Stack - Fast Packet Assembler using 'zero copy' approach.
• Stack - MAC address manipulation library - Compatible with buffer protocol (Memoryview).
• Stack - IPv4 address manipulation library - Compatible with buffer protocol (Memoryview) and not dependent on Python standard library.
• Stack - IPv6 address manipulation library - Compatible with buffer protocol (Memoryview) and not dependent on Python standard library.
• Code - Unit testing for some of the libraries and modules (based on Facebook's Testslide framework)
• Ethernet protocol - Support of Ethernet II standard frames.
• Ethernet protocol - Unicast, IPv4 multicast, IPv6 multicast and broadcast addressing.
• ARP protocol - Replies, queries, ARP cache mechanism.
• ARP protocol - ARP Probe/Announcement IP conflict detection (ACD) mechanism.
• IPv4 protocol - Default routing, the stack can talk to hosts over the Internet using IPv4 protocol.
• IPv4 protocol - Automatic IPv4 address configuration using DHCPv4 protocol.
• IPv4 protocol - Inbound packet defragmentation, robust mechanism able to handle out-of-order and overlapping data fragments.
• IPv4 protocol - Outbound packet fragmentation.
• IPv4 protocol - IPv4 options accepted but not supported.
• IPv4 protocol - Multiple stack's IPv4 addresses supported, each of them acts as it was assigned to separate VRF
• ICMPv4 protocol - The echo-request, echo-reply, and port-unreachable messages.
• IPv6 protocol - Default routing, the stack can talk to hosts over the Internet using IPv6 protocol.
• IPv6 protocol - Automatic link-local address configuration using EUI64 and Duplicate Address Detection.
• IPv6 protocol - Automatic GUA address configuration using Router Advertisement / EUI64.
• IPv6 protocol - Automatic assignment of Solicited Node Multicast addresses.
• IPv6 protocol - Automatic assignment of IPv6 multicast MAC addresses.
• IPv6 protocol - Inbound packet defragmentation, robust mechanism able to handle out-of-order and overlapping data fragments.
• IPv6 protocol - Outbound packet fragmentation.
• ICMPv6 protocol - The echo-request, echo-reply, and port-unreachable messages.
• ICMPv6 protocol - Neighbor Discovery, Duplicate Address Detection.
• ICMPv6 protocol - Neighbor Discovery cache mechanism.
• ICMPv6 protocol - Multicast Listener Discovery v2 (MLDv2) protocol implementation (only messages needed by stack).
• UDP protocol - Full support. The stack can exchange data with other hosts using the UDP protocol.
• UDP sockets - Full support, stack's 'end user' API similar to Berkeley sockets.
• UDP services - The Echo, Discard, and Daytime services implemented for testing purposes (in 'examples').
• TCP protocol - Full implementation of TCP Finite State Machine. At this point, the stack can exchange bulk data with other hosts over the TCP protocol.
• TCP protocol - TCP options support for: MSS, WSCALE, SACKPERM, TIMESTAMP.
• TCP protocol - TCP sliding window mechanism with and data retransmission (fast retransmit and time-based scenarios).
• TCP protocol - TCP backoff mechanism / basic congestion control.
• TCP protocol - TCP SYN/FIN packet retransmission.
• TCP sockets - Full support, stack's 'end user' API similar to Berkeley sockets

To be implemented:

• [ ] Ethernet II - Ensure the minimum value for the 'type' field is 0x600, part of integrity checks
• [ ] Protocols - Refactor integrity and sanity messages to be more consistent and provide more detail on the issue, eg. length of the bad packet, etc.
• [ ] IPv4/TCP/ICMPv6 - Reconsider mechanism of adding options after creating initial options object.
• [ ] IMCPv4 - Add proper assert messages.
• [ ] IMCPv6 - Add proper assert messages.
• [ ] ICMPv4 - *Refactor unit tests to test assembling and parsing