MemoryPack

Zero encoding extreme performance binary serializer for C# and Unity.

Compared with System.Text.Json, protobuf-net, MessagePack for C#, Orleans.Serialization. Measured by .NET 7 / Ryzen 9 5950X machine. These serializers have IBufferWriter<byte> method, serialized using ArrayBufferWriter<byte> and reused to avoid measure buffer copy.

For standard objects, MemoryPack is x10 faster and x2 ~ x5 faster than other binary serializers. For struct array, MemoryPack is even more powerful, with speeds up to x50 ~ x200 greater than other serializers.

MemoryPack is my 4th serializer, previously I've created well known serializers, ~~ZeroFormatter~~, ~~Utf8Json~~, MessagePack for C#. The reason for MemoryPack's speed is due to its C#-specific, C#-optimized binary format and a well tuned implementation based on my past experience. It is also a completely new design utilizing .NET 7 and C# 11 and the Incremental Source Generator (.NET Standard 2.1 (.NET 5, 6) and there is also Unity support).

Other serializers perform many encoding operations such as VarInt encoding, tag, string, etc. MemoryPack format uses a zero-encoding design that copies as much C# memory as possible. Zero-encoding is similar to FlatBuffers, but it doesn't need a special type, MemoryPack's serialization target is POCO.

Other than performance, MemoryPack has these features.

• Support modern I/O APIs (IBufferWriter<byte>, ReadOnlySpan<byte>, ReadOnlySequence<byte>)
• Native AOT friendly Source Generator based code generation, no Dynamic CodeGen (IL.Emit)
• Reflectionless non-generics APIs
• Deserialize into existing instance
• Polymorphism (Union) serialization
• Limited version-tolerant (fast/default) and full version-tolerant support
• Circular reference serialization
• PipeWriter/Reader based streaming serialization
• TypeScript code generation and ASP.NET Core Formatter
• Unity (2021.3) IL2CPP Support via .NET Source Generator

Installation

This library is distributed via NuGet. For best performance, recommend to use .NET 7. Minimum requirement is .NET Standard 2.1.

PM> Install-Package MemoryPack

And also a code editor requires Roslyn 4.3.1 support, for example Visual Studio 2022 version 17.3, .NET SDK 6.0.401. For details, see the Roslyn Version Support document.

For Unity, the requirements and installation process are completely different. See the Unity section for details.

Quick Start

Define a struct or class to be serialized and annotate it with the [MemoryPackable] attribute and the partial keyword.

using MemoryPack;

[MemoryPackable]
public partial class Person
{
    public int Age { get; set; }
    public string Name { get; set; }
}

Serialization code is generated by the C# source generator feature which implements the IMemoryPackable<T> interface. In Visual Studio you can check a generated code by using a shortcut Ctrl+K, R on the class name and select *.MemoryPackFormatter.g.cs.

Call MemoryPackSerializer.Serialize<T>/Deserialize<T> to serialize/deserialize an object instance.

var v = new Person { Age = 40, Name = "John" };

var bin = MemoryPackSerializer.Serialize(v);
var val = MemoryPackSerializer.Deserialize<Person>(bin);

Serialize method supports a return type of byte[] as well as it can serialize to IBufferWriter<byte> or Stream. Deserialize method supports ReadOnlySpan<byte>, ReadOnlySequence<byte> and Stream. And there are alse non-generics versions.

Built-in supported types

These types can be serialized by default:

• .NET primitives (byte, int, bool, char, double, etc.)
• Unmanaged types (Any enum, Any user-defined struct which doesn't contain reference types)
• string, decimal, Half, Int128, UInt128, Guid, Rune, BigInteger
• TimeSpan, DateTime, DateTimeOffset, TimeOnly, DateOnly, TimeZoneInfo
• Complex, Plane, Quaternion Matrix3x2, Matrix4x4, Vector2, Vector3, Vector4
• Uri, Version, StringBuilder, Type, BitArray, CultureInfo
• T[], T[,], T[,,], T[,,,], Memory<>, ReadOnlyMemory<>, ArraySegment<>, ReadOnlySequence<>
• Nullable<>, Lazy<>, KeyValuePair<,>, Tuple<,...>, ValueTuple<,...>
• List<>, LinkedList<>, Queue<>, Stack<>, HashSet<>, SortedSet<>, PriorityQueue<,>
• Dictionary<,>, SortedList<,>, SortedDictionary<,>, ReadOnlyDictionary<,>
• Collection<>, ReadOnlyCollection<>, ObservableCollection<>, ReadOnlyObservableCollection<>
• IEnumerable<>, ICollection<>, IList<>, IReadOnlyCollection<>, IReadOnlyList<>, ISet<>
• IDictionary<,>, IReadOnlyDictionary<,>, ILookup<,>, IGrouping<,>,
• ConcurrentBag<>, ConcurrentQueue<>, ConcurrentStack<>, ConcurrentDictionary<,>, BlockingCollection<>
• Immutable collections (ImmutableList<>, etc.) and interfaces (IImmutableList<>, etc.)

Define [MemoryPackable] class / struct / record / record struct

[MemoryPackable] can annotate to any class, struct, record, record struct and interface. If a type is struct or record struct which contains no reference types (C# Unmanaged types) any additional annotation (ignore, include, constructor, callbacks) is not used, that serialize/deserialize directly from the memory.

Otherwise, by default, [MemoryPackable] serializes public instance properties or fields. You can use [MemoryPackIgnore] to remove serialization target, [MemoryPackInclude] promotes a private member to serialization target.

[MemoryPackable]
public partial class Sample
{
    // these types are serialized by default
    public int PublicField;
    public readonly int PublicReadOnlyField;
    public int PublicProperty { get; set; }
    public int PrivateSetPublicProperty { get; private set; }
    public int ReadOnlyPublicProperty { get; }
    public int InitProperty { get; init; }
    public required int RequiredInitProperty { get; init; }

    // these types are not serialized by default
    int privateProperty { get; set; }
    int privateField;
    readonly int privateReadOnlyField;

    // use [MemoryPackIgnore] to remove target of a public member
    [MemoryPackIgnore]
    public int PublicProperty2 => PublicProperty + PublicField;

    // use [MemoryPackInclude] to promote a private member to serialization target
    [MemoryPackInclude]
    int privateField2;
    [MemoryPackInclude]
    int privateProperty2 { get; set; }
}

MemoryPack's code generator adds information about what members are serialized to the <remarks /> section. This can be viewed by hovering over the type with Intellisense.

All members must be memorypack-serializable, if not the code generator will emit an error.

MemoryPack has 35 diagnostics rules (MEMPACK001 to MEMPACK035) to be defined comfortably.

If target type is defined MemoryPack serialization externally and registered, use [MemoryPackAllowSerialize] to silent diagnostics.

[MemoryPackable]
public partial class Sample2
{
    [MemoryPackAllowSerialize]
    public NotSerializableType? NotSerializableProperty { get; set; }
}

Member order is important, MemoryPack does not serialize the member-name or other information, instead serializing fields in the order they are declared. If a type is inherited, serialization is performed in the order of parent → child. The order of members can not change for the deserialization. For the schema evolution, see the Version tolerant section.

The default order is sequential, but you can choose the explicit layout with [MemoryPackable(SerializeLayout.Explicit)] and [MemoryPackOrder()].

// serialize Prop0 -> Prop1
[MemoryPackable(SerializeLayout.Explicit)]
public partial class SampleExplicitOrder
{
    [MemoryPackOrder(1)]
    public int Prop1 { get; set; }
    [MemoryPackOrder(0)]
    public int Prop0 { get; set; }
}

Constructor selection

MemoryPack supports both parameterized and parameterless constructors. The selection of the constructor follows these rules. (Applies to classes and structs).

• If there is [MemoryPackConstructor], use it.
• If there is no explicit constructor (including private), use a parameterless one.
• If there is one parameterless/parameterized constructor (including private), use it.
• If there are multiple constructors, then the [MemoryPackConstructor] attribute must be applied to the desired constructor (the generator will not automatically choose one), otherwise the generator will emit an error.
• If using a parameterized constructor, all parameter names must match corresponding member names (case-insensitive).

[MemoryPackable]
public partial class Perso