BinaryCodable

<p align="center"> <img src="assets/logo.png" width="500" max-width="90%" alt="BinaryCodable" /> </p> <p align="center"> <img src="assets/swift.svg" /> <!-- https://img.shields.io/badge/Swift-6-orange.svg --> <img src="assets/platforms.svg" /> <!-- https://img.shields.io/badge/Platforms-iOS_|_macOS_|_Linux_|_tvOS_|_watchOS-green.svg --> <a href="https://github.com/christophhagen/BinaryCodable/actions/workflows/tests.yml" style="text-decoration: none;"><img src="https://github.com/christophhagen/BinaryCodable/actions/workflows/tests.yml/badge.svg" /></a> <a href="https://docs.christophhagen.de/documentation/binarycodable" style="text-decoration: none;"> <img src="assets/docs.svg" /> <!-- https://img.shields.io/badge/documentation-100%25-green.svg --> </a> </p>

This package provides convenient encoding and decoding to/from binary data for all Swift Codable types.

Use cases

There are only few encoders and decoders available for Swift's Codable format, and Apple provides a JSONEncoder and a PropertyListEncoder for basic encoding. While these can cover some use cases (especially when interacting with Web Content through JSON), they lack encoding efficiency when designing APIs within an ecosystem. JSON, for example, is notoriously inefficient when it comes to binary data.

One very popular alternative for binary data are Google's Protocol Buffers, which offer broad support across different platforms and programming languages. But they don't support Swift's Codable protocol, and thus require manual message definitions, the Protobuf compiler, and a lot of copying between data structures during encoding and decoding.

So if you're looking for a decently efficient binary encoder in a pure Swift project, then BinaryCodable may be right for you. Simply make your structs (or classes!) conform to Codable, and BinaryCodable does the rest!

Alternatives

Protocol Buffers

Efficient binary format, and broad support across different platforms and programming languages. Much more limited than Codable in terms of allowed types. Message definitions must be written in separate files and compiled to Swift code. The library for swift is swift-protobuf.

ProtobufCodable

A Codable-compatible implementation of the Protocol Buffer binary format. It has the same limitations on the allowed types as swift-protobuf, but message definitions can be written directly in Swift. Slower during encoding and decoding than swift-protobuf.

LegacyBinaryCodable

A mirror of the older V2 version of BinaryCodable, which should only be used to migrate encoded data to a different format.

CBORCoding

If you're looking for a Codable-compatible alternative which is also available on other platforms, with a well-defined spec. It appears to have nearly the same encoding efficiency as BinaryCodable. Suffers from the double-optional bug.

PotentCodables

Also offers CBOR encoding for Codable types, plus a bunch of other things related to Codable. Suffers from the double-optional bug.

Swift BSON

Encoding according to the BSON specification. Less efficient binary represenation than Protocol Buffers and BinaryCodable, but mature. Used for MongoDB. There is also another implementation.

JSONEncoder/JSONDecoder

The Foundation module provides JSON encoding, which is String-based and therefore not efficient. Also has the double-optional bug.

Installation

Swift Package Manager

Simply include in your Package.swift:

dependencies: [
    .package(url: "https://github.com/christophhagen/BinaryCodable", from: "4.0.0")
],
targets: [
    .target(name: "MyTarget", dependencies: [
        .product(name: "BinaryCodable", package: "BinaryCodable")
    ])
]

Xcode project

Select your Project, navigate to the Package Dependencies tab, and add https://github.com/christophhagen/BinaryCodable using the + button.

Usage

Documentation of the library is available here (generated using Swift DocC).

First steps

Let's assume a message definition:

struct Message: Codable {

    var sender: String
    
    var isRead: Bool
    
    var unreadCount: Int
}

Simply import the module where you need to encode or decode a message:

import BinaryCodable

Encoding

Construct an encoder when converting instances to binary data, and feed the message(s) into it:

let message: Message = ...

let encoder = BinaryEncoder()
let data = try encoder.encode(message)

It's also possible to encode single values, arrays, optionals, sets, enums, dictionaries, and more, so long as they conform to Codable.

Decoding

Decoding instances from binary data works much the same way:

let decoder = BinaryDecoder()
let message = try decoder.decode(Message.self, from: data)

Alternatively, the type can be inferred:

let message: Message = try decoder.decode(from: data)

Custom encoding and decoding

BinaryCodable supports the use of custom encoding and decoding routines by implementing encode(to:) and init(from:).

There is only one aspect that's handled differently than the Codable documentation specifies, which is the explicit encoding of nil in keyed containers. Calling encodeNil(forKey:) on a keyed container has no effect, there is no explicit nil value encoded for the key. This results in the contains() function during decoding returning false for the key. This is different to e.g. JSON, where calling encodeNil(forKey:) would cause the following encoding:

{ 
    "myProperty" : null
}

The implementation of encodeNil(forKey:) and decodeNil(forKey:) handles this case differently, because the alternatives are not optimal: It would be possible to explicitly encode nil for a key, but this would cause problems with double optionals in structs (e.g. Int??), which could no longer distinguish between .some(nil) and nil. To fix this issue, an additional nil indicator would be needed for all values in keyed containers, which would decrease the efficiency of the binary format. That doesn't seem reasonable just to support a rarely used feature, since encodeNil(forKey:) is never called for automatically synthesized Codable conformances.

The recommendation therefore is to use encodeIfPresent(_, forKey:) and decodeIfPresent(_, forKey:). Another option would be to use a double optional, since this is basically the information encodeNil provides: nil, if the key is not present, .some(nil), if the key is present with nil, and value, if the key is present with a value.

Errors

It's possible for both encoding and decoding to fail. Encoding can produce EncodingError errors, while unsuccessful decoding produces DecodingErrors. Both are the default Errors provided by Swift, supplied with additional information describing the nature of the error. See the documentation of the types to learn more about the different error conditions.

Handling corrupted data

The binary format provides no provisions to detect data corruption, and various errors can occur as the result of added, changed, or missing bytes and bits. Additional external measures (checksums, error-correcting codes, ...) should be applied if there is an increased risk of data corruption.

As an example, consider the simple encoding of a String inside a struct, which consists of a key followed by the length of the string in bytes, and the string content. The length of the string is encoded using variable-length encoding, so a single bit flip (in the MSB of the length byte) could result in a very large length being decoded, causing the decoder to wait for a very large number of bytes to decode the string. This simple error would cause much data to be skipped, potentially corrupting the data stream indefinitely. At the same time, it is not possible to determine with certainty where the error occured, making error recovery difficult without additional information about boundaries between elements.

The decoding errors provided by the library are therefore only hints about errors likely occuring from non-conformance to the binary format or version incompatibility, which are not necessarily the true causes of the failures when data corruption is present.

Coding Keys

The Codable protocol uses CodingKey definitions to identify properties of instances. By default, coding keys are generated using the string values of the property names.

Similar to JSON encoding, BinaryCodable can embed the property names in the encoded data.

Unlike JSON (which is human-readable), the binary representation produced by BinaryCodable is intended for cases when efficient encoding is important. Codable allows the use of integer keys for each property, which significantly increases encoding efficiency. You can specify integer keys by adding an `