METARESC: META data and RESource library for C language

METARESC is a software library that implements reflection in the C programming language. It allows to derive fields metadata from declarations of structures and unions. You may use a special METARESC grammar for types declaration or get metadata from DWARF debug symbols or __builtin_dump_struct function available in Clang. The variables of discovered types may be subsequently serialized into various formats (XML, JSON, YAML, XDR) and deserialized later. To achieve this the library adds metadata annotations to types. These annotations may be used for many purposes far beyond just achieving persistence: deep copy, recursive memory deallocation, automatic hashing and comparation of structures, generic sorting.

<!-- markdown-toc start - Don't edit this section. Run M-x markdown-toc-refresh-toc -->

Table of Contents

• METARESC: META data and RESource library for C language
  • How to build library:
    • Ubuntu
    • MacOs
    • FreeBSD
    • Windows
  • How to build a sample app
  • Serialization/deserialization interface
    • Serialization
    • Deserialization
    • MR_PRINT helper macro
  • Types declaration macro language
    • Structure type declaration
      • Fields of a basic types
      • Extended semantics for fields declaration
      • Non-serializable fields
      • Field declaration
      • Pointer declaration
      • Function pointer declaration
      • Bitfields declaration
      • Union declaration
      • Array declaration
      • Text metadata and resource information
      • NULL-terminated strings vs character arrays
    • Union type declaration
    • Enumeration type declaration
    • Function pointer type declaration
    • Metadata and resources definition for types
  • Extra features
    • Deep copy
    • Recursive memory deallocation
    • Objects hashing
    • Comparation of structures
    • Generic sorting
    • Structure of serialization graph
    • Access and structure of type descriptor
    • Access and structure of field descriptor
    • Enumerations metadata
  • Internals
    • Error handling
    • Memory allocation
    • How to make certain field in type non-serializable
    • Formatted output to a resizable array
    • Indexing framework

<!-- markdown-toc end -->

How to build library:

Install external dependencies autoconf, automake, libtool, pkg-config, flex, bison, libxml2-dev, libyaml-dev, libtirpc-dev, check. Clone Metaresc from Github, configure and build according to the standard autoconf/automake process.

Ubuntu

On Ubuntu, run the following command:

# apt-get install git autoconf automake libtool pkg-config flex bison libxml2-dev libyaml-dev libtirpc-dev check

Checkout Metaresc from github

$ git clone https://github.com/alexanderchuranov/Metaresc.git
$ cd Metaresc
$ git submodule update --init --recursive --remote

Run autoconf/automake generators:

$ ./autogen.sh

Configure project for target system

$ ./configure CFLAGS=-I/usr/include/tirpc LIBS=-ltirpc

Build and check library

$ make -j 4 check

MacOs

On MacOs install developer tools first:

$ xcode-select --install

Install external dependencies:

# brew install autoconf automake libtool pkg-config flex bison libxml2 libyaml check

Checkout and build Metaresc:

$ git clone https://github.com/alexanderchuranov/Metaresc.git
$ cd Metaresc
$ git submodule update --init --recursive --remote
$ ./autogen.sh
$ ./configure HAVE_BISON=yes YACC=$(brew --prefix)/opt/bison/bin/bison
$ make -j 4 check

FreeBSD

On FreeBSD 12.1 install external dependencies:

# pkg install git autoconf automake libtool pkgconf flex bison libxml2 libyaml check

Checkout and build Metaresc:

$ git clone https://github.com/alexanderchuranov/Metaresc.git
$ cd Metaresc
$ git submodule update --init --recursive --remote
$ ./autogen.sh
$ ./configure CFLAGS=-I/usr/local/include/
$ make -j 4 check

Windows

Download and install msys2 from https://www.msys2.org/

Install external dependencies:

# pacman --noconfirm -S --needed --overwrite "*" base-devel git autoconf automake libtool pkg-config flex bison libyaml-devel mingw-w64-x86_64-check mingw-w64-x86_64-toolchain

Checkout and build Metaresc:

$ git clone https://github.com/alexanderchuranov/Metaresc.git
$ cd Metaresc
$ git submodule update --init --recursive --remote
$ ./autogen.sh
$ ./configure --without-libxml2 --disable-static --enable-shared
$ sed -i.bak -e "s/\(allow_undefined=\)yes/\1no/" libtool
$ make -j 4 check

How to build a sample app

Instruction below will cover scenario when Metaresc is build locally and is not installed into the system.

Create a folder at the same level as Metaresc. Start with hello world stub that will use metaresc.h.

#include <metaresc.h>
  
int main (int argc, char * argv[])
{
  printf ("Hello world!\n");
  return (EXIT_SUCCESS);
}

Compiler should be instructed where to find Metaresc header file, so we should add -I../Metaresc/src as a compilation flag. Minimal Makefile for this app should be as follows:

all: sample

CFLAGS += -I../Metaresc/src `xml2-config --cflags` -lyaml

As a next step we will add some custom structure type definition, variable of that type and serialization of this variable into format consumable by C compiler as a static initialization of that type (CINIT). For demonstration purposes sample app will define type for the binary tree node with char * value.

Metaresc provides two basic layers that enables reflection in C. First is a macro language that replaces standard type definition semantics. Second is an API to introspect meta-data generated by macro language and serialize structures according to this metadata. Macro language is pretty similar to standard semantics of types definition in C, but still requires some learning curve. In the following example it should be self-explanatory.

#include <metaresc.h>

TYPEDEF_STRUCT (tree_node_t,
		(char *, value),
		(tree_node_t *, left),
		(tree_node_t *, right),
		);
  
int main (int argc, char * argv[])
{
  tree_node_t root = {
    "root",
    (tree_node_t[]){ { "left" } },
    (tree_node_t[]){ { "right" } },
  };
  
  MR_PRINT ("tree = ", (tree_node_t, &root));
  return (EXIT_SUCCESS);
}

At this point we need to link our application against statically build Metaresc library files. Makefile should be extended as follows:

all: sample

CFLAGS += -I../Metaresc/src `xml2-config --cflags`
LDLIBS += ../Metaresc/src/.libs/libmetaresc.a `xml2-config --libs` -lyaml

Output of this sample application is as follows:

tree = {
  .value = "root",
  .left = (tree_node_t[]){
    {
      .value = "left",
      .left = NULL,
      .right = NULL
    }
  },
  .right = (tree_node_t[]){
    {
      .value = "right",
      .left = NULL,
      .right = NULL
    }
  }
}

You may find custom macro language for types definition as an overkill. In this case you could use Metaresc macro language only for metadata generation and keep definitions of your types in a plain C. It basically means that you need to duplicate type definition: once in standard C and one more time with a Metaresc macro language. In this case global preprocessor variable MR_MODE should be defined into DESC. Example above will look as follows:

#include <metaresc.h>

typedef struct tree_node_t {
  char * value;
  struct tree_node_t * left;
  struct tree_node_t * right;
} tree_node_t;

#define MR_MODE DESC
TYPEDEF_STRUCT (tree_node_t,
		(char *, value),
		(tree_node_t *, left),
		(tree_node_t *, right),
		);
  
int main (int argc, char * argv[])
{
  tree_node_t root = {
    "root",
    (tree_node_t[]){ { "left" } },
    (tree_node_t[]){ { "right" } },
  };
  
  MR_PRINT ("tree = ", (tree_node_t, &root));
  return (EXIT_SUCCESS);
}

Metadata generation macro will validate at compile time that provided types matches types of the structure fields, but it is still on user to synchronize the list of fields with the original declaration. Order of the fields could be arbitrary and library will reorder them based on offset for use in serialization. Necessity to keep types of fields in sync might be annoying and overwhelming. The good news is that you could omit types specification in macro language for all fields of basic types (boolean, integer, float, complex, single characters,