Ioscript

ioscript

ioscript is an interface for calling code written in higher level scripting languages directly from C++. The library motivates writing small function objects that embed useful scripting behaviour, either to be called independently or as small scripting 'snippets' to compose as a sequence. Function objects may also be bound to C++ types and so provide a method to invoke script code directly on C++ objects.

The original use case was for quickly plotting C++ objects to debug scientific code using matplotlib and Gnuplot, however you might find if useful in any context where it's helpful to bind script code directly with C++ types for further processing.

    // Example usage: Plot vec1 & vec2 using the python code wrapped in a LineChart,
    //  and vec3 using the python code wrapped in a BarChart, then Show to screen
    Script<Python,MyTypes> script;
    script.run(LineChart{}, vec1, vec2, BarChart{}, vec3, Show{});

The library is agnostic to the scripting language, it should work with any runtime that's installed on your system and accepts code via it's standard input. There is a set of examples for using the library with Python, Ruby and Gnuplot, but depending on the language it should be reasonably straightforward to set up with different runtimes.

Use

The library is a single header plus optional definitions for each language's runtime. For C++17/1z there are no further dependencies, otherwise there is a toggle to switch to boost for [Boost.Variant][link_boost_variant] (also header-only) (edit: but currently broken).

The interface is currently implemented in terms of POSIX pipes and so unfortunately there is no Windows support. Tested only on macOS, although I expect it to work fine on linux.

Example

The following example plots the values of a std::vector with Python and the matplotlib plotting library.

#include "ioscript/ioscript.h"
#include "ioscript/python.h"
#include <vector>

using namespace iosc;

struct LineChart
{
    template <typename T>
    void operator()(Process<Python>& python, const T& obj) const
    {
        python << R"(
import matplotlib.pyplot as plt
vals = map(int, iosc_in[0].readline().split())
plt.plot(vals, 'o-')
)";
        for (auto& elem : obj) {
            python.data_out(0) << elem << " ";
        }
        python.data_out(0) << std::endl;
    }
};

struct Show {
    void operator()(Process<Python>& python) const { python << "plt.show()\n"; }
};

// Associate std::vector with a LineChart
template <>
struct binds_to<std::vector<int>> { using type = variant<LineChart>; };

void main()
{
    std::vector<int> series1{0,1,1,2,3,5,8,13,21,34,55,89};
    std::vector<int> series2{0,1,3,6,10,15,21,28,36,45,55,66,78,91};

    using MyTypes = std::tuple<std::vector<int>>;
    Script<Python,MyTypes> script;

    script.run(LineChart{}, series1, series2, Show{});
}

The sections below outline the basic parts of the above example and for using ioscript in general.

Wrapping script code in C++ function objects

Function objects like LineChart embed the scripting code to define the intended behaviour, for example the Python code used to describe how to plot a data type. For functors that take an object as a second parameter, immediately following is a set of C++ statements to accept the object and describe how that data is sent to the Python subprocess. In simple cases, since the two parts are kept together there's a direct continuity between how the data is sent and how the data is received.

In the above example, the call python.iosc_out(0) returns a std::ostream to send the data to the Python subprocess on 'channel 0'. In Python, the read end of channel 0 is represented by the file object iosc_in[0].

If the embedded script code doesn't operate on any C++ object the second parameter to the overloaded operator() can be omitted. Since the library only requires that the function object is callable, you could write the code inline with a lambda:

script.run(LineChart{}, series1, series2, [](Process<Python>& py) {
    py << "plt.savefig('vals1.png')" << std::endl;
});

Alternatively keep it with a name:

struct Title {
    void operator()(Process<Python>& python) const {
        python << "plt.title('" << title << "')" << std::endl;
    }
    const char* title = "Default title";
};

Function objects passed to ioscript must be default, copy and copy-assign constructable, but in broad terms the library imposes few requirements on the nature of the functor classes.

Binding script code to C++ types

The following specifies the relationship that should we pass an object of type std::vector to run(), we want ioscript to call a LineChart function object on that std::vector. To do this, first wrap LineChart in a variant (of which we only have one alternative) and then add a specialization for the type trait binds_to<std::vector<int>> to refer to this variant:

// std::vector currently binds only to one alternative (LineChart)
template <>
struct binds_to<std::vector<int>> { using type = variant<LineChart>; };

Should you want to associate more data types to work with a LineChart, add additional binds_to<> statements. Templates work as expected, too. The following makes a std::array<T,N> of any type T and size N bind for use with our LineChart 'snippet':

template <typename  T, size_t N>
template<> struct binds_to<std::array<T,N>> { using type = variant<LineChart>; }

To call out one piece of housekeeping, you must register the types you expect to use in calls to run(). That is, you must register at least one of every type such that all snippet types are visible to ioscript. This isn't ideal, but is the fundamental mechanism by which the static binding can work. To do so, define a std::tuple that lists your types, and use this as the second parameter to instantiating a Script<Process,MyTypes>:

using MyTypes = std::tuple<std::vector<int>
                           std::array<int,0>>;  // `0` is sufficient for all array<int,N>

// Later
Script<Python, MyTypes> script;
script.run(...);

If you forget to do so ioscript should provide a useful error message to that effect, rather than leave you at the fate of your compiler's template error messages<sup id="br1">[1]</sup>.

Calling script code

To execute a sequence of script snippets, instantiate a Script<Python, MyTypes> object and call its run() member function, passing objects of your snippet types. The call to run() takes any number of parameters, with arguments always parsed from left to right and processed in sequential order:

std::vector<int> vec1, vec2;
std::array<int,10> arr1;
// Fill containers ...

Script<Python, MyStyles> script;
script.run(LineChart{}, Title{"Number sequences"}, vec1, vec2, arr1, Show{});

In the above example, the run() call will fork a new Python subprocess, keep a copy of our default-constructed LineChart, send our Title code to the Python interpreter and then call the LineChart copy with each vector and array argument in turn.

Notice each snippet object can have its own state. For example, you might modify LineChart to take a line colour parameter:

script.run(LineChart{'r'}, vec1, LineChart{'b'}, vec2, Show{});

The basic approach adopted is

• Any snippet that defines the one-parameter operator()(Process<P>&) will be called immediately in sequence.
• Any snippet that defines the two parameter form operator()(Process<P>&, const T&) will instead be called once with each object of type T that binds to it.
• A function object can contain either form of operator(), or both, with the above semantics applied to each form in turn.

Adding additional variations

It's possible to declare a C++ object to be bound to more than one snippet alternative, with the choice of functor to be selected as part of each call to run(). To do so, add alternative snippets as additional types to the relevant variant<>'s parameter list. For example, had we defined a BarChart and PieChart along similar lines to LineChart, then

// A std::vector<int> is now available for use with three alternative snippet types
struct binds_to<std::vector<int>> { using type = variant<LineChart,BarChart,PieChart>; };

Where more than one alternative exists, the alternative chosen is the 'most recent' type passed that binds to that object (that is, as if reading from left to right). For example

script.run(LineChart{}, vec1, vec2, BarChart{}, vec2, Show{});

will plot vec1 and vec2 as a LineChart, and vec2 as a BarChart.

If no snippet is explicitly specified, a default-initialized snippet of the type of the first alternative is constructed. This follows immediately from [std::variant][link_std_variant] or [boost::variant][link_boost_variant].

By default, objects passed that do not associate with any snippets will cause a deliberate compile error (under the assumption that there is never a purposeful reason to do so). You can change this behaviour in ioscript.h - the error will lead you directly to the line to change.

The Script constructor

Each call to run() forks a new subprocess, sends the sequence of code snippets and terminates the process. No state is shared between run calls. Whilst this is noticeably slower, this significantly helps reason about each plot call independently.<sup id="br2">[2]</sup>

If you do want to share state that's common to a number of successive runs, pass the snippets in the Script constructor:

Script<Gnuplot,MyTypes> script(ImageSize{800,600});

script.run(vec1);                       // plots 800x600
script.run(ImageSize{640,480}, vec2);   // plots 640x480
script.run(vec3);                       // plots 800x600

All snippets passed in the constructor are stored in a common 'header' that's replayed at the beginning of each