GitHub

Pipes are small components for writing expressive code when working on collections. Pipes chain together into a pipeline that receives data from a source, operates on that data, and send the results to a destination.

This is a header-only library, implemented in C++14.

The library is under development and subject to change. Contributions are welcome. You can also log an issue if you have a wish for enhancement or if you spot a bug.

A First Example
A Second Example
Doesn't it look like ranges?
Operating on several collections
End pipes
Easy integration with STL algorithms
Streams support
List of available pipes

A First Example

Here is a simple example of a pipeline made of two pipes: transform and filter:

auto const source = std::vector<int>{0, 1, 2, 3, 4, 5, 6, 7, 8, 9};
auto destination = std::vector<int>{};

source >>= pipes::filter([](int i){ return i % 2 == 0; })
       >>= pipes::transform([](int i){ return i * 2; })
       >>= pipes::push_back(destination);

// destination contains {0, 4, 8, 12, 16};

What's going on here:

Each elements of source is sent to filter.
Every time filter receives a piece of data, it sends its to the next pipe (here, transform) only if that piece of data satisfies filter's' predicate.
transform then applies its function on the data its gets and sends the result to the next pipe (here, pipes::push_back).
pipes::push_back push_backs the data it receives to its vector (here, destination).

A Second Example

Here is a more elaborate example with a pipeline that branches out in several directions:

A >>= pipes::transform(f)
  >>= pipes::filter(p)
  >>= pipes::unzip(pipes::push_back(B),
                   pipes::fork(pipes::push_back(C),
                               pipes::filter(q) >>= pipes::push_back(D),
                               pipes::filter(r) >>= pipes::push_back(E));

Here, unzip takes the std::pairs or std::tuples it receives and breaks them down into individual elements. It sends each element to the pipes it takes (here pipes::push_back and fork).

fork takes any number of pipes and sends the data it receives to each of them.

Since data circulates through pipes, real life pipes and plumbing provide a nice analogy (which gave its names to the library). For example, the above pipeline can be graphically represented like this:

Doesn't it look like ranges?

Pipes sort of look like ranges adaptors from afar, but those two libraries have very different designs.

Range views are about adapting ranges with view layers, and reading through those layers in lazy mode. Ranges are "pull based", in that components ask for the next value. Pipes are about sending pieces of data as they come along in a collection through a pipeline, and let them land in a destination. Pipes are "push based", in that components wait for the next value.

Ranges and pipes have overlapping components such as transform and filter. But pipes do things like ranges can't do, such as pipes::mux, pipes::fork and pipes:unzip, and ranges do things that pipes can't do, like infinite ranges.

It is possible to use ranges and pipes in the same expression though:

ranges::view::zip(dadChromosome, momChromosome)
    >>= pipes::transform(crossover) // crossover takes and returns a tuple of 2 elements
    >>= pipes::unzip(pipes::push_back(gameteChromosome1),
                     pipes::push_back(gameteChromosome2));

Operating on several collections

The pipes library allows to manipulate several collections at the same time, with the pipes::mux helper. Note that contrary to range::view::zip, pipes::mux doesn't require to use tuples:

auto const input1 = std::vector<int>{1, 2, 3, 4, 5};
auto const input2 = std::vector<int>{10, 20, 30, 40, 50};

auto results = std::vector<int>{};

pipes::mux(input1, input2) >>= pipes::filter   ([](int a, int b){ return a + b < 40; })
                           >>= pipes::transform([](int a, int b) { return a * b; })
                           >>= pipes::push_back(results);

// results contains {10, 40, 90}

Operating on all the possible combinations between several collections

pipes::cartesian_product takes any number of collections, and generates all the possible combinations between the elements of those collections. It sends each combination successively to the next pipe after it.

Like pipes::mux, pipes::cartesian_product doesn't use tuples but sends the values directly to the next pipe:

auto const inputs1 = std::vector<int>{1, 2, 3};
auto const inputs2 = std::vector<std::string>{"up", "down"};

auto results = std::vector<std::string>{};

pipes::cartesian_product(inputs1, inputs2)
    >>= pipes::transform([](int i, std::string const& s){ return std::to_string(i) + '-' + s; })
    >>= pipes::push_back(results);

// results contains {"1-up", "1-down", "2-up", "2-down", "3-up", "3-down"}

Operating on adjacent elements of a collection

pipes::adjacent allows to send adjacent pairs of element from a range to a pipeline:

auto const input = std::vector<int>{1, 2, 4, 7, 11, 16};

auto results = std::vector<int>{};

pipes::adjacent(input)
    >>= pipes::transform([](int a, int b){ return b - a; })
    >>= pipes::push_back(results);

// result contains {1, 2, 3, 4, 5};

Operating on all combinations of elements of one collection

pipes::combinations sends each possible couple of different elements of a range to a pipeline:

auto const inputs = std::vector<int>{ 1, 2, 3, 4, 5 };

auto results = std::vector<std::pair<int, int>>{};

 pipes::combinations(inputs)
     >>= pipes::transform([](int i, int j){ return std::make_pair(i, j); })
     >>= pipes::push_back(results);
     
 /*
 results contains:
 {
    { 1, 2 }, { 1, 3 }, { 1, 4 }, { 1, 5 },
              { 2, 3 }, { 2, 4 }, { 2, 5 },
                        { 3, 4 }, { 3, 5 },
                                  { 4, 5 }
 }
 /*

End pipes

This library also provides end pipes, which are components that send data to a collection in an elaborate way. For example, the map_aggregate pipe receives std::pair<Key, Value>s and adds them to a map with the following rule:

if its key is not already in the map, insert the incoming pair in the map,
otherwise, aggregate the value of the incoming pair with the existing one in the map.

Example:

std::map<int, std::string> entries = { {1, "a"}, {2, "b"}, {3, "c"}, {4, "d"} };
std::map<int, std::string> entries2 = { {2, "b"}, {3, "c"}, {4, "d"}, {5, "e"} };
std::map<int, std::string> results;

// results is empty

entries >>= pipes::map_aggregator(results, concatenateStrings);

// the elements of entries have been inserted into results

entries2 >>= pipes::map_aggregator(results, concatenateStrings);

// the new elements of entries2 have been inserter into results, the existing ones have been concatenated with the new values 
// results contains { {1, "a"}, {2, "bb"}, {3, "cc"}, {4, "dd"}, {5, "e"} }

All components are located in the namespace pipes.

Easy integration with STL algorithms

All pipes can be used as output iterators of STL algorithms:

std::set_difference(begin(setA), end(setA),
                    begin(setB), end(setB),
                    transform(f) >>= filter(p) >>= map_aggregator(results, addValues));

Streams support

The contents of an input stream can be sent to a pipe by using read_in_stream. The end pipe to_out_stream sends data to an output stream.

The following example reads strings from the standard input, transforms them to upper case, and sends them to the standard output:

std::cin >>= pipes::read_in_stream<std::string>{}
         >>= pipes::transform(toUpper)
         >>= pipes::to_out_stream(std::cout);

List of available pipes

General pipes
- dev_null
- drop
- drop_while
- filter
- fork
- partition
- read_in_stream
- switch
- stride
- take
- tee
- transform
- unzip
End pipes

General pipes

`dev_null`

dev_null is a pipe that doesn't do anything with the value it receives. It is useful for selecting only some data coming out of an algorithm that has several outputs. An example of such algorithm is set_segregate:

std::set<int> setA = {1, 2, 3, 4, 5};
std::set<int> setB = {3, 4, 5, 6, 7};

std::vector<int> inAOnly;
std::vector<int> inBoth;

sets::set_seggregate(setA, setB,
                     pipes::push_back(inAOnly),

Pipes

Install / Use

README

Contents