FuncFrog

FuncFrog is a library for performing efficient, parallel, lazy map, reduce, filter and many other operations on slices and other data sequences in a pipeline. The sequence can be set by a variety of generating functions. Everything is supported to be executed in parallel with minimal overhead on copying and locks. There is a built-in support of error handling with Yeet/Snag methods
The library is easy to use and has a clean, intuitive API.
You can measure performance comparing to vanilla for loop on your machine using cd perf/; make (spoiler: FuncFrog is better when multithreading).

Table of Contents

• Getting Started
• Basic information
• Supported functions list
  • Constructors
  • Set Pipe length
  • Split evaluation into n goroutines
  • Transform data
  • Retrieve a single element or perform a boolean check
  • Evaluate the pipeline
  • Transform Pipe from one type to another
  • Easy type conversion for Pipe[any]
  • Error handling
  • To be done
• Using prefix Pipe to transform Pipe type
• Using ff package to write shortened pipes
• Look for useful functions in Pipies package
• Examples
  • Basic example
  • Example using Func and Take
  • Example using Func and Gen
  • Example difference between Take and Gen
  • Example using Filter and Map
  • Example using Map and Reduce
  • Example of Map and Reduce with the underlying array type change
  • Example using Sort
  • Example of infine sequence generation
  • Example using Range and Map
  • Example using Repeat and Map
  • Example using Cycle and Filter
  • Example using Erase and Collect
  • Example of simple error handling
  • Example of multiple error handling
• Is this package stable?
• Contributions
• What's next?

Getting Started

To use FuncFrog in your project, run the following command:

go get github.com/koss-null/funcfrog

Then, import the library into your Go code (basically you need the pipe package):

import "github.com/koss-null/funcfrog/pkg/pipe"

You can then use the pipe package to create a pipeline of operations on a slice:

res := pipe.Slice(a).
    Map(func(x int) int { return x * x }).
    Filter(func(x *int) bool { return *x > 100 }).
    Parallel(12).
    Do()

All operations are carefully fenced with interfaces, so feel free to use anything, autosuggestion suggests you.

If you want it fast and short, you may use ff:

import "github.com/koss-null/funcfrog/pkg/ff"

res := ff.Map(strArr, strings.ToUpper).Do()

To see some code snippets, check out the Examples.

Basic information

The Piper (or PiperNoLen for pipes with undetermined lengths) is an interface that represents a lazy-evaluated sequence of data. The Piper interface provides a set of methods that can be used to transform, filter, collect and analyze data in the sequence. Every pipe can be conveniently copied at every moment just by equating it to a variable. Some methods (as Take or Gen) lead from PiperNoLen to Piper interface making wider method range available.

Supported functions list

The following functions can be used to create a new Pipe (this is how I call the inner representation of a sequence ofelements and a sequence operations on them):

Constructors

• :frog: Slice([]T) Piper: creates a Pipe of a given type T from a slice, the length is known.
• :frog: Func(func(i int) (T, bool)) PiperNL: creates a Pipe of type T from a function. The function returns an element which is considered to be at ith position in the Pipe, as well as a boolean indicating whether the element should be included (true) or skipped (false), the length is unknown.
• :frog: Fn(func(i int) (T)) PiperNL: creates a Pipe of type T from a function. The function should return the value of the element at the ith position in the Pipe; to be able to skip values use Func.
• :frog: FuncP(func(i int) (*T, bool)) PiperNL: creates a Pipe of type T from a function. The function returns a pointer to an element which is considered to be at ith position in the Pipe, as well as a boolean indicating whether the element should be included (true) or skipped (false), the length is unknown.
• :frog: Cycle(data []T) PiperNL: creates a new Pipe that cycles through the elements of the provided slice indefinitely. The length is unknown.
• :frog: Range(start, end, step T) Piper: creates a new Pipe that generates a sequence of values of type T from start to end (exclusive) with a fixed step value between each element. T can be any numeric type, such as int, float32, or float64. The length is known.
• :frog: Repeat(x T, n int) Piper: creates a new Pipe that generates a sequence of values of type T and value x with the length of n. The length is known.

Set Pipe length

• :frog: Take(n int) Piper: if it's a Func-made Pipe, expects n values to be eventually returned. Transforms unknown length to known.
• :frog: Gen(n int) Piper: if it's a Func-made Pipe, generates a sequence from [0, n) and applies the function to it. Transforms unknown length to known.

Split evaluation into n goroutines

• :frog: Parallel(n int) Pipe: sets the number of goroutines to be executed on (1 by default). This function can be used to specify the level of parallelism in the pipeline. Availabble for unknown length.

Transform data

• :frog: Map(fn func(x T) T) Pipe: applies the function fn to every element of the Pipe and returns a new Pipe with the transformed data. Available for unknown length.
• :frog: Filter(fn func(x *T) bool) Pipe: applies the predicate function fn to every element of the Pipe and returns a new Pipe with only the elements that satisfy the predicate. Available for unknown length.
• :frog: MapFilter(fn func(T) (T, bool)) Piper[T]: applies given function to each element of the underlying slice. If the second returning value of fn is false, the element is skipped (may be useful for error handling).
• :frog: Reduce(fn func(x, y *T) T) *T: applies the binary function fn to the elements of the Pipe and returns a single value that is the result of the reduction. Returns nil if the Pipe was empty before reduction.
• :frog: Sum(plus func(x, y *T) T) T: makes parallel reduce with associative function plus.
• :frog: Sort(less func(x, y *T) bool) Pipe: sorts the elements of the Pipe using the provided less function as the comparison function.

Retrieve a single element or perform a boolean check

• :frog: Any() T: returns a random element existing in the pipe. Available for unknown length.
• :frog: First() T: returns the first element of the Pipe, or nil if the Pipe is empty. Available for unknown length.
• :frog: Count() int: returns the number of elements in the Pipe. It does not allocate memory for the elements, but instead simply returns the number of elements in the Pipe.

Evaluate the pipeline

• :frog: Do() []T function is used to execute the pipeline and return the resulting slice of data. This function should be called at the end of the pipeline to retrieve the final result.

Transform Pipe from one type to another

• :frog: Erase() Pipe[any]: returns a pipe where all objects are the objects from the initial Pipe but with erased type. Basically for each x it returns any(&x). Use pipe.Collect[T](Piper[any]) PiperT to collect it back into some type (or pipe.CollectNL for slices with length not set yet).

Easy type conversion for Pipe[any]

• :frog: pipe.Collect[T](Piper[any]) PiperNoLen[T]
• :frog: pipe.CollectNL[T](PiperNoLen[any]) PiperNoLen[T] This functions takes a Pipe of erased interface{} type (which is pretty useful if you have a lot of type conversions along your pipeline and can be achieved by calling Erase() on a Pipe). Basically, for each element x in a sequence Collect returns *(x.(*T)) element.

Error handling

• :frog: Yeti(yeti) Pipe[T]:set a yeti - an object tha