NetFabric.Hyperlinq

NetFabric.Hyperlinq contains alternative implementations of many operations found in the System.Linq namespace:

Uses value-types to improve performance by making method calls non-virtual and reducing GC collections by not allocating on the heap.
Adds support for Span<>, ReadOnlySpan<>, Memory<> and ReadOnlyMemory<>.
Nullable reference type annotations.
One single NuGet package support for both sync and async enumerables.

This implementation favors performance in detriment of assembly binary size (lots of overloads).

Fast enumeration
Reduced heap allocations
Benchmarks
Usage
Documentation
Supported operations
References
Credits
License

Fast enumeration

NetFabric.Hyperlinq can enumerate faster the results of a query than System.Linq by performing all of the following:

Merges multiple enumerators into a single one for several more scenarios.
It does not box value-type enumerators so, calls to the Current property and the MoveNext() method are non-virtual.
All the enumerables returned by operations define a value-type enumerator.
Whenever possible, the enumerator returned by the public GetEnumerator() or GetAsyncEnumerator() does not implement IDisposable. This allows the foreach that enumerates the result to be inlinable.
Operations enumerate the source using the indexer when the source is an array, ArraySegment<>, Span<>, ReadOnlySpan<>, Memory<>, ReadOnlyMemory<>, or implements IReadOnlyList<>.
Range() and Repeat() return enumerables that implement IReadOnlyCollection<> and ICollection<>. Return() and Select() return enumerables that implement IReadOnlyList<> and IList<>.
Use of buffer pools in operations like Distinct(), ToArray() and ToList().
Use of SIMD in Sum() and SelectVector().
Elimination of conditional branchs in Where().Count().
Allows the JIT compiler to perform optimizations on array enumeration whenever possible.
Takes advantage of EqualityComparer<>.Default devirtualization whenever possible.

The performance is equivalent when the enumerator is a reference-type. This happens when the enumerable is generated using yield or when it's cast to one of the BCL enumerable interfaces (IEnumerable, IEnumerable<>, IReadOnlyCollection<>, ICollection<>, IReadOnlyList<>, IList<>, or IAsyncEnumerable<>). In the case of operation composition, this only affects the first operation. The subsequent operations will have value-type enumerators.

Reduced heap allocations

NetFabric.Hyperlinq allocates as much as possible on the stack. Enumerables and enumerators are defined as value-types. Generics constraints are used for the operation parameters so that the value-types are not boxed.

It only allocates on the heap for the following cases:

Operations that use ICollection<>.CopyTo(), ICollection<>.Contains(), or IList<>.IndexOf() will box enumerables that are value-types.
ToArray() and ToList() allocate their results on the heap. You can use the ToArray() overload that take an buffer pool as parameter so that its result is not managed by the garbage collector.

Benchmarks

The results of the benchmarks comparing multiple LINQ libraries can be found in the LinqBenchmarks repository.

The results of the benchmarks included in this repository can be found in the Benchmarks folder.

The names of the benchmarks are structured as follow:

The library used:
- Linq - the System.Linq namespace (includes System.Linq.Async, System.Interactive, and System.Interactive.Async)
- StructLinq - StructLinq
- Hyperlinq - NetFabric.Hyperlinq
The type of collection used as source:
- Array - an array
- Span - a Span<>
- Memory - a Memory<>
- Enumerable - implements IEnumerable<>
- Collection - implements IReadOnlyCollection<> and ICollection<>
- List - implements IReadOnlyList<> and IList<> but is not an array
- AsyncEnumerable - implements IAsyncEnumerable<>
The type of enumerator provided by the source:
- Value - the enumerator is a value type
- Reference - the enumerator is a reference type
How the result of the operation is iterated:
- For - a for loop is used to call the indexer
- Foreach - a foreach loop is used to call the enumerator
Has a variant:
- SIMD - using SIMD

Usage

Add the NetFabric.Hyperlinq NuGet package to your project.
Optionally, also add the NetFabric.Hyperlinq.Analyzer NuGet package to your project. It's a Roslyn analyzer that suggests performance improvements on your enumeration source code. No dependencies are added to your assemblies.
Add an using NetFabric.Hyperlinq directive to all source code files where you want to use NetFabric.Hyperlinq. It can coexist with System.Linq and System.Linq.Async directives:

using System;
using System.Linq;
using NetFabric.Hyperlinq; // add this directive

Use the methods AsValueEnumerable() to make any collection usable with NetFabric.Hyperlinq. This includes arrays, Memory<>, ReadOnlyMemory<>, Span<>, ReadOnlySpan<>, BCL collections, and any other implementation of IEnumerable<>. Use AsAsyncValueEnumerable() for any implementation of IAsyncEnumerable<>.

public static void Example(IReadOnlyList<int> list)
{
  var result = list
    .AsValueEnumerable()
    .Where(item => item > 2)
    .Select(item => item * 2);

  foreach(var value in result)
    Console.WriteLine(value);
}

Netfabric.Hyperlinq contains special versions of AsValueEnumerable() for better performance with all collections in the System.Collections.Immutable namespace. Projects targetting .NET Framework, netcoreapp2.1 or netstandard2.0, require the addition of the NetFabric.Hyperlinq.Immutable NuGet package dependency.
Most enumerables returned by NetFabric.Hyperlinq are compatible with System.Linq. The exception is enumerables for Span<> or ReadOnlySpan<>.

This allows the use of System.Linq operators on NetFabric.Hyperlinq enumerables. OrderByDescending() is not yet available in Netfabric.Hyperlinq but can still be used without requiring any conversion:

public static void Example(IReadOnlyList<int> list)
{
  var result = list
    .AsValueEnumerable()
    .Where(item => item > 2)
    .OrderByDescending(item => item) // is not yet available in Netfabric.Hyperlinq
    .AsValueEnumerable()
    .Select(item => item * 2);

  foreach(var value in result)
    Console.WriteLine(value);
}

To add NetFabric.Hyperlinq operations after a System.Linq operation, simply add one more AsValueEnumerable() or AsAsyncValueEnumerable().

Value delegates

Calling a lambda expression for each item of the collection is very expensive. NetFabric.Hyperlinq supports an alternative that is not as practical but that has much better performance.

Declare a struct that implements IFunction<>. Here's two examples of how to implement:

readonly struct MultiplyBy2
    : IFunction<int, int>
{
    public int Invoke(int element)
        => element * 2;
}

readonly struct LessThan
    : IFunction<int, bool>
{
	readonly int value;
	
    public LessThan(int value)
		=> this.value = value;
    
    public bool Invoke(int element)
        => element < value;
}

Pass an instance as a parameter or just add the type to the generics arguments list (uses the default constructor):

public static void Example(IReadOnlyList<int> list)
{
  var result = list
    .AsValueEnumerable()
    .Where(new LessThan(10))
    .Select<int, MultiplyBy2>();

  foreach(var value in result)
    Console.WriteLine(value);
}

The instances are allocated on the stack and the methods calls are non-virtual.

Generation operations

In NetFabric.Hyperlinq, the generation operations like Empty(), Range(), `Repeat(

NetFabric.Hyperlinq

Install / Use

README

NetFabric.Hyperlinq

Contents

Fast enumeration

Reduced heap allocations

Benchmarks

Usage

Value delegates

Generation operations