Turbit

High-speed Multicore Computing

Turbit is an advanced high-speed multicore computing library in Node.js, designed to optimize performance for computationally intensive operations by leveraging parallel processing across multiple CPU cores.

Create powerful applications, scripts, and automations with enhanced performance through parallel processing. Turbit was designed for the multi-core era.

Key Applications

Turbit excels at optimizing resource-intensive operations, making it ideal for:

| Application | Description | |-------------|-------------| | Data Processing and Analysis | Efficiently handle and analyze large volumes of data | | Scientific Computations | Accelerate complex scientific calculations and simulations | | Batch Processing | Process large datasets quickly and efficiently | | Complex Algorithmic Operations | Speed up execution of complex algorithms and mathematical operations |

Inspiration

As a researcher, I often struggled to find a truly user-friendly implementation for parallel processing in Node.js. Most existing solutions were not user-friendly, complex, and unwieldy, making it challenging to efficiently harness CPU power for high-performance tasks. This inspired me to create Turbit, a library that simplifies parallel processing in applications and processes, allowing developers to easily tap into the full potential of their hardware without getting bogged down in implementation details. With AI advancements, many people have chosen to exclusively use the GPU to process data and perform complex tasks. However, although the CPU is not always directly compared to the GPU in terms of performance for certain tasks, it also has great potential that should not be underestimated. Turbit facilitates the harnessing of this potential, enabling efficient parallel processing on the CPU.

One of my goals when creating things is always to simplify them as much as possible for others and, of course, to make them unique.

The code is art, akin to music and other creative fields. I created Turbit out of passion and love for coding. Writing code is not just a daily habit for me, it's a task of focus and discipline, driven by my lifelong love for technology. During the final stages of developing this library and its two powerful use cases, throughout the code writing process, I constantly listened to some songs that I'd like to share:

• Monaco - Bad Bunny: "Dime (dime), dime, ¿esto es lo que tú querías?"
• Los Pits - Bad Bunny: "Te lo dije que me hacía millo antes de los treinta."
• 1 of 1 - Maluma: "Mi carrera es una chimba 1 of 1, me voy y vuelvo y los bajo de number one."
• Mírame - Blessd: "No cualquiera puede aguantar mi ritmo."
• Vuelve candy b - Bad Bunny: "¿Cómo tú vas a dudar del más cabrón que le mete?"
• Otra noche en miami - Bad Bunny: "Todo es superficial, nada real, nada raro que el dinero no pueda comprar."
• Decisions - Borgore & Miley Cyrus: "Decisions, but I want it all."
• Si tú supieras - Feid: "Baby, perdón, pero el tiempo que no estoy contigo es tiempo perdido."
• Brickell - Feid: "Ojalá me hubieras dicho que era la última vez."
• Meek Mill - Drake: "Playin' with my name, that's lethal, dawg (who you say you was?)"
• La Jumpa - Arcangel & Bad Bunny: "Hoy tú te vas con una leyenda que no va a volver a nacer."

As I reflect on this project, I can't help but wonder: Perhaps these are among the last lines of code written 100% by a human. Regardless, I hope you enjoy this code as much as I enjoyed writing it.

Always the same creative hacker, but more evolved. - Jose Pino

Getting Started

Turbit's only dependency is the built-in child_process module in Node.js for managing child processes.

To use Turbit, you need to have Node.js installed. Then, you can install Turbit using npm:

npm install turbit

After installation, import it in your code as follows:

const Turbit = require("turbit");
// Create a Turbit instance for parallel processing
const turbit = Turbit();

Usage

RUN (Main Function)

Turbit provides a run function to execute tasks across multiple cores for parallel processing.

turbit.run(func, options)

Parameters

1. func (Function): The function to be executed in parallel. This is the task you want to distribute across multiple cores.

2. options (Object): Configuration options for the execution.

   • type (required, string): Specifies the type of execution. Can be either "simple" or "extended".

     • simple: Use this when you have a single task that doesn't require input data.
     • extended: Use this when you have a set of data that needs to be processed in parallel. It divides the input data into chunks and distributes them across multiple processes for simultaneous processing.

   • data (optional, array): The data to be processed in parallel. This is required for "extended" type executions. Each item in this array will be passed as an argument to your function.

   • args (optional, object): Additional arguments to pass to the function. These will be passed to your function along with the data item (for "extended" type).

   • power (required, number): Controls the intensity of parallel processing. Values range from 1 to 100 for normal system resource utilization. Values above 100 enable an intentional system overload mode, allowing for more processes than the system would typically handle. Default is 70.

     • power: 1-100: Normal utilization of system resources.
     • power: > 100: Overload mode. Allows generating more processes than the system would normally handle.

     Note: Using power > 100 can lead to increased RAM usage, significant CPU utilization, and potential system instability. Use this feature with caution and only when necessary for extremely intensive processing tasks. Suitable use cases include:

     • Processing large datasets in limited time
     • Complex simulations requiring multiple parallel iterations
     • High-intensity rendering or scientific calculations
     • Intensive cryptographic operations and brute-force processes

Returns

• A promise that resolves with the execution results and statistics.
  • data (Array): The results of the executed function.
  • stats (Object): Performance statistics of the execution.
    • timeTakenSeconds: The total execution time in seconds.
    • numProcessesUsed: The number of processes used for the execution.
    • dataProcessed: The amount of data items processed.
    • memoryUsed: The amount of memory used during execution.

The stats object provides valuable insights into the performance of your parallel processing tasks. It allows you to monitor execution time, resource utilization, and processing efficiency, which can be crucial for optimizing your applications.

KILL (Function)

Turbit provides a kill function to terminate all active child processes and free system resources.

turbit.kill();

Architecture

Turbit utilizes a parallel processing architecture based on Node.js child_process module. Here's a brief overview of how it works:

1. Initialization: Turbit creates a pool of child processes up to the maximum number of available CPU cores when the Turbit instance is created.

2. Task Distribution: When turbit.run() is called, the library prepares the task for parallel execution. For 'simple' type, it replicates the task across processes. For 'extended' type, it divides the input data into chunks for distribution across processes. The number of processes used is determined by the power parameter.

3. Process Management: Turbit manages the pool of child processes, creating additional processes or terminating excess ones as needed based on the power parameter for each run() call.

4. Parallel Execution: Tasks are distributed across child processes, allowing for simultaneous utilization of multiple CPU cores.

5. Inter-Process Communication: Turbit uses Node.js built-in messaging system to send function definitions and arguments to child processes, and to receive results back.

6. Result Collection: As child processes complete their tasks, results are gathered and consolidated into a single output.

7. Performance Tracking: Turbit monitors and reports execution time, number of processes used, amount of data processed, and memory consumption.

Examples

Here are some examples of how to use Turbit:

Simple Execution

// Define a simple task function
const task = function() {
    return "Hello, humans and intelligent machines!";
}

// Implement Turbit for parallel processing
turbit.run(task, { type: "simple", power: 100 })
    .then(result => {
        console.log("Simple execution result:", result.data);
        turbit.kill(); // Cleans up child processes after completing the task
    })
    .catch(error => {
        console.error("Error in simple execution:", error);
        turbit.kill(); // Make sure to call kill even if there's an error
    });

Extended Execution

// Define a calculation function to be executed in parallel
function calculate(item) {
    return item * 12;
}

// Implement Turbit for parallel processing
tur