<HTML> <head> <meta charset="UTF-8"> <meta name="description" content="DataFrame is a templatized and heterogeneous C++ container designed for data analysis for statistical, machine-learning, or financial applications. You can think of data-frame as a two-dimensional data structure of columns and rows just like an Excel spreadsheet, or a SQL table. But in case of C++ DataFrame, your data needn't be two-dimensional necessarily. Columns in the C++ DataFrame could be vectors of any type, including DataFrames or other containers. So, a C++ DataFrame can be of any dimension. Columns are the first-class citizens of DataFrame, meaning operations and access to columns is far more efficient and easier than dealing with data row by row. That's the logical layout of the data. C++ DataFrame also includes an intuitive API for data analysis and analytics. The API is designed to be open-ended meaning you can easily include your own custom algorithms."> <meta name="keywords" content="CPP, C++, DataFrame, data-science, machine-learning, AI, statistical-analysis, data-analysis, trading-strategies, statistical, numerical-analysis, tensorboard, financial-engineering, large-data, tensorboard, financial-data-analysis, multidimensional-data"> <meta name="author" content="Hossein Moein"> <meta name="viewport" content="width=device-width, initial-scale=1.0">  <meta property="og:title" content="DataFrame — High-Performance Data Analysis in C++"> <meta property="og:description" content="Fast C++ DataFrame library for tabular data analysis, filtering, grouping, and numerical computation."> <meta property="og:type" content="website"> <meta property="og:url" content="https://hosseinmoein.github.io/DataFrame/"> <meta property="og:image" content="https://github.com/hosseinmoein/DataFrame/blob/master/docs/LionLookingUp.jpg"> <link rel="icon" type="image/jpeg" href="https://github.com/hosseinmoein/DataFrame/blob/master/docs/LionLookingUp.jpg"> <link rel="canonical" href="https://hosseinmoein.github.io/DataFrame/docs/HTML/DataFrame.html"> </head> </HTML>

<a href="https://hosseinmoein.github.io/DataFrame/docs/HTML/DataFrame.html" target="_blank">DataFrame documentation with code samples</a>

DataFrame is a high-performance C++ library for in-memory data exploration, transformation, and statistical analysis — designed for data scientists, quant traders, and C++ developers who need efficient tabular data processing without Python overhead. This library is designed to provide similar functionalities to data manipulation and analysis tools found in other languages, such as Python's Pandas or R's data.frame. It aims to offer a robust and efficient way to handle tabular data in C++. The depth and breadth of functionalities offered by C++ DataFrame alone are greater than functionalities offered by packages such as Pandas, data.frame, and Polars combined. You can slice the data in many different ways. You can join, merge, group-by the data. You can run various statistical, summarization, financial, and ML algorithms on the data. You can add your custom algorithms easily. You can multi-column sort, custom pick and delete the data. And more … DataFrame also includes a large collection of analytical algorithms in form of visitors. These are from basic stats such as Mean, STDEV, Moving Averages, ... to more involved analysis such as PCA, Polynomial Fit, FFT, Eigens ... including a good collection of trading indicators. You can also easily add your own algorithms. Many of these algorithms work seamlessly with both scalar and multidimensional datasets. DataFrame also employs extensive multithreading in almost all its API’s, for large datasets. That makes DataFrame especially suitable for analyzing large datasets. For basic operations to start you off, see Hello World and/or Cheat Sheet. For a complete list of features with code samples, see <a href="https://hosseinmoein.github.io/DataFrame/docs/HTML/DataFrame.html" target="_blank">documentation</a>.

I have followed a few principles in this library:

Performance

You have probably heard of Polars DataFrame. It is implemented in Rust and ported with zero-overhead to Python (as long as you don’t have a loop). I have been asked by many people to write a comparison for DataFrame vs. Polars. So, I finally found some time to learn a bit about Polars and write a very simple benchmark. I wrote the following identical programs for both Polars and C++ DataFrame (and Pandas). I used Polars version: 0.19.14 (Pandas version: 1.5.3, Numpy version: 1.24.2). And I used C++23 GCC-14 compiler with -O3 option. I ran both on my, somewhat outdated, MacBook Pro (Intel chip, 96GB RAM). In both cases, I created a dataframe with 3 random columns. The C++ DataFrame also required an additional index column of the same size. Polars doesn’t believe in index columns (that has its own pros and cons. I am not going through it here). Each program has three identical parts. First it generates and populates 3 columns with 300m random numbers each (in case of C++ DataFrame, it must also generate a sequential index column of the same size). That is the part I am not interested in. In the second part, it calculates the mean of the first column, the variance of the second column, and the Pearson correlation of the second and third columns. In the third part, it does a select (or filter as Polars calls it) on one of the columns.

Results: The maximum dataset I could load into Polars was 300m rows per column. Any bigger dataset blew up the memory and caused OS to kill it. I ran C++ DataFrame with 2b rows per column and I am sure it would have run with bigger datasets too. So, I was forced to run both with 300m rows to compare. I ran each test 4 times and took the best time. Polars numbers varied a lot from one run to another, especially calculation and selection times. C++ DataFrame numbers were significantly more consistent.

| | C++ DataFrame | [ Polars &nbs

DataFrame

Install / Use

README

<a href="https://hosseinmoein.github.io/DataFrame/docs/HTML/DataFrame.html" target="_blank"><B>DataFrame documentation with code samples</B></a>

Performance