Noj - a data science toolkit

Noj logo

Noj is an out-of-the-box Clojure library designed to streamline data science workflows for both newcomers and experienced users. Noj provides a tested and integrated collection of libraries that are known to work seamlessly together from day one, rather than requiring users to find, configure, and integrate multiple libraries separately.

Traditional data science setups involve piecing together various libraries for different tasks, which can be time-consuming and error-prone. Noj solves this problem by providing a curated collection of libraries that covers:

Bridges & Interop: Connect and interact with other language systems and libraries
Data Processing: Efficiently transform and prepare data for analysis
High-Performance Computing: Access optimized tools to tackle complex computations
Mathematics & Statistics: Perform comprehensive mathematical and statistical operations
Machine Learning: Access powerful machine learning tools and models
Data Visualization: Generate clear and compelling data visualizations

All included libraries are designed with consistent data handling as a unified architecture - they use tech.ml.dataset directly for tabular data structures or provide high interoperability with it, and support kindly framework for data visualization across different output formats.

Benefits and Foundation

Noj goes above and beyond as a comprehensive bundling solution:

Pre-integrated and Tested Libraries: Users receives a collection of carefully curated libraries that are guaranteed to work together, eliminating compatibility issues and reducing setup time.
Comprehensive Documentation: Noj provides extensive documentation and tutorials that demonstrate how to use different libraries in combination, with practical examples and useful resources.
Development-Ready Environment: For hassle-free setup, Noj includes a devcontainer that handles complex native dependencies automatically.

General Info

| Resource | Link | |:-----------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------| | Website | https://scicloj.github.io/noj/ | | Source Code | | | Dependencies | | | Build Status | | | License | EPL v1.0 | | Development Status | Beta stage | | Developer Chat | #noj-dev on Clojurians Zulip | | User Support | #data-science on Clojurians Zulip |

Getting Started with Noj

Noj is released in 3 different forms, namely

A Clojure library
An uberjar using Clay to enable 'live-reload' workflows
An uberjar to be used as Jupyter kernel (based on clojupyter)

Noj as Clojure library

Add this to your deps.edn:

org.scicloj/noj {:mvn/version "2-beta21"}

to get noj as a Clojure library. It comes with a curated list of various data-science libraries, see documentation.

Noj as Live-Reload app

Noj includes Clay, which transforms Clojure namespaces into interactive notebooks. This allows users to write and execute Clojure code within a notebook-like environment. For a more interactive experience, Clay can render Clojure files as live notebooks in your browser.

Download noj-<version>-clay.jar from Releases into a local directory of your choice (replace <version> with e.g. 2-beta21).

Launch Clay with live-reload functionality:

java -jar noj-<version>-clay.jar hello.clj

This command opens an interactive notebook in the browser. Users can make changes to the file using any text editor, and changes will update in the browser automatically.

Clay offers extensive integration with detailed REPL integration and editor support.

Noj as Jupyter Kernel

Noj provides a dedicated kernel for Jupyter, enabling notebook-style development with most Noj visualization capabilities. The Jupyter kernel provides familiar notebook interface for users comfortable with the Jupyter environment.

Note: There's also an ongoing effort to support Google Colab for cloud-based notebook usage.

Environment Setup Process

To use the Jupyter kernel, users need to set up a Python environment:

python3 -m venv python_venv
source python_venv/bin/activate
python3 -m pip install jupyterlab

Download noj-<version>-clojupyter.jar from Releases into a local directory of your choice (replace <version> with e.g. 2-beta21).
Navigate to the JAR-downloaded directory in a terminal.
Execute the following command to install the Noj Jupyter kernel:

java -cp noj-2-beta21-clojupyter.jar clojupyter.cmdline install --jarfile noj-2-beta21-clojupyter.jar --ident noj-2-beta21

a. Confirm the installation: java -cp noj-2-beta21-clojupyter.jar clojupyter.cmdline list-installs
Launch Jupyter Lab: jupyter lab

Technical Note: Each notebook instance starts with its own nREPL server. For technical support and detailed configuration questions, visit our Zulip community chat.

Learning Resources

The following list of resources provides comprehensive guidance for learning Noj, from interactive notebooks and video tutorials to documentation and real-world data examples.

| Resource Type | Link | |:--------------------------|:----------------------------------------------------------------------------------------------------| | 📖 Notebook/Documentation | Noj Getting Started Notebook | | 🐙 Repo | Noj Getting Started Repo | | 🎥 Video | Noj Getting Started Intro | | 🎥 Video | Noj Getting Started: VS Code, Calva, and Clay | | 💾 Data Source | Clojure Events Calendar Feed | | 🎥 Video | Noj in a JAR Intro | | 📖 Notebook/Documentation | Clay Documentation | | 🎥 Video | Clay Overview Demo |

License

Distributed under the Eclipse Public License version 1.0.

Noj

Install / Use

README