Narr
Scala.js abstraction for a common core of features shared by Array[T], js.Array[T], and the JavaScript TypedArray family of Arrays..
Install / Use
/learn @dragonfly-ai/NarrREADME
NArr
Pronounced: <i>(ˈnär, as in gnarly)</i>. Stands for: <i>Native Array</i><br />
Why Arrays? Because they have the lowest memory footprint and the deepest hardware optimization! As Daniel Spiewak famously understated the matter: <a href="https://youtu.be/n5u7DgFwLGE?t=720">"As good as you think Arrays are, they are better!"</a> Arrays are so light and fast that the <a href="https://www.biblegateway.com/quicksearch/?quicksearch=Array&version=KJV">Hebrew Bible</a> mentions them over 40 times! Unfortunately, this sacred and holy data structure causes a host of problems in cross-compiled Scala projects, mostly because of JavaScript idiosyncracies. More specifically, although Scala Native and Scala JVM share a single unified Array type, Scala.js presents no fewer than 14: scala.Array, js.Array, Int8Array, Uint8Array, Uint8ClampedArray, Int16Array, Uint16Array, Int32Array, Uint32Array, Float16Array, Float32Array, Float64Array, BigInt64Array, and BigUint64Array. This wide diversity begets a web of frustrating design tradeoffs, but fear not! The following text not only maps them out but also shows how NArr addresses them all.
Choosing the Right Array Type.
"Scala supports scala.Array[T] on all compilation targets, so what's the harm in always using that?" True, but in Scala.js, scala.Array[T] wraps either JavaScript's signature associative array or the most relevant TypedArray depending on the value of T. As a system of aliases for native types, NArr wraps nothing! As such, it not only eliminates wrapper related memory overhead, but also all friction related to native interoperability. For more about how NArr streamlines native interop, see the section about it below.<br />
"What about js.Array[T] then?" That improves JavaScript interop and eliminates overhead for Arrays of objects, Bytes, Chars, and Longs, but disqualifies lighter alternatives for Arrays where T ∈ {Byte, Short, Int, Float, Double}. Worse, Scala JVM and Native don't support js.Array[T] so using it necessitates parallel implementations of methods, classes, or even entire programs. NArr, by contrast, proliferates the most optimized possible Array type across an entire codebase, eliminating the need for any platform specific code.<br />
Maybe one of the TypedArrays? Again, these aren't supported on JVM or Native. Trying to rely on them in a cross project requires a lot of duplicate code.<br /><br />
Instead of these, use narr.NArray[T] as a drop in replacement for any other Array type because it always reduces to the most optimized native array type available on the compilation target platform. As a system of type aliases, narr.NArray[T] introduces no runtime costs on any platform, necessitates no parallel implementations of Array related methods and classes, and provides seamless interoperability with native code. The following table articulates the system of type aliases across all three platforms:
NArray[Byte]
NArray[Short]
NArray[Int]
NArray[Float]
NArray[Double]
NArray[Long]
NArray[String]
NArray[AnyRef]
NArray[NArray[Int]]
NArray[NArray[AnyRef]]
Int8Array
Int16Array
Int32Array
Float32Array
Float64Array
js.Array[Long]
js.Array[String]
js.Array[AnyRef]
js.Array[Int32Array]
js.Array[js.Array[AnyRef]]
scala.Array[Byte]
scala.Array[Short]
scala.Array[Int]
scala.Array[Float]
scala.Array[Double]
scala.Array[Long]
scala.Array[String]
scala.Array[AnyRef]
scala.Array[Array[Int]]
scala.Array[Array[AnyRef]]
👣 Memory Footprint.
Because narr.NArray[T] at its core, consists only of type aliases, it will always select the most memory efficient available TypedArray or, for objects and Scala's value types that have no native equivalent in JavaScript, it will resort to JavaScript's signature associative Array which benefits from a long history as the only data structure in JavaScript and in turn, extensive optimization. The system of type aliases itself consists of match types which reduce to scala.Array[T] on JVM and Native platforms. The following code snippet illustrates how they reduce in Scala.js:
type NArray[T] = T match
case Byte => scala.scalajs.js.typedarray.Int8Array
case Short => scala.scalajs.js.typedarray.Int16Array
case Int => scala.scalajs.js.typedarray.Int32Array
case Float => scala.scalajs.js.typedarray.Float32Array
case Double => scala.scalajs.js.typedarray.Float64Array
case _ => scala.scalajs.js.Array[T]
🏎 Speed.
As discussed in Choosing the Right Array Type, NArr always reduces to the most optimized possible Array type available to Scala.js. By simply typing NArray instead of Array a cross compiled code base automatically benefits from minimum memory footprint and maximum hardware acceleration.<br />
Native Interoperability.
Imagine trying to make a cross compiled Scala library accessible to JavaScript developers. Scala.js makes that possible through annotations like @JSExport("..."), @JSExportAll, and @JSExportTopLevel("..."). Now consider a method that accepts an Array as a parameter and/or returns an Array:
@JSExportTopLevel("fooBarMagic")
def fooBarMagic(a:scala.Array[Int]): scala.Array[Int] = ...
How will a native JavaScript developer procure an array of type: scala.Array[Int]? How will she make use of the return value or pass it onto other JavaScript code? Traditionally, Scala.js developers handle this in one of two ways: either by writing a separate implementation of the library specially for JavaScript, or by providing a conversion method to the js project which calls the shared code. Although carefully writing a separate implementation specially for JavaScript can preserve performance it doubles production and maintenance costs. Most Scala.js projects simply abandon the idea of supporting native JavaScript accessibility, but for the sake of convenience some Scala.js developers opt for writing conversion methods like so:
@JSExportTopLevel("fooBarMagic")
def fooBarMagicHelper(a:scala.scalajs.js.typedarray.Int32Array): scala.scalajs.js.typedarray.Int32Array = {
// convert to Array[Int]
val temp0 = new scala.Array[Int](a.length)
var i = 0
while (i < a.length) {
temp0(i) = a(i)
i = i + 1
}
// invoke fooBarMagic
val temp1 = fooBarMagic(temp0)
// convert back to Int32Array
val out = new scala.scalajs.js.typedarray.Int32Array(a.length)
i = 0
while (i < a.length) {
out(i) = temp(i)
i = i + 1
}
out
}
Although this approach makes use of shared code, and increases developer convenience somewhat, it abandons performance by trippling memory footrpint and requiring two separate O(n) array conversions that can't benefit from SIMD capable hardware. NArr by contrast, provides the best of both approaches for the one time cost of a simple refactor of the original code:
@JSExportTopLevel("fooBarMagic")
def fooBarMagic(a:narr.NArray[Int]): narr.NArray[Int] = ...
In this way, all platforms share the exact same code without any conversions or wrappers. What's more, Java, C/C++, and JavaScript developers can seamlessly interact with the library using the native Array types most familiar to their respective platforms.
</li> <li>Code Redundancy.
As described in Native Interoperability, NArr eliminates the need for platform specific Array optimizations.
</li> <li>ArrayOps: Mixed Support for Scala Semantics.
A major impediment to using JavaScript TypedArrays in Scala.js projects comes from the fact that while scala.Array[T] and js.Array[T] have their respective ArrayOps utilities, no such functionality has ever existed for Int8Array, Int16Array, Int32Array, Float32Array, and Float64Array. Fortunately NArr polyfills almost all of these so Scala developers can enjoy highly optimized Scala semantics on every kind of Array.
<br /><a href="https://dragonfly-ai.github.io/narr/FeatureGrid">Click here to compare NArr features to those built into Scala JVM/Native and Scala.js</a>.
In short, NArr shrinks code bases and memory footprint; saves time: run, code, and maintenance; and also simplifies native interoperability.<br />
<br />Caveats:
<ul> <li>NArr relies heavily on Scala 3 features and sadly offers no support for Scala 2.</li> <li>In some cases, type inference fails on higher kinds, for example, consider the following class:
class Foo[T](a:NArray[T])
We might expect the compiler to infer that T = Int from statements like:
val a = NArray(1, 2, 3)
val f = new Foo(a)
or even:
val a = NArray[Int](1, 2, 3)
val f = new Foo(a)
however, the compiler fails to infer the correct type for T when its evidence spans multiple lines. Luckily, we can avoid these situations by providing type parameters explicitly. The following alternatives all compile and run correctly:
// multi-line
val a = NArray(1, 2, 3)
val f = new Foo[Int](a)
val a = NArray[Int](1, 2, 3)
val f = new Foo[Int](a)
// single line
val f = new Foo(NArray(1, 2, 3))
val f = new Foo(NArray[Int](1, 2, 3))
val f = new Foo[Int](NArray[Int](1, 2, 3))
Although the TypedArray family of data structures avoids the following issu
