NeuralNetwork Library For Microcontrollers

Nothing "Import ant", just a simple library for implementing both MLP & Recurrent Neural-Networks easily and effectively on most Arduino compatible boards and microcontrollers, with partial bare-metal & native-os support.

📚 Summary

| NN<span> </span>Functions | Input<span> </span>Type<span> </span>(x)|Output<span> </span>Type<span> </span>(Y) |<span>                      </span>Action<span>                       </span>| | ------ | ------ | ------ | ------ | | BackProp(x)| DFLOAT Array| - | <details><summary>Trains the Neural-network</summary>"Tells" to the NN if the output was correct/the-expected/X-inputs and then, "teaches" it.</details>| | *FeedForward(x)| Template Array| DFLOAT Array| <details><summary>Returns the output of it</summary>"Feeds" the NN with X-input values and returns Y-Output Values, If needed.</details>| |getMeanSqrdError(x)| Unsigned Int| DFLOAT| <details><summary> Returns the Mean Squared Error</summary> MSE, is SSE (Sum Squared Error) divided by the Product of number-οf-οutputs and inputs-per-epoch aka batch-size. </details>|

Understanding the Basics of a Neural Network:
EXM 0 1 2 3 4 5 6 7 8 9 10 11

📦 Features

• Current

• • + Use of activation-functions per layer-to-layer.
• • + Optimizations based on user's preference.
• • + Support for custom activation functions.
• • + ESP32-S3 DSP-accelerated dot-prod.
• • + Both 16 and 8 bit, int quantization.
• • + MSE/BCE/CCE loss-functions.
• • + Support for double precision.
• • + Many activation-functions.
• • + Use of storage medias.
• • + Exceptional solutions.
• • + RNN GRU LSTM.
• • + Simplicity!

• In the future, i want

• • - Your support.
• • - Better overall code.
• • - Other training methods.
• • - More Activation Functions.
• • - Training with int quantization.
• • - Support for external EEPROM.
• • - Support for convolutional layers.
• • - Support for fixed-point arithmetics.
• • - Different weight initialization methods.
• • - Running NNs entirely off external memmory.
• • - Even more properties, for many different needs.

✏️ Examples

✨ (See also: training with Tensorflow section & native examples)

• 🔤 Basic: MLP | RNN | GRU | LSTM
• 💾 Media: MLP | RNN | GRU | LSTM
• 🎲 Other: MLP | RNN | GRU | LSTM

🦾 Bare & Native support

• To disable NN.print(), simply #define DISABLE_NN_SERIAL_SUPPORT
• To use NN.print() with bare-metal, you should map printf to UART
• USE_PROGMEM in native-os applications, simply allows the use of const
• To USE_INTERNAL_EEPROM with bare-metal, an EEPROM.h-alike is needed
• FS supports objects that implement: read() write() seek() seekp()

📌 Important

1. <details><summary><b>MOST IMPORTANT</b>, DESTRUCTOR WONT FREE LAST-LAYER'S OUTPUTS !!!!</summary><b>By design</b>, the destructor won't free\deallocate the last layer's outputs, allowing you to continue using <a href="https://github.com/GiorgosXou/NeuralNetworks/blob/9ffc36f6e897fe486e2d58ecf8d2cbb9848f71e9/examples/Basic/FeedForward_double_Xor/FeedForward_double_Xor.ino#L7">these outputs</a> through the pointer in your sketch. <b>To fully delete</b> the neural-network and free the associated resources, <b>it's your responsibility to:</b> either <code>delete[] outputs</code> <b>or</b> <code>delete[] NN.layers[NN.numberOflayers - 1].outputs;</code> <ins>at the end of the scope</ins>. <b>Additionally</b>, with <code>NN.load(file)</code>: ensure you deleted last-layer's <code>*outputs</code> in your sketch, in case you plan to re-use the same pointer for capturing the outputs of the newly-loaded-NN's feedforward.</details>
2. If you need back-propagation without hidden-layers see _3_OPTIMIZE 0B00001000
3. Ensure to use (32-bit) float-precision during training unless you USE_64_BIT_DOUBLE
4. bias simply means biases when you enable MULTIPLE_BIASES_PER_LAYER
5. In case you have an error realated to 'POINTER_REGS' check this link here
6. Bare-metal and Native-OS support are still on an experimental state!
7. Last but not least,I am NOT a professional in any of those fields.

🔬 Tested on

<details><summary><strong>Arduino UNO</strong></summary>

• Can't USE_64_BIT_DOUBLE: double is the same as float on the UNO

</details