Metronome
Suite of parallel iterative algorithms built on top of Iterative Reduce
Install / Use
/learn @jpatanooga/MetronomeREADME
Metronome is a suite of parallel iterative algorithms that run natively on Hadoop's Next Generation YARN platform.
- Algorithms
- Parallel Neural Networks
- Parallel linear regression
- Parallel logistic regression
- Parallel K-means
- Restricted Boltzmann Machines
- Parallel Deep Belief Networks
- Developed directly in concert with [Adam Gibson and the DL4J project] (http://deeplearning4j.org)
- [Running Deep Learning with Metronome on Hadoop] (https://github.com/jpatanooga/Metronome/wiki/Running-Deep-Learning-on-Metronome)
- [Fundamental Architecture of our Deep Learning Library over at DL4J] (http://deeplearning4j.org/overview.html)
- Scales linearly with input size
- Built on top of BSP-style computation framework "Iterative Reduce" (Hadoop / YARN)
- Packaged in a new suite of parallel iterative algorithms called Metronome on [IterativeReduce] (https://github.com/emsixteeen/IterativeReduce)
- 100% Java, ASF 2.0 Licensed, on github
Project Next Steps
- 6/5/2014 - we're working on cutting a 0.1 tag and download for the current project snapshot for both
- IterativeReduce
- Metronome
Presentations
Hadoop Summit EU 2013
-
[Hadoop Summit EU 2013: Parallel Linear Regression, IterativeReduce, and YARN] (http://www.slideshare.net/jpatanooga/hadoop-summit-eu-2013-parallel-linear-regression-iterativereduce-and-yarn)
-
[Video of Presentation from Hadoop Summit EU 2013] (http://www.youtube.com/watch?feature=player_embedded&v=rwiy_YhbxiI)
BigDataTechCon 2013
- Slides soon
MLConf 2013
- [Parallel Neural Networks on Hadoop Slides] (http://www.slideshare.net/jpatanooga/metronome-ml-confnov2013v20131113)
Hadoop Summit 2014 (San Jose)
-
[Slides for Intro To Deep Learning on Hadoop] (http://www.slideshare.net/jpatanooga/hadoop-summit-2014-san-jose-introduction-to-deep-learning-on-hadoop)
-
[Video of Hadoop Summit Talk] (https://www.youtube.com/watch?v=h2Ixpfn-DTg&list=PLSAiKuajRe2k0Aw_njj74rIlbdk3JTQ8Y)
Deep Learning Coverage in Wired Magazine
-
[Wired - Google Data Center AI] (http://www.wired.com/2014/05/google-data-center-ai/)
-
[Wired - Skymind Deep Learning] (http://www.wired.com/2014/06/skymind-deep-learning/)
Resources
- [General guide on running yarn jobs] (https://github.com/jpatanooga/Metronome/wiki/Running-Jobs-on-YARN-Clusters)
- [Running Deep Learning with Metronome on Hadoop] (https://github.com/jpatanooga/Metronome/wiki/Running-Deep-Learning-on-Metronome)
- [IterativeReduce Programming Model] (https://github.com/emsixteeen/IterativeReduce/wiki/Iterative-Reduce-Programming-Guide)
- Using IRUnit - the IterativeReduce Unit Testing Framework
- [Running parallel linear regression with IRUnit on synthetic test data] (https://github.com/jpatanooga/Metronome/wiki/Running-Parallel-Linear-Regression)
Contributors and Special Thanks
- Adam Gibson
- [Github] (https://github.com/agibsonccc/)
- Michael Katzenellenbollen
- [Github] (https://github.com/emsixteeen)
- Dr. Jason Baldridge
- Dr. James Scott
- Paul Wilkinson
- David Kale
- Professional Hipster / Hobbit
Related Skills
node-connect
338.0kDiagnose OpenClaw node connection and pairing failures for Android, iOS, and macOS companion apps
frontend-design
83.4kCreate distinctive, production-grade frontend interfaces with high design quality. Use this skill when the user asks to build web components, pages, or applications. Generates creative, polished code that avoids generic AI aesthetics.
openai-whisper-api
338.0kTranscribe audio via OpenAI Audio Transcriptions API (Whisper).
commit-push-pr
83.4kCommit, push, and open a PR
