Omnitool.ai - Your Open Source AI Desktop

Discover, Learn, Evaluate and Build with thousands of Generative AI Models.

Omnitool.ai is an open-source, downloadable "AI Lab in a box" built for learners, enthusiasts and anyone with interest in the current wave of AI innovation. It provides an extensible browser based desktop environment for streamlined, hands-on interacting with the latest AI models from OpenAI, replicate.com, Stable Diffusion, Google, or other leading providers through a single, unified interface.

Watch the demo! and see more videos on our Youtube channel.

Why Omnitool?

With thousands of preprints and countless "AI tools" released each week, it is incredibly challenging to stay on top of the rapidly evolving AI ecosystem, to separate hype and facts and to extract durable long term skills and learning. PapersWithCode and GitHub repositories attached to arXiv papers provide ability to hands-on validate and apply the latest discoveries, but the fragile nature of the Python ecosystem and often steep hardware requirments dramatically limits accessibility. Likewise implementing and testing cloud based models requires delving deep into API documentation and wrestling with connecting code.

We believe that is a serious problem. AI may represent the first large scale technological disruption unbounded by logistical challenges, scaling along existing wires, API infastructure and app delivery platforms. Meanwhile, market pressure to adopt AI is felt by many businesses and teams.

Without educated decision makers and technical experts, businesses and public organisations alike are at high risk of falling for hype and magical narratives and expensive misadventures.

Omnitool is our attempt to improve this situation: A single, unified interface capable of connecting with as many AI models as possible and to reduce the "time to hands on AI" to an absolute minimum.

Omnitool is highly extensible and interoperable. Most OpenAPI3 based services can be connected and turned into "blocks" without writing code. It's extension framework enables deeper integrations of anything from custom UIs (Like Stability Dream Studio) to Game Engines (like BabyonJS or Phaser) to Image manipulation libraries.

What Omnitool is NOT

• Omnitool is not a multi-user cloud SaaS product. It's a downloadable, locally installed product.
• Omnitool is NOT a no-code solution meant to replace coding or enable non engineers to code. It's focused on interacting with AI use cases, not writing general purpose software.
• Omnitool is not production/enterprise software. (Yet.) It's a lab optimizing for access to the latest technologies over stability and, as with any lab, things may blow up from time to time.

Table of Contents

• Key Features
• Quickstart
• Manual Install
• PocketBase DB Admin (ADVANCED)
• Next Steps
• Changelist

Key Features

Self-hosted and Open Source

• Omnitool is local self-hosted software that turns your machine into a powerful AI Lab.

  • You install Omnitool and it runs on your Mac, Windows or Linux notebook, desktop or server, not cloud servers.
  • Data stores it's data locally on your machine and is only transmitted to the third party provider APIs you choose to access. Updates are managed via github.
  • A Docker Image is forthcoming.
  • If you are interested in running Omnitool in the cloud, please get in touch with us at contact@omnitool.ai

• Open Source and Open Standards

  • Omnitool is licensed as open source software and heavily leverages open standards, such as OpenAPI, making it interoperable and extensible.

Rapid Access to the world of generative AI without GPU, Managing Python installations and learning dozens of APIs and interfaces

• Minimal Time-to-AI: It allows you to try out models and services in minutes without having to study API docs, write boilerplate code, manage python venvs or figuring out new user interfaces. Because of it's integration of many leading AI platforms, the lag time between "paper with code" to hands on experimentation often is cut down to days.

• It presents the vast world of generative AI - image, video, audio, text, and classification APIS - through a single, unified user interface without oversimplifying or hiding the power of the APIs.

Comprehensive AI Provider Support

• Seamlessly provides access to 1000s of AI model and utility APIs from an rapidly growing list leading AI providers and aggregators, exposing them all via interoperable blocks.

Currently supported:

• Civitai.com (Model metadata access)
• Deepl.com (Document translation)
• ElevenLabs.io (Multilingual voice generation)
• Getimg.ai (Image generation and manipulation APIs)
• Github.com (Various)
• Google.com
  • Gmail
  • Vertex (AI)
  • Google Translate
  • Google TTS (Text to Speech)
  • Google Vision (Computer Vision)
• Huggingface.com (1000's of models, including free inference models)
• OpenAI.com (Image/Text/Audio Generation including GPT3/4/Visual, Whisper, Dall-e 2, Dall-e 3, Moderation APIs and more)
• OpenRouter.ai (100s of LLM APIs)
• Perplexity.ai (Text Generation)
• Stability.ai (Image Generation and Manipulation APIs)
• TextSynth.com (LLM, translation, and classification APIs)
• Replicate.com (1000s of models across all modalities)
• Uberduck.com (Voice Generation, Music centric offerings)
• Unsplash.com (Stock imagery)
• with many more APIs in testing...

Currently supported Local APIs:

• LM-studio
• Ollama
• Oobabooga Text Generation UI
• Automatic1111/SD-Next
• Paperless-ng
• Qdrant Vector Store

Omnitool is able to generate blocks from any openapi.json definitions via URL or directly supplied file. We support a number of custom x- annotations that can be added to openapi definitions to allow omnitool to guide the block generation. It also supports creating "patches" on top of existing APIs to create customized blocks. With integrated JSONATA support, it is possible to build powerful data processing blocks using pure data.

Extensible Architecture

• Inspired by the common modding architecture found in video game toolsets, Omnitool is built, from the ground up, to be extensible via multiple mechanisms:

  • Simple Client and Server scripts allowing addition of /commands that are hot-reloaded, so editing and building is a breeze.
  • Client Extensions - any web-app/webpage can be turned into an extension and integrated directly on Omnitool's desktop via it's window system. Omnitool's client SDK exposes the full range of platform functionality to extensions, allowing you to write apps or tools using every API or recipe enabled in Omnitool.
  • Server Extensions - Server extensions written in javascript that can add new blocks, API and core functionality.

• Some examples of currently available extensions:

  • omni-core-replicate, a core extensions that allows import of any AI model on replicate.com into a ready to use block in Omnitool
  • omni-extension-sharp, an extension adding an array of Image Manipulation blocks such as format conversion, masking, composition and more based on the powerful sharp image processing library.
  • omni-extension-minipaint, a powerful photo editing tool useful for quickly creating and editing images without switching out of the app.
  • omni-extension-openpose, a OpenPose based pose estimation and generation toolkit useful for creating guidance images for controlnet/diffusion models.
  • omni-extension-tldraw, a whiteboarding/sketching extension built on TLDraw, useful for generating input for visual transformers and diffusion models
  • omni-extension-wavacity, a full wasm implementation of Audacity, a state of the art audio recorder/editor useful for generating and editing audio content.

• Visit the Extension tab in app or see our [Omnitool Community Git