76 skills found · Page 1 of 3
openai / Tiktokentiktoken is a fast BPE tokeniser for use with OpenAI's models.
jimmc414 / OnefilellmSpecify a github or local repo, github pull request, arXiv or Sci-Hub paper, Youtube transcript or documentation URL on the web and scrape into a text file and clipboard for easier LLM ingestion
dqbd / TiktokenizerOnline playground for OpenAPI tokenizers
pkoukk / Tiktoken Gogo version of tiktoken
niieani / Gpt TokenizerThe fastest JavaScript BPE Tokenizer Encoder Decoder for OpenAI's GPT models (gpt-5, gpt-o*, gpt-4o, etc.). Port of OpenAI's tiktoken with additional features.
M4THYOU / TokenDaggerHigh-Performance Implementation of OpenAI's TikToken.
tiktoken-go / TokenizerPure Go implementation of OpenAI's tiktoken tokenizer
karpathy / RustbpeThe missing tiktoken training code
zurawiki / Tiktoken RsReady-made tokenizer library for working with GPT and tiktoken
dmitry-brazhenko / SharpTokenSharpToken is a C# library for tokenizing natural language text. It's based on the tiktoken Python library and designed to be fast and accurate.
daulet / TokenizersGo, Wasm bindings for HF Tokenizers and Tiktoken
CNSeniorious000 / Free ChatAn elegant LLM chat UI forked from chatgpt-demo of @anse-app. Index site at https://free-chat.asia
IAPark / Tiktoken RubyUnofficial ruby binding for tiktoken by way of rust
yethee / Tiktoken PhpThis is a port of the tiktoken
saschaschramm / ChatgptAnalysis of OpenAI's ChatGPT
johannschopplich / Tokenx📐 Fast token estimation at 96% accuracy of a full tokenizer in a 2kB bundle
aiqinxuancai / TiktokenSharpToken calculation for OpenAI models, using `o200k_base` `cl100k_base` `p50k_base` encoding.
ceifa / Tiktoken NodeOpenAI's tiktoken but with node bindings
MNeMoNiCuZ / ComfyUI Mnemic NodesNodes: Wildcard Processor, Get File Path, Save Text File, Download Image from URL, Tiktoken Tokenizer, String Cleaning, String Text Splitter, String Text Extractor, Format Date Time, Load Text-Image Pairs, Metadata Extractor, Audio Visualizer, Load Image Advanced, Colorful Starting Image, Groq LLM, VLM, ALM API
openshieldai / OpenshieldOpenShield is a new generation security layer for AI models
