54 skills found · Page 1 of 2
Axym-Labs / LetoReaderA free self-hostable speed reader. Highly customizable. Implements chunking (RSVP), pacing and highlighting. Modern UI and local-storage only.
constantinpape / Z5Lightweight C++ and Python interface for datasets in zarr and N5 format
instructure / Idb CacheIndexedDB-based caching library with encryption and chunked storage, designed for performance and security. Implements AsyncStorage interface.
mafintosh / Abstract Chunk StoreA test suite and interface you can use to implement a chunk based storage backend
QubitProducts / Gcs Browser UploadResumable chunked uploads to Google Cloud Storage from the browser
orangain / Scrapy S3pipelineScrapy pipeline to store chunked items into Amazon S3 or Google Cloud Storage bucket.
jivco / Cassandra Nginx CdnSome config files and POC code to use Apache Cassandra as distributed storage for HLS chunks accross multiple datacenters and scripts for converting/transcoding UDP MPEG-TS to HLS and vice versa. The idea is take from Globo.com’s Live Video Platform for FIFA World Cup ’14.
Juiix / EntitiesDbA high-performance, cache-efficient Entity Component System (ECS) for C#. Features chunk-based storage, fast queries, source-generated iteration, multithreading, SIMD support, and zero external dependencies.
kinstephen / Angular Azure Blob UploadAngularJS mobule for uploading to azure blob storage in chunks.
zoj613 / Zarr MlAn implementation of the Zarr storage format specification for chunked & compressed multidimensional arrays.
rbuch703 / CoordsCOORDS - Chunk-Organized OSM Render-Data Storage
arimanyus / HentaividCulturally-compliant video storage. Embeds searchable text chunks into pixelated media for lightning-fast semantic search. Zero-database, maximum compliance.
killcod3 / Temps3A secure CLI application for temporary file storage using AWS S3 with automatic expiration, intelligent chunking, and local history tracking.
HORNET-Storage / Scionic Merkle TreeScionic Merkle Trees contain small branches like Classic Merkle Trees, the folder storage support of Merkle DAGs, and numbered Merkle leaves so anyone can sync by requesting a range of missing leaf numbers that correspond to missing file chunks.
Azure-Samples / Fabric Sqldb AI RagpipelineThe repository contains Python code for a data pipeline that ingests files from Azure Blob Storage, extracts and chunks text, redacts sensitive information, and generates embeddings. The embeddings and redacted chunks are stored in a SQL database in Fabric
sjtutim / AI Writing MasterAWM is a TypeScript RAG writing system with Nuxt 3 frontend, Express backend, pgvector search, and MinIO storage. It ingests Word/text to Markdown, chunks and embeds with bge-m3, and streams grounded generations using chosen prompts, styles, and scoped collections. Outputs stay auditable with metadata and RBAC.
Nixy1234 / Dfdfd# All paths in this configuration file are relative to Dynmap's data-folder: minecraft_server/dynmap/ # All map templates are defined in the templates directory # To use the HDMap very-low-res (2 ppb) map templates as world defaults, set value to vlowres # The definitions of these templates are in normal-vlowres.txt, nether-vlowres.txt, and the_end-vlowres.txt # To use the HDMap low-res (4 ppb) map templates as world defaults, set value to lowres # The definitions of these templates are in normal-lowres.txt, nether-lowres.txt, and the_end-lowres.txt # To use the HDMap hi-res (16 ppb) map templates (these can take a VERY long time for initial fullrender), set value to hires # The definitions of these templates are in normal-hires.txt, nether-hires.txt, and the_end-hires.txt # To use the HDMap low-res (4 ppb) map templates, with support for boosting resolution selectively to hi-res (16 ppb), set value to low_boost_hi # The definitions of these templates are in normal-low_boost_hi.txt, nether-low_boost_hi.txt, and the_end-low_boost_hi.txt # To use the HDMap hi-res (16 ppb) map templates, with support for boosting resolution selectively to vhi-res (32 ppb), set value to hi_boost_vhi # The definitions of these templates are in normal-hi_boost_vhi.txt, nether-hi_boost_vhi.txt, and the_end-hi_boost_vhi.txt # To use the HDMap hi-res (16 ppb) map templates, with support for boosting resolution selectively to xhi-res (64 ppb), set value to hi_boost_xhi # The definitions of these templates are in normal-hi_boost_xhi.txt, nether-hi_boost_xhi.txt, and the_end-hi_boost_xhi.txt deftemplatesuffix: lowres # Map storage scheme: only uncommoent one 'type' value # filetree: classic and default scheme: tree of files, with all map data under the directory indicated by 'tilespath' setting # sqlite: single SQLite database file (this can get VERY BIG), located at 'dbfile' setting (default is file dynmap.db in data directory) # mysql: MySQL database, at hostname:port in database, accessed via userid with password # mariadb: MariaDB database, at hostname:port in database, accessed via userid with password # postgres: PostgreSQL database, at hostname:port in database, accessed via userid with password storage: # Filetree storage (standard tree of image files for maps) type: filetree # SQLite db for map storage (uses dbfile as storage location) #type: sqlite #dbfile: dynmap.db # MySQL DB for map storage (at 'hostname':'port' in database 'database' using user 'userid' password 'password' and table prefix 'prefix' #type: mysql #hostname: localhost #port: 3306 #database: dynmap #userid: dynmap #password: dynmap #prefix: "" components: - class: org.dynmap.ClientConfigurationComponent - class: org.dynmap.InternalClientUpdateComponent sendhealth: true sendposition: true allowwebchat: true webchat-interval: 5 hidewebchatip: false trustclientname: false includehiddenplayers: false # (optional) if true, color codes in player display names are used use-name-colors: false # (optional) if true, player login IDs will be used for web chat when their IPs match use-player-login-ip: true # (optional) if use-player-login-ip is true, setting this to true will cause chat messages not matching a known player IP to be ignored require-player-login-ip: false # (optional) block player login IDs that are banned from chatting block-banned-player-chat: true # Require login for web-to-server chat (requires login-enabled: true) webchat-requires-login: false # If set to true, users must have dynmap.webchat permission in order to chat webchat-permissions: false # Limit length of single chat messages chatlengthlimit: 256 # # Optional - make players hidden when they are inside/underground/in shadows (#=light level: 0=full shadow,15=sky) # hideifshadow: 4 # # Optional - make player hidden when they are under cover (#=sky light level,0=underground,15=open to sky) # hideifundercover: 14 # # (Optional) if true, players that are crouching/sneaking will be hidden hideifsneaking: false # If true, player positions/status is protected (login with ID with dynmap.playermarkers.seeall permission required for info other than self) protected-player-info: false # If true, hide players with invisibility potion effects active hide-if-invisiblity-potion: true # If true, player names are not shown on map, chat, list hidenames: false #- class: org.dynmap.JsonFileClientUpdateComponent # writeinterval: 1 # sendhealth: true # sendposition: true # allowwebchat: true # webchat-interval: 5 # hidewebchatip: false # includehiddenplayers: false # use-name-colors: false # use-player-login-ip: false # require-player-login-ip: false # block-banned-player-chat: true # hideifshadow: 0 # hideifundercover: 0 # hideifsneaking: false # # Require login for web-to-server chat (requires login-enabled: true) # webchat-requires-login: false # # If set to true, users must have dynmap.webchat permission in order to chat # webchat-permissions: false # # Limit length of single chat messages # chatlengthlimit: 256 # hide-if-invisiblity-potion: true # hidenames: false - class: org.dynmap.SimpleWebChatComponent allowchat: true # If true, web UI users can supply name for chat using 'playername' URL parameter. 'trustclientname' must also be set true. allowurlname: false # Note: this component is needed for the dmarker commands, and for the Marker API to be available to other plugins - class: org.dynmap.MarkersComponent type: markers showlabel: false enablesigns: false # Default marker set for sign markers default-sign-set: markers # (optional) add spawn point markers to standard marker layer showspawn: true spawnicon: world spawnlabel: "Spawn" # (optional) layer for showing offline player's positions (for 'maxofflinetime' minutes after logoff) showofflineplayers: false offlinelabel: "Offline" offlineicon: offlineuser offlinehidebydefault: true offlineminzoom: 0 maxofflinetime: 30 # (optional) layer for showing player's spawn beds showspawnbeds: false spawnbedlabel: "Spawn Beds" spawnbedicon: bed spawnbedhidebydefault: true spawnbedminzoom: 0 spawnbedformat: "%name%'s bed" # (optional) Show world border (vanilla 1.8+) showworldborder: true worldborderlabel: "Border" - class: org.dynmap.ClientComponent type: chat allowurlname: false - class: org.dynmap.ClientComponent type: chatballoon focuschatballoons: false - class: org.dynmap.ClientComponent type: chatbox showplayerfaces: true messagettl: 5 # Optional: set number of lines in scrollable message history: if set, messagettl is not used to age out messages #scrollback: 100 # Optional: set maximum number of lines visible for chatbox #visiblelines: 10 # Optional: send push button sendbutton: false - class: org.dynmap.ClientComponent type: playermarkers showplayerfaces: true showplayerhealth: true # If true, show player body too (only valid if showplayerfaces=true showplayerbody: false # Option to make player faces small - don't use with showplayerhealth smallplayerfaces: false # Optional - make player faces layer hidden by default hidebydefault: false # Optional - ordering priority in layer menu (low goes before high - default is 0) layerprio: 0 # Optional - label for player marker layer (default is 'Players') label: "Players" #- class: org.dynmap.ClientComponent # type: digitalclock - class: org.dynmap.ClientComponent type: link - class: org.dynmap.ClientComponent type: timeofdayclock showdigitalclock: true #showweather: true # Mouse pointer world coordinate display - class: org.dynmap.ClientComponent type: coord label: "Location" hidey: false show-mcr: false show-chunk: false # Note: more than one logo component can be defined #- class: org.dynmap.ClientComponent # type: logo # text: "Dynmap" # #logourl: "images/block_surface.png" # linkurl: "http://forums.bukkit.org/threads/dynmap.489/" # # Valid positions: top-left, top-right, bottom-left, bottom-right # position: bottom-right #- class: org.dynmap.ClientComponent # type: inactive # timeout: 1800 # in seconds (1800 seconds = 30 minutes) # redirecturl: inactive.html # #showmessage: 'You were inactive for too long.' #- class: org.dynmap.TestComponent # stuff: "This is some configuration-value" # Treat hiddenplayers.txt as a whitelist for players to be shown on the map? (Default false) display-whitelist: false # How often a tile gets rendered (in seconds). renderinterval: 1 # How many tiles on update queue before accelerate render interval renderacceleratethreshold: 60 # How often to render tiles when backlog is above renderacceleratethreshold renderaccelerateinterval: 0.2 # How many update tiles to work on at once (if not defined, default is 1/2 the number of cores) tiles-rendered-at-once: 2 # If true, use normal priority threads for rendering (versus low priority) - this can keep rendering # from starving on busy Windows boxes (Linux JVMs pretty much ignore thread priority), but may result # in more competition for CPU resources with other processes usenormalthreadpriority: true # Save and restore pending tile renders - prevents their loss on server shutdown or /reload saverestorepending: true # Save period for pending jobs (in seconds): periodic saving for crash recovery of jobs save-pending-period: 900 # Zoom-out tile update period - how often to scan for and process tile updates into zoom-out tiles (in seconds) zoomoutperiod: 30 # Control whether zoom out tiles are validated on startup (can be needed if zoomout processing is interrupted, but can be expensive on large maps) initial-zoomout-validate: true # Default delay on processing of updated tiles, in seconds. This can reduce potentially expensive re-rendering # of frequently updated tiles (such as due to machines, pistons, quarries or other automation). Values can # also be set on individual worlds and individual maps. tileupdatedelay: 30 # Tile hashing is used to minimize tile file updates when no changes have occurred - set to false to disable enabletilehash: true # Optional - hide ores: render as normal stone (so that they aren't revealed by maps) #hideores: true # Optional - enabled BetterGrass style rendering of grass and snow block sides #better-grass: true # Optional - enable smooth lighting by default on all maps supporting it (can be set per map as lighting option) smooth-lighting: true # Optional - use world provider lighting table (good for custom worlds with custom lighting curves, like nether) # false=classic Dynmap lighting curve use-brightness-table: true # Optional - render specific block names using the textures and models of another block name: can be used to hide/disguise specific # blocks (e.g. make ores look like stone, hide chests) or to provide simple support for rendering unsupported custom blocks block-alias: # "minecraft:quartz_ore": "stone" # "diamond_ore": "coal_ore" # Default image format for HDMaps (png, jpg, jpg-q75, jpg-q80, jpg-q85, jpg-q90, jpg-q95, jpg-q100, webp, webp-q75, webp-q80, webp-q85, webp-q90, webp-q95, webp-q100), # Note: any webp format requires the presence of the 'webp command line tools' (cwebp, dwebp) (https://developers.google.com/speed/webp/download) # # Has no effect on maps with explicit format settings image-format: jpg-q90 # If cwebp or dwebp are not on the PATH, use these settings to provide their full path. Do not use these settings if the tools are on the PATH # For Windows, include .exe # #cwebpPath: /usr/bin/cwebp #dwebpPath: /usr/bin/dwebp # use-generated-textures: if true, use generated textures (same as client); false is static water/lava textures # correct-water-lighting: if true, use corrected water lighting (same as client); false is legacy water (darker) # transparent-leaves: if true, leaves are transparent (lighting-wise): false is needed for some Spout versions that break lighting on leaf blocks use-generated-textures: true correct-water-lighting: true transparent-leaves: true # ctm-support: if true, Connected Texture Mod (CTM) in texture packs is enabled (default) ctm-support: true # custom-colors-support: if true, Custom Colors in texture packs is enabled (default) custom-colors-support: true # Control loading of player faces (if set to false, skins are never fetched) #fetchskins: false # Control updating of player faces, once loaded (if faces are being managed by other apps or manually) #refreshskins: false # Customize URL used for fetching player skins (%player% is macro for name) skin-url: "http://skins.minecraft.net/MinecraftSkins/%player%.png" # Control behavior for new (1.0+) compass orientation (sunrise moved 90 degrees: east is now what used to be south) # default is 'newrose' (preserve pre-1.0 maps, rotate rose) # 'newnorth' is used to rotate maps and rose (requires fullrender of any HDMap map - same as 'newrose' for FlatMap or KzedMap) compass-mode: newnorth # Triggers for automatic updates : blockupdate-with-id is debug for breaking down updates by ID:meta # To disable, set just 'none' and comment/delete the rest render-triggers: - blockupdate #- blockupdate-with-id #- lightingupdate - chunkpopulate - chunkgenerate #- none # Title for the web page - if not specified, defaults to the server's name (unless it is the default of 'Unknown Server') #webpage-title: "My Awesome Server Map" # The path where the tile-files are placed. tilespath: web/tiles # The path where the web-files are located. webpath: web # The path were the /dynmapexp command exports OBJ ZIP files exportpath: export # The network-interface the webserver will bind to (0.0.0.0 for all interfaces, 127.0.0.1 for only local access). # If not set, uses same setting as server in server.properties (or 0.0.0.0 if not specified) #webserver-bindaddress: 0.0.0.0 # The TCP-port the webserver will listen on. webserver-port: 8123 # Maximum concurrent session on internal web server - limits resources used in Bukkit server max-sessions: 30 # Disables Webserver portion of Dynmap (Advanced users only) disable-webserver: false # Enable/disable having the web server allow symbolic links (true=compatible with existing code, false=more secure (default)) allow-symlinks: true # Enable login support login-enabled: false # Require login to access website (requires login-enabled: true) login-required: false # Period between tile renders for fullrender, in seconds (non-zero to pace fullrenders, lessen CPU load) timesliceinterval: 0.0 # Maximum chunk loads per server tick (1/20th of a second) - reducing this below 90 will impact render performance, but also will reduce server thread load maxchunkspertick: 200 # Progress report interval for fullrender/radiusrender, in tiles. Must be 100 or greater progressloginterval: 100 # Parallel fullrender: if defined, number of concurrent threads used for fullrender or radiusrender # Note: setting this will result in much more intensive CPU use, some additional memory use. Caution should be used when # setting this to equal or exceed the number of physical cores on the system. #parallelrendercnt: 4 # Interval the browser should poll for updates. updaterate: 2000 # If nonzero, server will pause fullrender/radiusrender processing when 'fullrenderplayerlimit' or more users are logged in fullrenderplayerlimit: 0 # If nonzero, server will pause update render processing when 'updateplayerlimit' or more users are logged in updateplayerlimit: 0 # Target limit on server thread use - msec per tick per-tick-time-limit: 50 # If TPS of server is below this setting, update renders processing is paused update-min-tps: 18.0 # If TPS of server is below this setting, full/radius renders processing is paused fullrender-min-tps: 18.0 # If TPS of server is below this setting, zoom out processing is paused zoomout-min-tps: 18.0 showplayerfacesinmenu: true # Control whether players that are hidden or not on current map are grayed out (true=yes) grayplayerswhenhidden: true # Set sidebaropened: 'true' to pin menu sidebar opened permanently, 'pinned' to default the sidebar to pinned, but allow it to unpin #sidebaropened: true # Customized HTTP response headers - add 'id: value' pairs to all HTTP response headers (internal web server only) #http-response-headers: # Access-Control-Allow-Origin: "my-domain.com" # X-Custom-Header-Of-Mine: "MyHeaderValue" # Trusted proxies for web server - which proxy addresses are trusted to supply valid X-Forwarded-For fields trusted-proxies: - "127.0.0.1" - "0:0:0:0:0:0:0:1" joinmessage: "%playername% joined" quitmessage: "%playername% quit" spammessage: "You may only chat once every %interval% seconds." # format for messages from web: %playername% substitutes sender ID (typically IP), %message% includes text webmsgformat: "&color;2[WEB] %playername%: &color;f%message%" # Control whether layer control is presented on the UI (default is true) showlayercontrol: true # Enable checking for banned IPs via banned-ips.txt (internal web server only) check-banned-ips: true # Default selection when map page is loaded defaultzoom: 0 defaultworld: world defaultmap: flat # (optional) Zoom level and map to switch to when following a player, if possible #followzoom: 3 #followmap: surface # If true, make persistent record of IP addresses used by player logins, to support web IP to player matching persist-ids-by-ip: true # If true, map text to cyrillic cyrillic-support: false # Messages to customize msg: maptypes: "Map Types" players: "Players" chatrequireslogin: "Chat Requires Login" chatnotallowed: "You are not permitted to send chat messages" hiddennamejoin: "Player joined" hiddennamequit: "Player quit" # URL for client configuration (only need to be tailored for proxies or other non-standard configurations) url: # configuration URL #configuration: "up/configuration" # update URL #update: "up/world/{world}/{timestamp}" # sendmessage URL #sendmessage: "up/sendmessage" # login URL #login: "up/login" # register URL #register: "up/register" # tiles base URL #tiles: "tiles/" # markers base URL #markers: "tiles/" # Snapshot cache size, in chunks snapshotcachesize: 500 # Snapshot cache uses soft references (true), else weak references (false) soft-ref-cache: true # Player enter/exit title messages for map markers # # Processing period - how often to check player positions vs markers - default is 1000ms (1 second) #enterexitperiod: 1000 # Title message fade in time, in ticks (0.05 second intervals) - default is 10 (1/2 second) #titleFadeIn: 10 # Title message stay time, in ticks (0.05 second intervals) - default is 70 (3.5 seconds) #titleStay: 70 # Title message fade out time, in ticks (0.05 seocnd intervals) - default is 20 (1 second) #titleFadeOut: 20 # Enter/exit messages use on screen titles (true - default), if false chat messages are sent instead #enterexitUseTitle: true # Set true if new enter messages should supercede pending exit messages (vs being queued in order), default false #enterReplacesExits: true # Set to true to enable verbose startup messages - can help with debugging map configuration problems # Set to false for a much quieter startup log verbose: false # Enables debugging. #debuggers: # - class: org.dynmap.debug.LogDebugger # Debug: dump blocks missing render data dump-missing-blocks: false
tigusigalpa / Dropbox PhpModern PHP SDK for Dropbox API v2 with full Laravel 8-12 support. Provides clean, intuitive interface for cloud storage operations: file upload/download, sharing, batch operations, OAuth 2.0 flow. Built with PHP 8.1+, includes chunked uploads, comprehensive documentation, and PHPUnit tests. Works standalone or with Laravel.
eihwaz / Anvil RegionRegion file format storage for chunks
NetApp / SS CDCThe code provides an example C language based implementation of the idea proposed in the paper "SS-CDC: a two-stage parallel content-defined chunking for deduplicating backup storage". It provides an efficient parallel content-defined chunking (CDC) implementation for deduplication systems with Intel AVX-512 instructions, which can achieve high chunking speed as well as zero deduplication ratio reduction compared to sequential CDC methods.
