Cacache
💩💵 but for your data. If you've got the hash, we've got the cache ™ (moved)
Install / Use
/learn @zkat/CacacheREADME
cacache 
NOTE: This repository has moved to https://github.com/npm/cacache
cacache is a Node.js library for managing
local key and content address caches. It's really fast, really good at
concurrency, and it will never give you corrupted data, even if cache files
get corrupted or manipulated.
It was originally written to be used as npm's local cache, but can just as easily be used on its own.
Translations: español
Install
$ npm install --save cacache
Table of Contents
- Example
- Features
- Contributing
- API
- Using localized APIs
- Reading
- Writing
- Utilities
- Integrity
Example
const cacache = require('cacache/en')
const fs = require('fs')
const tarball = '/path/to/mytar.tgz'
const cachePath = '/tmp/my-toy-cache'
const key = 'my-unique-key-1234'
// Cache it! Use `cachePath` as the root of the content cache
cacache.put(cachePath, key, '10293801983029384').then(integrity => {
console.log(`Saved content to ${cachePath}.`)
})
const destination = '/tmp/mytar.tgz'
// Copy the contents out of the cache and into their destination!
// But this time, use stream instead!
cacache.get.stream(
cachePath, key
).pipe(
fs.createWriteStream(destination)
).on('finish', () => {
console.log('done extracting!')
})
// The same thing, but skip the key index.
cacache.get.byDigest(cachePath, integrityHash).then(data => {
fs.writeFile(destination, data, err => {
console.log('tarball data fetched based on its sha512sum and written out!')
})
})
Features
- Extraction by key or by content address (shasum, etc)
- Subresource Integrity web standard support
- Multi-hash support - safely host sha1, sha512, etc, in a single cache
- Automatic content deduplication
- Fault tolerance (immune to corruption, partial writes, process races, etc)
- Consistency guarantees on read and write (full data verification)
- Lockless, high-concurrency cache access
- Streaming support
- Promise support
- Pretty darn fast -- sub-millisecond reads and writes including verification
- Arbitrary metadata storage
- Garbage collection and additional offline verification
- Thorough test coverage
- There's probably a bloom filter in there somewhere. Those are cool, right? 🤔
Contributing
The cacache team enthusiastically welcomes contributions and project participation! There's a bunch of things you can do if you want to contribute! The Contributor Guide has all the information you need for everything from reporting bugs to contributing entire new features. Please don't hesitate to jump in if you'd like to, or even ask us questions if something isn't clear.
All participants and maintainers in this project are expected to follow Code of Conduct, and just generally be excellent to each other.
Please refer to the Changelog for project history details, too.
Happy hacking!
API
<a name="localized-api"></a> Using localized APIs
cacache includes a complete API in English, with the same features as other
translations. To use the English API as documented in this README, use
require('cacache/en'). This is also currently the default if you do
require('cacache'), but may change in the future.
cacache also supports other languages! You can find the list of currently
supported ones by looking in ./locales in the source directory. You can use
the API in that language with require('cacache/<lang>').
Want to add support for a new language? Please go ahead! You should be able to
copy ./locales/en.js and ./locales/en.json and fill them in. Translating the
README.md is a bit more work, but also appreciated if you get around to it. 👍🏼
<a name="ls"></a> > cacache.ls(cache) -> Promise<Object>
Lists info for all entries currently in the cache as a single large object. Each
entry in the object will be keyed by the unique index key, with corresponding
get.info objects as the values.
Example
cacache.ls(cachePath).then(console.log)
// Output
{
'my-thing': {
key: 'my-thing',
integrity: 'sha512-BaSe64/EnCoDED+HAsh=='
path: '.testcache/content/deadbeef', // joined with `cachePath`
time: 12345698490,
size: 4023948,
metadata: {
name: 'blah',
version: '1.2.3',
description: 'this was once a package but now it is my-thing'
}
},
'other-thing': {
key: 'other-thing',
integrity: 'sha1-ANothER+hasH=',
path: '.testcache/content/bada55',
time: 11992309289,
size: 111112
}
}
<a name="ls-stream"></a> > cacache.ls.stream(cache) -> Readable
Lists info for all entries currently in the cache as a single large object.
This works just like ls, except get.info entries are
returned as 'data' events on the returned stream.
Example
cacache.ls.stream(cachePath).on('data', console.log)
// Output
{
key: 'my-thing',
integrity: 'sha512-BaSe64HaSh',
path: '.testcache/content/deadbeef', // joined with `cachePath`
time: 12345698490,
size: 13423,
metadata: {
name: 'blah',
version: '1.2.3',
description: 'this was once a package but now it is my-thing'
}
}
{
key: 'other-thing',
integrity: 'whirlpool-WoWSoMuchSupport',
path: '.testcache/content/bada55',
time: 11992309289,
size: 498023984029
}
{
...
}
<a name="get-data"></a> > cacache.get(cache, key, [opts]) -> Promise({data, metadata, integrity})
Returns an object with the cached data, digest, and metadata identified by
key. The data property of this object will be a Buffer instance that
presumably holds some data that means something to you. I'm sure you know what
to do with it! cacache just won't care.
integrity is a Subresource
Integrity
string. That is, a string that can be used to verify data, which looks like
<hash-algorithm>-<base64-integrity-hash>.
If there is no content identified by key, or if the locally-stored data does
not pass the validity checksum, the promise will be rejected.
A sub-function, get.byDigest may be used for identical behavior, except lookup
will happen by integrity hash, bypassing the index entirely. This version of the
function only returns data itself, without any wrapper.
Note
This function loads the entire cache entry into memory before returning it. If
you're dealing with Very Large data, consider using get.stream
instead.
Example
// Look up by key
cache.get(cachePath, 'my-thing').then(console.log)
// Output:
{
metadata: {
thingName: 'my'
},
integrity: 'sha512-BaSe64HaSh',
data: Buffer#<deadbeef>,
size: 9320
}
// Look up by digest
cache.get.byDigest(cachePath, 'sha512-BaSe64HaSh').then(console.log)
// Output:
Buffer#<deadbeef>
<a name="get-stream"></a> > cacache.get.stream(cache, key, [opts]) -> Readable
Returns a Readable Stream of the cached data identified by key.
If there is no content identified by key, or if the locally-stored data does
not pass the validity checksum, an error will be emitted.
metadata and integrity events will be emitted before the stream closes, if
you need to collect that extra data about the cached entry.
A sub-function, get.stream.byDigest may be used for identical behavior,
except lookup will happen by integrity hash, bypassing the index entirely. This
version does not emit the metadata and integrity events at all.
Example
// Look up by key
cache.get.stream(
cachePath, 'my-thing'
).on('metadata', metadata => {
console.log('metadata:', metadata)
}).on('integrity', integrity => {
console.log('integrity:', integrity)
}).pipe(
fs.createWriteStream('./x.tgz')
)
// Outputs:
metadata: { ... }
integrity: 'sha512-SoMeDIGest+64=='
// Look up by digest
cache.get.stream.byDigest(
cachePath, 'sha512-SoMeDIGest+64=='
).pipe(
fs.createWriteStream('./x.tgz')
)
<a name="get-info"></a> > cacache.get.info(cache, key) -> Promise
Looks up key in the cache index, returning information about the entry if
one exists.
Fields
key- Key the entry was looked up under. Matches thekeyargument.integrity- Subresource Integrity hash for the content this entry refers to.path- Filesystem path where content is stored, joined withcacheargument.time- Timestamp the entry was first added on.metadata- User-assigned metadata associated with the entry/content.
Example
cacache.get.info(cachePath, 'my-thing').then(console.log)
// Output
{
key: 'my-thing'
