cacache

cacache is a Node.js library for managing local key and content address caches. It's really fast, really good at concurrency, and it will never give you corrupted data, even if cache files get corrupted or manipulated.

On systems that support user and group settings on files, cacache will match the uid and gid values to the folder where the cache lives, even when running as root.

It was written to be used as npm's local cache, but can just as easily be used on its own.

Install

$ npm install --save cacache

Table of Contents

• Example
• Features
• Contributing
• API
  • Using localized APIs
  • Reading
    • ls
    • ls.stream
    • get
    • get.stream
    • get.info
    • get.hasContent
  • Writing
    • put
    • put.stream
    • rm.all
    • rm.entry
    • rm.content
    • index.compact
    • index.insert
  • Utilities
    • clearMemoized
    • tmp.mkdir
    • tmp.withTmp
  • Integrity
    • Subresource Integrity
    • verify
    • verify.lastRun

Example

const cacache = require('cacache')
const fs = require('fs')

const cachePath = '/tmp/my-toy-cache'
const key = 'my-unique-key-1234'

// Cache it! Use `cachePath` as the root of the content cache
cacache.put(cachePath, key, '10293801983029384').then(integrity => {
  console.log(`Saved content to ${cachePath}.`)
})

const destination = '/tmp/mytar.tgz'

// Copy the contents out of the cache and into their destination!
// But this time, use stream instead!
cacache.get.stream(
  cachePath, key
).pipe(
  fs.createWriteStream(destination)
).on('finish', () => {
  console.log('done extracting!')
})

// The same thing, but skip the key index.
cacache.get.byDigest(cachePath, integrityHash).then(data => {
  fs.writeFile(destination, data, err => {
    console.log('tarball data fetched based on its sha512sum and written out!')
  })
})

Features

• Extraction by key or by content address (shasum, etc)
• Subresource Integrity web standard support
• Multi-hash support - safely host sha1, sha512, etc, in a single cache
• Automatic content deduplication
• Fault tolerance (immune to corruption, partial writes, process races, etc)
• Consistency guarantees on read and write (full data verification)
• Lockless, high-concurrency cache access
• Streaming support
• Promise support
• Fast -- sub-millisecond reads and writes including verification
• Arbitrary metadata storage
• Garbage collection and additional offline verification
• Thorough test coverage
• There's probably a bloom filter in there somewhere. Those are cool, right? 🤔

Contributing

The cacache team enthusiastically welcomes contributions and project participation! There's a bunch of things you can do if you want to contribute! Please don't hesitate to jump in if you'd like to, or even ask us questions if something isn't clear.

All participants and maintainers in this project are expected to follow Code of Conduct, and just generally be excellent to each other.

Please refer to the Changelog for project history details, too.

Happy hacking!

API

<a name="ls"></a> > cacache.ls(cache) -> Promise<Object>

Lists info for all entries currently in the cache as a single large object. Each entry in the object will be keyed by the unique index key, with corresponding get.info objects as the values.

Example

cacache.ls(cachePath).then(console.log)
// Output
{
  'my-thing': {
    key: 'my-thing',
    integrity: 'sha512-BaSe64/EnCoDED+HAsh=='
    path: '.testcache/content/deadbeef', // joined with `cachePath`
    time: 12345698490,
    size: 4023948,
    metadata: {
      name: 'blah',
      version: '1.2.3',
      description: 'this was once a package but now it is my-thing'
    }
  },
  'other-thing': {
    key: 'other-thing',
    integrity: 'sha1-ANothER+hasH=',
    path: '.testcache/content/bada55',
    time: 11992309289,
    size: 111112
  }
}

<a name="ls-stream"></a> > cacache.ls.stream(cache) -> Readable

Lists info for all entries currently in the cache as a single large object.

This works just like ls, except get.info entries are returned as 'data' events on the returned stream.

Example

cacache.ls.stream(cachePath).on('data', console.log)
// Output
{
  key: 'my-thing',
  integrity: 'sha512-BaSe64HaSh',
  path: '.testcache/content/deadbeef', // joined with `cachePath`
  time: 12345698490,
  size: 13423,
  metadata: {
    name: 'blah',
    version: '1.2.3',
    description: 'this was once a package but now it is my-thing'
  }
}

{
  key: 'other-thing',
  integrity: 'whirlpool-WoWSoMuchSupport',
  path: '.testcache/content/bada55',
  time: 11992309289,
  size: 498023984029
}

{
  ...
}

<a name="get-data"></a> > cacache.get(cache, key, [opts]) -> Promise({data, metadata, integrity})

Returns an object with the cached data, digest, and metadata identified by key. The data property of this object will be a Buffer instance that presumably holds some data that means something to you. I'm sure you know what to do with it! cacache just won't care.

integrity is a Subresource Integrity string. That is, a string that can be used to verify data, which looks like <hash-algorithm>-<base64-integrity-hash>.

If there is no content identified by key, or if the locally-stored data does not pass the validity checksum, the promise will be rejected.

A sub-function, get.byDigest may be used for identical behavior, except lookup will happen by integrity hash, bypassing the index entirely. This version of the function only returns data itself, without any wrapper.

See: options

Note

This function loads the entire cache entry into memory before returning it. If you're dealing with Very Large data, consider using get.stream instead.

Example

// Look up by key
cache.get(cachePath, 'my-thing').then(console.log)
// Output:
{
  metadata: {
    thingName: 'my'
  },
  integrity: 'sha512-BaSe64HaSh',
  data: Buffer#<deadbeef>,
  size: 9320
}

// Look up by digest
cache.get.byDigest(cachePath, 'sha512-BaSe64HaSh').then(console.log)
// Output:
Buffer#<deadbeef>

<a name="get-stream"></a> > cacache.get.stream(cache, key, [opts]) -> Readable

Returns a Readable Stream of the cached data identified by key.

If there is no content identified by key, or if the locally-stored data does not pass the validity checksum, an error will be emitted.

metadata and integrity events will be emitted before the stream closes, if you need to collect that extra data about the cached entry.

A sub-function, get.stream.byDigest may be used for identical behavior, except lookup will happen by integrity hash, bypassing the index entirely. This version does not emit the metadata and integrity events at all.

See: options

Example

// Look up by key
cache.get.stream(
  cachePath, 'my-thing'
).on('metadata', metadata => {
  console.log('metadata:', metadata)
}).on('integrity', integrity => {
  console.log('integrity:', integrity)
}).pipe(
  fs.createWriteStream('./x.tgz')
)
// Outputs:
metadata: { ... }
integrity: 'sha512-SoMeDIGest+64=='

// Look up by digest
cache.get.stream.byDigest(
  cachePath, 'sha512-SoMeDIGest+64=='
).pipe(
  fs.createWriteStream('./x.tgz')
)

<a name="get-info"></a> > cacache.get.info(cache, key) -> Promise

Looks up key in the cache index, returning information about the entry if one exists.

Fields

• key - Key the entry was looked up under. Matches the key argument.
• integrity - Subresource Integrity hash for the content this entry refers to.
• path - Filesystem path where content is stored, joined with cache argument.
• time - Timestamp the entry was first added on.
• metadata - User-assigned metadata associated with the entry/content.

Example

cacache.get.info(cachePath, 'my-thing').then(console.log)

// Output
{
  key: 'my-thing',
  integrity: 'sha256-MUSTVERIFY+ALL/THINGS=='
  path: '.testcache/content/deadbeef',
  time: 12345698490,
  size: 849234,
  metadata: {
    name: 'blah',
    version: '1.2.3',
    description: 'this was once a package but now it is my-thing'
  }
}

<a name="get-hasContent"></a> > cacache.get.hasContent(cache, integrity) -> Promise

Looks up a Subresource Integrity hash in the cache. If content exists for this integrity, it will return an object, with the specific single integrity hash that was found in sri key, and the size of the found content as size. If no content exists for this integrity, it will return false.

Example

cacache.get.hasContent(cachePath, 'sha256-MUSTVERIFY+ALL/THINGS==').then(console.log)

// Output
{
  sri: {
    source: 'sha256-MUSTVERIFY+ALL/THINGS==',
    algorithm: 'sha256',
    d