idmp

An elegant, lightweight (~1KB gzipped) utility to deduplicate concurrent calls to the same async function, providing idempotent behavior for network and async requests.

English | 简体中文

Demo https://idmp.js.org

Breaking Changes

The following breaking changes are introduced in recent major versions. Please review if you're upgrading from older versions.

v4.x: node-fs/redis persistence uses json-web3, cached data from serialize-javascript is not compatible
v3.x: not export { _globalStore as g } any more
v2.x:: remove the "type": "module" field in Package.json

Usage

Basic Usage

import idmp from 'idmp'

const getInfo = async () => {
  const API = `https://idmp.js.org/api?/your-info`
  return await fetch(API).then((d) => d.text())
}

// Only one line need to change
export const getInfoIdmp = () => idmp('/api/your-info', getInfo)

for (let i = 0; i < 10; ++i) {
  getInfoIdmp().then((d) => {
    console.log(d)
  })
}

Check the network console, there will be only 1 network request, but 10 callbacks will be triggered correctly.

Advanced Usage

const getInfoById = async (id: string) => {
  const API = `https://idmp.js.org/api?/your-info&id=${id}`
  return await fetch(API).then((d) => d.json())
}

// Handle params
export const getInfoByIdIdmp = (id: string) =>
  idmp(`/api/your-info?${id}`, () => getInfoById(id))

// Or a more generic type juggling, for complex params, idmp will infer the return type automatically, keep it consistent with the original function
export const getInfoByIdIdmp = (...args: Parameters<typeof getInfoById>) =>
  idmp(`/api/your-info?${JSON.stringify(args)}`, () => getInfoById(...args))

// More options
export const getInfoByIdIdmp = (id: string) =>
  idmp(`/api/your-info?${id}`, () => getInfoById(id), {
    maxAge: 86400 * 1000,
  })

Then replace getInfoByIdIdmp with getInfoById.

Plugins

idmp has a powerful plugin system. The following plugins are officially maintained, and you can also reference the source code to create your own plugins:

The analogy to higher-order functions elegantly conveys that plugins can extend idmp's core functionality in a non-invasive way, similar to mathematical functions $g(f)(x)$. This provides great flexibility and extensibility to the plugin system.

Data Persistence with node-fs (Persist data to the file system)
Data Persistence with localStorage
Data Persistence with sessionStorage
Data Persistence with redis
Data Persistence with indexedDB // TODO
Data Persistence with chrome-extension // TODO

Options

declare const idmp: {
  <T>(
    globalKey: IdmpGlobalKey,
    promiseFunc: IdmpPromise<T>,
    options?: IdmpOptions,
  ): Promise<T>
  flush: (globalKey: IdmpGlobalKey) => void
  flushAll: () => void
}

type IdmpPromise<T> = () => Promise<T>
type IdmpGlobalKey = string | number | symbol | false | null | undefined

IdmpOptions:

| Property | Type | Default | Description | | --------------- | ---------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | maxRetry | number | 30 | Maximum number of retry attempts. | | minRetryDelay | number | 50(ms) | Minimum retry interval in milliseconds. The default value is 50 ms. | | maxAge | number | 3000(ms) | Maximum age in milliseconds. The maximum value is 604800000 ms (7 days). | | onBeforeRetry | function | - | Function to be executed before a retry attempt. Takes two parameters: err (any type) and extra (an object with properties globalKey of type IdmpGlobalKey and retryCount of type number). Returns void. |

flush

flush is a static method of idmp that will immediately clear the cache so that the next call shortly after will not use the cache.

flush takes a globalKey as parameter, has no return value. Calling it repeatedly or with a non-existing globalKey will not have any prompts.


const fetchData = () => idmp('key', async () => data)

idmp.flush('key')
fetchData().then(...) // will skip cache

flushAll

flushAll is a static method of idmp that will immediately clear all caches so that the next calls shortly after will not use caches.

flushAll is idempotent like flush, no params or return value. Calling it multiple times will not have any prompts.


const fetchData1 = () => idmp('key1', async () => data1)
const fetchData2 = () => idmp('key2', async () => data2)

idmp.flushAll()

fetchData1().then(...) // will skip cache
fetchData2().then(...) // will skip cache

You can do some works with flush or flushAll, for example, auto refresh list after clicking the save button, should fetch the latest data from server forcibly.

Disable debug logs

In development mode, debug information is displayed by default. Most modern frameworks and build tools—such as React, Vue, Webpack, and Vite etc. will automatically set process.env.NODE_ENV to production in production builds.

In production, idmp prunes debug logic to reduce bundle size and improve performance.

If you prefer not to see debug information even in development, you can disable it manually by setting the following in the browser console: localStorage.idmp_debug = false.

Deduplication in React

In React, you can share requests using SWR, Provider and more complex state management libraries. But there are some problems:

SWR: Requires requests to be encapsulated in hooks, which may limit conditional or nested usage patterns. Migrating legacy codebases can be non-trivial in some cases.
Provider: Needs centralized data management. The data center can't perceive which modules will consume the data, need to maintain the data for a long time, and dare not delete it in time
Redux: Should focus on state changes and sequences, not data sharing. idmp lets you focus more on local state

See demo and source code

So when module A or module B's code is deleted, there is no need to maintain their cache.

Module A and B have greater independence, can be reused across projects, without having to be wrapped in a specific Provider.

Limitations of requesting data in Hooks

import useSWR from 'swr'

function Profile() {
  const { data, error, isLoading } = useSWR('/api/user', fetcher)

  if (error) return <div>failed to load</div>
  if (isLoading) return <div>loading...</div>
  return <div>hello {data.name}!</div>
}

The example on SWR's homepage is very elegant, but in practice a view is likely to come from more than one data source. Because Hooks can't be nested and have conditional branches. Assume there are two interfaces, B depends on the result of A as params, the code will quickly deteriorate to the following form:

...
const { data: dataA } = useSWR('/api/a', fetchA)
const { data: dataB } = useSWR(dataA ? `/api/b${JSON.stringify(dataA)}` : null, () => dataA ? fetchB(dataA): null)
...

This doesn't handle exception cases yet, and there are only 2 interfaces. If there are n related interfaces, the code complexity deteriorates at a rate of $O(2^n)$

$$ C_{n}^{0} + C_{n}^{1} + C_{n}^{2} + ... + C_{n}^{n} = 2^n $$

There are several optimization forms:

Abandon SWR and use request in useEffect, so the benefits of SWR are lost, and there may still be duplicate requests issues even if passing empty array as the second param of useEffect, see https://github.com/ha0z1/idmp/blob/main/demo/Item.tsx#L10
Wrap fetchAB method to request sequentially and return at one time. In Hooks just call the single fetchAB. Here the views that only rely on dataA have to wait for completion before rendering. In addition, dataA is often some common data that may need to handle

Idmp

Install / Use

README