Store

<table> <tr> <td> <h3>DEPRECATED</h3> Store(3) is deprecated. No more development will be taking place. For an up-to-date version, please use <a href="https://github.com/dropbox/store">Store(4)</a>. Thanks for all your support! </td> </tr> </table> <br/><br/>

Store is a Java library for effortless, reactive data loading.

The Problems:

• Modern software needs data representations to be fluid and always available.
• Users expect their UI experience to never be compromised (blocked) by new data loads. Whether an application is social, news, or business-to-business, users expect a seamless experience both online and offline.
• International users expect minimal data downloads as many megabytes of downloaded data can quickly result in astronomical phone bills.

A Store is a class that simplifies fetching, parsing, storage, and retrieval of data in your application. A Store is similar to the Repository pattern [https://msdn.microsoft.com/en-us/library/ff649690.aspx] while exposing a Reactive API built with RxJava that adheres to a unidirectional data flow.

Store provides a level of abstraction between UI elements and data operations.

Overview

A Store is responsible for managing a particular data request. When you create an implementation of a Store, you provide it with a Fetcher, a function that defines how data will be fetched over network. You can also define how your Store will cache data in-memory and on-disk, as well as how to parse it. Since Store returns your data as an Observable, threading is a breeze! Once a Store is built, it handles the logic around data flow, allowing your views to use the best data source and ensuring that the newest data is always available for later offline use. Stores can be customized to work with your own implementations or use our included middleware.

Store leverages RxJava and multiple request throttling to prevent excessive calls to the network and disk cache. By utilizing Store, you eliminate the possibility of flooding your network with the same request while adding two layers of caching (memory and disk).

How to include in your project

Include gradle dependency

implementation 'com.nytimes.android:store3:3.1.1'

Set the source & target compatibilities to 1.8

Starting with Store 3.0, retrolambda is no longer used. Therefore to allow support for lambdas the Java sourceCompatibility and targetCompatibility need to be set to 1.8

android {
    compileOptions {
        sourceCompatibility 1.8
        targetCompatibility 1.8
    }
    ...
}

Fully Configured Store

Let's start by looking at what a fully configured Store looks like. We will then walk through simpler examples showing each piece:

Store<ArticleAsset, Integer> articleStore = StoreBuilder.<Integer, BufferedSource, ArticleAsset>parsedWithKey()
        .fetcher(articleId -> api.getArticleAsBufferedSource(articleId))  // OkHttp responseBody.source()
        .persister(FileSystemPersister.create(FileSystemFactory.create(context.getFilesDir()), pathResolver))
        .parser(GsonParserFactory.createSourceParser(gson, ArticleAsset.Article.class))
        .open();
        

With the above setup you have:

• In-memory caching for rotation
• Disk caching for when users are offline
• Parsing through streaming API to limit memory consumption
• Rich API to ask for data whether you want cached, new or a stream of future data updates.

And now for the details:

Creating a Store

You create a Store using a builder. The only requirement is to include a Fetcher<ReturnType, KeyType> that returns a Single<ReturnType> and has a single method fetch(key)

Store<ArticleAsset, Integer> store = StoreBuilder.<>key()
        .fetcher(articleId -> api.getArticle(articleId))  // OkHttp responseBody.source()
        .open();

Stores use generic keys as identifiers for data. A key can be any value object that properly implements toString(), equals() and hashCode(). When your Fetcher function is called, it will be passed a particular Key value. Similarly, the key will be used as a primary identifier within caches (Make sure to have a proper hashCode()!!).

Our Key implementation - Barcodes

For convenience, we included our own key implementation called a BarCode. Barcode has two fields String key and String type

BarCode barcode = new BarCode("Article", "42");

When using a Barcode as your key, you can use a StoreBuilder convenience method

Store<ArticleAsset, BarCode> store = StoreBuilder.<ArticleAsset>barcode()
        .fetcher(articleBarcode -> api.getAsset(articleBarcode.getKey(), articleBarcode.getType()))
        .open();

Public Interface - Get, Fetch, Stream, GetRefreshing

Single<Article> article = store.get(barCode);

The first time you subscribe to store.get(barCode), the response will be stored in an in-memory cache. All subsequent calls to store.get(barCode) with the same Key will retrieve the cached version of the data, minimizing unnecessary data calls. This prevents your app from fetching fresh data over the network (or from another external data source) in situations when doing so would unnecessarily waste bandwidth and battery. A great use case is any time your views are recreated after a rotation, they will be able to request the cached data from your Store. Having this data available can help you avoid the need to retain this in the view layer.

So far our Store’s data flow looks like this:

By default, 100 items will be cached in memory for 24 hours. You may pass in your own instance of a Guava Cache to override the default policy.

Busting through the cache

Alternatively you can call store.fetch(barCode) to get an Observable that skips the memory (and optional disk cache).

Fresh data call will look like: store.fetch()

In the New York Times app, overnight background updates use fetch() to make sure that calls to store.get() will not have to hit the network during normal usage. Another good use case for fetch() is when a user wants to pull to refresh.

Calls to both fetch() and get() emit one value and then call onCompleted() or throw an error.

Stream

For real-time updates, you may also call store.stream() which returns an Observable that emits each time a new item is added to the Store. You can think of stream as an Event Bus-like feature that allows you to know when any new network hits happen for a particular Store. You can leverage the Rx operator filter() to only subscribe to a subset of emissions.

Get Refreshing

There is another special way to subscribe to a Store: getRefreshing(key). This method will subscribe to get() which returns a single response, but unlike get(), getRefreshing(key) will stay subscribed. Anytime you call store.clear(key) anyone subscribed to getRefreshing(key) will resubscribe and force a new network response.

Inflight Debouncer

To prevent duplicate requests for the same data, Store offers an inflight debouncer. If the same request is made within a minute of a previous identical request, the same response will be returned. This is useful for situations when your app needs to make many async calls for the same data at startup or when users are obsessively pulling to refresh. As an example, The New York Times news app asynchronously calls ConfigStore.get() from 12 different places on startup. The first call blocks while all others wait for the data to arrive. We have seen a dramatic decrease in the app's data usage after implementing this inflight logic.

Adding a Parser

Since it is rare for data to arrive from the network in the format that your views need, Stores can delegate to a parser by using a StoreBuilder.<BarCode, BufferedSource, Article>parsedWithKey()

Store<Article, Integer> store = StoreBuilder.<Integer, BufferedSource, Article>parsedWithKey()
        .fetcher(articleId -> api.getArticle(articleId)) 
        .parser(source -> {
            try (InputStreamReader reader = new InputStreamReader(source.inputStream())) {
                return gson.fromJson(reader, Article.class);
            } catch (IOException e) {
                throw new RuntimeException(e);
            }
        })
        .open();

Our updated data flow now looks like this:

store.get() ->

Middleware - GsonSourceParser

There are also separate middleware libraries with parsers to help in cases where your fetcher is a Reader, BufferedSource or String and your parser is Gson:

• GsonReaderParser
• GsonSourceParser
• GsonStringParser

These can be accessed via a Factory class (GsonParserFactory).

Our example can now be rewritten as:

Store<Article, Integer> store = StoreBuilder.<Integer, BufferedSource, Article>parsedWithKey()
        .fetcher(articleId -> api.getArticle(articleId)) 
        .parser(GsonParserFactory.createSourceParser(gson, Article.class))
        .open();

In some cases you may need to parse a top level JSONArray, in which case you can provide a TypeToken.

Store<List<Article>, Integer> store = StoreBuilder.<Integer, BufferedSource, List<Article>>parsedWithKey()
        .fetcher(articleId -> api.getArticles()) 
        .parser(GsonParserFactory.createSourceParser(gson, new TypeToken<List<Article>>() {}))