Slow Reader

Web app to combine feeds from social networks and RSS and to help read more meaningful and deep content.

Right now, it is just a prototype. We plan to have features:

Combine all feeds (social media, RSS) in a single app.
Track how each subscription is useful for you.
Split subscriptions to slow (something useful, deep) and fast-food (fun and small).
Spend more time on slow content by blocking fast in the evening, etc.

Pre-alpha prototype: dev.slowreader.app

↬ How to contribute and join the team

To ask any question: h+h lab Discord

License
Principles
Project Structure
Tools
Scripts
Synchronization Protocol
Client Storage
Test Strategy
Visual Language
Dependencies
- JS
- GitHub Actions
- Update
Deploy
Guides

License

Slow Reader is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License (version 3 or any later). For details, see the LICENSE.md file in this directory.

Principles

Local-First: Clients Do All the Job

Local-first means the client stores all data locally and does most of the job. Even if we decide to close the cloud, you can still use the Slow Reader client.

In our case, it means that:

Client stores all feeds and posts. You can read posts offline.
Client checks feeds for new posts.
You need the cloud to sync data between clients.

Local-first manifest tells more philosophy behind that idea.

Zero-Knowledge Synchronization

Clients use end-to-end encryption during cloud sync. It means the cloud can’t know what feeds you are reading or what posts you like.

You can check encryption source code.

We have a proxy to make HTTP requests to other servers from the website. But this proxy is only for development, bypassing government censorship, or the initial test of the app. The web client should mostly use an upcoming web extension to bypass the CORS limit. Upcoming native clients will use direct HTTP requests since they don’t have a CORS limitation.

Event-Sourcing and CRDT

The source of truth in the client is a list of changes (action log). An action is a JSON object like:

{
  "type": "feeds/changed",
  "id": "kc4VfXpvw3vZZBu_ugGlC",
  "fields": {
    "reading": "slow"
  }
}

To render the UI, the client reduces actions from the log into the state (objects of feeds, posts, etc.). We store the state cache in Nano Stores.

The log simplifies synchronization. We just need to track the last synchronized action and send all actions after that one.

We use Logux to work with log and synchronization.

Client Core: Reusing All Logic Between Different Platforms

For now, we have only a web client. But we want to have native clients for different platforms.

To make client porting easier, we separate core logic and UI. Core logic is the same for every client. The client just needs to bind this logic to the UI using native components.

We write logic in TypeScript as smart stores in the Nano Stores state manager. The client needs to subscribe to stores and render UI according to the store.

We try to move to the store as much as possible: app routing, validations, and UI helpers. The client should be as thin as possible. The ideal client is just a UI renderer.

Core depends on the platform environment (like storage to store settings). Before using any store, the client must call setEnvironment() to define how the core should interact with the platform.

Project Structure

Slow Reader is a local-first app. Clients do most of the work, and the server just syncs data between users’ devices (with end-to-end encryption).

Clients:
- core/: client’s logic and i18n translations. Clients for specific platforms is just a UI around this core to simplify porting
  - See core/README.md for core architecture.
- web/: the client to be run in the browser. Both for desktop and mobile.
  - See web/README.md for web client architecture.
server/: a small server that syncs data between users’ devices.
proxy/: HTTP proxy server to bypass censorship or to try web clients before they install the upcoming extensions (to bypass the CORS limit of the web apps).
extension/: browser’s extension to avoid CORS limits in web client.
api/: types and constants shared between clients and server.
docs/: guides for developers.
scripts/: scripts to test project and configure Google Cloud. Check the script’s descriptions for further details.
loader-tests/: integration tests for each social network or news format.
.devcontainer/: Dockerfile and configs to run project in Docker/Podman image on developer’s machine. It increases security (malicious dependency will not have access to the whole machine) and simplify onboarding. We have configs for Docker and Podman (more secure version of Docker).
.github/: scripts to test projects on CI.
.husky/: scripts to call on git commit command to avoid popular errors.
.vscode/: VS Code settings to reduce code format errors for new contributors.

We are using pnpm monorepo. Each project has its dependencies, tools, and configs. Read README.md in each project for project’s files and architecture.

Tools

Global development tools:

Dev Container to use the same environment for all developers and isolate project from developer’s machine.
Prettier to use the same code style formatting.
TypeScript for strict type checking.
ESLint to check for popular mistakes in JavaScript.
remark to find mistakes in .md files.
Multiocular to review dependencies updates.

Each project has its own tools, too.

Scripts

pnpm test: run all tests.
pnpm quick: run quick checks for changed files.
pnpm offline: run all tests expect which need Internet. It is useful for developing in airplane/train.
pnpm start: run proxy and web client development server.
pnpm format: fix code style in all files.
pnpm clean: remove all temporary files.
pnpm check-opml: test loaders with user’s OPML RSS export.
pnpm test-loaders: test loaders with different blogging platforms.
pnpm unused-messages: check that all messages are used.
pnpm update-env: check for Node.js and pnpm updates.
pnpm update-review: run Multiocular to review updates.
pnpm update-ci: update CI actions.
pnpm update-browsers: update target browsers of web client.

We use pnpm feature to run scripts in parallel, having scripts like test:types and test:audit. Then, we run all scripts in all projects by test:* prefix.

Synchronization Protocol

We use Logux WebSocket protocol to synchronize actions between clients and server.

Clients keep a list of changes (action log) as the source of truth and then send new actions to the server. The server then sends all new actions to other clients.

The server doesn’t see those actions because clients encrypt them before sending and decrypt them upon receiving. The server sees only actions like:

// Add encrypted action to the server log
{
  "type": "0",
  "d": "encrypted data",
  "iv": "random vector used together with password in encryption"
}

// Remove action from the server log
{
  "type": "0/clean",
  "id": "action ID"
}

Client Storage

The clients store a list of changes (action log). Durin console.log(client1.log.entries())g the start, the client reduces all necessary actions from the log to the Logux SyncMap stores.

For simple things like client local settings, we use Nano Store Persistent.

The web client uses IndexedDB to store log and localStorage for the client’s settings.

Test Strategy

If any mistake happens a few times, we should add an automatic tool to prevent mistakes in the future. Possible strategies:

Types.
Scripts, custom ESLint or Stylelint plugins.
Tests.
Pull request checklist.

Any code-style rule should be implemented as a pre-commit hook or linter’s rule.

Types should try to use precise types and explain data relations with them:

- { type: string, error?: Error }
+ { type: 'ok' } | { type: 'error', error: Error }

We are using integration tests for client core. We mock network requests and the platform environment, but emulate user interaction and test the composition of all stores. As the result it is a little closer to integration tests rather than unit tests (by using high level APIs of pages

Slowreader

Install / Use

README