Firestore / Firebase Typesense Search Extension ⚡ 🔍

A Firebase extension to sync data from your Firestore collection to Typesense, to be able to do full-text fuzzy search on your Firestore data, with typo tolerance, faceting, filtering, sorting, curation, synonyms, geosearch and more.

This extension listens to your specified Firestore collection and syncs Firestore documents to Typesense on creation, updates and deletes. It also provides a function to help you backfill data.

What is Typesense?

If you're new to Typesense, it is an open source search engine that is simple to use, run and scale, with clean APIs and documentation. Think of it as an open source alternative to Algolia and an easier-to-use, batteries-included alternative to ElasticSearch. Get a quick overview from this guide.

⚙️ Usage

Step 1️⃣ : Setup Prerequisites

Before installing this extension, make sure that you have:

1. Set up a Cloud Firestore database in your Firebase project.

   If using Google Workspace for Business, ensure that your default cloud compute based service account has the following roles (which can be found in the Google Cloud Console IAM section):

   • Artifact Registry Administrator
   • Artifact Registry Create-on-Push Writer
   • Artifact Registry Service Agent
   • Logs Writer
   • Storage Object Viewer

2. Set up a Typesense cluster on Typesense Cloud or Self-Hosted (free).

3. Set up a Typesense Collection either through the Typesense Cloud dashboard or through the API.

[!IMPORTANT] ☝️ #3 above is a commonly missed step. This extension does not create the Typesense Collection for you. Instead it syncs data to a Typesense collection you've already created. If you see an HTTP 404 in the extension logs, it's most likely because of missing this step.

Step 2️⃣ : Install the Extension

You can install this extension either through the Firebase Web console or through the Firebase CLI.

Firebase Console

Firebase CLI

firebase ext:install typesense/firestore-typesense-search --project=[your-project-id]

Learn more about installing extensions in the Firebase Extensions documentation: Console, CLI.

Syncing Multiple Firestore collections

[!TIP] You can install this extension multiple times in your Firebase project by clicking on the installation link above multiple times, and use a different Firestore collection path in each installation instance. Here is a screenshot of how this looks.

[!NOTE] This release also supports the legacy single-collection params (FIRESTORE_COLLECTION_PATH, TYPESENSE_COLLECTION_NAME, FIRESTORE_COLLECTION_FIELDS, and FLATTEN_NESTED_DOCUMENTS). They are deprecated and will log a warning when used. If both config shapes are fully set, the extension uses the multi-collection params and warns once. If only part of either shape is set, the install fails with a clear error. For new installs, prefer the multi-collection params (FIRESTORE_COLLECTION_PATHS, TYPESENSE_COLLECTION_NAMES, FIRESTORE_COLLECTION_FIELDS_LIST, and FLATTEN_NESTED_DOCUMENTS_LIST).

🎛️ Configuration Parameters

When you install this extension, you'll be able to configure the following parameters:

| Parameter | Description | |-------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Cloud Functions location | Where do you want to deploy the functions created for this extension? You usually want a location close to your database. For help selecting a location, refer to the location selection guide. | | Firestore Database region | Where the Firestore Database that holds the Firestore collection you want to sync into Typesense is located. Refer to the Cloud Firestore locations guide. | | Firestore Collection Path | The Firestore collection that needs to be indexed into Typesense. | | Firestore Collection Fields | A comma separated list of fields that need to be indexed from each Firestore document. Leave blank to index all fields. | | Typesense Hosts | A comma-separated list of Typesense Hosts (only domain without https or port number). For single node clusters, a single hostname is sufficient. For multi-node Highly Available or (Search Delivery Network) SDN Clusters, please be sure to mention all hostnames in a comma-separated list. | | Typesense API Key | A Typesense API key with admin permissions. Click on "Generate API Key" in cluster dashboard in Typesense Cloud. | | Typesense Collection Name | Typesense collection name to index data into (you need to create this collection in Typesense yourself. This extension does not create the Typesense Collection for you). | | Flatten Nested Documents | Should nested documents in Firestore be flattened before they are indexed in Typesense? Set to "Yes" for Typesense Server versions v0.23.1 and below, since indexing Nested objects is natively supported only in Typesense Server v0.24 and above. | | Log Typesense Inserts for Debugging | Should data inserted into Typesense be logged in Cloud Logging? This can be useful for debugging, but should not be enabled in production. |

⚠️ You'll notice that there is no way to configure the port number or protocol. This is because this extension only supports connecting to Typesense running HTTPS on Port 443, since your data goes from Firebase to Typesense over the public internet and we want your data to be encrypted in transit. For Typesense Cloud, HTTPS is already configured for you.

When self-hosting Typesense, you want to make sure you set --api-port=443 and also get an SSL certificate from say LetsEncrypt or any registrar and configure Typesense to use it using the --ssl-certificate and --ssl-certificate-key server parameters. Alternatively, if you're running Typesense on your local machine, you can also set up a local HTTPS tunnel using something like ngrok (ngrok http 8108) and use the ngrok hostname in the extension.

Example

If you have a Firestore database called (default) with a collection inside it called users in the nam5 region like this:

<img src="assets/firestore_db_example.png" alt="Firestore DB Example" width="800"/>

Here's the extension configuration screen with all the options filled out, if you want to sync the users Firestore collection to Typesense:

<img src="assets/extension_configuration_example.png" alt="Firestore DB Example" width="500" />

Step 3️⃣ : [Optional] Backfill existing data

This extension only syncs data that was created or changed in Firestore, after it was installed. In order to backfill data that already exists in your Firestore collection to your Typesense Collection:

• Create a new Firestore collection called typesense_sync through the Firestore UI.
• Create a new document with the ID backfill and contents of {trigger: true}
• [Optional] If you have multiple instances of the extension installed to sync multiple collections, you can specify which particular collections are backfilled by setting the contents of the backfill document in the previous step to `{trigger: true, firestore_collections: ["path/