We previously discussed methods and techniques for testing Cloud Firestore functions and security rules, and we’ll be expanding on that knowledge in later sections. Give that article a read first, then join us back here.

Overview

A huge tenant of any mobile application is displaying, filtering, and very likely full-text searching across large lists of data. Querying is relatively straightforward in Cloud Firestore, but full-text searching is not (which is another issue entirely). The cost of reading ever-increasing amounts of documents (even using paginated queries) can quickly add up with a traditional ‘query all documents in a collection’ approach that would be the go-to in most applications.

Given the above, our criteria are as follows:

  • Provide the minimum amount of data to the client that can be displayed, filtered, and searched through
  • Use as little Cloud Firestore quota as possible
  • Don’t generate documents that exceed storage capabilities
  • Require as little manual intervention as possible

We could consider using a third-party service such as Algolia, but that wouldn’t check all of our boxes and would come with additional fee structures. Plus it does nothing to help our Cloud Firestore quotas.

For this exercise we’re going to explore a technique that uses resources already available to us: client-side logic to manage the retrieved data and Cloud Firestore functions to generate the data.

Data structure

Expanding on the structure from the previous cloud functions article, we’re going to add the concept of a plant variety. The requirements for plant varieties are as follows:

  • There is a root collection of plant varieties available to all homesteads
  • Each homestead has a private collection of plant varieties
  • It should be easy to copy a plant variety between homesteads
  • Plant varieties should track their origin company or homestead

Multiple collection approach

One potential structure would be a root collection for plant varieties, and a subcollection existing on each homestead for the private plant varieties. Managing permissions for private plant varieties becomes easier, but targeting the correct index (more on that below) becomes more complicated, as does triggering cloud functions on the appropriate collection.

Single collection approach

A single, flattened, root collection requires extra data on each plant variety in order to associate it with a homestead. However, copying plant varieties becomes much easier since all of the documents are contained in the same collection.

The chosen approach

Honestly, either approach is viable, and will depend on your specific requirements. Very likely whether or not you need a shared root collection or only have private collections. For the needs outlined here one root collection for all documents ends up the slight favorite.

Here’s what our root plantVariety collection looks like:

plantVarieties/{plantVarietyId}

{
  homesteadId: string,
  commonName: string,
  variety: string,
  originType: 'company' | 'homestead'
  originId: string,
  originName: string,
  /* other fields that need to be stored */
}

#cloud #firebase #developer

Efficiently Manage Large Lists in Cloud Firestore
3.80 GEEK