Lazy-Loading and Querying Firestore with SvelteKit

Contents

• Introduction
• Firestore Security Rules
• Typescript Typing for Stores
• Store Subscriptions
• Querying for Orders by User
• Adding Query Parameters
• More Complex Scenarios

Introduction

If you followed the previous article about Lazy-Loading Firebase with SvelteKit you’ll now have a Svelte Store that contains your auth state. You can use it wherever you need access to the user in a Svelte-reactive way and the act of subscribing to it will trigger it to load, making it lazy for good Lighthouse and Core Web Virtals performance scores.

One common use for knowing the current user is to load the data that they should have access to, and we’re going to assume that data is coming from Firestore. In case you don’t know, there’s a lot to love about Firestore - it’s a serverless “Database as a Service” that lets you subscribe to receive live updates as data changes remotely and also has support for working offline, which can provide a great experience for users with spotty connectivity.

There’s a generous free-tier and if you use it correctly it can be inexpensive. Well, in the sense of $, it is more expensive in terms of JS bytes because it’s one of the larger modules in the Firebase Web SDK.

So we’re back to a similar issue with the auth - we want to use Firestore, but we don’t want to bundle all that JS directly into our app so it loads eagerly at startup. We want to be able to load our app and then load the Firebase code and our data lazily. In a future article, I’ll show how you can incorporate Server Side Rendering so we get the best of all worlds.

But for now, let’s focus on the lazy-loading on the client and for this we’re going to start with something similar to what we did with Auth, but use some other features of Svelte Stores to build up something that is fully queryable as well.

Let’s start with the basics of a Svelte store for our data. What is the minimum we want from it? Data … so it’s going to be something like this:

import { firestore } from './firestore' // initiated firestore instance
import { collection, query, orderBy, limit, onSnapshot } from 'firebase/firestore'
import { readable } from 'svelte/store'
import type { Order } from './models'

function createOrders() {
  const { subscribe } = readable([] as Order[], set => {
    let unsubscribe = () => {}

    if (browser) {
      let q = query(collection(firestore, 'orders'))
      q = query(q, orderBy('ordered', 'desc'))
      q = query(q, limit(10))

      unsubscribe = onSnapshot(q, snap => set(snap.docs.map(doc => ({ ...doc.data(), id: doc.id }))))
    }

    return unsubscribe
  })

  return { subscribe }
}

export const orders = createOrders()

This is requesting the last 10 orders from the orders collection but right away it’s giving us lazy-loading. So it will only load Firestore if we’re on a page that subscribes to this store, and it will return an empty array until the SDK has loaded and the data has been fetched. An added benefit is that if a new order is written or an order is updated while the user is on a page showing this data, it will auto-update with the new values.

As for the auth state, how you model the store contents matters to how you consume it - if you need to know the difference between an uninitialized store and an initialized store that happens to be empty (has no orders) then you might want to have the store initially contained undefined vs an empty array, so you can test for that and show a loading spinner in the UI, or maybe have the store contain a Promise instead. We’re keeping it simple for this example.

Firestore Security Rules

If we’re intending to use this store to provide data to a “My Account” page, showing a user’s order history, we’re missing something though - we’re not querying for just the current user’s orders!

NOTE: This is where Firestore Security Rules come in - if this worked, and returned any orders at all then you likely have something wrong! The security rules should be setup to protect your data and limit queries of orders to those owned by the user. We’re going to assume those rules are in place and focus on how to handle querying with that in mind.

So we need to incorporate our auth state into the query, how can we do that? This is a perfect use-case for a Svelte Derived store. A derived store can take other stores as input and then output its own state based on what they contain. Because our store needs to know the auth state, we’ll pass that in to a derived store instead. These are the basic types and structure:

import { derived } from 'svelte/store'
import { auth } from './auth'
import type { Order } from './models'

function createOrders() {
  const { subscribe } = derived(auth, ($auth, set) => {
    // implementation here
  }, [] as Order[])

  return { subscribe }
}

export const orders = createOrders()

Typescript Typing for Stores

As an aside, there are two ways to approach using Typescript with Svelte Stores. You can defined explicit types or you can allow typescript to handle implicit typing for you by infering the types. The code above uses implicit typing, and relies on Typescript being able to know the type of each store which is why we initialize the state using [] as Order[]. If we didn’t do that, it wouldn’t know what our store contained and we’d then need to use explicit typing instead.

Here is the same example using explicit typing, where we define the types on the derived function for the dependent stores and the returned store itself. It’s quite verbose because we now have to be importing types for things we don’t really depend on directly (like the firebase User) and I think is the kind of thing that most people who object to Typescipt as being “more work” are really refering to.

import type { Readable } from 'svelte/store'
import type { User } from 'firebase/auth'
import { auth } from './auth'

function createOrders(auth: Readable<User>) {
  const { subscribe } = derived<Readable<User>, Order[]>(auth, ($auth, set) => {
    // implement here
  }, [])
}

We’re going to stick with the implicit approach, and let vscode do the work for us.

Store Subscriptions

The other thing to note is that our function is passed the value of the auth store that our derived store depends on. As a convention, I tend to use the $ prefix just to differentiate the value in the store from the store itself. Although there is no ‘auto subscription’ going on such as happens using $auth in a Svelte Component, our derived store will only subscribe to the auth store if something subscribes to it, so the concept is similar.

This also means we’re safe to import the auth store and won’t trigger loading the Firebase Auth module unless our store is used. Another reason that it’s the place to put the lazy-loading mechanism rather than at the UI component level.

Querying for Orders by User

Now we have our user auth status, we can include it in the query and of course avoid querying anything if they are not signed in:

import { firestore } from './firestore'
import { collection, query, orderBy, limit, onSnapshot } from 'firebase/firestore'
import { derived } from 'svelte/store'
import { auth } from './auth'
import type { Order } from './models'

function createOrders() {
  const { subscribe } = derived(auth, ($auth, set) => {
    let unsubscribe = () => {}

    if (browser) {
      // auth is a user object if the user is authenticated
      if ($auth) {
        let q = query(collection(firestore, 'orders'))
        q = query(q, where('user', '==', $auth.uid))
        q = query(q, orderBy('ordered', 'desc'))

        unsubscribe = onSnapshot(q, snap => set(snap.docs.map(doc => ({ ...doc.data(), id: doc.id }))))
      } else {
        // whether the user auth hasn't been resolved yet or is signed out, we don't need to subscribe
        // to any orders, and we can set the state to empty
        set([])
      }
    }

    return unsubscribe
  }, [] as Order[])

  return { subscribe }
}

export const orders = createOrders()

Great, we now have an orders store that we can use on a “My Account” page (which would probably be protected by an auth check and sign-in). It doesn’t matter what order the Firebase SDK modules load (and they only load if they user views a page that needs them), once they are loaded and the auth status is resolved we’ll be subscribed to the orders. And anytime the store is no longer subscribed (including if the auth state changes) any previous subscription will automatically be cleaned up for us because of the unsubscribe return function (this is called whenever the derived store parameters change or the last subscriber disconnects).

But oh dear, queries like this can be dangerous. If you have an open-ended query with heavily accessed pages you might be loading a lot more data than you need or realize. This is where the “horror stories” where people rack up a large expensive bill with Firebase originate. And realistically, users rarely want all data (imagine a heavy Amazon users always having to wait to see everything they have ever purchased to load?). In this example most people would usually be interested in the latest orders only so adding a limit back is an easy first fix, but why not allow them to also paginate through the data? For this we need to control the query that we use to subscribe to Firestore, but that query is buried inside our store …

Adding Query Parameters

Think about it, we already depend on the auth store for one parameter of the query, why not make the pagination and any other filtering parameters we need another dependency too?

interface OrderFilter {
  start: any;
  limit: number;
}

function createOrderFilter() {
  const INITIAL: OrderFilter = { start: null, limit: 10 }
  const STATE: OrderFilter = { ...INITIAL }

  const { subscribe, set } = writable(STATE)

  function update(part: Partial<OrderFilter>) {
    Object.assign(STATE, part)
    set(STATE)
  }

  return {
    subscribe,
    reset: () => set(INITIAL),
    first: () => update({ start: null }),
    next: (start) => update({ start }),
    size: (limit) => update({ limit, start: null }),
  }
}

export const orderFilter = createOrderFilter()

Although we could get away with a simple writable store that the app can update, this tends to spread the logic of querying for data into Components. I like to have everything encapsulated in the store and in one place and this provides an easier API to use when binding components.

However we do it, we end up with a store that contains the filter state, and we can pass that store as an additional dependency to our data fetching store:

import { firestore } from './firestore'
import { collection, query, orderBy, limit, onSnapshot } from 'firebase/firestore'
import { derived } from 'svelte/store'
import { auth } from './auth'
import type { Order } from './models'

// ... filter store defined above ...

function createOrders() {
  const { subscribe } = derived([auth, filter], ([$auth, $filter], set) => {
    let unsubscribe = () => {}

    if (browser) {
      if ($auth) {
        let q = query(collection(firestore, 'orders'))
        q = query(q, where('user', '==', $auth.uid))
        q = query(q, orderBy('ordered', 'desc'))
        q = query(q, startAfter('ordered', $limit.start))
        q = query(q, limit($filter.limit))

        unsubscribe = onSnapshot(q, snap => set(snap.docs.map(doc => ({ ...doc.data(), id: doc.id }))))
      } else {
        set([])
      }
    }

    return unsubscribe
  }, [] as Order[])

  return { subscribe }
}

export const orders = createOrders()

Note that the dependencies to the derived store are now defined in an array. This is the point where having the explicit type definitions becomes quite painful to work with, which is another to reason to use implicit type inference.

Also, although we have them as two separate stores, we could combine them into a single custom stores that have the writable and derived stores internally. That would allow a single store to contain the data and also have the methods on for navigating (by manipulating the internal query store).

More Complex Scenarios

The filtering in this example is simple, it would allow for simple “Next” pagination, which in truth maybe all that is needed for a view of orders. But the approach works for more complex queries there different sort fields can be selected, page sizes changed and full First / Previous / Next / Last navigation is provided (tip: you’ll be looking at using Firestore’s startAfter and endBefore to support navigating in different navigations).

If you are providing multiple sort options, it can be simpler to always pass the actual Firestore document to start at rather than the specific sort field values. If doing this you can consider splitting this derived store into one that returns the raw Firestore documents, which makes this easier. Then use another derived store dependent on that to handle converting the docs into the Model used by your app. The query store would then be passed the first or last value from the raw document store as part of the custom methods. Again, the entire thing can be wrapped up in a single custom store which only exposes the transformed Order array and the custom navigation methods.