Storage Adapters

Storage adapters in SignalDB provide the mechanism for storing and retrieving data, ensuring that your data is kept safe across sessions and reloads of your application.

These adapters interact with the underlying storage medium, such as localStorage, IndexedDB, or even a remote server, and handle the specifics of those storage systems while providing a consistent interface for data operations in your application.

Storage adapters are responsible for transforming the high-level operations you perform on your data (such as saving a document or loading a collection) into the low-level operations that the specific storage system can understand and perform.

The main benefit of using storage adapters is the abstraction they provide. They allow SignalDB to remain agnostic to the underlying storage system. This means that you can switch between different systems without changing the rest of your code.

Not sure which browser storage to use? See localStorage vs IndexedDB vs OPFS for a comparison.

The following storage adapters are currently available:

• IndexedDB
• localStorage
• OPFS
• FileSystem
• Generic file system — the foundation behind the FileSystem and OPFS adapters, for building your own file-based storage

A storage adapter is reached through a data adapter, which is what decides where the data operations run.

Building your own storage adapter for your specific use case is also possible and pretty straightforward. See createStorageAdapter for more information.

Why query semantics stay in SignalDB

A storage adapter has to answer three questions: all items, items by id, and the contents of one index. That is deliberate, and it is the reason the required interface is as small as it is:

readAll(): Promise<T[]>
readIds(ids: I[]): Promise<T[]>
readIndex(field: string): Promise<Map<string | null, Set<I>>>

An index is keyed by serializeValue(value), not by the raw field value — that is what SignalDB looks it up with. An adapter that stores raw keys answers nothing for every non-string field and everything for a $ne on one.

Query semantics live in SignalDB, in one place. A storage backend only has to be able to store things and find them again — it never has to understand $in, $gt, $regex, or how they combine under $and and $or.

That keeps the set of possible backends wide open. Anything that can persist bytes and look something up can be a storage adapter: a file system that maps each index onto a directory, an archive file, a key-value store, a remote endpoint that only offers a fetch-by-key. None of them are excluded for lacking a query language, and none of them can quietly disagree with another about what a selector means. One implementation of the semantics means one place to fix a bug in them and one suite to test them.

Letting adapters answer whole queries instead would put the selector semantics into every adapter that opts in, make an adapter's capabilities unpredictable from the outside, and turn "which adapter am I using" into something that changes results rather than just performance.

Narrowing a read with query

A backend that can read less than everything may implement the optional query method. It receives the whole query — selector, sort, window and projection — and answers as much of it as it can, saying what it did and handing back the part of the selector it did not apply. SignalDB applies everything the adapter declined, so the result is the same with or without query; only the cost differs. @signaldb/indexeddb uses it to turn an equality on the primary key or an indexed field into a key-range read instead of reading the whole store.

query is consulted by the data adapters that read from storage on every query — AsyncDataAdapter, AutoFetchDataAdapter and the WorkerDataAdapterHost. The DefaultDataAdapter loads everything into memory once and does not use it.

What this means for your collections

The cost of this design is that SignalDB can only narrow a query down to the indices you declared. A selector that no index covers is answered by reading the whole collection out of storage and filtering it in JavaScript. That is fine for small collections and for one-off reads, and it is the wrong thing for a collection that grows.

So for anything that keeps growing, declare the fields you actually select on as indices when you create the collection:

import { AsyncDataAdapter, Collection } from '@signaldb/core'
import createFilesystemAdapter from '@signaldb/fs'

const dataAdapter = new AsyncDataAdapter({
  storage: name => createFilesystemAdapter(`./data/${name}`),
})

const posts = new Collection('posts', dataAdapter, {
  indices: ['authorId', 'status'],
})

Selecting a single item by id needs no index. It is resolved directly through the adapter's readIds — every storage can look an item up by its id, which is why that is part of the interface rather than something you have to declare.