Store interface

Store is a generic interface exposed by DataHandlerContext at .store. It is used in the batch handler to persist data. Its concrete type is inferred from the Database argument of the run() processor method:

run<Store>(
  db: Database<Store>,
  handler: (ctx: DataHandlerContext<Store, F extends FieldSelection>)
    => Promise<void>
): void

The Database implementation supplied here defines the store and the way the processor persists its status. Squid SDK supports Database implementations for:

Support for more databases and analytics storage will be added in the future.

Custom `Database`

A custom implementation of the Database interface is the recommended solution for squids with multiple data sinks and/or for non-transactional or non-relational databases. In such a case, the inferred Store facade exposed to the handlers may provide multiple targets for persisting the data. Database.transact should handle potential edge-cases when writes to either of the data sinks fail. In order to implement a custom data sink adapter, it suffices to implement the Database interface:

export type Database<S> = FinalDatabase<S> | HotDatabase<S>

export interface FinalDatabase<S> {
    supportsHotBlocks?: false
    connect(): Promise<HashAndHeight>
    transact(info: FinalTxInfo, cb: (store: S) => Promise<void>): Promise<void>
}

export interface HotDatabase<S> {
    supportsHotBlocks: true
    connect(): Promise<HotDatabaseState>
    transact(info: FinalTxInfo, cb: (store: S) => Promise<void>): Promise<void>

    /** @deprecated Implement transactHot2 when possible. */
    transactHot(info: HotTxInfo, cb: (store: S, block: HashAndHeight) => Promise<void>): Promise<void>

    transactHot2?(
        info: HotTxInfo,
        cb: (store: S, blockSliceStart: number, blockSliceEnd: number) => Promise<void>
    ): Promise<void>
}

HotTxInfo describes a fork-aware update:

export interface HotTxInfo {
    finalizedHead: HashAndHeight
    baseHead: HashAndHeight
    newBlocks: HashAndHeight[]
}

transactHot2() lets the database group several consecutive blocks into one store transaction. Each callback receives a half-open [blockSliceStart, blockSliceEnd) range of indexes into info.newBlocks. Calling cb(store, 0, info.newBlocks.length) processes the entire update as one slice. An adapter may call the callback more than once when it needs smaller transactions. The processor maps the same slice of fetched blocks into the handler. The database implementation is responsible for rolling back to baseHead, invoking the callback for the blocks it commits, and persisting the resulting finalized and unfinalized heads atomically where the storage system supports transactions.

The interface above is the compatibility surface used by chain-specific processors such as @subsquid/evm-processor: deprecated transactHot() remains required and transactHot2() is optional. The Portal-native @subsquid/batch-processor interface requires transactHot2() and does not include the deprecated fallback.

Consult this file for details. The interface only defines how the processor advances with the indexing and connects to the data sink. The following is left as implementation details:

persisting the indexing status
opening and rolling back the transaction (if applicable)

Consult these implementations for details:

(Transactional) Postgres-based TypeormDatabase
(Non-transactional) File-based database

Overview

Pipes SDK

Squid SDK

Alternative clients

Custom `Database`

​Custom Database

Custom `Database`