EVM Portal stream

@subsquid/evm-stream is the EVM data source for Squid SDK: a DataSourceBuilder that streams pre-filtered data from a SQD Network Portal dataset, including real-time unfinalized blocks. Pass the built source to run() or Processor, or consume its stream directly.

Portal is public — no API key is needed. Migrating an older gateway-and-RPC squid? Follow Migrate a squid to Portal.

Install

npm i @subsquid/evm-stream @subsquid/evm-objects @subsquid/batch-processor @subsquid/logger @subsquid/typeorm-store

Complete example

import {run} from '@subsquid/batch-processor'
import {DataSourceBuilder, type FieldSelection} from '@subsquid/evm-stream'
import {augmentBlock} from '@subsquid/evm-objects'
import {createLogger} from '@subsquid/logger'
import {TypeormDatabase} from '@subsquid/typeorm-store'

const USDC = '0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48'
const TRANSFER = '0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef'

const fields = {
  block: {timestamp: true},
  log: {
    address: true,
    topics: true,
    data: true,
    transactionHash: true,
  },
} satisfies FieldSelection

const dataSource = new DataSourceBuilder()
  .setPortal('https://portal.sqd.dev/datasets/ethereum-mainnet')
  .setFields(fields)
  .setBlockRange({from: 18_000_000})
  .addLog({
    where: {address: [USDC], topic0: [TRANSFER]},
  })
  .build()

const log = createLogger('sqd:processor:mapping')
const db = new TypeormDatabase({supportHotBlocks: true})

run(dataSource, db, async (ctx) => {
  const blocks = ctx.blocks.map(augmentBlock)
  log.info({blocks: blocks.length}, 'received batch')
  // Decode and persist data here with ctx.store.
})

The handler context and the block data shape are described in Context interfaces.

`DataSourceBuilder`

All builder methods return the same builder unless noted.

`setPortal(portal)`

Required. Accepts a Portal URL, PortalClientOptions, or an existing PortalClient.

.setPortal('https://portal.sqd.dev/datasets/ethereum-mainnet')

`setBlockRange({from, to?})`

Applies a global range to all requests. EVM ranges use block heights.

`setFields(fields)`

Selects the fetched fields and narrows the inferred block type. There are no default optional fields — list every field the handler reads. See Field selection for the full per-item field lists.

Data requests

Each request method takes {where, include, range} — item filters, related items to fetch alongside, and an optional block range overriding the global one:

addLog(options) — event logs, filtered by contract address and topics
addTransaction(options) — transactions, filtered by from/to/sighash/type
addTrace(options) — execution traces (calls, creates, suicides, rewards)
addStateDiff(options) — storage, balance, code, and nonce changes

`includeAllBlocks(range?)`

Includes blocks without matching items. Without it, Portal may omit empty blocks.

`addQuery(query)`

Adds a Query or QueryBuilder when several filters should share one range:

import {QueryBuilder} from '@subsquid/evm-stream'

source.addQuery(
  new QueryBuilder()
    .setRange({from: 18_000_000})
    .includeAllBlocks()
    .addLog({where: {address: [USDC]}})
    .addTransaction({where: {to: [USDC]}, include: {logs: true}})
)

QueryBuilder also has addTrace(), addStateDiff(), and build().

`build()`

Returns an EVMDataSource. It throws Portal settings not set if setPortal() was not called. Pass the result to run() or Processor for restart-safe processing. You can also call getHead(), getFinalizedHead(), getStream(), getFinalizedStream(), and the optional getBlocksCountInRange() helper directly. See Read a data source directly.

Real-time and unfinalized blocks

The Portal stream serves blocks all the way to the chain head, including unfinalized ones. Run the source with a hot-block-aware database (new TypeormDatabase({supportHotBlocks: true})) and run() consumes the fork-aware stream: on a chain reorganization the store rolls back the affected changes automatically. See Indexing unfinalized blocks. For endpoint-outage resilience, wrap the source in an EVM fallback.

Raw and augmented block types

The data source returns raw blocks from @subsquid/evm-stream. augmentBlock() from @subsquid/evm-objects adds IDs and navigation helpers such as log.getTransaction(), trace.getParent(), and transaction logs, traces, and stateDiffs collections — see Context interfaces. Use header.number for the block height. header.height remains available only as a deprecated alias.

Overview

Pipes SDK

Squid SDK

Alternative clients

Install

Complete example

`DataSourceBuilder`

`setPortal(portal)`

`setBlockRange({from, to?})`

`setFields(fields)`

Data requests

`includeAllBlocks(range?)`

`addQuery(query)`

`build()`

Real-time and unfinalized blocks

Raw and augmented block types

​Install

​Complete example

​DataSourceBuilder

​setPortal(portal)

​setBlockRange({from, to?})

​setFields(fields)

​Data requests

​includeAllBlocks(range?)

​addQuery(query)

​build()

​Real-time and unfinalized blocks

​Raw and augmented block types

Install

Complete example

`DataSourceBuilder`

`setPortal(portal)`

`setBlockRange({from, to?})`

`setFields(fields)`

Data requests

`includeAllBlocks(range?)`

`addQuery(query)`

`build()`

Real-time and unfinalized blocks

Raw and augmented block types