Migrate SDK squids to Portal

← Back to Portal Setup

Return to the Portal setup overview to explore other deployment options.

This guide walks you through migrating EVM indexer setups that use RPC for real-time data ingestion to ingesting real-time data from SQD Network Portals. The guide works regardless of whether you have already migrated to using a portal for historical data.

Prerequisites

Before starting, ensure you have:

An existing EVM indexer using @subsquid/evm-processor
Node.js and npm installed
Access to an SQD Portal

Migration Steps

Install New Packages

Replace the old processor package with the new packages:

npm uninstall @subsquid/evm-processor
npm i @subsquid/evm-stream @subsquid/evm-objects @subsquid/batch-processor @subsquid/logger

Verify Portal Support

Make sure you have an SQD portal URL for your dataset and that real-time data is supported.Public Portal URLs follows this pattern:

https://portal.sqd.dev/datasets/<dataset-slug>

where <dataset-slug> is found on the networks page.

Update Processor Configuration

Replace your processor configuration (typically at src/processor.ts or src/main.ts) with a data source configuration.

Update Imports

Replace the processor imports with the new data source and object imports:

-import {EvmBatchProcessor} from '@subsquid/evm-processor'
+import {DataSourceBuilder} from '@subsquid/evm-stream'
+import {augmentBlock} from '@subsquid/evm-objects'
+import {createLogger} from '@subsquid/logger'

If your project re-exports data types (typically from src/processor.ts), update those imports too. The data types are now split across two packages:

@subsquid/evm-stream exports FieldSelection.
@subsquid/evm-objects exports the augmented block data types: BlockHeader, Block (the parent of transactions/logs/etc., previously called BlockData), Log, Transaction, Trace, StateDiff.

-import {
-  BlockHeader,
-  DataHandlerContext,
-  EvmBatchProcessor,
-  Log as _Log,
-  Transaction as _Transaction,
-  BlockData as _BlockData,
-  FieldSelection,
-} from '@subsquid/evm-processor'
+import * as evmObjects from '@subsquid/evm-objects'
+import {DataSourceBuilder, FieldSelection} from '@subsquid/evm-stream'
+import type {DataHandlerContext as BaseDataHandlerContext} from '@subsquid/batch-processor'
+import type {Logger} from '@subsquid/logger'

Replace Processor Initialization

Replace EvmBatchProcessor initialization with DataSourceBuilder:

-const processor = new EvmBatchProcessor()
-  .setGateway('https://v2.archive.subsquid.io/network/ethereum-mainnet')
-  .setRpcEndpoint('https://rpc.ankr.com/eth')
-  .setFinalityConfirmation(75)
+const dataSource = new DataSourceBuilder()
+  .setPortal('https://portal.sqd.dev/datasets/ethereum-mainnet')

RPC endpoint and finality confirmation settings are no longer needed.

Update Type Exports

If src/processor.ts re-exports Fields/Block/BlockData/Log/Transaction and a per-project context alias (often called ProcessorContext), rewrite them in terms of the new packages. There are three changes to be aware of:

BlockData<F> is renamed to Block<F> in @subsquid/evm-objects. The single-block type that used to be called Block (just the header) is now BlockHeader<F>.
The upstream DataHandlerContext lives in @subsquid/batch-processor. Its generic arguments are flipped — it is DataHandlerContext<Block, Store>, not DataHandlerContext<Store, Fields>. It also has only {store, blocks, isHead} — no log and no _chain. Add those yourself when defining your project’s enriched context type. With the original DataHandlerContext no longer in your imports, this is also a natural time to rename your project alias to DataHandlerContext — the name fits the role better than ProcessorContext now that there is no processor object.
The Fields type alias should be derived from the same fields constant you pass to .setFields().

-const fields = {
-  log: {
-    transactionHash: true,
-  },
-} satisfies FieldSelection
-
-export type Fields = typeof fields
-export type Block = BlockHeader<Fields>
-export type BlockData = _BlockData<Fields>
-export type Log = _Log<Fields>
-export type Transaction = _Transaction<Fields>
-export type ProcessorContext<Store> = DataHandlerContext<Store, Fields>
+const fields = {
+  log: {
+    transactionHash: true,
+  },
+} satisfies FieldSelection
+
+export type Fields = typeof fields
+export type Block = evmObjects.BlockHeader<Fields>
+export type BlockData = evmObjects.Block<Fields>
+export type Log = evmObjects.Log<Fields>
+export type Transaction = evmObjects.Transaction<Fields>
+export type DataHandlerContext<Store> = BaseDataHandlerContext<BlockData, Store> & {
+  log: Logger
+  // include `_chain` here as well if you make direct RPC calls — see the optional step below
+}

Expand the Field Selection

Every field your handler reads must be listed in .setFields(). With EvmBatchProcessor, common fields like log.address, log.topics, log.data, block.timestamp and transaction.from/to/hash were merged in from a built-in default set, so partial selections worked even when the handler accessed unlisted fields. In DataSourceBuilder only the fields you list are fetched from the portal and, at the type level, Log<F> / BlockHeader<F> / Transaction<F> expose only those fields. TypeScript will reject log.address, log.topics, log.data or block.header.timestamp accesses if you forgot to request them.

If your previous fields constant relied on the merged defaults, expand it to enumerate every field your handler actually reads. A minimum set typical for an EVM logs indexer looks like:

const fields = {
  block: {
    timestamp: true,
  },
  log: {
    address: true,
    topics: true,
    data: true,
    transactionHash: true, // and anything else you were already selecting
  },
} satisfies FieldSelection

Add transaction/trace/state-diff entries as needed — TypeScript errors at compile time will point you at any field still missing from the selection.

Update Data Requests

Rewrite data requests using the new where-include-range syntax:

// Old flat object syntax
.addLog({
  address: [USDC_CONTRACT_ADDRESS],
  topic0: [usdcAbi.events.Transfer.topic],
  transaction: true, // include the parent transaction
  range: { from: 6_082_465 },
})

Build the Data Source

Include a .build() call at the end of the data source initialization:

.setFields({
  log: {
    transactionHash: true,
  },
})
+.build()

If you pass your processor object between source files (e.g., from src/processor.ts to src/main.ts), pass the dataSource object in the same way.

Update the Run Function

Replace the processor.run() call with the unified run function.

Import the Run Function

Add the run function import to your main file:

+import {run} from '@subsquid/batch-processor'

Create a Logger

Manually create a logger for your batch handler:

const logger = createLogger('sqd:processor:mapping')

Replace processor.run()

Update the run call and enrich the context:

-processor.run(db, async (ctx) => {
+run(dataSource, db, async (simpleCtx) => {
+  const ctx = {
+    ...simpleCtx,
+    blocks: simpleCtx.blocks.map(augmentBlock),
+    log: logger
+  }

View Complete Diff Example

Here’s a full example of the changes up to this point:

--- a/src/main.ts
+++ b/src/main.ts
@@ -1,29 +1,43 @@
-import {EvmBatchProcessor} from '@subsquid/evm-processor'
+import {DataSourceBuilder} from '@subsquid/evm-stream'
+import {augmentBlock} from '@subsquid/evm-objects'
+import {run} from '@subsquid/batch-processor'
+import {createLogger} from '@subsquid/logger'
 import {TypeormDatabase} from '@subsquid/typeorm-store'
 import * as usdcAbi from './abi/usdc'
 import {UsdcTransfer} from './model'
 
 const USDC_CONTRACT_ADDRESS = '0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48'
 
-const processor = new EvmBatchProcessor()
-  .setGateway('https://v2.archive.subsquid.io/network/ethereum-mainnet')
-  .setRpcEndpoint('https://rpc.ankr.com/eth')
-  .setFinalityConfirmation(75)
+const dataSource = new DataSourceBuilder()
+  .setPortal('https://portal.sqd.dev/datasets/ethereum-mainnet')
   .addLog({
+    where: {
+      address: [USDC_CONTRACT_ADDRESS],
+      topic0: [usdcAbi.events.Transfer.topic],
+    },
+    include: {
+      transaction: true, // include the parent transaction
+    },
     range: { from: 6_082_465 },
-    address: [USDC_CONTRACT_ADDRESS],
-    topic0: [usdcAbi.events.Transfer.topic],
-    transaction: true, // include the parent transaction
   })
   .setFields({
     log: {
       transactionHash: true,
     },
   })
+  .build()
 
 const db = new TypeormDatabase({supportHotBlocks: true})
 
-processor.run(db, async (ctx) => {
+const logger = createLogger('sqd:processor:mapping')
+
+run(dataSource, db, async (simpleCtx) => {
+  const ctx = {
+    ...simpleCtx,
+    blocks: simpleCtx.blocks.map(augmentBlock),
+    log: logger,
+  }
+
   const transfers: UsdcTransfer[] = []
   for (let block of ctx.blocks) {
     for (let log of block.logs) {

Add RPC Client (Optional)

Only complete this step if you use direct RPC calls in your batch handler code. If you don’t make direct contract state queries, skip this step.

If you use direct RPC calls in your batch handler, you’ll need to add an RPC client to your context.

Install RPC Client

npm i @subsquid/rpc-client

Initialize RPC Client

Import and initialize an RpcClient:

import {RpcClient} from '@subsquid/rpc-client'

const rpcClient = new RpcClient({
  url: 'https://my_rpc_url',
  rateLimit: 100
})

Enrich Context

In your batch handler, add the _chain field to the context:

  const ctx = {
    ...simpleCtx,
    blocks: simpleCtx.blocks.map(augmentBlock),
    log: logger,
+    _chain: {
+      client: rpcClient
+    }
  }

If you keep a DataHandlerContext<Store> alias in src/processor.ts, extend it so the _chain field is part of the type your handlers see:

 export type DataHandlerContext<Store> = BaseDataHandlerContext<BlockData, Store> & {
   log: Logger
+  _chain: { client: RpcClient }
 }

import type {RpcClient} from '@subsquid/rpc-client'

Now your contract state queries will work as before:

const usdcContract = new usdcAbi.Contract(
  ctx,
  blocks[0].header,
  USDC_CONTRACT_ADDRESS
)
// Query decimals via direct call to state at blocks[0].header.height
const decimals = await usdcContract.decimals()

Migration Complete

Your indexer is now ready to source real-time data from an SQD Network portal. The portal provides improved performance and reliability compared to RPC endpoints.

Example Migrations

Complete migration examples for a simple USDC transfers indexer are available:

Without RPC Client

Basic migration example without RPC client integration

With RPC Client

Migration example including RPC client for state queries

Next Steps

Portal API Reference

Explore the complete Portal API documentation

EVM Examples

See more examples of Portal API usage

Overview

Pipes SDK

Squid SDK

← Back to Portal Setup

Prerequisites

Migration Steps

Update Imports

Replace Processor Initialization

Update Type Exports

Expand the Field Selection

Update Data Requests

Build the Data Source

Import the Run Function

Create a Logger

Replace processor.run()

Install RPC Client

Initialize RPC Client

Enrich Context

Migration Complete

Example Migrations

Without RPC Client

With RPC Client

Next Steps

Portal API Reference

EVM Examples

Overview

Pipes SDK

Squid SDK

Documentation Index

← Back to Portal Setup

​Prerequisites

​Migration Steps

​Update Imports

​Replace Processor Initialization

​Update Type Exports

​Expand the Field Selection

​Update Data Requests

​Build the Data Source

​Import the Run Function

​Create a Logger

​Replace processor.run()

​Install RPC Client

​Initialize RPC Client

​Enrich Context

​Migration Complete

​Example Migrations

Without RPC Client

With RPC Client

​Next Steps

Portal API Reference

EVM Examples

Prerequisites

Migration Steps

Update Imports

Replace Processor Initialization

Update Type Exports

Expand the Field Selection

Update Data Requests

Build the Data Source

Import the Run Function

Create a Logger

Replace processor.run()

Install RPC Client

Initialize RPC Client

Enrich Context

Migration Complete

Example Migrations

Next Steps