Field selection

`setFields(options)`

Set the fields to be retrieved for data items of each supported type. The options object has the following structure:

{
  instruction?:  // field selector for instructions
  transaction?:  // field selector for transactions
  log?:          // field selector for log messages
  balance?:      // field selector for balances
  tokenBalance?: // field selector for token balances
  reward?:       // field selector for reward
  block?:        // field selector for block headers
}

Every field selector is a collection of boolean fields, typically mapping one-to-one to the fields of data items within the batch context iterables. Defining a field of a field selector of a given type and setting it to true will cause the processor to populate the corresponding field of all data items of that type. Here is a definition of a data source that requests signatures and err fields for transactions and the data field for instructions:

const dataSource = new DataSourceBuilder()
  .setFields({
    transaction: {
      signatures: true,
      err: true
    },
    instruction: {
      data: true
    }
  })

There are no default optional fields: apart from a small set of always-present fields (listed per item type below), a field that is not requested with setFields() will not be fetched and will be absent from the TypeScript types. Request every field your batch handler reads.

Same fields will be available for all data items of any given type, including the items accessed via nested references. Suppose we used the data source defined above to subscribe to some transactions as well as some instructions, and for each instruction we requested a parent transaction:

dataSource
  .addInstruction({
    where: {
      // some instruction data requests
    },
    include: {
      transaction: true
    }
  })
  .addTransaction({
    where: {
      // some transaction data requests
    },
    include: {
      instructions: true
    }
  })
  .build()

After populating the convenience reference fields with augmentBlock() from @subsquid/solana-objects, signatures and err fields would be available both within the transaction items of the transactions iterable of block data and within the transaction items that provide parent transaction information for the instructions matching the addInstruction() data request:

run(dataSource, database, async (ctx) => {
  let blocks = ctx.blocks.map(augmentBlock)
  for (let block of blocks) {
    for (let txn of block.transactions) {
      let sigs = txn.signatures; // OK
    }
    for (let ins of block.instructions) {
      if (/* ins matches the data request */) {
        let parentTxSigs = ins.getTransaction().signatures; // also OK!
      }
    }
  }
})

Fields that were not requested are missing from the data and from the item types, so reading them fails to compile:

const dataSource = new DataSourceBuilder()
  .setFields({
    transaction: {
      err: true
    }
  })
  .build()

run(dataSource, database, async (ctx) => {
  for (let block of ctx.blocks) {
    for (let txn of block.transactions) {
      let signatures = txn.signatures; // ERROR: no such field
    }
  }
})

Requesting only the fields you need improves sync performance, as unrequested fields are not fetched from the SQD Portal.

Data item types and field selectors

Most IDEs support smart suggestions to show the possible field selectors. For VS Code, press Ctrl+Space.

All addresses and pubkeys are represented as base58-encoded strings.

Here we describe the data item types as functions of the field selectors. Unless otherwise mentioned, each data item type field maps to the eponymous field of its corresponding field selector. Item fields are divided into two categories:

Fields that are always present regardless of the setFields() call.
Fields that must be requested with setFields().

Instruction

Instruction data items may have the following fields:

Instruction {
  // independent of field selectors
  transactionIndex: number
  instructionAddress: number[]

  // can be requested with field selectors
  programId: string
  accounts: string[]
  data: string
  isCommitted: boolean
  computeUnitsConsumed?: bigint
  error?: string
  hasDroppedLogMessages: boolean
}

Here, instruction address is an array of tree indices addressing the instruction in the call tree:

Top level instructions get addresses [0], [1], …, [N-1], where N is the number of top level instructions in the transaction.
An address of length 2 indicates an inner instruction directly called by one of the top level instructions. For example, [3, 0] is the first inner instruction called by the fourth top level instruction.
Addresses of length 3 or more indicate inner instructions invoked by other inner instructions.

Transaction

Transaction data items may have the following fields:

Transaction {
  // independent of field selectors
  transactionIndex: number

  // can be requested with field selectors
  signatures: string[]
  err: null | object
  version: 'legacy' | number
  accountKeys: string[]
  addressTableLookups: AddressTableLookup[]
  numReadonlySignedAccounts: number
  numReadonlyUnsignedAccounts: number
  numRequiredSignatures: number
  recentBlockhash: string
  computeUnitsConsumed: bigint
  costUnits: bigint
  fee: bigint
  loadedAddresses: {
    readonly: string[]
    writable: string[]
  } // request the whole struct with loadedAddresses: true
  hasDroppedLogMessages: boolean
}

LogMessage

LogMessage data items may have the following fields:

LogMessage {
  // independent of field selectors
  transactionIndex: number
  logIndex: number
  instructionAddress: number[]

  // can be requested with field selectors
  programId: string
  kind: 'log' | 'data' | 'other'
  message: string
}

Balance

Balance data items may have the following fields:

Balance {
  // independent of field selectors
  transactionIndex: number
  account: string

  // can be requested with field selectors
  pre: bigint
  post: bigint
}

Each Balance item describes a single account: pre is the account’s SOL balance before the transaction and post is its balance after it.

TokenBalance

Field selection for token balances data items is more nuanced, as depending on the subtype of the token balance some fields may be undefined. PostTokenBalance and PreTokenBalance both represent token balances, however PreTokenBalance will have postProgramId, postMint, postDecimals, postOwner and postAmount as undefined. PostTokenBalance will have preProgramId, preMint, preDecimals, preOwner and preAmount as undefined. TokenBalance data items may have the following fields:

TokenBalance {
  // independent of field selectors
  transactionIndex: number
  account: string

  // can be requested with field selectors
  preProgramId?: string
  preMint: string
  preDecimals: number
  preOwner?: string
  preAmount: bigint
  postProgramId?: string
  postMint: string
  postDecimals: number
  postOwner?: string
  postAmount: bigint
}

Reward

Reward data items may have the following fields:

Reward {
  // independent of field selectors
  pubkey: string

  // can be requested with field selectors
  lamports: bigint
  postBalance: bigint
  rewardType?: string
  commission?: number
}

Block header

BlockHeader data items may have the following fields:

BlockHeader {
  // independent of field selectors
  number: number     // Solana slot
  hash: string
  parentHash: string

  // can be requested with field selectors
  parentNumber: number
  timestamp: number  // Unix milliseconds
}

slot and parentSlot are not field names: use number for the Solana slot (also in setBlockRange() values) and parentNumber for the parent slot.

@subsquid/[email protected] accepts height in the TypeScript field selection, but the Portal source does not return it.

The stream converts Portal’s raw Solana block time from seconds to milliseconds. Pass timestamp directly to new Date(timestamp) without multiplying it by 1000.

A complete example

Complete example

import { run } from "@subsquid/batch-processor";
import { augmentBlock } from "@subsquid/solana-objects";
import { DataSourceBuilder } from "@subsquid/solana-stream";
import { TypeormDatabase } from "@subsquid/typeorm-store";
import * as whirlpool from "./abi/whirlpool";

const dataSource = new DataSourceBuilder()
  .setPortal("https://portal.sqd.dev/datasets/solana-mainnet")
  .setBlockRange({ from: 259_984_950 })
  .setFields({
    block: {
      timestamp: true
    },
    transaction: {
      version: true,
      fee: true
    },
    instruction: {
      programId: true,
      data: true,
      error: true
    },
    tokenBalance: {
      postProgramId: true,
      postMint: true,
      postAmount: true
    }
  })
  .addInstruction({
    // select instructions that
    where: {
      // were executed by the Whirlpool program
      programId: [whirlpool.programId],
      // have the first 8 bytes of .data equal to swap descriptor
      d8: [whirlpool.instructions.swap.d8],
      // were successfully committed
      isCommitted: true,
    },
    include: {
      // inner instructions
      innerInstructions: true,
      // transaction that executed the given instruction
      transaction: true,
      // all token balance records of executed transaction
      transactionTokenBalances: true
    },
  })
  .build();

run(dataSource, new TypeormDatabase(), async (ctx) => {
  // Simply output all the items in the batch.
  // It is guaranteed to have all the data matching the data requests,
  // but not guaranteed to not have any other data.
  console.log(ctx.blocks.map(augmentBlock));
})

Pipes SDK

Squid SDK

`setFields(options)`

Data item types and field selectors

Instruction

Transaction

LogMessage

Balance

TokenBalance

Reward

Block header

A complete example

​setFields(options)

​Data item types and field selectors

​Instruction

​Transaction

​LogMessage

​Balance

​TokenBalance

​Reward

​Block header

​A complete example

`setFields(options)`

Data item types and field selectors

Instruction

Transaction

LogMessage

Balance

TokenBalance

Reward

Block header

A complete example