Substrate Batch General Settings

The method documentation is also available inline and can be accessed via suggestions in most IDEs.

The following setters configure the global settings of SubstrateBatchProcessor. They return the modified instance and can be chained. Calling setRpcEndpoint() is a hard requirement on Substrate, as chain RPC is used to retrieve chain metadata. Adding a SQD Network gateway with setGateway() is optional but highly recommended, as it greatly reduces RPC usage. To reduce it further, you can explicitly disable RPC ingestion by calling setRpcDataIngestionSettings({ disabled: true }): in this scenario the RPC will only be used for metadata retrieval and to perform any direct RPC queries you might be doing in your squid code. This will, however, introduce a delay of a few thousands of blocks between the chain head and the highest block available to your squid.

SQD Network v2 gateways require an API key. Set SQD_API_KEY in the process environment or pass apiKey in GatewaySettings.

`setGateway(url: string | GatewaySettings)`

Adds a SQD Network data source. The argument is either a string URL of a SQD Network gateway or

{
  url: string // gateway URL
  apiKey?: string // defaults to process.env.SQD_API_KEY
  requestTimeout?: number // in milliseconds
}

See Substrate gateways.

`setRpcEndpoint(rpc: ChainRpc)`

Adds a RPC data source. If added, it will be used for

RPC ingestion (unless explicitly disabled with setRpcDataIngestionSettings())
any direct RPC queries you make in your squid code

A node RPC endpoint can be specified as a string URL or as an object:

type ChainRpc = string | {
  url: string // http, https, ws and wss are supported
  capacity?: number // num of concurrent connections, default 10
  maxBatchCallSize?: number // default 100
  rateLimit?: number // requests per second, default is no limit
  requestTimeout?: number // in milliseconds, default 30_000
  headers?: Record<string, string> // http headers
}

Setting maxBatchCallSize to 1 disables batching completely.

We recommend using private endpoints for better performance and stability of your squids. For SQD Cloud deployments you can use the RPC addon. If you use an external private RPC, keep the endpoint URL in a Cloud secret.

`setDataSource(ds: {archive?: string, chain?: ChainRpc})` (deprecated)

Replaced by setGateway() and setRpcEndpoint().

`setRpcDataIngestionSettings(settings: RpcDataIngestionSettings)`

Specify the RPC ingestion settings.

type RpcDataIngestionSettings = {
  disabled?: boolean
  headPollInterval?: number
  newHeadTimeout?: number
}

Here,

disabled: Explicitly disables data ingestion from an RPC endpoint. RPC remains required on Substrate because SubstrateBatchProcessor uses it for metadata. The processor continues to follow the gateway dataset head; it does not terminate just because it catches up.
headPollInterval: Poll interval for new blocks in milliseconds. Poll mechanism is used to get new blocks via HTTP connections. Default: 5000.
newHeadTimeout: When ingesting from a websocket, this setting specifies the timeout in milliseconds after which the connection will be reset and subscription re-initiated if no new blocks were received. Default: no timeout.

`setBlockRange({from: number, to?: number})`

Limits the range of blocks to be processed. When the upper bound is specified, processor will terminate with exit code 0 once it reaches it. Note that block ranges can also be specified separately for each data request. This method sets global bounds for all block ranges in the configuration.

`setFinalityConfirmation(nBlocks: number)`

Overrides finalized-head tracking with a fixed distance from the chain head. The default uses chain_getFinalizedHead. Setting 0 treats all blocks as final.

`includeAllBlocks(range?: {from: number, to?: number})`

By default, processor will fetch only blocks which contain requested items. This method modifies such behavior to fetch all chain blocks. Optionally a range of blocks can be specified for which the setting should be effective.

`setTypesBundle(bundle: string | OldTypesBundle | OldSpecsBundle | PolkadotjsTypesBundle)`

Sets a types bundle. Types bundle is only required for historical blocks which have metadata version below 14 and only if we don’t have built-in support for the chain in question. Most chains listed in the polkadot.js app are supported. SQD project has its own types bundle format, however, most of polkadotjs types bundles will work as well. Types bundles can be specified in 2 different ways:

as a name of a JSON file:

processor.setTypesBundle('typesBundle.json')

as an OldTypesBundle/OldSpecsBundle or PolkadotjsTypesBundle object:

// OldTypesBundle object
processor.setTypesBundle({
  types: {
    Foo: 'u8'
  }
})

There a mini-guide on how to obtain type bundles for Substrate chains without relying on SQD tools.

`setPrometheusServer(server: PrometheusServer)`

Sets a custom Prometheus server from @subsquid/util-internal-processor-tools.

`setPrometheusPort(port: string | number)` (deprecated)

Sets the port for the built-in Prometheus metrics server. Use setPrometheusServer() for new code. PROCESSOR_PROMETHEUS_PORT takes precedence over PROMETHEUS_PORT; if neither is set, the processor selects an ephemeral port.

Substrate batch processor

Squid SDK

Substrate Batch General Settings

`setGateway(url: string | GatewaySettings)`

`setRpcEndpoint(rpc: ChainRpc)`

`setDataSource(ds: {archive?: string, chain?: ChainRpc})` (deprecated)

`setRpcDataIngestionSettings(settings: RpcDataIngestionSettings)`

`setBlockRange({from: number, to?: number})`

`setFinalityConfirmation(nBlocks: number)`

`includeAllBlocks(range?: {from: number, to?: number})`

`setTypesBundle(bundle: string | OldTypesBundle | OldSpecsBundle | PolkadotjsTypesBundle)`

`setPrometheusServer(server: PrometheusServer)`

`setPrometheusPort(port: string | number)` (deprecated)

​setGateway(url: string | GatewaySettings)

​setRpcEndpoint(rpc: ChainRpc)

​setDataSource(ds: {archive?: string, chain?: ChainRpc}) (deprecated)

​setRpcDataIngestionSettings(settings: RpcDataIngestionSettings)

​setBlockRange({from: number, to?: number})

​setFinalityConfirmation(nBlocks: number)

​includeAllBlocks(range?: {from: number, to?: number})

​setTypesBundle(bundle: string | OldTypesBundle | OldSpecsBundle | PolkadotjsTypesBundle)

​setPrometheusServer(server: PrometheusServer)

​setPrometheusPort(port: string | number) (deprecated)

`setGateway(url: string | GatewaySettings)`

`setRpcEndpoint(rpc: ChainRpc)`

`setDataSource(ds: {archive?: string, chain?: ChainRpc})` (deprecated)

`setRpcDataIngestionSettings(settings: RpcDataIngestionSettings)`

`setBlockRange({from: number, to?: number})`

`setFinalityConfirmation(nBlocks: number)`

`includeAllBlocks(range?: {from: number, to?: number})`

`setTypesBundle(bundle: string | OldTypesBundle | OldSpecsBundle | PolkadotjsTypesBundle)`

`setPrometheusServer(server: PrometheusServer)`

`setPrometheusPort(port: string | number)` (deprecated)