> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sqd.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# RPC ingestion and reorgs

> How Squid SDK handles unfinalized blocks and reorgs — RPC ingestion, finality assumptions, and the rollback model for chain consensus changes.

Starting with the ArrowSquid release, the processor can ingest data either from a [SQD Network](/en/network) gateway or directly from an RPC endpoint. If both a network gateway and a RPC endpoint are provided, the processor will use network data until it reaches the highest block available there, then index the few remaining blocks using the RPC endpoint. This allows squids to combine low sync times with near real-time chain data access. It is, however, possible to use just the RPC endpoint (e.g. for [local development](/en/sdk/squid-sdk/tutorials/evm-local)). On [EVM](/en/sdk/squid-sdk/reference/processors/evm-batch) it is also possible to use just the gateway (e.g. for non-realtime analytics).

RPC ingestion can create a heavy load on node endpoints. The load is typically short in duration when a SQD Network gateway is used, and the total number of requests is low. However, the request rate may be sufficient to trigger HTTP 429 responses. Use private endpoints and rate limit your requests with the `rateLimit` option ([EVM](/en/sdk/squid-sdk/reference/processors/evm-batch/general#set-rpc-endpoint)/[Substrate](/en/sdk/squid-sdk/reference/processors/substrate-batch/general#set-rpc-endpoint)).

## Indexing unfinalized blocks

When ingesting from RPC, Squid SDK can index blocks before they are [finalized](https://info.etherscan.com/epoch-in-ethereum/), enabling real-time use cases. If a blockhain reorganization happens, [processor](/en/sdk/squid-sdk/reference/processors/architecture) will roll back any changes to the database made due to orphaned blocks, then re-run its batch handler on consensus blocks.

To take advantage of this feature, a squid processor must

1. have a [node RPC endpoint](/en/sdk/squid-sdk/reference/processors/evm-batch/general#set-rpc-endpoint) as one of its data sources and
2. use a [`Database`](/en/sdk/squid-sdk/resources/persisting-data/overview) with unfinalized blocks support (e.g. [`TypeormDatabase`](/en/sdk/squid-sdk/resources/persisting-data/typeorm)) in its [`processor.run()`](/en/sdk/squid-sdk/reference/processors/architecture#processorrun) call, and
3. not disable RPC ingestion explicitly with [`setRpcDataIngestionSettings({ disabled: true })`](/en/sdk/squid-sdk/reference/processors/evm-batch/general#set-rpc-data-ingestion-settings).

With this, the processor will consider some blocks to be "hot":

<Tabs>
  <Tab title="EVM">
    All blocks with fewer confirmations than the number set by the [`setFinalityConfirmations()`](/en/sdk/squid-sdk/reference/processors/evm-batch/general#set-finality-confirmation) setting
  </Tab>

  <Tab title="Substrate">
    All blocks above the latest finalized block provided by the [`chain_getFinalizedHead()`](https://polkadot.js.org/docs/substrate/rpc/#getfinalizedhead-blockhash) RPC method
  </Tab>
</Tabs>

The processor will periodically (interval setting for [EVM](/en/sdk/squid-sdk/reference/processors/evm-batch/general#set-rpc-data-ingestion-settings), [Substrate](/en/sdk/squid-sdk/reference/processors/substrate-batch/general#set-rpc-data-ingestion-settings)) poll the RPC endpoint for changes in consensus. When the consensus changes, it will re-run the [batch handler](/en/sdk/squid-sdk/reference/processors/architecture#processorrun) with the new consensus data and ask the `Database` to adjust its state. The `Database` then must roll back the changes made due to orphaned blocks and apply the new changes. With this, the state of the `Database` reflects the current blockchain consensus at all times.