Use this guide when a squid works but is slow, consumes unpredictable resources,
or behaves differently after a restart or reorg.
Separate ingestion from processing
Before changing RPC settings or scaling the database, measure where time is
spent. Temporarily replace the body of the
batch handler
with a block counter:
Run the probe over the same block range and in the same environment as the slow
run:
- If throughput improves substantially, optimize transformations and database
access. Start with batch processing
and query optimization.
- If throughput stays similar, investigate the data source, network connection,
RPC rate limits, and RPC provider.
An empty handler still advances processor status. Use a disposable database or
a unique stateSchema, as shown above. Do not replace a production handler and
reuse its processor status.
Request only data you use
Narrow requests by contract and block range whenever possible. Disable optional
fields that are not used — this reduces data transfer and the amount of data
held in each batch.
setFields()
controls the shape of returned items. It selects fields on items that were
already requested elsewhere in the configuration; it does not add new data items
to the batch by itself. Request each data item type and its related data
explicitly, and enable only the fields your handler reads.
Make batch handlers deterministic
Batch boundaries are an implementation detail and can change between runs. The
same block range can arrive as several different batch groupings.
- Do not let results depend on mutable module-level or global state carried from
one batch to the next.
- Persist correctness-critical state in the database, or derive it again from
the current batch and durable data.
- Keep in-memory caches bounded and safe to rebuild after a restart.
- Test important changes with multiple clean syncs and compare the resulting
rows or a stable export.
Batch handlers should also be safe to retry. TypeormDatabase can run a
handler more than once for the same data when a transaction is retried or the
chain reorganizes. See the
typeorm-store transaction notes.
Keep batch work bounded
Processors can receive large batches, and the number of blocks and selected
items per batch varies. Keep memory and CPU use close to linear in the number
of selected items:
- collect IDs in a
Set, then load matching rows with one findBy() call;
- update entities in a
Map, then save them as an array;
- avoid nested scans over the complete batch;
- clear temporary collections at the end of each handler call;
- load test with realistic event density, not only a short quiet range.
Do not replace batching with a database read or write per event. The
batch-processing patterns
show how to load and persist entities in groups.
Run periodic work intentionally
By default, a processor fetches only blocks that contain requested items. Add
includeAllBlocks() when logic must run on every block:
includeAllBlocks()
accepts an optional block range, so limit it when the requirement is historical
only. Fetching every block increases work.
Make external side effects reorg-safe
When a reorg replaces unfinalized blocks, a compatible database rolls back SQL
changes made for orphaned blocks and the processor runs the handler again on
the canonical chain. Messages already sent to Kafka, HTTP requests, and other
external effects are not rolled back.
For effects derived from unfinalized blocks:
- include block height and block hash in the external record;
- make writes idempotent;
- let consumers replace or retract provisional data when the canonical hash at
a height changes;
- delay irreversible actions until the required confirmation or finality
threshold has been reached.
The unfinalized blocks and reorgs guide
explains which blocks are considered hot and how database rollback works.
Debug a processor near the failure
Use a narrow debug namespace before enabling all logs:
Debug output can be large. If the issue occurs near the chain head, first sync
close to the head with the normal log level, then restart with debug logging for
a short reproduction. See
overriding the log level
and Cloud logging.
For slow or oversized GraphQL requests, use the
query optimization procedure, add
schema indexes, paginate
collections, and configure
OpenReader DoS protection.
Common errors
Processor errors
Event decoding errors on Substrate
-
Ensure that the runtime version matches the block being indexed. Substrate events can change between runtime versions.
-
Verify that the event name and pallet match the runtime metadata for the block range being indexed.
-
Check that event arguments are being decoded in the correct order according to the runtime metadata.
Database errors
QueryFailedError: relation "..." does not exist
It is likely that the generated migrations in the db/migrations folder are outdated and do not match the schema file.
Recreate the migrations from scratch as detailed on this page. The relevant commands are listed in the squid-typeorm-migration reference.
If your squid saves its data to a database, all operations with ctx.store are asynchronous. Make sure you await on all store operations like upsert, update, find, save etc.
You may find the require-await eslint rule to be helpful.
QueryFailedError: invalid byte sequence for encoding "UTF8": 0x00
PostgreSQL doesn’t support storing NULL (\0x00) characters in text fields. Usually the error occurs when a raw bytes string (like UIntArray or Bytes) is inserted into a String field. If this is the case, use hex encoding, e.g. using util-internal-hex library. For addresses, use the ss58 encoding library.
GraphQL errors
response might exceed the size limit
Make sure the input query has limits set or the entities are decorated with @cardinality. We recommend using XXXConnection queries for pagination. For configuring limits and max response sizes, see DoS protection. 