Squid SDK Overview

A squid is an indexing project built with Squid SDK to retrieve and process blockchain data from the SQD Network (either permissioned or decentralized instance). The Squid SDK is a set of open source Typescript libraries that retrieve, decode, transform and persist the data. It can also make the transformed data available via an API. All stages of the indexing pipeline, from the data extraction to transformation to persistence are performed on batches of blocks to maximize the indexing speed. Modular architecture of the SDK makes it possible to extend indexing projects (squids) with custom plugins and data targets.

​

Required squid components

​

Processor

Processor is the word used for

1. The main NodeJS process of the squid.
2. The main object (processor) of this process: its method call processor.run() is the entry point.

Processor code handles data retrieval and transformation; data persistence is handled by a separate object called Store. Current Squid SDK data sources include:

• @subsquid/evm-stream with @subsquid/batch-processor for Portal-native EVM indexing
• @subsquid/solana-stream with @subsquid/batch-processor for Portal-native Solana indexing
• EvmBatchProcessor from @subsquid/evm-processor for gateway and RPC ingestion on Ethereum-compatible networks
• SubstrateBatchProcessor from @subsquid/substrate-processor for Substrate networks
• TronBatchProcessor from @subsquid/tron-processor for Tron

The Portal-native stack uses the standalone run(dataSource, database, handler) function. Its handler context contains {store, blocks, isHead}. The older processor classes expose .run() and also inject ctx.log and, where supported, ctx._chain.

​

Store

A store is an object that processors use to persist their data. SQD offers three store classes:

• TypeormStore for saving data to PostgreSQL, via
  • @subsquid/typeorm-store
  • @subsquid/typeorm-codegen (a code generator, install with --save-dev)
  • @subsquid/typeorm-migration
  Install all three packages to use this store.
• file-store via @subsquid/file-store - for saving data to filesystems. It is a modular system with a variety of extensions for various formats and destinations.
• bigquery-store via @subsquid/bigquery-store - for saving data to Google BigQuery.

You can mix and match any store class with any processor class.

​

Optional squid components

​

Typegen

A typegen is a tool for generating utility code for technology-specific operations such as decoding. Here are the typegens available: Install these with --save-dev.

​

GraphQL server

Squids that store their data in PostgreSQL can subsequently make it available as a GraphQL API via a variety of supported servers. See Serving GraphQL. Among other alternatives, SQD provides its own server called OpenReader via the @subsquid/graphql-server package. The server runs as a separate process. Core API is automatically derived from the schema file; it is possible to extend it with custom queries and basic access control.

​

Misc utilities