Overview

A squid is an indexing project built with Squid SDK to retrieve and process blockchain data from the SQD Network (either permissioned or decentralized instance). The Squid SDK is a set of open source Typescript libraries that retrieve, decode, transform and persist the data. It can also make the transformed data available via an API. All stages of the indexing pipeline, from the data extraction to transformation to persistence are performed on batches of blocks to maximize the indexing speed. Modular architecture of the SDK makes it possible to extend indexing projects (squids) with custom plugins and data targets.

Required squid components

Processor

Processor is the word used for

1. The main NodeJS process of the squid.
2. The main object (processor) of this process: its method call processor.run() is the entry point.

processor objects handle data retrieval and transformation; data persistence is handled by a separate object called Store. Squid SDK offers two processor classes:

• EvmBatchProcessor via the @subsquid/evm-processor NPM package - for Ethereum-compatible networks
• SubstrateBatchProcessor via @subsquid/substrate-processor - for networks based on Substrate such as Polkadot

Store

A store is an object that processors use to persist their data. SQD offers three store classes:

• TypeormStore for saving data to PostgreSQL, via

  • @subsquid/typeorm-store
  • @subsquid/typeorm-codegen (a code generator, install with --save-dev)
  • @subsquid/typeorm-migration

  Install all three packages to use this store.

• file-store via @subsquid/file-store - for saving data to filesystems. It is a modular system with a variety of extensions for various formats and destinations.

• bigquery-store via @subsquid/bigquery-store - for saving data to Google BigQuery.

You can mix and match any store class with any processor class.

Optional squid components

Typegen

A typegen is a tool for generating utility code for technology-specific operations such as decoding. Here are the typegens available:

On EVM

• squid-evm-typegen via @subsquid/evm-typegen:
  • decodes smart contract data
  • handles direct calls to contract methods
  • exposes useful constants such as event topics and function signature hashes The generated code depends on @subsquid/evm-abi, SQD's own high performance, open source EVM codec.

On Substrate

• squid-substrate-typegen via @subsquid/substrate-typegen:
  • general purpose pallet data decoding (aware of runtime versions)
  • handles direct storage queries
• squid-ink-typegen via @subsquid/ink-typegen for decoding the data of ink! contracts

Install these with --save-dev.

GraphQL server

Squids that store their data in PostgreSQL can subsequently make it available as a GraphQL API via a variety of supported servers. See Serving GraphQL.

Among other alternatives, SQD provides its own server called OpenReader via the @subsquid/graphql-server package. The server runs as a separate process. Core API is automatically derived from the schema file; it is possible to extend it with custom queries and basic access control.

Misc utilities

On EVM

• Squid CLI is a utility for retrieving squid templates, managing chains of commands commonly used in development and running all squid processes at once. It can also be used for deploying to SQD Cloud.

On Substrate

• Squid CLI is a utility for retrieving squid templates, managing chains of commands commonly used in development and running all squid processes at once. It can also be used for deploying to SQD Cloud.
• @subsquid/ss58 handles encoding and decoding of SS58 addresses
• @subsquid/frontier decodes events and calls of the Frontier EVM pallet to make them decodable with squid-evm-typegen