Indexer from scratch

Here's an example of how SDK packages can be combined into a working indexer (called squid).

This page goes through all the technical details to make the squid architecture easier to understand. If you would like to get to a working indexer ASAP, bootstrap from a template.

USDT transfers API

Pre-requisites: NodeJS 20.x or newer, Docker.

Suppose the task is to track transfers of USDT on Ethereum, then save the resulting data to PostgreSQL and serve it as a GraphQL API. From this description we can immediately put together a list of packages:

@subsquid/evm-processor - for retrieving Ethereum data
the triad of @subsquid/typeorm-store, @subsquid/typeorm-codegen and @subsquid/typeorm-migration - for saving data to PostgreSQL

We also assume the following choice of optional packages:

@subsquid/evm-typegen - for decoding Ethereum data and useful constants such as event topic0 values
@subsquid/evm-abi - as a peer dependency for the code generated by @subsquid/evm-typegen
@subsquid/graphql-server / OpenReader

To make the indexer, follow these steps:

Create a new folder and initialise a new project
- create package.json
```
npm init
```
- add .gitignore
```
touch .gitignore
```
  .gitignore
```
/node_modules
package-lock.json
.env
```

install the packages:

npm i dotenv typeorm @subsquid/evm-processor @subsquid/typeorm-store @subsquid/typeorm-migration @subsquid/graphql-server @subsquid/evm-abi

npm i typescript @subsquid/typeorm-codegen @subsquid/evm-typegen --save-dev

Add a minimal tsconfig.json:

tsconfig.json
{
  "compilerOptions": {
    "rootDir": "src",
    "outDir": "lib",
    "module": "commonjs",
    "target": "es2020",
    "esModuleInterop": true,
    "skipLibCheck": true,
    "experimentalDecorators": true,
    "emitDecoratorMetadata": true
  }
}

Define the schema for both the database and the core GraphQL API in schema.graphql:

schema.graphql
type Transfer @entity {
  id: ID!
  from: String! @index
  to: String! @index
  value: BigInt!
}

Generate TypeORM classes based on the schema:
```
npx squid-typeorm-codegen
```
The TypeORM classes are now available at src/model/index.ts.

Prepare the database:

create .env and docker-compose.yaml files

.env
DB_NAME=squid
DB_PORT=23798

docker-compose.yaml
version: "3"
services:
  db:
    image: postgres:15
    environment:
      POSTGRES_DB: "${DB_NAME}"
      POSTGRES_PASSWORD: postgres
    ports:
      - "${DB_PORT}:5432"

start the database container
```
docker compose up -d
```
compile the TypeORM classes
```
npx tsc
```
generate the migration file
```
npx squid-typeorm-migration generate
```
apply the migration with
```
npx squid-typeorm-migration apply
```

Generate utility classes for decoding USDT contract data based on its ABI downloaded from Etherscan:
```
npx \
squid-evm-typegen \
src/abi \
0xdAC17F958D2ee523a2206206994597C13D831ec7#usdt
```
The utility classes are now available at src/abi/usdt.ts

Tie all the generated code together with a src/main.ts executable with the following code blocks:

Imports

import { EvmBatchProcessor } from '@subsquid/evm-processor'
import { TypeormDatabase } from '@subsquid/typeorm-store'
import * as usdtAbi from './abi/usdt'
import { Transfer } from './model'

EvmBatchProcessor object definition

const processor = new EvmBatchProcessor()
  .setGateway('https://v2.archive.subsquid.io/network/ethereum-mainnet')
  .setRpcEndpoint({
    // set RPC endpoint in .env
    url: process.env.RPC_ETH_HTTP,
    rateLimit: 10
  })
  .setFinalityConfirmation(75) // 15 mins to finality
  .addLog({
    address: [ '0xdAC17F958D2ee523a2206206994597C13D831ec7' ],
    topic0: [ usdtAbi.events.Transfer.topic ]
  })

TypeormDatabase object definition
```
const db = new TypeormDatabase()
```

A call to processor.run() with an inline definition of the batch handler

processor.run(db, async ctx => {
  const transfers: Transfer[] = []
  for (let block of ctx.blocks) {
    for (let log of block.logs) {
      let {from, to, value} = usdtAbi.events.Transfer.decode(log)
      transfers.push(new Transfer({
        id: log.id,
        from, to, value
      }))
    }
  }
  await ctx.store.insert(transfers)
})

Note how supplying a TypeormDatabase to the function caused ctx.store to be a PostgreSQL-compatible Store object.

Compile the project and start the processor process
```
npx tsc
```
```
node -r dotenv/config lib/main.js
```
In a separate terminal, configure the GraphQL port and start the GraphQL server:

.env
 DB_NAME=squid
 DB_PORT=23798
+GQL_PORT=4350

npx squid-graphql-server

The finished GraphQL API with GraphiQL is available at localhost:4350/graphql.

USDT transfers API​

USDT transfers API