---
name: Sqd
description: Use when building blockchain data indexers, querying blockchain data via HTTP APIs, or deploying indexers to production. Reach for SQD when you need to extract, transform, and persist blockchain data from 225+ networks (EVM, Solana, Substrate, Bitcoin, Tron, etc.) into databases or data lakes.
metadata:
    mintlify-proj: sqd
    version: "1.0"
---

# SQD Skill

## Product Summary

SQD is a blockchain data indexing platform with three complementary products: **Portal API** (HTTP access to raw blockchain data across 225+ networks), **Squid SDK** (TypeScript framework for building full-stack indexers with PostgreSQL and GraphQL), and **Pipes SDK** (lightweight streaming library for custom pipelines). Agents use SQD to extract, decode, transform, and persist blockchain data. Key files: `squid.yaml` (deployment manifest), `schema.graphql` (data schema), `src/main.ts` (processor entry point), `src/processor.ts` (data source config). CLI: `sqd init`, `sqd deploy`, `sqd run`. Primary docs: https://docs.sqd.dev

## When to Use

Reach for this skill when:

- **Building indexers**: User asks to "create an indexer for X contract" or "index all transfers on Y chain"
- **Querying blockchain data**: User needs to extract logs, transactions, traces, or state from any supported network
- **Deploying to production**: User wants to deploy an indexer to SQD Cloud with monitoring and scaling
- **Transforming on-chain data**: User needs to decode events, normalize data, and persist to a database
- **Real-time data pipelines**: User wants streaming data from blockchain into PostgreSQL, BigQuery, or files
- **Migrating from Subgraph**: User has a Graph Protocol subgraph and wants to move to SQD

Do NOT use this skill for: wallet operations, smart contract deployment, transaction signing, or off-chain application logic unrelated to data indexing.

## Quick Reference

### Three Ways to Access Blockchain Data

| Method | Best For | Setup | Output |
|--------|----------|-------|--------|
| **Portal API** | Analytics, data lakes, any language | HTTP requests, no setup | Raw JSON, streaming |
| **Pipes SDK** | Custom pipelines, non-Postgres databases | TypeScript, lightweight | Flexible (Postgres, ClickHouse, files) |
| **Squid SDK** | Full-stack dApps, GraphQL APIs | TypeScript, PostgreSQL | Auto-generated GraphQL API |

### Essential CLI Commands

```bash
sqd init my-squid -t evm              # Scaffold from template
sqd run                               # Run locally (processor + API)
sqd deploy .                          # Deploy to SQD Cloud
sqd logs -f                           # Stream logs from Cloud
sqd secrets set KEY value             # Store sensitive data
```

### Project Structure (Squid SDK)

```
src/
  main.ts              # Entry point, processor.run() call
  processor.ts         # Data source config (gateway, RPC, filters)
  model/generated/     # TypeORM entities (auto-generated)
  abi/                 # EVM contract ABIs (auto-generated)
schema.graphql         # Entity definitions (PostgreSQL + GraphQL)
db/migrations/         # Database migrations
squid.yaml             # Deployment manifest
.env                   # Local environment variables
docker-compose.yml     # PostgreSQL container
```

### Processor Configuration Checklist (EVM)

```typescript
const processor = new EvmBatchProcessor()
  .setGateway('https://v2.archive.subsquid.io/network/ethereum-mainnet')  // Fast sync
  .setRpcEndpoint('https://rpc.ankr.com/eth')                             // Real-time data
  .setFinalityConfirmation(75)                                            // Fork handling
  .addLog({ address: [CONTRACT], topic0: [EVENT_TOPIC] })                 // Filter logs
  .setFields({ log: { transactionHash: true } })                          // Select fields
```

### Deployment Manifest Essentials

```yaml
manifest_version: subsquid.io/v0.1
name: my-squid
version: 1

build:

deploy:
  addons:
    postgres:
  processor:
    cmd: ["sqd", "process:prod"]
  api:
    cmd: ["sqd", "serve:prod"]

scale:
  processor:
    profile: medium
  api:
    replicas: 3
```

## Decision Guidance

### When to Use Portal API vs Squid SDK vs Pipes SDK

| Scenario | Portal API | Pipes SDK | Squid SDK |
|----------|-----------|-----------|-----------|
| One-off data extraction | ✓ | | |
| Custom pipeline in Python/Go | ✓ | | |
| Real-time GraphQL API | | | ✓ |
| Non-Postgres database | | ✓ | |
| Rapid prototyping | | ✓ | ✓ |
| Production dApp backend | | | ✓ |
| Complex transformations | | ✓ | ✓ |
| Minimal dependencies | ✓ | ✓ | |

### When to Use PostgreSQL vs File Store vs BigQuery

| Use Case | PostgreSQL | File Store | BigQuery |
|----------|-----------|-----------|----------|
| GraphQL API needed | ✓ | | |
| Real-time queries | ✓ | | |
| Batch analytics | | ✓ | ✓ |
| Cost-sensitive | | ✓ | |
| Large datasets (100GB+) | | | ✓ |
| S3 export | | ✓ | |

## Workflow

### Building a Squid (Squid SDK)

1. **Understand requirements**: What data? Which chain? What output format? Off-chain data needed?
2. **Choose template**: `sqd init my-squid -t evm` (or substrate, ink, multichain, etc.)
3. **Install dependencies**: `cd my-squid && npm i`
4. **Generate utilities**: `npx squid-evm-typegen src/abi ./abi/*.json` (EVM) or configure `typegen.json` (Substrate)
5. **Configure processor** (`src/processor.ts`):
   - Set gateway (SQD Network for fast sync)
   - Set RPC endpoint (for real-time data)
   - Add data requests (logs, transactions, traces, state diffs)
   - Select fields needed
6. **Define schema** (`schema.graphql`): Entity types, relations, indexes
7. **Generate models**: `npx squid-typeorm-codegen`
8. **Write batch handler** (`src/main.ts`):
   - Decode data using generated utilities
   - Transform and normalize
   - Persist to `ctx.store`
9. **Test locally**:
   - `docker compose up -d` (start Postgres)
   - `npx squid-typeorm-migration apply` (run migrations)
   - `node -r dotenv/config lib/main.js` (run processor)
   - `npx squid-graphql-server` (start API)
10. **Deploy to Cloud**: `sqd deploy .`

### Deploying to SQD Cloud

1. **Prepare squid**: Test locally, ensure `squid.yaml` is correct
2. **Create account**: Visit https://app.subsquid.io
3. **Create organization**: Set up professional org with billing
4. **Configure manifest**: Update `squid.yaml` with processor/API commands, addons, scale settings
5. **Set secrets**: `sqd secrets set RPC_ENDPOINT "https://..."`
6. **Deploy**: `sqd deploy .`
7. **Monitor**: Check logs and metrics in Cloud console

### Querying with Portal API

1. **Identify dataset**: Find network at https://docs.sqd.dev/en/data/all-networks
2. **Construct request**: Specify `fromBlock`, `toBlock`, filters (logs, transactions, etc.), fields
3. **Handle streaming**: Read response in batches; continue with `fromBlock = lastBlock + 1` until HTTP 204
4. **Parse NDJSON**: Each line is one block; decode as JSON
5. **Handle reorgs**: Check `parentBlockHash` on subsequent requests; handle HTTP 409

## Common Gotchas

- **Forgot to `await` on store operations**: All `ctx.store` calls are async. Missing `await` causes "Query runner already released" error. Use eslint `require-await` rule.
- **Schema mismatch with migrations**: After changing `schema.graphql`, regenerate migrations: `rm -r db/migrations && npx squid-typeorm-migration generate`
- **Decoder on wrong data**: "data out-of-bounds" usually means decoder received unexpected data type. Verify event filtering and sorting in batch handler.
- **Missing `includeAllBlocks` on Portal**: Without filters and without `includeAllBlocks: true`, Portal returns only boundary blocks, not full range.
- **RPC endpoint too old for Multicall**: Batch state queries fail if Multicall contract is newer than blocks being indexed. Use older Multicall or delay queries.
- **NULL bytes in text fields**: PostgreSQL rejects `\0x00` in text. Use hex encoding for raw bytes; use `ss58` for Substrate addresses.
- **Processor crashes on unfinalized blocks**: Set `setFinalityConfirmation()` to handle forks properly; use RPC for real-time data.
- **Slow sync**: Check batch processing patterns; avoid per-item handlers; use indexed fields in database lookups; see Cloud best practices guide.
- **Outdated ABI**: If contract was upgraded, regenerate typegen with implementation contract ABI, not proxy.
- **Forgetting to build**: Always run `npm run build` after code changes before running processor.

## Verification Checklist

Before submitting work:

- [ ] Processor runs locally without errors: `node -r dotenv/config lib/main.js`
- [ ] GraphQL API responds: `npx squid-graphql-server` and query at `localhost:4350/graphql`
- [ ] Schema matches migrations: `schema.graphql` and `db/migrations` are in sync
- [ ] All `ctx.store` calls are awaited
- [ ] Data is decoded correctly (spot-check a few rows in database)
- [ ] Processor handles forks: `setFinalityConfirmation()` is set
- [ ] No hardcoded secrets in code; use `.env` or Cloud secrets
- [ ] `squid.yaml` has correct processor/API commands and addon config
- [ ] Deployment manifest version is `subsquid.io/v0.1`
- [ ] For Cloud: squid name matches manifest, organization is professional (not playground)

## Resources

**Comprehensive navigation**: https://docs.sqd.dev/llms.txt (page index) and https://docs.sqd.dev/llms-full.txt (full content)

**Critical pages**:
- [Squid SDK Overview](/en/sdk/squid-sdk/overview) — Architecture, processors, stores, components
- [Development Flow](/en/sdk/squid-sdk/how-to-start/squid-development) — End-to-end workflow with templates
- [Deployment Guide](/en/cloud/deployment-guide) — Step-by-step Cloud deployment

---

> For additional documentation and navigation, see: https://docs.sqd.dev/llms.txt