---
name: Sqd
description: Use when building blockchain data indexers, querying blockchain data via HTTP API, or deploying data pipelines to production. Reach for SQD when you need to extract, transform, and persist blockchain data from 140+ networks including Ethereum, Solana, Bitcoin, and others.
metadata:
    mintlify-proj: sqd
    version: "1.0"
---

# SQD Skill

## Product summary

SQD is a blockchain data indexing platform with three layers: **Portal API** (HTTP interface for raw blockchain data), **SDKs** (TypeScript libraries for decoding and transformation), and **Cloud** (managed hosting). Portal streams data from 140+ networks (EVM, Solana, Bitcoin, Substrate, Tron, Hyperliquid) with native finality awareness and reorg handling. The Pipes SDK is the recommended starting point for new TypeScript indexers; it includes built-in targets for PostgreSQL, ClickHouse, BigQuery, and Parquet. The Squid SDK is a schema-first framework that generates GraphQL APIs and deploys to SQD Cloud. All SDKs consume Portal internally. Key files: `squid.yaml` (deployment manifest), `pipes.config.json` (Pipes CLI config), `schema.graphql` (Squid SDK schema). CLI: `sqd` (Squid), `@subsquid/pipes-cli` (Pipes). Primary docs: https://docs.sqd.dev

## When to use

- **Portal API**: Query blockchain data from any language (Python, Go, Rust, etc.) without SDK dependencies. Use for prototypes, data warehouses, or non-TypeScript pipelines.
- **Pipes SDK**: Building a new TypeScript indexer that writes to your own database (PostgreSQL, ClickHouse, BigQuery, Parquet). Recommended for most new projects.
- **Squid SDK**: Building a dApp backend that needs a GraphQL API, or indexing Substrate/Fuel/Starknet networks. Deploy to SQD Cloud with `sqd deploy`.
- **SQD Cloud**: Running Squid SDK indexers in production with managed infrastructure, monitoring, and zero-downtime deployments.

Trigger conditions: User asks to "index blockchain data," "extract contract events," "track token transfers," "build an indexer," "deploy to production," or "query blockchain history."

## Quick reference

### Portal API (HTTP)

| Task | Command |
|------|---------|
| Query EVM logs | `POST /datasets/ethereum-mainnet/stream` with `type: "evm"`, `logs: [...]` |
| Query Solana instructions | `POST /datasets/solana-mainnet/stream` with `type: "solana"`, `instructions: [...]` |
| Get finalized data only | Use `/finalized-stream` instead of `/stream` |
| Resolve timestamp to block | `GET /datasets/{dataset}/timestamps/{timestamp}/block` |
| Get dataset metadata | `GET /datasets/{dataset}/metadata` |

### Pipes SDK (TypeScript)

| Task | Command |
|------|---------|
| Scaffold project | `pnpx @subsquid/pipes-cli@beta init` |
| Regenerate from config | `pnpx @subsquid/pipes-cli@beta init --config pipes.config.json` |
| Run locally | `pnpm run dev` (with `docker compose up -d` for database) |
| Build for production | `pnpm build` |
| Query EVM events | `evmEventDecoder({ contracts: [...], events: {...} })` |
| Query Solana instructions | `solanaInstructionDecoder({ programs: [...], instructions: {...} })` |
| Write to PostgreSQL | `drizzleTarget({ db, tables, onData })` |
| Write to ClickHouse | `clickhouseTarget({ db, tables, onData })` |

### Squid SDK (TypeScript)

| Task | Command |
|------|---------|
| Scaffold project | `sqd init hello-squid -t evm` |
| Build | `sqd build` |
| Start database | `sqd up` |
| Run locally | `sqd run .` |
| Deploy to Cloud | `sqd deploy .` |
| View logs | `sqd logs -n <name> -s <slot>` |
| Add production tag | `sqd tags add production -n <name> -s <slot>` |
| Define schema | Edit `schema.graphql`, run `sqd codegen`, `sqd migration:generate` |

### SQD Cloud deployment manifest (`squid.yaml`)

```yaml
manifest_version: subsquid.io/v0.1
name: my-squid
build:
  node_version: "20"
deploy:
  addons:
    postgres:
      version: "18"
  processor:
    cmd: ["sqd", "process:prod"]
  api:
    cmd: ["sqd", "serve:prod"]
scale:
  dedicated: true
  addons:
    postgres:
      profile: medium
      storage: 100Gi
  processor:
    profile: medium
  api:
    profile: medium
    replicas: 2
```

## Decision guidance

### When to use Portal API vs Pipes SDK vs Squid SDK

| Scenario | Use | Reason |
|----------|-----|--------|
| Non-TypeScript pipeline (Python, Go, Rust) | Portal API | Plain HTTP, no SDK dependency |
| Streaming into your own database | Pipes SDK | Built-in targets, cursor management, reorg handling |
| Building a dApp backend with GraphQL | Squid SDK | Auto-generated GraphQL API, schema-first workflow |
| Prototype or one-off query | Portal API | Zero setup, no authentication |
| Indexing Substrate/Fuel/Starknet | Squid SDK | Pipes SDK only supports EVM, Solana, Bitcoin, Tron, Hyperliquid |
| Production deployment with monitoring | Squid SDK + Cloud | `sqd deploy` handles scaling, logging, zero-downtime updates |
| Embedding indexing in existing app | Pipes SDK | Lightweight library, not a separate process |

### When to use finalized vs unfinalized streams

| Scenario | Use |
|----------|-----|
| Production indexer requiring certainty | `/finalized-stream` (only finalized blocks, no reorg handling needed) |
| Real-time monitoring or alerts | `/stream` (includes unfinalized blocks, handle reorgs in code) |
| Historical backfill | Either (finalized is safer, unfinalized is faster) |

### When to migrate from Gateway to Portal

| Condition | Action |
|-----------|--------|
| Squid using `v2.archive.subsquid.io` gateway | Migrate to Portal (gateways require API keys since May 2026) |
| Self-hosted squid on gateway endpoint | Migrate to Portal or use RPC fallback |
| SQD Cloud deployment | Already using Portal, no action needed |

## Workflow

### Building a Pipes SDK indexer

1. **Scaffold the project**: Run `pnpx @subsquid/pipes-cli@beta init` and answer prompts (network, database, templates).
2. **Review generated code**: Check `src/index.ts` for the decoder and target, `schemas.ts` for table definitions.
3. **Customize the decoder**: Edit contract addresses, block ranges, event filters in `pipes.config.json`, then re-run `init`.
4. **Test locally**: Run `docker compose up -d` for the database, then `pnpm run dev` to start the pipeline.
5. **Verify data**: Query the database table to confirm data is landing correctly.
6. **Deploy**: Push to a self-hosted Node.js environment or use SQD Cloud (Pipes SDK does not support Cloud yet).

### Building a Squid SDK indexer

1. **Scaffold the project**: Run `sqd init hello-squid -t evm`.
2. **Define the schema**: Edit `schema.graphql` with your entities and relations.
3. **Generate models**: Run `sqd codegen` to generate TypeORM models from the schema.
4. **Configure the data source**: Edit `src/main.ts` to select blocks, transactions, logs, or traces.
5. **Write the batch handler**: Transform data and insert into the store.
6. **Test locally**: Run `sqd up` (database), then `sqd run .` (processor + GraphQL).
7. **Query the API**: Open `http://localhost:4350/graphql` and test your queries.
8. **Deploy to Cloud**: Run `sqd deploy .` to create a new slot, validate, then `sqd tags add production -n <name> -s <slot>`.

### Querying with Portal API

1. **Choose the dataset**: Pick from 140+ networks (e.g., `ethereum-mainnet`, `solana-mainnet`).
2. **Construct the request**: POST to `/datasets/{dataset}/stream` with `type`, `fromBlock`, `toBlock`, `fields`, and filters.
3. **Stream the response**: Read newline-delimited JSON, one block per line.
4. **Handle reorgs**: Check for rollback events in the response; use `/finalized-stream` to avoid them.
5. **Decode ABI data**: Parse `topics` and `data` fields using your contract ABI.

### Deploying to SQD Cloud

1. **Test locally**: Run the same commit, lockfile, and manifest commands locally.
2. **Create squid.yaml**: Define services (processor, api), addons (postgres), and scale profile.
3. **Deploy to a new slot**: Run `sqd deploy .` to build, init, and start services.
4. **Monitor progress**: Check `sqd logs -n <name> -s <slot>` and processor height.
5. **Validate output**: Query the GraphQL API or database to confirm correctness.
6. **Attach production tag**: Run `sqd tags add production -n <name> -s <slot>` to route traffic.
7. **Clean up old slots**: Remove previous slots after a rollback window.

## Common gotchas

- **Pipes SDK does not support SQD Cloud**: Deploy Pipes indexers to self-hosted Node.js environments (Railway, Heroku, etc.), not Cloud.
- **Cursor must be stable**: In Pipes SDK, the `id` field in `evmPortalStream()` must not change between runs, or the target will lose its resume position.
- **Reorg handling is automatic for built-in targets**: ClickHouse, PostgreSQL, BigQuery, and Parquet targets roll back reorganized blocks automatically. Custom targets must implement `onRollback()`.
- **Field selection is required**: In Portal API and SDKs, only request fields you actually use. Requesting unnecessary fields slows queries and increases bandwidth.
- **Batch handler filtering is recommended**: Even though Portal filters server-side, re-check addresses and topics in your batch handler to catch future config changes.
- **Schema changes require planning**: Changing `schema.graphql` in Squid SDK may require a migration, backfill, or fresh-slot replay. Plan before deploying.
- **Update-heavy tables bloat databases**: If your squid repeatedly rewrites the same rows (prices, balances, TVL), tune those tables separately or use background jobs.
- **Secrets are not injected until restart**: Changing a secret in SQD Cloud does not update running processes until you run `sqd restart`.
- **Portal API responses are newline-delimited JSON**: Do not parse the entire response as a single JSON object; read line by line.
- **Finalized blocks are chain-specific**: Finality rules differ per network (Ethereum ~15 min, Solana ~30 sec). Check the dataset metadata for your chain.
- **Migrations must be idempotent**: In Squid SDK, migrations run on every deployment. Ensure they can run multiple times without error.
- **Do not use mutable global state in batch handlers**: State that survives only in process memory will diverge after a restart or reorg.

## Verification checklist

Before submitting work:

- [ ] **Portal API**: Response is newline-delimited JSON; each line parses as valid JSON.
- [ ] **Pipes SDK**: `pnpm run dev` runs without errors; data lands in the target database.
- [ ] **Pipes SDK**: The `id` field in `evmPortalStream()` is stable and unique per pipeline.
- [ ] **Squid SDK**: `sqd build` completes without TypeScript errors.
- [ ] **Squid SDK**: `sqd run .` starts processor and GraphQL server; test queries return expected results.
- [ ] **Squid SDK**: `schema.graphql` has `@index` decorators on frequently queried fields.
- [ ] **Squid SDK**: Batch handler filters data (e.g., re-checks contract addresses).
- [ ] **Cloud deployment**: `squid.yaml` uses `manifest_version: subsquid.io/v0.1`.
- [ ] **Cloud deployment**: Secrets are referenced with `${{ secrets.NAME }}`, not hardcoded.
- [ ] **Cloud deployment**: `sqd deploy .` succeeds; processor height advances; API responds.
- [ ] **Cloud deployment**: Production tag is attached only after validation in a new slot.
- [ ] **Performance**: Slow queries are identified with `log_min_duration_statement` and indexed.
- [ ] **Monitoring**: Processor height, lag, and logs are checked, not just the service health badge.

## Resources

- **Comprehensive navigation**: https://docs.sqd.dev/llms.txt (index) and https://docs.sqd.dev/llms-full.txt (full content)
- **Portal API quickstart**: https://docs.sqd.dev/en/portal/evm/quickstart
- **Pipes SDK quickstart**: https://docs.sqd.dev/en/sdk/pipes-sdk/evm/quickstart
- **Squid SDK quickstart**: https://docs.sqd.dev/en/sdk/squid-sdk/evm/quickstart
- **SQD Cloud deployment guide**: https://docs.sqd.dev/en/cloud/deployment-guide
- **Choosing your tool**: https://docs.sqd.dev/en/sdk/options-comparison
- **Best practices**: https://docs.sqd.dev/en/cloud/resources/best-practices
- **Troubleshooting**: https://docs.sqd.dev/en/cloud/troubleshooting

---

> For additional documentation and navigation, see: https://docs.sqd.dev/llms.txt