# Feedback

If you encounter incorrect, outdated, or confusing documentation on any page, submit feedback:

POST https://docs.sqd.dev/feedback

```json
{
  "path": "/current-page-path",
  "feedback": "Description of the issue"
}
```

Only submit feedback when you have something specific and actionable to report.

# Home
Source: https://docs.sqd.dev/en/home

Build blockchain data pipelines with SQD — Portal HTTP APIs across 225+ chains, Squid and Pipes SDKs in TypeScript, and managed Cloud hosting for indexers.

<div>
  <div>
    <div>
      <div>
        <span>
          The new blockchain data standard
        </span>
      </div>

      <h1>
        High-Throughput Blockchain Data Access. Finally, an Alternative to RPC
      </h1>

      <p>
        Portal is a high-performance HTTP API for querying blockchain data at
        scale. With native finality, real-time streaming, and deep historical
        access. Query arbitrary block ranges in a single request with automatic reorg handling.
      </p>

      <div>
        <div>
          <svg>
            <path />
          </svg>

          <div>
            <div>
              Arbitrary block ranges
            </div>

            <div>
              Query any block range in a single request, no manual pagination needed,
              even on high-throughput chains
            </div>
          </div>
        </div>

        <div>
          <svg>
            <path />
          </svg>

          <div>
            <div>
              Native finalization
            </div>

            <div>
              Automatic chain-specific finality & reorg handling. Skip the
              custom rollback logic.
            </div>
          </div>
        </div>

        <div>
          <svg>
            <path />
          </svg>

          <div>
            <div>
              Streaming responses
            </div>

            <div>
              Ingest high-throughput data ingestion with constant memory
              footprint
            </div>
          </div>
        </div>

        <div>
          <svg>
            <path />
          </svg>

          <div>
            <div>
              No vendor lock-in
            </div>

            <div>
              Self-host or use decentralized infrastructure. Run the same code
              anywhere.
            </div>
          </div>
        </div>
      </div>

      <div>
        <a href="/en/portal/evm/quickstart">
          Build with Portal API

          <svg>
            <path />
          </svg>
        </a>

        <a href="/en/sdk/overview">
          Use with SDK
        </a>
      </div>
    </div>
  </div>
</div>

<div />

<div>
  <div>
    <div>
      <h2>
        The Complete Data Stack
      </h2>

      <p>
        Four integrated products that work together to power your blockchain
        data infrastructure
      </p>

      <div>
        <span>
          Portal
        </span>

        <span>
          •
        </span>

        <span>
          SDK
        </span>

        <span>
          •
        </span>

        <span>
          Cloud
        </span>

        <span>
          •
        </span>

        <span>
          Network
        </span>
      </div>
    </div>

    <div>
      <a href="/en/portal/evm/quickstart">
        <img alt="Portal - HTTP API for blockchain data" />

        <div>
          <h3>
            Portal
          </h3>

          <p>
            HTTP API for raw blockchain data with arbitrary ranges, streaming,
            and finality handling
          </p>

          <span>
            Learn more →
          </span>
        </div>
      </a>

      <a href="/en/sdk/overview">
        <img alt="SDK - TypeScript libraries for data transformation" />

        <div>
          <h3>
            SDK
          </h3>

          <p>
            TypeScript libraries for data transformation, decoding, and
            persistence to any database
          </p>

          <span>
            Learn more →
          </span>
        </div>
      </a>

      <a href="/en/cloud/overview">
        <img alt="Cloud - Managed deployment platform" />

        <div>
          <h3>
            Cloud
          </h3>

          <p>
            Managed deployment platform with monitoring, scaling, and zero
            DevOps required
          </p>

          <span>
            Learn more →
          </span>
        </div>
      </a>

      <a href="/en/network/overview">
        <img alt="Network - Decentralized data lake" />

        <div>
          <h3>
            Network
          </h3>

          <p>
            Decentralized data lake powering Portal with 200+ networks and
            self-hosting options
          </p>

          <span>
            Learn more →
          </span>
        </div>
      </a>
    </div>
  </div>
</div>

<div />

<div>
  <div>
    <div>
      <h2>
        Query the blockchain instantly
      </h2>

      <p>
        SQD Portal provides an HTTP API for extracting blockchain data with
        arbitrary block ranges, streaming, and reorg handling
      </p>
    </div>

    <QueryInterface />

    <div>
      <a href="/en/portal/evm/quickstart">
        Try Portal API

        <svg>
          <path />
        </svg>
      </a>

      <a href="/en/portal/overview">
        Learn about Portal

        <svg>
          <path />
        </svg>
      </a>
    </div>
  </div>
</div>

<div />

<div>
  <div>
    <h2>
      200+ Networks Supported
    </h2>

    <p>
      Access data from all major blockchains through Portal
    </p>

    <CardGroup>
      <Card href="/en/data/evm/ethereum-mainnet">
        <img alt="Ethereum - EVM compatible chains" />
      </Card>

      <Card href="/en/data/solana/solana-mainnet">
        <img alt="Solana - High-performance blockchain" />
      </Card>

      <Card href="/en/data/evm/base-mainnet">
        <img alt="Base - Coinbase L2 on Ethereum" />
      </Card>

      <Card href="/en/data/evm/arbitrum-one">
        <img alt="Arbitrum - Ethereum L2 scaling solution" />
      </Card>

      <Card href="/en/data/evm/binance-mainnet">
        <img alt="Binance Smart Chain - BSC" />
      </Card>

      <Card href="/en/data/evm/plasma-mainnet">
        <img alt="Plasma - High-performance blockchain" />
      </Card>

      <Card href="/en/data/evm/monad-testnet">
        <img alt="Monad - Next-gen parallel EVM" />
      </Card>

      <Card href="/en/data/evm/hyperevm-mainnet">
        <img alt="HyperEVM - High-performance blockchain" />
      </Card>

      <Card href="/en/data/evm/megaeth-mainnet">
        <img alt="MegaETH - Ultra-fast real-time blockchain" />
      </Card>

      <Card href="/en/portal/hyperliquid/overview">
        <img alt="HyperCore - Hyperliquid high-performance blockchain" />
      </Card>

      <Card href="/en/portal/bitcoin/overview">
        <img alt="Bitcoin - The original blockchain" />
      </Card>

      <Card href="/en/data/evm">
        <img alt="View All Networks - 200+ supported blockchains" />
      </Card>
    </CardGroup>
  </div>
</div>

<div />

<div>
  <div>
    <div>
      <h2>
        Production Use Cases
      </h2>

      <p>
        Leading Web3 projects trust SQD to power their blockchain data infrastructure
      </p>
    </div>

    <div>
      <div>
        <div>
          <a href="https://blog.sqd.dev/case-study-how-sqd-supports-pancakeswap-with-large-scale-access-to-multichain-data/">
            <img alt="PancakeSwap case study" />

            <div>
              <div>
                <span>
                  DeFi
                </span>
              </div>

              <h3>
                PancakeSwap
              </h3>

              <p>
                Large-scale multichain data indexing for top DEX
              </p>
            </div>
          </a>

          <a href="https://blog.sqd.dev/how-gmx-accelerated-cross-chain-indexing-with-sqd/">
            <img alt="GMX case study" />

            <div>
              <div>
                <span>
                  DeFi
                </span>
              </div>

              <h3>
                GMX
              </h3>

              <p>
                Accelerated cross-chain indexing for decentralized exchange
              </p>
            </div>
          </a>

          <a href="https://blog.sqd.dev/case-study-chainsafe-using-subsquid-to-power-fast-evm-deployments/">
            <img alt="ChainSafe case study" />

            <div>
              <div>
                <span>
                  Gaming
                </span>
              </div>

              <h3>
                ChainSafe
              </h3>

              <p>
                Seamless game development across multiple chains
              </p>
            </div>
          </a>

          <a href="https://blog.sqd.dev/case-study-railgun-leveraging-sqd-trace-data-to-enable-privacy-pools/">
            <img alt="RAILGUN case study" />

            <div>
              <div>
                <span>
                  Privacy
                </span>
              </div>

              <h3>
                RAILGUN
              </h3>

              <p>
                Crosschain privacy with trace data indexing
              </p>
            </div>
          </a>

          <a href="https://blog.sqd.dev/case-study-how-hydration-uses-sqd-data-access-to-facilitate-its-defi-success/">
            <img alt="Hydration case study" />

            <div>
              <div>
                <span>
                  DeFi
                </span>
              </div>

              <h3>
                Hydration
              </h3>

              <p>
                Data-powered growth for liquidity protocol
              </p>
            </div>
          </a>

          <a href="https://blog.sqd.dev/how-polimec-uses-sqd-data-to-provide-compliant-defi/">
            <img alt="Polimec case study" />

            <div>
              <div>
                <span>
                  DeFi
                </span>
              </div>

              <h3>
                Polimec
              </h3>

              <p>
                Compliant DeFi with SQD data infrastructure
              </p>
            </div>
          </a>

          <a href="https://blog.sqd.dev/skybreach-case-study/">
            <img alt="Skybreach case study" />

            <div>
              <div>
                <span>
                  Gaming
                </span>
              </div>

              <h3>
                Skybreach
              </h3>

              <p>
                SQD data access powering blockchain games
              </p>
            </div>
          </a>

          <a href="https://blog.sqd.dev/how-fantasy-top-relies-on-sqd-to-provide-a-seamless-social-gaming-experience/">
            <img alt="Fantasy.Top case study" />

            <div>
              <div>
                <span>
                  Gaming
                </span>
              </div>

              <h3>
                Fantasy.Top
              </h3>

              <p>
                Seamless social gaming experience with real-time data
              </p>
            </div>
          </a>

          <a href="https://blog.sqd.dev/chillwhales-x-sqd-partner-case-study/">
            <img alt="Chillwhales case study" />

            <div>
              <div>
                <span>
                  Social
                </span>
              </div>

              <h3>
                Chillwhales
              </h3>

              <p>
                Expanding NFT browsing on LUKSO with SQD
              </p>
            </div>
          </a>

          <a href="https://blog.sqd.dev/levr-bet-x-sqd-partner-case-study/">
            <img alt="Levr.Bet case study" />

            <div>
              <div>
                <span>
                  Social
                </span>
              </div>

              <h3>
                Levr.Bet
              </h3>

              <p>
                Real-time sports betting on Monad powered by SQD
              </p>
            </div>
          </a>

          <a href="https://blog.sqd.dev/how-coolwallet-uses-sqd-to-secure-its-data-back-end/">
            <img alt="CoolWallet case study" />

            <div>
              <div>
                <span>
                  Social
                </span>
              </div>

              <h3>
                CoolWallet
              </h3>

              <p>
                Securing data back-end infrastructure with SQD
              </p>
            </div>
          </a>

          <a href="https://blog.sqd.dev/how-interface-uses-sqd-to-enable-people-to-explore-ethereum/">
            <img alt="Interface case study" />

            <div>
              <div>
                <span>
                  Social
                </span>
              </div>

              <h3>
                Interface
              </h3>

              <p>
                Enabling people to explore Ethereum with SQD
              </p>
            </div>
          </a>

          <a href="https://blog.sqd.dev/case-study-how-guru-network-powers-its-ai-processors-with-data-from-sqd/">
            <img alt="Guru Network case study" />

            <div>
              <div>
                <span>
                  AI
                </span>
              </div>

              <h3>
                Guru Network
              </h3>

              <p>
                Powering AI processors with blockchain data from SQD
              </p>
            </div>
          </a>
        </div>
      </div>
    </div>

    <div>
      <a href="/en/other/case-studies">
        View All Case Studies

        <svg>
          <path />
        </svg>
      </a>
    </div>
  </div>
</div>

<div />

<div>
  <div>
    <div>
      <h2>
        Get Started
      </h2>

      <p>
        Choose your path based on your needs, from quick data queries to full
        production deployments
      </p>
    </div>

    <div>
      <a href="/en/portal/evm/quickstart">
        <div>
          <div>
            <div>
              <svg>
                <path />
              </svg>
            </div>

            <h3>
              Query with Portal
            </h3>
          </div>

          <svg>
            <path />
          </svg>
        </div>

        <p>
          Extract blockchain data in minutes using Portal's HTTP API, no setup
          required, just HTTP requests
        </p>

        <div>
          <span>5 min</span>
          <span>•</span>
          <span>Beginner</span>
        </div>
      </a>

      <a href="/en/sdk/pipes-sdk/evm/prod-ready-pipe">
        <div>
          <div>
            <div>
              <svg>
                <path />
              </svg>
            </div>

            <h3>
              Build an Indexer
            </h3>
          </div>

          <svg>
            <path />
          </svg>
        </div>

        <p>
          Create type-safe indexers with TypeScript SDK and stream data to any
          database
        </p>

        <div>
          <span>15 min</span>
          <span>•</span>
          <span>Intermediate</span>
        </div>
      </a>

      <a href="/en/cloud/overview">
        <div>
          <div>
            <div>
              <svg>
                <path />
              </svg>
            </div>

            <h3>
              Deploy to Production
            </h3>
          </div>

          <svg>
            <path />
          </svg>
        </div>

        <p>
          Ship to SQD Cloud with managed infrastructure, monitoring, and zero
          DevOps
        </p>

        <div>
          <span>15 minutes</span>
          <span>•</span>
          <span>Intermediate</span>
        </div>
      </a>

      <a href="/en/ai/ai-development">
        <div>
          <div>
            <div>
              <svg>
                <path />
              </svg>
            </div>

            <h3>
              AI Development
            </h3>
          </div>

          <svg>
            <path />
          </svg>
        </div>

        <p>
          Build AI agents with SQD using MCP servers, agent skills, and LLM-optimized documentation
        </p>

        <div>
          <span>10 min</span>
          <span>•</span>
          <span>Intermediate</span>
        </div>
      </a>
    </div>

    <div>
      <p>
        Need help choosing? Check out our FAQ or join our community
      </p>

      <div>
        <a href="/en/other/faq">
          View FAQ
        </a>

        <a href="https://t.me/subsquid">
          <svg>
            <path />
          </svg>

          Join Telegram
        </a>
      </div>
    </div>
  </div>
</div>


# Getting Started with Portal
Source: https://docs.sqd.dev/en/portal/migration

Choose your SQD Portal migration path — migrate Cloud squids, develop locally with the Portal API, or self-host a Portal instance for full control.

Portal provides access to blockchain data from the permissionless SQD Network. Whether you're migrating existing squids or setting up Portal for the first time, choose the path that matches your deployment scenario.

## Choose Your Setup Path

Select the option that best describes your situation:

<CardGroup>
  <Card title="Cloud Migration" icon="cloud" href="/en/portal/migration/cloud-portal-evm">
    Migrate your Cloud squids from gateways to Portal for improved performance and stability.
  </Card>

  <Card title="SDK Migration (EVM)" icon="code" href="/en/sdk/migration/evm-gateway-to-portal">
    Migrate EVM SDK-based squids running locally or self-hosted from gateways to Portal.
  </Card>

  <Card title="SDK Migration (Solana)" icon="code" href="/en/sdk/migration/solana-gateway-to-portal">
    Migrate Solana SDK-based squids from gateways and RPC to Portal.
  </Card>

  <Card title="RPC Migration" icon="rotate" href="/en/sdk/migration/rpc-to-portal">
    Migrate your EVM indexer from RPC-based real-time data to Portal API.
  </Card>
</CardGroup>

<CardGroup>
  <Card title="Local Development" icon="laptop-code" href="/en/data/evm-local-setup/overview">
    Set up Portal locally for development and testing with your own devnet or testnet.
  </Card>

  <Card title="Self-Hosting" icon="server" href="/en/portal/self-hosting">
    Run your own Portal instance for complete control over your data infrastructure.
  </Card>
</CardGroup>

## Cloud Migration by Network Type

If you're using SQD Cloud, choose your network type:

<CardGroup>
  <Card title="EVM & Substrate" icon="ethereum" href="/en/portal/migration/cloud-portal-evm">
    Migrate squids running on EVM or Substrate networks to Cloud Portal.
  </Card>

  <Card title="Solana" icon="circle-nodes" href="/en/cloud/resources/migrate-to-portal-on-solana">
    Migrate Solana squids to Portal with real-time data support.
  </Card>
</CardGroup>

## Why Migrate to Portal?

<Tip>
  Migrating to Portal provides significant benefits over traditional gateways and centralized services.
</Tip>

* **Reduced reliance on centralized services:** The permissionless SQD Network consists of [over 2500](https://arbiscan.io/address/0x36e2b147db67e76ab67a4d07c293670ebefcae4e#readContract#F6) nodes [ran by independent operators](subsquid-network/worker)
* **Improved stability:** With a total capacity of roughly 2Pb, the permissionless SQD Network provides significant redundancy
* **Improved speed:** Portals use available bandwidth more effectively than gateways. Data fetching is 5-10 times faster in our tests
* **Future-proof:** All future development will focus on portals and the permissionless SQD Network

## Additional Resources

<CardGroup>
  <Card title="API Reference" icon="book-open" href="/en/portal/evm/overview">
    Complete Portal API documentation with examples and field definitions.
  </Card>

  <Card title="Portal Overview" icon="network-wired" href="/en/portal/overview">
    Learn about Portal architecture and core capabilities.
  </Card>
</CardGroup>


# Choosing Your SQD Tool
Source: https://docs.sqd.dev/en/sdk/options-comparison

Compare the Portal API, Pipes SDK, and Squid SDK to choose the right SQD blockchain indexing tool — features, complexity, and use-case fit.

This guide helps you understand the differences between SQD's three offerings and choose the right tool for your project.

## Overview: Three Ways to Access Blockchain Data

SQD provides three tools for working with blockchain data, each serving different use cases:

1. **Portal API** - Direct HTTP access to raw blockchain data
2. **Pipes SDK** - Lightweight streaming library for custom data pipelines
3. **Squid SDK** - Complete framework with built-in PostgreSQL and GraphQL

## How They Work Together

<Frame>
  <img alt="Portal data flow showing raw blockchain data being transformed by SDK and stored in your database" />
</Frame>

* **Portal** provides the raw blockchain data through HTTP API
* **SDKs** (Pipes and Squid) use Portal as their data source and add:
  * Event and transaction decoding
  * Type-safe data transformation
  * Batch processing capabilities
  * Database persistence
  * Real-time data ingestion

<Info>
  Both SDKs use Portal under the hood. When you use an SDK, you're still
  accessing blockchain data through Portal, the SDK just makes it easier to work
  with.
</Info>

## Raw API vs SDKs

Portal and SDKs serve different use cases in the blockchain data indexing pipeline.

### Portal: Direct Data Access

Portal provides raw blockchain data access through a simple HTTP API. You query specific blocks, transactions, logs, or traces and receive the raw data directly.

<CodeGroup>
  ```bash Portal Query Example theme={"system"}
  curl --compress -X POST 'https://portal.sqd.dev/datasets/ethereum-mainnet/stream' \
    -H 'Content-Type: application/json' \
    -d '{
      "type": "evm",
      "fromBlock": 18000000,
      "toBlock": 18001000,
      "fields": {
        "log": {
          "address": true,
          "topics": true,
          "data": true
        }
      },
      "logs": [{
        "address": ["0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48"]
      }]
    }'
  ```

  ```python Portal Python Example theme={"system"}
  import requests
  import json

  url = "https://portal.sqd.dev/datasets/ethereum-mainnet/stream"
  response = requests.post(url, json={
      "type": "evm",
      "fromBlock": 18000000,
      "toBlock": 18001000,
      "fields": {"log": {"address": True, "topics": True}},
      "logs": [{"address": ["0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48"]}]
  })

  for line in response.text.strip().split('\n'):
      data = json.loads(line)
      # Process raw blockchain data
  ```
</CodeGroup>

### SDKs: Data Transformation Frameworks

SDKs (Pipes and Squid) build on top of Portal to provide data transformation, decoding, and persistence capabilities.

<CodeGroup>
  ```typescript Pipes SDK Example theme={"system"}
  import { createTarget } from "@subsquid/pipes";
  import { evmPortalSource, EvmQueryBuilder } from "@subsquid/pipes/evm";

  const queryBuilder = new EvmQueryBuilder()
  .addFields({ block: { number: true, hash: true }, log: { data: true } })
  .addLog({
    request: { address: ["0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48"] },
    range: { from: 20000000, to: 20001000 }
  });

  const source = evmPortalSource({
  portal: "https://portal.sqd.dev/datasets/ethereum-mainnet",
  query: queryBuilder
  });

  const target = createTarget({
  write: async ({ read }) => {
    for await (const { data } of read()) {
      // Transform and persist to your database
    }
  }
  });

  await source.pipeTo(target);
  ```

  ```typescript Squid SDK Example theme={"system"}
  import { EvmBatchProcessor } from "@subsquid/evm-processor";
  import { TypeormDatabase } from "@subsquid/typeorm-store";

  const processor = new EvmBatchProcessor()
    .setGateway("https://v2.archive.subsquid.io/network/ethereum-mainnet")
    .addLog({
      address: ["0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48"],
      topic0: [usdcAbi.events.Transfer.topic]
    });

  const db = new TypeormDatabase();

  processor.run(db, async (ctx) => {
    const transfers: Transfer[] = [];
    for (let block of ctx.blocks) {
      for (let log of block.logs) {
        let { from, to, value } = usdcAbi.events.Transfer.decode(log);
        transfers.push(new Transfer({ from, to, value }));
      }
    }
    await ctx.store.insert(transfers);
  });
  ```
</CodeGroup>

### Detailed Comparison

| Aspect                  | Portal                                  | SDKs (Pipes/Squid)                       |
| ----------------------- | --------------------------------------- | ---------------------------------------- |
| **Purpose**             | Raw data retrieval                      | Data transformation and persistence      |
| **Output**              | Raw blockchain data (JSON)              | Processed data in databases/APIs         |
| **Performance**         | 10-50x faster than RPC                  | Additional processing overhead           |
| **Setup Complexity**    | Minimal (HTTP requests only)            | Project scaffolding, TypeScript required |
| **Data Decoding**       | Manual (you handle ABI decoding)        | Built-in ABI decoding                    |
| **Database**            | Bring your own                          | PostgreSQL (Squid), custom (Pipes)       |
| **GraphQL API**         | Not included                            | Auto-generated (Squid only)              |
| **Type Safety**         | Limited (JSON responses)                | Full TypeScript support                  |
| **State Management**    | Manual                                  | Built-in with fork handling              |
| **Best For**            | Analytics, data lakes, custom pipelines | dApps, APIs, traditional backends        |
| **Languages Supported** | Any (HTTP API)                          | TypeScript                               |
| **Learning Curve**      | Low (HTTP + your language)              | Medium (TypeScript + framework concepts) |

### When to Use Portal

<CardGroup>
  <Card title="Data Analytics" icon="chart-line">
    Stream raw data directly into analytics platforms like ClickHouse, BigQuery, or Snowflake
  </Card>

  <Card title="Custom Pipelines" icon="diagram-project">
    Build data pipelines in Python, Go, Rust, or any language with HTTP support
  </Card>

  <Card title="Data Lakes" icon="database">
    Populate data warehouses with raw blockchain data for long-term analysis
  </Card>

  <Card title="Rapid Prototyping" icon="bolt">
    Quick experiments without setting up a full indexing framework
  </Card>
</CardGroup>

### When to Use SDKs

<CardGroup>
  <Card title="dApp Backends" icon="server">
    Build GraphQL APIs that power decentralized applications
  </Card>

  <Card title="Complex Transformations" icon="gears">
    Decode events, track relationships, and maintain state across blocks
  </Card>

  <Card title="Production APIs" icon="cloud">
    Deploy scalable indexers with built-in monitoring and deployment tools
  </Card>

  <Card title="Type Safety" icon="shield">
    Leverage TypeScript for compile-time safety and better developer experience
  </Card>
</CardGroup>

## Pipes SDK vs Squid SDK

Both SDKs consume data from Portal but offer different levels of abstraction and flexibility.

### Architecture Differences

<Tabs>
  <Tab title="Pipes SDK">
    **Streaming Library Approach**

    Pipes SDK is a lightweight streaming library that gives you maximum flexibility:

    * You define the data flow using a pipe-and-target pattern
    * You bring your own database and persistence logic
    * You control exactly how data is processed and stored
    * No CLI tools or scaffolding - integrate into existing projects

    ```typescript theme={"system"}
    // Minimal setup - you control everything
    const source = evmPortalSource({ portal: url, query });
    const target = createTarget({ write: yourCustomLogic });
    await source.pipeTo(target);
    ```
  </Tab>

  <Tab title="Squid SDK">
    **Complete Framework Approach**

    Squid SDK is a full-featured framework with built-in conventions:

    * CLI tools for project scaffolding and management
    * PostgreSQL database with automatic migrations
    * Auto-generated GraphQL API from schema
    * Built-in deployment tools for SQD Cloud

    ```typescript theme={"system"}
    // Opinionated structure with built-in features
    const processor = new EvmBatchProcessor()
      .setGateway(url)
      .addLog({ ... });

    const db = new TypeormDatabase();
    processor.run(db, async (ctx) => {
      // Framework handles state, rollbacks, persistence
    });
    ```
  </Tab>
</Tabs>

### Feature Comparison Matrix

| Feature               | Pipes SDK                     | Squid SDK                             |
| --------------------- | ----------------------------- | ------------------------------------- |
| **Type**              | Streaming library             | Complete framework                    |
| **Installation**      | `npm install @subsquid/pipes` | `npm i -g @subsquid/cli`              |
| **Project Setup**     | Manual integration            | CLI scaffolding (`sqd init`)          |
| **Database**          | Bring your own (any)          | PostgreSQL (built-in)                 |
| **GraphQL API**       | Not included                  | Auto-generated from schema            |
| **Migrations**        | You implement                 | Auto-generated                        |
| **Type Generation**   | Manual                        | Auto-generated from ABI and schema    |
| **Event Decoding**    | Built-in codec                | Built-in codec                        |
| **State Rollbacks**   | You implement                 | Built-in fork handling                |
| **CLI Tools**         | None                          | Full CLI suite                        |
| **Cloud Deployment**  | Manual                        | `sqd deploy` command                  |
| **Local Development** | Standard Node.js              | Docker Compose included               |
| **Data Targets**      | Custom (you implement)        | TypeORM, BigQuery, CSV, JSON, Parquet |
| **Real-time Support** | Yes (streaming)               | Yes (unfinal blocks)                  |
| **Best For**          | Custom pipelines, flexibility | Full-stack dApps, rapid development   |
| **Learning Curve**    | Lower (simpler API)           | Higher (more concepts)                |
| **Bundle Size**       | Smaller                       | Larger (includes framework)           |
| **Customization**     | Maximum flexibility           | Opinionated but extensible            |

### Decision Matrix

<Tabs>
  <Tab title="Choose Pipes SDK">
    You should use Pipes SDK if:

    * ✅ You want maximum flexibility and control
    * ✅ You're integrating into an existing codebase
    * ✅ You want to use a specific database (MongoDB, ClickHouse, etc.)
    * ✅ You don't need a GraphQL API
    * ✅ You prefer lightweight dependencies
    * ✅ You want to build custom data pipelines
    * ✅ You're experienced with database design and management

    **Example use cases:**

    * Real-time dashboards with custom databases
    * Data pipelines feeding multiple systems
    * Microservices that need blockchain data
    * Custom analytics engines
  </Tab>

  <Tab title="Choose Squid SDK">
    You should use Squid SDK if:

    * ✅ You want a complete solution with minimal setup
    * ✅ You need a GraphQL API
    * ✅ You're building a dApp backend
    * ✅ You want automatic type generation
    * ✅ You prefer PostgreSQL
    * ✅ You need built-in deployment tools
    * ✅ You want comprehensive documentation and examples

    **Example use cases:**

    * dApp backends with GraphQL APIs
    * NFT marketplaces
    * DeFi analytics platforms
    * Governance tools
  </Tab>
</Tabs>

## Why We Built Pipes SDK

Pipes SDK represents a fundamental rethinking of how blockchain indexing SDKs should work, informed by years of experience with Squid SDK 1.0 and feedback from the developer community.

### Challenges with Squid SDK 1.0

While Squid SDK 1.0 successfully served many production use cases, we identified several architectural limitations that hindered developer experience and community adoption:

<AccordionGroup>
  <Accordion title="Tight Coupling and Complexity">
    Although designed with modularity in mind, many components became tightly coupled in practice. This made it extremely difficult to replace or extend parts of the system without understanding the entire SDK's internal workings, creating a steep learning curve for developers attempting customization.
  </Accordion>

  <Accordion title="Lack of Documentation and Tests">
    Insufficient documentation and internal test coverage made introducing changes
    risky and time-consuming. Developers faced significant friction when trying to
    extend or modify the SDK's behavior.
  </Accordion>

  <Accordion title="Standalone Design Limitations">
    The SDK was built to function as a standalone process, making it challenging
    to embed into existing applications or workflows. This architectural decision
    limited flexibility for teams wanting to integrate blockchain indexing into
    their existing codebases.
  </Accordion>

  <Accordion title="Rigid and Opinionated Approach">
    The framework enforced a very opinionated development style with limited room
    for customization. Developers often found themselves constrained by the SDK's
    assumptions rather than supported by its abstractions.
  </Accordion>

  <Accordion title="No Client-Side Testing Framework">
    The absence of a built-in testing framework for client applications increased
    friction for developers who needed to validate their integrations and business
    logic.
  </Accordion>

  <Accordion title="Limited Community Adoption">
    These combined limitations significantly hindered external contributions, resulting in minimal meaningful engagement from the developer community.
  </Accordion>
</AccordionGroup>

### Pipes SDK Design Goals

Pipes SDK was built from the ground up to address these challenges and provide a modern, flexible foundation for blockchain data indexing:

<CardGroup>
  <Card title="Focus on Business Logic" icon="code">
    Enable developers to concentrate on application-specific business logic rather than dealing with low-level blockchain implementation details.
  </Card>

  <Card title="Code Reusability" icon="recycle">
    Promote code sharing and maintainability by extracting common functionality
    into reusable, composable packages.
  </Card>

  <Card title="Built-in Extensions" icon="puzzle-piece">
    Provide ready-to-use extensions for common tasks including Portal caching
    layers, factory contract handling, and database integrations (PostgreSQL,
    ClickHouse, Kafka).
  </Card>

  <Card title="Community-Friendly Architecture" icon="users">
    Design a plugin system that makes it easy for developers to create and share
    extensions, fostering a vibrant ecosystem of community contributions.
  </Card>

  <Card title="Better Observability" icon="chart-line">
    Include first-class support for custom metrics, profiling tools, and
    centralized logging services to help developers monitor and optimize their
    indexers.
  </Card>

  <Card title="Modern Runtime Support" icon="bolt">
    Ensure compatibility with modern JavaScript runtimes like Bun for faster development and improved performance.
  </Card>
</CardGroup>

<Info>
  Pipes SDK maintains backward compatibility with Squid SDK's data sources
  (Portal) while offering a completely redesigned developer experience focused
  on flexibility, composability, and ease of use.
</Info>

## Key Benefits by Tool

### Portal API Benefits

* **Language Agnostic** - Use any programming language with HTTP support
* **Minimal Setup** - Start querying in minutes with simple HTTP requests
* **Maximum Control** - Full control over data processing and storage
* **High Performance** - 10-50x faster than RPC for historical data

### Pipes SDK Benefits

* **Lightweight** - Minimal dependencies and bundle size
* **Flexible** - Use any database or data target
* **Streaming** - Built-in backpressure handling and memory efficiency
* **Type-Safe** - Full TypeScript support with auto-generated types

### Squid SDK Benefits

* **Complete Solution** - Everything you need in one framework
* **Rapid Development** - CLI tools and scaffolding for quick starts
* **Auto-Generated APIs** - GraphQL API generated from your schema
* **Production Ready** - Built-in deployment, monitoring, and scaling tools
* **Comprehensive Tooling** - Hot reload, migrations, type generation

## Next Steps

Now that you understand the differences, choose your path:

<CardGroup>
  <Card title="Try Portal" icon="network-wired" href="/en/portal/evm/quickstart">
    Start querying raw blockchain data in minutes
  </Card>

  <Card title="Try Pipes SDK" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Build a lightweight data pipeline
  </Card>

  <Card title="Try Squid SDK" icon="server" href="/en/sdk/squid-sdk/quickstart">
    Create a full-featured indexer with GraphQL API
  </Card>
</CardGroup>


# SQD SDK Overview
Source: https://docs.sqd.dev/en/sdk/overview

Choose how to access blockchain data with SQD — direct HTTP queries via the Portal API, lightweight Pipes SDK pipelines, or full Squid SDK GraphQL stacks.

## Choose Your Data Access Method

SQD offers three ways to access blockchain data. Each is designed for different use cases and developer preferences.

<CardGroup>
  <Card title="Portal API" icon="network-wired" href="/en/portal/evm/quickstart">
    Direct HTTP access to raw blockchain data. Query using simple HTTP requests in any programming language.
  </Card>

  <Card title="Pipes SDK" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Lightweight TypeScript streaming library for custom data pipelines with
    automatic decoding and type safety.
  </Card>

  <Card title="Squid SDK" icon="server" href="/en/sdk/squid-sdk/quickstart">
    Complete TypeScript framework with built-in PostgreSQL and auto-generated GraphQL API.
  </Card>
</CardGroup>

## Quick Comparison

<Tabs>
  <Tab title="Portal API">
    **What it is:** Language-agnostic HTTP API for raw blockchain data

    **Best for:** Analytics, data lakes, custom pipelines in any language

    **Key advantage:** 10-50x faster than RPC with minimal setup
  </Tab>

  <Tab title="Pipes SDK">
    **What it is:** TypeScript streaming library with flexible architecture

    **Best for:** Custom pipelines, existing codebases, non-PostgreSQL databases

    **Key advantage:** Full control over data flow with minimal dependencies
  </Tab>

  <Tab title="Squid SDK">
    **What it is:** Full-stack TypeScript framework with database and API

    **Best for:** dApp backends, GraphQL APIs, rapid development

    **Key advantage:** Built-in PostgreSQL + auto-generated GraphQL API
  </Tab>
</Tabs>

## Next Steps

<CardGroup>
  <Card title="Compare Options" icon="scale-balanced" href="/en/sdk/options-comparison">
    See feature matrices, code examples, and technical trade-offs between Portal API, Pipes SDK, and Squid SDK.
  </Card>

  <Card title="Start Building" icon="code" href="/en/portal/evm/quickstart">
    Follow the quickstart to set up Portal API and run your first query.
  </Card>
</CardGroup>

<Info>
  **Enterprise Custom Development**

  For enterprise clients requiring custom indexer development, please contact our team to discuss your specific requirements. [Schedule a Consultation →](https://calendly.com/t-tyrie-subsquid/30min)
</Info>


# Factory transformers
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/evm/guides/advanced-topics/factory-transformers

Index dynamic contracts with the factory pattern

Use the factory pattern when you need to index events from contracts that are deployed dynamically by a known factory contract — for example, Uniswap V3 pools created by the `UniswapV3Factory`.

<Note>
  The examples below use typegen-generated ABI modules. See [Specifying events](../basic-development/handling-events#specifying-events) for how to generate them from a JSON ABI.
</Note>

## Basic factory

Track events from contracts created by a factory. The `factory()` helper discovers child contracts from the factory's creation events and maintains the address list in a local SQLite database.

```ts theme={"system"}
import { evmPortalStream, evmDecoder, factory, factorySqliteDatabase } from "@subsquid/pipes/evm";
import { createTarget } from "@subsquid/pipes";
import * as factoryAbi from "./abi/uniswap-v3-factory";
import * as poolAbi from "./abi/uniswap-v3-pool";

await evmPortalStream({
  portal: "https://portal.sqd.dev/datasets/ethereum-mainnet",
  outputs: evmDecoder({
    range: { from: 12369621 },
    contracts: factory({
      address: "0x1f98431c8ad98523631ae4a59f267346ea31f984",
      event: factoryAbi.events.PoolCreated,
      parameter: "pool",
      database: factorySqliteDatabase({ path: "./uniswap-v3-pools.sqlite" }),
    }),
    events: { swap: poolAbi.events.Swap },
  }),
}).pipeTo(createTarget({
  write: async ({ logger, read }) => {
    for await (const { data } of read()) {
      logger.info(`Parsed ${data.swap.length} swaps`);
    }
  },
}));
```

## Filtering factory events

To narrow which child contracts are tracked, pass an `event` object with a `params` field. Only creation events matching the specified parameter values are stored — unmatched contracts are ignored at both the portal and the local database level.

```ts theme={"system"}
contracts: factory({
  address: "0x1f98431c8ad98523631ae4a59f267346ea31f984",
  event: {
    event: factoryAbi.events.PoolCreated,
    params: {
      token0: "0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2", // WETH
    },
  },
  parameter: "pool",
  database: factorySqliteDatabase({ path: "./uniswap-v3-weth-pools.sqlite" }),
})
```

**Filter rules:**

* Only **indexed parameters** can be used for filtering.
* Multiple parameters are combined with AND logic.
* Passing an **array** of values for a parameter matches any of them (OR logic).
* Address matching is case-insensitive.

<Expandable title="Full example: filter by token0 with WETH pools">
  ```ts theme={"system"}
  import { evmPortalStream, evmDecoder, factory, factorySqliteDatabase } from "@subsquid/pipes/evm";
  import { createTarget } from "@subsquid/pipes";
  import * as factoryAbi from "./abi/uniswap-v3-factory";
  import * as poolAbi from "./abi/uniswap-v3-pool";

  const WETH = "0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2";

  await evmPortalStream({
    portal: "https://portal.sqd.dev/datasets/ethereum-mainnet",
    outputs: evmDecoder({
      range: { from: 12369621 },
      contracts: factory({
        address: "0x1f98431c8ad98523631ae4a59f267346ea31f984",
        event: {
          event: factoryAbi.events.PoolCreated,
          params: { token0: WETH },
        },
        parameter: "pool",
        database: factorySqliteDatabase({ path: "./uniswap-v3-weth-pools.sqlite" }),
      }),
      events: { swap: poolAbi.events.Swap },
    }),
  }).pipeTo(createTarget({
    write: async ({ logger, read }) => {
      for await (const { data } of read()) {
        logger.info(`Parsed ${data.swap.length} swaps from WETH pools`);
      }
    },
  }));
  ```
</Expandable>

## Including factory event data

`DecodedEvent<T, F>` carries a `.factory` field with the creation event. Use it when you need to include factory context (e.g. pool token addresses) alongside each decoded event.

<Expandable title="Full example: access factory event metadata">
  ```ts theme={"system"}
  import {
    evmPortalStream, evmDecoder, factory, DecodedEvent, factorySqliteDatabase,
  } from "@subsquid/pipes/evm";
  import { createTarget } from "@subsquid/pipes";
  import * as factoryAbi from "./abi/uniswap-v3-factory";
  import * as poolAbi from "./abi/uniswap-v3-pool";

  function addFactoryMetadata<T, F>(event: DecodedEvent<T, F>) {
    return {
      ...event.event,
      blockNumber: event.block.number,
      factoryEvent: event.factory?.event,
    };
  }

  const decoder = evmDecoder({
    range: { from: 12369621 },
    contracts: factory({
      address: "0x1f98431c8ad98523631ae4a59f267346ea31f984",
      event: factoryAbi.events.PoolCreated,
      parameter: "pool",
      database: factorySqliteDatabase({ path: "./uniswap-v3-pools.sqlite" }),
    }),
    events: { swap: poolAbi.events.Swap, mint: poolAbi.events.Mint },
  }).pipe(({ swap, mint }) => ({
    swap: swap.map(addFactoryMetadata),
    mint: mint.map(addFactoryMetadata),
  }));

  await evmPortalStream({
    portal: "https://portal.sqd.dev/datasets/ethereum-mainnet",
    outputs: decoder,
  }).pipeTo(createTarget({
    write: async ({ logger, read }) => {
      for await (const { data } of read()) {
        for (const s of data.swap) {
          logger.info({
            pool: s.factoryEvent?.pool,
            token0: s.factoryEvent?.token0,
            token1: s.factoryEvent?.token1,
            amount0: s.amount0.toString(),
            amount1: s.amount1.toString(),
          });
        }
      }
    },
  }));
  ```
</Expandable>

## Multiple factories

Pass separate `evmDecoder` outputs to track contracts from different factory addresses in a single pipeline.

<Expandable title="Full example: Uniswap V2 and V3 in one pipeline">
  ```ts theme={"system"}
  import {
    evmPortalStream, evmDecoder, factory, factorySqliteDatabase,
  } from "@subsquid/pipes/evm";
  import { createTarget } from "@subsquid/pipes";
  import * as uniswapV3FactoryAbi from "./abi/uniswap-v3-factory";
  import * as uniswapV2FactoryAbi from "./abi/uniswap-v2-factory";
  import * as poolAbi from "./abi/pool";

  await evmPortalStream({
    portal: "https://portal.sqd.dev/datasets/ethereum-mainnet",
    outputs: {
      v3: evmDecoder({
        range: { from: 12369621 },
        contracts: factory({
          address: "0x1f98431c8ad98523631ae4a59f267346ea31f984",
          event: uniswapV3FactoryAbi.events.PoolCreated,
          parameter: "pool",
          database: factorySqliteDatabase({ path: "./v3-pools.sqlite" }),
        }),
        events: { swap: poolAbi.events.Swap },
      }),
      v2: evmDecoder({
        range: { from: 10000835 },
        contracts: factory({
          address: "0x5C69bEe701ef814a2B6a3EDD4B1652CB9cc5aA6f",
          event: uniswapV2FactoryAbi.events.PairCreated,
          parameter: "pair",
          database: factorySqliteDatabase({ path: "./v2-pairs.sqlite" }),
        }),
        events: { swap: poolAbi.events.Swap },
      }),
    },
  }).pipeTo(createTarget({
    write: async ({ logger, read }) => {
      for await (const { data } of read()) {
        logger.info({ v3Swaps: data.v3.swap.length, v2Swaps: data.v2.swap.length });
      }
    },
  }));
  ```
</Expandable>

## Pre-indexing factory <Badge>Experimental</Badge>

Pre-populate the factory database before the main pipeline to ensure all historical child contracts are known before live indexing begins.

<Warning>
  This is an experimental feature. The pre-indexing request is limited and this approach won't work for thousands of addresses.
</Warning>

<Expandable title="Full example: pre-index all pools then run main pipeline">
  ```ts theme={"system"}
  import {
    evmPortalStream, evmDecoder, factory, factorySqliteDatabase,
  } from "@subsquid/pipes/evm";
  import { createTarget } from "@subsquid/pipes";
  import * as factoryAbi from "./abi/uniswap-v3-factory";
  import * as poolAbi from "./abi/uniswap-v3-pool";

  const factoryDb = factorySqliteDatabase({ path: "./uniswap-v3-pools.sqlite" });

  // Step 1: pre-index historical pool creations
  await evmPortalStream({
    portal: "https://portal.sqd.dev/datasets/ethereum-mainnet",
    outputs: evmDecoder({
      range: { from: 12369621, to: 20000000 },
      contracts: ["0x1f98431c8ad98523631ae4a59f267346ea31f984"],
      events: { poolCreated: factoryAbi.events.PoolCreated },
    }),
  }).pipeTo(createTarget({
    write: async ({ logger, read }) => {
      for await (const { data } of read()) {
        for (const event of data.poolCreated) {
          await factoryDb.add(event.event.pool);
          logger.info(`Added pool: ${event.event.pool}`);
        }
      }
    },
  }));

  // Step 2: run main pipeline with populated factory database
  await evmPortalStream({
    portal: "https://portal.sqd.dev/datasets/ethereum-mainnet",
    outputs: evmDecoder({
      range: { from: 20000000 },
      contracts: factory({
        address: "0x1f98431c8ad98523631ae4a59f267346ea31f984",
        event: factoryAbi.events.PoolCreated,
        parameter: "pool",
        database: factoryDb,
      }),
      events: { swap: poolAbi.events.Swap },
    }),
  }).pipeTo(createTarget({
    write: async ({ logger, read }) => {
      for await (const { data } of read()) {
        logger.info(`Processed ${data.swap.length} swaps`);
      }
    },
  }));
  ```
</Expandable>


# Data freshness monitoring
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/evm/guides/advanced-topics/latency-monitoring

Compare Portal data freshness against external RPC providers

The `evmRpcLatencyWatcher` subscribes to RPC endpoints via WebSocket and measures when blocks arrive at the Portal versus when they appear at the RPC endpoints.

<Warning>
  The measured values include client-side network data freshness. For RPC endpoints, only the arrival time of blocks is measured—this does not capture the node’s internal processing or response latency if queried directly. Results represent end-to-end delays as experienced by the client, not pure Portal or RPC processing performance.
</Warning>

```ts theme={"system"}
import { formatBlock } from "@subsquid/pipes";
import { evmPortalSource, evmRpcLatencyWatcher } from "@subsquid/pipes/evm";
import { metricsServer } from "@subsquid/pipes/metrics/node";

async function main() {
  const stream = evmPortalSource({
    portal: "https://portal.sqd.dev/datasets/base-mainnet",
    outputs: evmDecoder({ range: { from: 'latest' }, events: {} }),
    metrics: metricsServer({ port: 9090 }),
  }).pipe(
    evmRpcLatencyWatcher({
      rpcUrl: ["https://base.drpc.org", "https://base-rpc.publicnode.com"],
    }).pipe((data, { metrics }) => {
      if (!data) return;

      // Update Prometheus metrics for each RPC endpoint
      for (const rpc of data.rpc) {
        metrics
          .gauge({
            name: "rpc_latency_ms",
            help: "RPC Latency in ms",
            labelNames: ["url"],
          })
          .set({ url: rpc.url }, rpc.portalDelayMs);
      }

      return data;
    })
  );

  for await (const { data } of stream) {
    if (!data) continue;

    console.log(`Block: ${formatBlock(data.number)} / ${data.timestamp}`);
    console.table(data.rpc);
  }
}

void main()
```

## Output format

Data freshness data includes:

* `url`: RPC endpoint URL
* `receivedAt`: Timestamp when the RPC endpoint received the block
* `portalDelayMs`: Milliseconds between RPC arrival and Portal availability

```
Block: 36,046,611 / Fri Sep 26 2025 14:29:29 GMT+0400
┌───┬─────────────────────────────────┬──────────────────────────┬───────────────┐
│   │ url                             │ receivedAt               │ portalDelayMs │
├───┼─────────────────────────────────┼──────────────────────────┼───────────────┤
│ 0 │ https://base.drpc.org           │ 2025-09-26T10:29:29.134Z │ 646           │
│ 1 │ https://base-rpc.publicnode.com │ 2025-09-26T10:29:29.130Z │ 642           │
└───┴─────────────────────────────────┴──────────────────────────┴───────────────┘
```

## Custom metrics integration

Export data freshness metrics to Prometheus or other monitoring systems.

```ts expandable theme={"system"}
import { solanaPortalSource, solanaRpcLatencyWatcher } from '@subsquid/pipes/solana'
import { Registry, Counter } from 'prom-client'

const registry = new Registry()
const latencyCounter = new Counter({
  name: 'portal_latency_ms_total',
  help: 'Total Portal latency in milliseconds',
  registers: [registry],
})

const stream = solanaPortalSource({
  portal: 'https://portal.sqd.dev/datasets/solana-mainnet',
  outputs: new SolanaQueryBuilder()
    .addFields({ block: { number: true, hash: true, timestamp: true } })
    .includeAllBlocks({ from: 'latest' })
    .build(),
}).pipe(
  solanaRpcLatencyWatcher({
    rpcUrl: ['https://api.mainnet-beta.solana.com'],
  }).pipe({
    transform: (data) => {
      if (!data) return

      // Record latency metrics
      for (const rpc of data.rpc) {
        latencyCounter.inc(rpc.portalDelayMs)
      }

      return data
    },
  }),
)

// Expose metrics endpoint
import http from 'http'
const server = http.createServer(async (req, res) => {
  if (req.url === '/metrics') {
    res.setHeader('Content-Type', registry.contentType)
    res.end(await registry.metrics())
  } else {
    res.end('OK')
  }
})
server.listen(9090)

for await (const { data } of stream) {
  // Process blocks
}
```


# Logging
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/evm/guides/advanced-topics/logging

Customize logging with Pino-compatible transports

[evmPortalSource()](../../reference/basic-components/source) accepts a Pino-compatible `logger`, allowing you to integrate custom log transports and send logs to external services like GCP Cloud Logging, Sentry, or any other Pino-compatible destination.

## Basic custom logger

Pass a custom logger to the source to configure logging for your entire pipeline.

```ts theme={"system"}
import { createTarget } from "@subsquid/pipes";
import { evmPortalSource } from "@subsquid/pipes/evm";
import pino from "pino";

async function main() {
  const transport = pino.transport({
    target: "pino-pretty",
    options: {
      colorize: true,
      translateTime: "HH:MM:ss",
    },
  });

  const source = evmPortalSource({
    portal: "https://portal.sqd.dev/datasets/ethereum-mainnet",
    logger: pino(transport),
  });

  const target = createTarget({
    write: async ({ logger, read }) => {
      for await (const { data } of read()) {
        logger.info({ count: data.length }, "Processed batch");
      }
    },
  });

  await source.pipeTo(target);
}

void main()
```

## Integration with cloud services

You can use any Pino transport to send logs to cloud services. Pass the configured logger to the source.

<Tabs>
  <Tab title="GCP Cloud Logging">
    ```ts theme={"system"}
    import { createTarget } from "@subsquid/pipes";
    import { evmPortalSource } from "@subsquid/pipes/evm";
    import pino from "pino";

    async function main() {
      const transport = pino.transport({
        target: "@google-cloud/logging-pino",
        options: {
          projectId: "your-project-id",
          logName: "pipes-indexer",
        },
      });

      const source = evmPortalSource({
        portal: "https://portal.sqd.dev/datasets/ethereum-mainnet",
        logger: pino(transport),
      });

      const target = createTarget({
        write: async ({ logger, read }) => {
          for await (const { data } of read()) {
            logger.info(
              {
                blocksProcessed: data.blocks?.length,
                eventsCount: data.transfer?.length,
              },
              "Batch processed"
            );
          }
        },
      });

      await source.pipeTo(target);
    }

    void main()
    ```
  </Tab>

  <Tab title="Sentry">
    ```ts theme={"system"}
    import { createTarget } from "@subsquid/pipes";
    import { evmPortalSource } from "@subsquid/pipes/evm";
    import pino from "pino";

    async function main() {
      const transport = pino.transport({
        target: "pino-sentry-transport",
        options: {
          sentry: {
            dsn: process.env.SENTRY_DSN,
            environment: "production",
          },
          level: "error", // Only send errors to Sentry
        },
      });

      const source = evmPortalSource({
        portal: "https://portal.sqd.dev/datasets/ethereum-mainnet",
        logger: pino(transport),
      });

      const target = createTarget({
        write: async ({ logger, read }) => {
          for await (const { data } of read()) {
            try {
              await processData(data);
              logger.info({ count: data.length }, "Batch processed");
            } catch (error) {
              logger.error({ error, data }, "Failed to process batch");
            }
          }
        },
      });

      await source.pipeTo(target);
    }

    void main()
    ```
  </Tab>

  <Tab title="Multiple Transports">
    ```ts theme={"system"}
    import { createTarget } from "@subsquid/pipes";
    import { evmPortalSource } from "@subsquid/pipes/evm";
    import pino from "pino";

    async function main() {
      const transport = pino.transport({
        targets: [
          {
            target: "pino-pretty",
            options: { colorize: true },
            level: "info",
          },
          {
            target: "@google-cloud/logging-pino",
            options: { projectId: "your-project-id" },
            level: "info",
          },
          {
            target: "pino-sentry-transport",
            options: { sentry: { dsn: process.env.SENTRY_DSN } },
            level: "error",
          },
        ],
      });

      const source = evmPortalSource({
        portal: "https://portal.sqd.dev/datasets/ethereum-mainnet",
        logger: pino(transport),
      });

      const target = createTarget({
        write: async ({ logger, read }) => {
          for await (const { data } of read()) {
            logger.info({ count: data.length }, "Processed batch");
          }
        },
      });

      await source.pipeTo(target);
    }

    void main()
    ```
  </Tab>
</Tabs>

<Tip>
  The `ctx.logger` in transformers and targets is the same logger instance passed to the source. Configure logging at the source level, then use `ctx.logger` throughout your pipeline for consistent logging.
</Tip>


# Metrics
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/evm/guides/advanced-topics/metrics

Track custom Prometheus metrics in EVM pipes

Pipes SDK can expose a Prometheus-compatible metrics server. You can customize it to add counters, gauges, histograms, and summaries.

```ts theme={"system"}
import { commonAbis, evmDecoder, evmPortalStream } from "@subsquid/pipes/evm";
import { metricsServer } from "@subsquid/pipes/metrics/node";

async function main() {
  const stream = evmPortalStream({
    id: 'evm-decoder',
    portal: 'https://portal.sqd.dev/datasets/ethereum-mainnet',
    outputs: evmDecoder({
      range: {
        from: 'latest',
      },
      events: {
        transfers: commonAbis.erc20.events.Transfer,
      },
    }),
    metrics: metricsServer({
      port: 9090
    }), // equivalent to metricsServer(), as 9090 is the default port
  })

  for await (const { data, ctx } of stream) {
    // Add custom counter metric
    ctx.metrics
      .counter({
        name: "my_transfers_counter",
        help: "Number of processed transactions",
      })
      .inc(data.transfers.length);
  }
}

void main()
```

<Check>
  Access metrics at `http://localhost:9090/metrics` to verify they're being exposed correctly.

  ```
  # HELP my_transfers_counter Number of processed transactions
  # TYPE my_transfers_counter counter
  my_transfers_counter 218598
  ```
</Check>

<Tip>
  Use Grafana dashboards to visualize block processing rate, error rates, and latency trends from your Prometheus metrics.
</Tip>

## Available metric types

You can create different types of Prometheus metrics:

```ts theme={"system"}
for await (const { data, ctx } of stream) {
  // Counter - monotonically increasing value
  ctx.metrics.counter({ name: "events_total", help: "Total events" }).inc();

  // Gauge - value that can go up or down
  ctx.metrics
    .gauge({ name: "queue_size", help: "Current queue size" })
    .set(queueSize);

  // Histogram - observations with configurable buckets
  ctx.metrics
    .histogram({ name: "batch_size", help: "Batch size distribution" })
    .observe(data.transfers.length);
}
```

<Info>
  Expose metrics with `metricsServer()` on your source, then visualize them with [Pipes UI](../basic-development/pipes-ui).
</Info>

See the [Profiling](./profiling) guide for the built-in per-batch profiler exposed on the same metrics endpoint.


# Profiling
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/evm/guides/advanced-topics/profiling

Measure where time is spent in a EVM pipe

Pipes SDK ships a built-in per-batch profiler. It records how long each part of the pipeline takes. When a [`metricsServer()`](../../reference/utility-components/metrics-server) is attached to the source, the profiler output is served as JSON at `http://localhost:<metrics_port>/profiler` and rendered live in [Pipes UI](../basic-development/pipes-ui).

## Enabling the profiler

The profiler is **on by default** when `process.env.NODE_ENV !== 'production'` and **off** otherwise. Override explicitly with `profiler: true` or `profiler: false` on the source.

```ts theme={"system"}
evmPortalStream({
  portal: 'https://portal.sqd.dev/datasets/ethereum-mainnet',
  outputs: /* ... */,
  metrics: metricsServer({ port: 9090 }),
  profiler: true,   // force on — useful in production
})
```

## Interpreting the output

The pipeline is represented as a tree. Each node reports how long was spent in that stage of a batch. A typical tree looks like:

```
batch
├── fetch data
├── apply transformers
│   ├── track progress
│   └── EVM decoder
├── clickhouse
│   ├── data handler
│   ├── insert cursor
│   └── cleanup cursors
└── metrics processing
```

## Custom spans

Wrap any code in your target or transformer to get it to appear as a tree node. `ctx.profiler.start()` is a no-op when the profiler is disabled, so the instrumentation is safe to leave in place.

```ts theme={"system"}
onData: async ({ data, ctx }) => {
  const span = ctx.profiler.start('my measure')
  await myDataProcessing(data)
  span.end()
},
```

The named span appears under its parent node in the tree:

```
batch
...
├── clickhouse
│   ├── data handler
│   │   └── my measure
...
```


# Railway deployment
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/evm/guides/advanced-topics/railway-deployment

Deploying a Pipes SDK project to Railway

This guide walks you through deploying a Pipes SDK indexer to [Railway](https://railway.app). You'll end up with four services running together: your indexer, a database (PostgreSQL or ClickHouse), and the Pipe UI dashboard.

## Prerequisites

* A [Railway account](https://railway.app)
* Your project pushed to a **public or private GitHub repository**
* Your project built with `pipes init` (which generates a `Dockerfile` and `docker-compose.yaml`)

***

## Option A — Drag & Drop via the Railway Dashboard

The quickest way to get started is to drop your `docker-compose.yaml` directly onto the Railway project canvas.

### Step 1 — Create a new project on Railway

1. Go to [railway.app/new](https://railway.app/new) and click **Empty Project**.

### Step 2 — Drag and drop your `docker-compose.yaml`

1. Open your project canvas.
2. Drag the `docker-compose.yaml` file from your project root and drop it anywhere on the canvas.

Railway parses the file and creates a service for each entry. For a typical Pipes SDK project this produces:

| Service                    | Image / Source                     |
| -------------------------- | ---------------------------------- |
| Your indexer               | Built from your local `Dockerfile` |
| `postgres` or `clickhouse` | Official Docker images             |

### Step 3 — Link the indexer service to your GitHub repository

1. Click the indexer service card on the canvas.
2. Go to **Settings → Source** and choose **GitHub Repo**.
3. Select your repository and branch.

Railway will now redeploy automatically on every push.

### Step 4 — Add the Pipe UI service

1. Click **+ New Service** on the canvas.
2. Choose **Docker Image** and enter `iankguimaraes/pipe-ui:latest`.
3. Go to the service's **Variables** tab and add:
   ```
   METRICS_SERVER_URL=${{Pipes.RAILWAY_PRIVATE_DOMAIN}}:9090
   ```
   Replace `Pipes` with the actual name Railway assigned to your indexer service.

### Step 5 — Generate public domains

For each service that needs a public URL:

1. Click the service card.
2. Go to **Settings → Networking → Public Networking**.
3. Click **Generate Domain** and set the correct port (`3000` for Pipe UI, `5432` for PostgreSQL, `8123` for ClickHouse).

***

## Option B — Railway CLI

If you prefer the terminal, the Railway CLI gives you full control over every service and environment variable.

### Step 1 — Install the Railway CLI

```bash theme={"system"}
# macOS / Linux
curl -fsSL https://railway.app/install.sh | sh

# or via npm
npm install -g @railway/cli
```

### Step 2 — Log in to Railway

```bash theme={"system"}
railway login
```

This opens a browser window for OAuth authentication. After approving, the CLI is authenticated for the current session.

### Step 3 — Initialize the Railway project

Run this from the root of your indexer project:

```bash theme={"system"}
railway init --name "your-project-name"
```

Use the same name as the `name` field in your `package.json`. This creates a new project on Railway and links the current directory to it.

### Step 4 — Add the database service

**If your project uses PostgreSQL** (projects with `drizzle.config.ts`):

```bash theme={"system"}
railway add -d postgres
```

Railway provisions a managed PostgreSQL instance and injects a `DATABASE_URL` variable automatically.

**If your project uses ClickHouse:**

```bash theme={"system"}
railway add \
  --service Clickhouse \
  --image clickhouse/clickhouse-server:latest \
  --variables CLICKHOUSE_DB=pipes \
  --variables CLICKHOUSE_USER=default \
  --variables CLICKHOUSE_PASSWORD=password
```

### Step 5 — Add the indexer service

Replace `your-org/your-repo` with your actual GitHub repository slug.

**PostgreSQL project:**

```bash theme={"system"}
railway add \
  --service Pipes \
  --repo your-org/your-repo \
  --variables "DB_CONNECTION_STR=\${{Postgres.DATABASE_URL}}"
```

**ClickHouse project:**

```bash theme={"system"}
railway add \
  --service Pipes \
  --repo your-org/your-repo \
  --variables "CLICKHOUSE_URL=http://\${{Clickhouse.RAILWAY_PRIVATE_DOMAIN}}:8123" \
  --variables CLICKHOUSE_DB=pipes \
  --variables CLICKHOUSE_USER=default \
  --variables CLICKHOUSE_PASSWORD=password
```

> **Note on variable syntax:** `${{ServiceName.VARIABLE}}` is Railway's cross-service reference syntax. The shell requires escaping the `$` as `\$` when passing it through the CLI; Railway resolves it at runtime.

The `--repo` flag links this service to your GitHub repository and enables automatic deployments on every push to the default branch.

### Step 6 — Add the Pipe UI dashboard

```bash theme={"system"}
railway add \
  --service PipeUI \
  --image iankguimaraes/pipe-ui:latest \
  --variables "METRICS_SERVER_URL=\${{Pipes.RAILWAY_PRIVATE_DOMAIN}}:9090"
```

The Pipe UI connects to your indexer's metrics endpoint (port 9090, exposed by the indexer at runtime).

### Step 7 — Generate public domains

Give each service a publicly accessible URL:

**PostgreSQL project:**

```bash theme={"system"}
# Expose the database
railway domain --service Postgres --port 5432

# Expose the UI
railway domain --service PipeUI --port 3000
```

**ClickHouse project:**

```bash theme={"system"}
# Expose the database
railway domain --service Clickhouse --port 8123

# Expose the UI
railway domain --service PipeUI --port 3000
```

### Step 8 — Open the Railway dashboard

```bash theme={"system"}
railway open
```

This opens your project in the Railway web UI where you can monitor deployments, view logs, and manage environment variables.

***

## Service Architecture

```
┌──────────────────────────────────────────────────────┐
│                   Railway Project                    │
│                                                      │
│  ┌─────────────┐     private network                 │
│  │   Database  │◄────────────────────┐               │
│  │  (Postgres  │                     │               │
│  │ /Clickhouse)│                     │               │
│  └──────┬──────┘                     │               │
│         │ public domain (optional)   │               │
│                               ┌──────┴──────┐        │
│                               │   Indexer   │        │
│                               │   (Pipes)   │        │
│                               └──────┬──────┘        │
│                                      │ :9090 metrics  │
│                               ┌──────▼──────┐        │
│                               │   Pipe UI   │        │
│                               │  (PipeUI)   │        │
│                               └──────┬──────┘        │
│                                      │ public domain  │
└──────────────────────────────────────┼───────────────┘
                                       ▼
                                  Browser / API
```

Services communicate over Railway's private network using `${{ServiceName.RAILWAY_PRIVATE_DOMAIN}}` references. Only the UI (and optionally the database) need public domains.

***

## Environment Variables Reference

### Indexer (PostgreSQL)

| Variable            | Value                        |
| ------------------- | ---------------------------- |
| `DB_CONNECTION_STR` | `${{Postgres.DATABASE_URL}}` |

### Indexer (ClickHouse)

| Variable              | Value                                                |
| --------------------- | ---------------------------------------------------- |
| `CLICKHOUSE_URL`      | `http://${{Clickhouse.RAILWAY_PRIVATE_DOMAIN}}:8123` |
| `CLICKHOUSE_DB`       | `pipes`                                              |
| `CLICKHOUSE_USER`     | `default`                                            |
| `CLICKHOUSE_PASSWORD` | `password`                                           |

### ClickHouse service

| Variable              | Value      |
| --------------------- | ---------- |
| `CLICKHOUSE_DB`       | `pipes`    |
| `CLICKHOUSE_USER`     | `default`  |
| `CLICKHOUSE_PASSWORD` | `password` |

### Pipe UI

| Variable             | Value                                    |
| -------------------- | ---------------------------------------- |
| `METRICS_SERVER_URL` | `${{Pipes.RAILWAY_PRIVATE_DOMAIN}}:9090` |

***

## Dockerfile Overview

Your project's `Dockerfile` (generated by `pipes init`) uses a two-stage build:

1. **Builder stage** — installs dependencies with `pnpm`, compiles TypeScript to `dist/`.
2. **Runner stage** — copies only the production build, runs migrations (PostgreSQL only), then starts the indexer.

The indexer exposes **port 9090** for metrics, which Pipe UI connects to.

```dockerfile theme={"system"}
EXPOSE 9090
CMD ["sh", "-lc", "pnpm db:generate && pnpm db:migrate && node dist/index.js"]
# (PostgreSQL only; ClickHouse projects skip the migration step)
```

***

## Troubleshooting

**Indexer fails to start — cannot connect to database**
The database service may not be healthy yet. Railway starts services in parallel; the indexer's health-check retry logic should handle this, but you can also set a startup delay under **Settings → Deploy → Start Command**.

**`${{...}}` variables show as literal strings**
Cross-service references are resolved at deploy time. Make sure both services are in the same Railway project and the referenced service name matches exactly (case-sensitive).

**ClickHouse connection refused**
Confirm `CLICKHOUSE_URL` uses the private domain (`RAILWAY_PRIVATE_DOMAIN`), not a public URL, and that port 8123 is correct.

**Pipe UI shows no data**
Check that `METRICS_SERVER_URL` points to the private domain of the indexer service and that port 9090 is included.


# Stateful transforms
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/evm/guides/advanced-topics/stateful-transforms

Six approaches to maintaining state across pipe batches, with trade-offs and examples

A stateful transform is any step in your pipeline that produces output based on more than the current batch — for example, running balances, sliding-window aggregates, or enrichment lookups. This page surveys the available approaches and when to choose each one.

## Before you add state

The cleanest solution is often to emit raw events and let the downstream database derive the state at query time. ClickHouse materialized views and Postgres views both work for this. If your logic can be expressed as SQL and you can tolerate slightly higher query latency, prefer this over transformer state — it eliminates crash recovery and fork handling entirely on the transformer side.

If your logic is hard to express in SQL, or if the derived state must be pre-computed before reaching the target, read on.

## At a glance

| Approach                                                | State lives in | Persistence across restarts    | Fork handling           | Extra infra   | Best for                                     |
| ------------------------------------------------------- | -------------- | ------------------------------ | ----------------------- | ------------- | -------------------------------------------- |
| [A. Pure in-RAM](#a-pure-in-ram)                        | JS heap        | rebuilt from portal on startup | `fork()` callback       | none          | sliding windows, candles, rolling aggregates |
| [B. ClickHouse MVs](#b-clickhouse-materialized-views)   | ClickHouse     | ✓                              | `sign = -1` rows        | ClickHouse    | SQL-expressible analytics                    |
| [C. SQLite transformer](#c-sqlite-transformer)          | Local file     | ✓ (delta table)                | `fork()` callback       | none          | moderate state                               |
| [D. Postgres/Drizzle target](#d-postgresdrizzle-target) | Postgres       | ✓                              | ✓ automatic             | Postgres      | atomic state + output, Postgres target       |
| [E. Apache Flink](#e-apache-flink)                      | Flink cluster  | ✓                              | via compensating events | Kafka + Flink | TB-scale distributed state                   |
| [F. External KV store](#f-external-kv-store)            | Redis / Valkey | ✓ (with AOF/RDB)               | `fork()` callback       | Redis         | µs-latency lookups, multi-process state      |

***

## A. Pure in-RAM

Keep state in a JavaScript `Map` or array inside the transformer closure. No external storage is involved.

**When to use:**

* State can be derived from a bounded window of recent blocks (e.g., last N blocks or last M seconds).
* You can afford to replay that window on restart (warm-up time ∝ window size).
* State loss is contained: at most one window's worth of history needs to be replayed.

**When not to use:**

* State grows without bound (e.g., all-time ERC-20 balances). Use [Postgres approach D](#d-postgresdrizzle-target) instead — specifically the in-memory + Postgres mirror sub-approach.
* The warm-up window is too large to replay quickly on every restart.

### The warm-up pattern

When the process restarts, the target's cursor tells you where the pipeline left off. The in-RAM state is gone. To rebuild it, call `portal.getStream()` in the `start()` callback — the raw portal client API, independent of the main stream:

```typescript theme={"system"}
start: async ({ portal, state, logger }) => {
  if (!state.current) return  // first ever run: start empty

  const warmupFrom = Math.max(state.initial, state.current.number - LOOKBACK_BLOCKS)
  if (warmupFrom >= state.current.number) return

  for await (const { blocks } of portal.getStream({
    type: 'evm',
    fromBlock: warmupFrom,
    toBlock: state.current.number,
    fields: { block: { number: true }, log: { data: true } },
    logs: [{ address: [CONTRACT], topic0: [EVENT_TOPIC] }],
  })) {
    for (const block of blocks) {
      for (const log of block.logs) {
        // rebuild in-RAM state from block and log fields
      }
    }
  }
}
```

`portal` is a live `PortalClient` already connected to the dataset. The warm-up query's `toBlock` is the saved cursor, so it terminates immediately after the pipeline resumes from `cursor + 1`. Multiple in-RAM transformers each run their own `start()` warm-up in parallel (the SDK calls child `start()` callbacks concurrently).

### Fork handling

`target.fork()` fires first (ClickHouse `onRollback` or drizzle snapshot rollback), then the transformer's `fork()` callback. At that point the database already reflects pre-fork state. In `fork()`, drop in-RAM entries for blocks beyond the rollback cursor:

```typescript theme={"system"}
fork: async (cursor, { logger }) => {
  recentEntries = recentEntries.filter(e => e.blockNumber <= cursor.number)
}
```

### Composability

Use the same `initQueue`/`WriteQueue` pattern as the Postgres examples. For ClickHouse targets the queue holds closures over `ClickhouseStore` instead of a Postgres `Transaction`:

```typescript theme={"system"}
type CHS = { insert(params: { table: string; values: unknown[]; format: string }): Promise<unknown> }

class WriteQueue {
  private ops: Array<(store: CHS) => Promise<void>> = []
  push(op: (store: CHS) => Promise<void>): void { this.ops.push(op) }
  async flush(store: CHS): Promise<void> { for (const op of this.ops) await op(store) }
}
```

<Expandable title="Complete example — rolling 1-hour SQD transfer volume">
  See [`13.stateful-transform-in-ram.example.ts`](https://github.com/subsquid-labs/pipes-sdk-docs/blob/master/src/advanced/evm/13.stateful-transform-in-ram.example.ts) for the full implementation: a rolling \~1-hour transfer volume tracker for the SQD token on Arbitrum, with portal warm-up, fork handling, and the WriteQueue composability pattern targeting ClickHouse.
</Expandable>

***

## B. ClickHouse materialized views

Write raw events to a base table; let ClickHouse compute derived state via materialized views (MVs). The transformer is stateless — it only emits events, not pre-computed state.

**When to use:**

* Your aggregation logic is expressible in SQL.
* ClickHouse is already your target.
* You want derived state updated automatically without any transformer code.

**When not to use:**

* Logic requires imperative iteration (e.g., order-dependent simulation).
* Each MV chain adds latency — avoid long dependency chains for latency-sensitive consumers.
* Very frequent writes on lightweight data: prefer plain (non-materialized) views if you have spare CPU on the database machine.

### The core limitation: MVs see only new rows

A materialized view fires when new rows are inserted into its source table. Its `SELECT` clause only operates on the **newly inserted batch**, not the full table. Running totals like cumulative balance cannot be written directly in the MV `SELECT`.

### Workaround: auxiliary aggregating tables

Maintain a separate "current state" table using `AggregatingMergeTree` with `argMaxState`. The MV reads this table alongside the new rows to resolve the latest value before the current batch:

```sql theme={"system"}
-- Stores latest balance per pool (AggregatingMergeTree = efficient upsert)
CREATE TABLE current_balances (
    pool_address        String,
    token_a_balance_raw AggregateFunction(argMax, Int256, Tuple(DateTime, UInt16, UInt16)),
    token_b_balance_raw AggregateFunction(argMax, Int256, Tuple(DateTime, UInt16, UInt16))
) ENGINE = AggregatingMergeTree()
ORDER BY pool_address;

-- MV that keeps current_balances up to date
CREATE MATERIALIZED VIEW current_balances_mv TO current_balances AS
SELECT
    pool_address,
    argMaxState(token_a_balance_raw, (timestamp, transaction_index, log_index)) AS token_a_balance_raw,
    argMaxState(token_b_balance_raw, (timestamp, transaction_index, log_index)) AS token_b_balance_raw
FROM balances_history
GROUP BY pool_address;
```

A downstream MV that needs the running balance queries `current_balances` with `argMaxMerge()`:

```sql theme={"system"}
latest_pool_balances AS (
    SELECT
        pool_address,
        argMaxMerge(token_a_balance_raw) AS balance_token_a_raw,
        argMaxMerge(token_b_balance_raw) AS balance_token_b_raw
    FROM current_balances
    WHERE pool_address IN (SELECT pool_address FROM unique_pools_to_insert)
    GROUP BY pool_address
)
```

### Temporal joins with ASOF JOIN

When you need "the most recent price before each event", use `ASOF JOIN`:

```sql theme={"system"}
SELECT ...
FROM liquidity_events_raw ml
ASOF JOIN latest_prices wp
    ON wp.pool_address = ml.pool_address
    AND wp.ts_num + wp.transaction_index * 100_000 + wp.log_index
        <= ml.ts_num + ml.transaction_index * 100_000 + ml.log_index
WHERE ml.protocol = 'uniswap_v4'
```

The `ASOF JOIN` selects the latest row in `latest_prices` whose ordering key is ≤ the current event's key — effectively "last price before this log".

### Fork rollback

ClickHouse is non-transactional. Use `CollapsingMergeTree` with a `sign` column: insert `sign = 1` rows on the way forward and `sign = -1` rows to cancel them on rollback. Your `onRollback` handler computes which blocks to cancel and inserts the negating rows.

<Expandable title="Complete example — DEX liquidity tracking">
  See [`pipes-sqdgn-dex-example/pipes/evm/liquidity/liquidity.sql`](https://github.com/subsquid-labs/pipes-sqdgn-dex-example/blob/master/pipes/evm/liquidity/liquidity.sql) for a full production SQL schema: `liquidity_events_raw` as the base table, `CollapsingMergeTree` for rollback, `AggregatingMergeTree` + `argMaxState` for current pool balances, an `ASOF JOIN` MV for V4 liquidity, and separate V2/V3/V4 MV chains targeting `balances_history`.
</Expandable>

***

## C. SQLite transformer

Keep transformer state in a local SQLite database. The transformer reads and writes SQLite; the downstream target (typically ClickHouse) receives the pre-computed rows.

**When to use:**

* State is too large for RAM.
* You need random-access lookups (e.g., "current balance of address X") that would be slow as a linear scan of in-RAM arrays.
* You're not using Postgres as your target (otherwise see [approach D](#d-postgresdrizzle-target) for better atomicity).
* The indexer runs on persistent infrastructure (SQLite file must survive restarts).

**When not to use:**

* The indexer runs on ephemeral infrastructure (containers, spot VMs). SQLite is lost on restart.
* State requires complex analytical SQL (window functions, multi-table joins) — consider DuckDB as a drop-in alternative with full analytical query support.

### The delta table pattern

SQLite and the downstream target commit separately — a crash between them leaves the two out of sync. A `balance_deltas` table records the net change per address per block, allowing `rollbackTo(blockNumber)` to invert any set of blocks atomically:

```typescript theme={"system"}
// Schema
db.exec(`
  CREATE TABLE IF NOT EXISTS balance_deltas (
    address      TEXT    NOT NULL,
    block_number INTEGER NOT NULL,
    delta        TEXT    NOT NULL,
    PRIMARY KEY (address, block_number)
  )
`)

function rollbackTo(blockNumber: number) {
  db.transaction(() => {
    const deltas = db.prepare('SELECT address, delta FROM balance_deltas WHERE block_number > ?').all(blockNumber)
    const net = new Map<string, bigint>()
    for (const { address, delta } of deltas) net.set(address, (net.get(address) ?? 0n) + BigInt(delta))
    for (const [address, delta] of net) {
      const { balance } = db.prepare('SELECT balance FROM balances WHERE address = ?').get(address) as any
      db.prepare('INSERT INTO balances (address, balance) VALUES (?, ?) ON CONFLICT(address) DO UPDATE SET balance = excluded.balance')
        .run(address, (BigInt(balance) - delta).toString())
    }
    db.prepare('DELETE FROM balance_deltas WHERE block_number > ?').run(blockNumber)
    db.prepare('DELETE FROM processed_blocks WHERE block_number > ?').run(blockNumber)
  })()
}
```

### Crash recovery in start()

Compare the SQLite high-water mark with the pipeline cursor. If SQLite is ahead, roll back to match:

```typescript theme={"system"}
start: async ({ state }) => {
  const sqliteLastBlock  = db.prepare('SELECT MAX(block_number) as m FROM processed_blocks').get().m ?? null
  const pipelineLastBlock = state.current?.number ?? null

  if (sqliteLastBlock !== null && (pipelineLastBlock === null || sqliteLastBlock > pipelineLastBlock)) {
    rollbackTo(pipelineLastBlock ?? -1)  // SQLite crashed ahead of cursor — roll back
  } else if (sqliteLastBlock !== pipelineLastBlock) {
    throw new Error(`State mismatch: SQLite=${sqliteLastBlock}, cursor=${pipelineLastBlock}. Delete the SQLite file to rebuild.`)
  }
}
```

### Historical-only variant

If you're indexing only finalized data and will never see forks, drop the delta table and accept that a crash requires rebuilding from scratch:

```typescript theme={"system"}
start: async ({ state }) => {
  if (sqliteLastBlock !== pipelineLastBlock) {
    throw new Error(`Delete ${SQLITE_DB_PATH} to rebuild.`)
  }
}
```

<Expandable title="Complete examples">
  * [`09.stateful-transform-on-sqlite.example.ts`](https://github.com/subsquid-labs/pipes-sdk-docs/blob/master/src/advanced/evm/09.stateful-transform-on-sqlite.example.ts) — full delta-table implementation with fork and crash recovery
  * [`10.stateful-transform-on-sqlite-no-forks.example.ts`](https://github.com/subsquid-labs/pipes-sdk-docs/blob/master/src/advanced/evm/10.stateful-transform-on-sqlite-no-forks.example.ts) — simpler historical-only variant
</Expandable>

***

## D. Postgres/Drizzle target

When your target is already Postgres, the `drizzleTarget` can commit transformer state and output rows inside the **same serializable transaction** as the cursor save. This gives the strongest atomicity guarantees of any approach: a crash between `transform()` and the cursor save is impossible because both commit together.

**When to use:**

* Postgres is your output target.
* You want zero crash recovery code (atomicity handles it automatically).
* Fork rollback should be automatic (drizzleTarget installs snapshot triggers).

**When not to use:**

* Your target is ClickHouse or another non-Postgres database.
* State is too large for Postgres (rare).

### The WriteQueue / initQueue pattern

Multiple stateful transformers all need to write inside the same transaction. The `WriteQueue` collects their write closures; `initQueue` wraps each batch in a `Piped<T>` with a fresh queue; `onData` flushes everything:

```typescript theme={"system"}
class WriteQueue {
  private ops: Array<(tx: Transaction) => Promise<void>> = []
  push(op: (tx: Transaction) => Promise<void>): void { this.ops.push(op) }
  async flush(tx: Transaction): Promise<void> { for (const op of this.ops) await op(tx) }
}

function initQueue<T>() {
  return createTransformer<T, Piped<T>>({
    transform: (data) => ({ payload: data, writes: new WriteQueue() }),
  })
}

// Pipeline:
stream
  .pipe(initQueue<DecodedBatch>())
  .pipe(transformerA(db))
  .pipe(transformerB(db))
  .pipeTo(drizzleTarget({
    db,
    tables: [tableA, tableB],
    onData: async ({ tx, data }) => { await data.writes.flush(tx) },
  }))
```

`onData` stays a one-liner regardless of how many transformers are chained.

### Sub-approach 1 — Stateless transform (per-batch DB reads)

`transform()` reads current state from Postgres (the last committed snapshot), computes the delta, and pushes write closures to the queue. No in-RAM Map survives between batches.

* ✓ No RAM limit on state size
* ✓ Zero fork handling code (snapshot triggers on `tables` cover rollback)
* ✗ One `SELECT … WHERE address IN (…)` per batch

<Expandable title="Complete example — stateless (example 11)">
  See [`11.stateful-transforms-postgres-stateless.example.ts`](https://github.com/subsquid-labs/pipes-sdk-docs/blob/master/src/advanced/evm/11.stateful-transforms-postgres-stateless.example.ts): two transformers (`BalanceTransformer` + `TransferCountTransformer`) reading from Postgres each batch and writing atomically via WriteQueue.
</Expandable>

### Sub-approach 2 — In-memory + Postgres mirror

`start()` loads the full state into in-RAM Maps. `transform()` reads/writes the Maps with no DB round trips per batch. `fork()` reloads the Maps from Postgres after drizzleTarget commits the snapshot rollback.

* ✓ No per-batch DB reads — all reads from memory after startup
* ✓ Fast for large batches with many distinct keys
* ✗ Full state must fit in RAM
* ✗ Startup time is O(state size)
* ✗ `fork()` callbacks required to resync Maps after rollback

<Expandable title="Complete example — in-memory (example 12)">
  See [`12.stateful-transforms-postgres-in-memory.example.ts`](https://github.com/subsquid-labs/pipes-sdk-docs/blob/master/src/advanced/evm/12.stateful-transforms-postgres-in-memory.example.ts): same two transformers with in-RAM Maps loaded from Postgres at startup and reloaded on fork.
</Expandable>

<Note>
  For both sub-approaches, all state tables must be listed in `drizzleTarget`'s `tables` array. This installs PostgreSQL snapshot triggers that roll them back automatically on a blockchain reorg. The `onStart` callback can run `CREATE TABLE IF NOT EXISTS` for quick setup; in production, use [drizzle-kit migrations](https://orm.drizzle.team/docs/migrations) instead.
</Note>

***

## E. Apache Flink

[Apache Flink](https://flink.apache.org) is a distributed stateful stream-processing framework. The Pipes SDK acts as a data source feeding Flink via Kafka or a direct connector.

**When to use:**

* State is too large for a single machine (terabytes).
* Your problem requires stateful joins across multiple independent streams (e.g., correlate DEX trades with lending liquidations across different chains).
* You need exactly-once semantics across multiple heterogeneous sinks.

**When not to use:**

* Single-node deployments — the operational overhead (JVM runtime, cluster management, ZooKeeper or KRaft, checkpoint storage) is only justified when the problem genuinely requires distributed state.

**Architecture:** The Pipes SDK emits raw events to Kafka (one topic per event type). On a blockchain fork, it emits compensating rows (e.g., `sign = -1`) that Flink sees as normal data and can handle with a subtract-and-recompute pattern. Flink manages its own checkpoints; crash recovery is handled entirely by Flink.

***

## F. External KV store

Use Redis, Valkey, or a similar key-value store as a fast external state backend.

**When to use:**

* Multiple parallel pipeline instances must share state (horizontal scaling of the indexer).
* Per-key lookups must complete in under 1 ms (e.g., enriching 50 k events per second with metadata from a 100 M-entry map that doesn't fit in RAM).

**When not to use:**

* A single-process indexer is sufficient — adding Redis increases operational complexity for no benefit.
* You need transactional state + output commits (use approach D instead).

**Fork handling:** The transformer's `fork()` callback must delete or undo the Redis keys written for rolled-back blocks. Keep a per-block write log (similar to the SQLite delta table) to know which keys to revert.

**Crash safety:** Redis is not durable by default. Enable AOF or RDB persistence, or treat Redis purely as a warm cache and accept that a Redis restart requires a replay from the pipeline cursor.

***

## Fork callbacks and crash recovery

The fork handling responsibilities differ by approach:

| Approach            | `fork()` needed in transformer        | How DB state is rolled back        |
| ------------------- | ------------------------------------- | ---------------------------------- |
| A. In-RAM           | ✓ — prune entries > cursor            | n/a (state is RAM-only)            |
| B. ClickHouse MVs   | ✗ — handled in `onRollback`           | `sign = -1` rows via `onRollback`  |
| C. SQLite           | ✓ — calls `rollbackTo(cursor.number)` | `rollbackTo()` reverts delta table |
| D. Postgres/drizzle | ✗ — automatic                         | snapshot triggers via `tables`     |
| E. Flink            | ✗ — compensating events               | Flink checkpoint rollback          |
| F. External KV      | ✓ — revert write log                  | manual key deletion                |

**Ordering guarantee:** `target.fork()` always fires before transformer `fork()` callbacks. By the time your transformer's `fork()` runs, the target (ClickHouse `onRollback`, drizzleTarget snapshot rollback) has already committed the database rollback. It is safe to read the database in `fork()`.

**Crash recovery** (approaches A, B, C only): a crash between the transformer's store write and the target's cursor save leaves state ahead of the pipeline cursor. Handle this in `start()` by comparing your local high-water mark to `state.current`:

```typescript theme={"system"}
start: async ({ state }) => {
  const localLastBlock    = /* read your checkpoint */
  const pipelineLastBlock = state.current?.number ?? null

  if (localLastBlock !== null && (pipelineLastBlock === null || localLastBlock > pipelineLastBlock)) {
    rollbackTo(pipelineLastBlock ?? -1)  // crash recovery: undo ahead-of-cursor writes
  }
}
```

Approaches D (Postgres/drizzle) and E (Flink) are immune to this problem: state and cursor commit atomically.

***

## Composing multiple stateful transformers

When multiple stateful transformers write to the same target, they must not each independently call the target's write API. Use the `WriteQueue` / `initQueue` pattern to collect all writes and flush them in a single `onData` call:

1. **`initQueue<T>()`** wraps the raw batch in `Piped<T>` with a fresh `WriteQueue`. Place it as the first `.pipe()`.
2. **Each transformer** receives `Piped<T>`, pushes closures to `writes`, and returns `Piped<T>` unchanged.
3. **`onData`** calls `data.writes.flush(store_or_tx)` — a one-liner that scales to any number of transformers.

```typescript theme={"system"}
type Piped<T> = { payload: T; writes: WriteQueue }

function initQueue<T>() {
  return createTransformer<T, Piped<T>>({
    transform: (data) => ({ payload: data, writes: new WriteQueue() }),
  })
}
```

Because every domain transformer takes `Piped<T>` as input and produces `Piped<T>` as output, none of them assume a fixed position in the chain — they are all order-independent and can be added or removed without touching the others.

For Postgres targets, `WriteQueue` closures take a `Transaction`; for ClickHouse, they take the structural `CHS` type shown in approach A. The pattern is identical in both cases.


# Cursor management
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/evm/guides/architecture-deep-dives/cursor-management

How pipelines track progress and resume after restarts

A cursor records the last successfully processed block. Built-in targets (ClickHouse, Drizzle) handle persistence automatically. When using `createTarget` directly you own the full lifecycle.

## The cursor object

```typescript theme={"system"}
type BlockCursor = {
  number: number      // stream resumes from number + 1
  hash?: string       // block hash — used as parentBlockHash for fork detection
  timestamp?: number  // block timestamp in seconds
}
```

`hash` is the fork detection tripwire: the SDK sends `parentBlockHash = cursor.hash` in each portal request. An absent hash silently skips fork detection for that request. See [cursor semantics](./fork-handling#5-cursor-semantics) for the full picture.

## Startup: range.from and stored cursors

`range.from` in the decoder sets where the stream begins on a first run — before any cursor exists:

```typescript theme={"system"}
evmDecoder({
  range: { from: 'latest' },         // chain head
  // range: { from: 20_000_000 },    // block number
  // range: { from: '2024-01-01' },  // ISO date string
  // range: { from: new Date() },    // Date object
})
```

Once a cursor is stored, `range.from` is ignored — the stream resumes from `cursor.number + 1`.

## The stream id

The `id` on `evmPortalStream` is the primary key for all stored state:

```typescript theme={"system"}
evmPortalStream({ id: 'my-pipeline', ... })
```

Both built-in targets use it to isolate state records, so multiple streams can share one physical table. **Never rename an active stream's id** — the stored cursor is keyed on it, and renaming causes the pipeline to restart from `range.from`.

## ClickHouse target

`clickhouseTarget` saves the cursor after every successful `onData` call and resolves fork and crash-recovery callbacks automatically.

```typescript theme={"system"}
clickhouseTarget({
  client,
  settings: {
    id: 'my-stream',      // overrides evmPortalStream id (default: 'stream')
    table: 'sync',        // state table name (default: 'sync')
    database: 'default',  // ClickHouse database
    maxRows: 10_000,      // cursor rows to keep per stream id (default: 10,000)
  },
  onData: ...,
  onRollback: ...,
})
```

**`onRollback` is called in two situations:**

* `type: 'offset_check'` — on every startup when a cursor exists. ClickHouse is non-transactional: a crash between `onData` and the cursor save leaves rows newer than the saved cursor. Delete them here. See [non-transactional databases](./fork-handling#4-state-rollback-atomicity).
* `type: 'blockchain_fork'` — when the portal signals a reorg. The rollback cursor is resolved automatically from stored history; your callback only needs to delete rows after `safeCursor.number`.

The same implementation typically serves both:

```typescript theme={"system"}
onRollback: async ({ store, safeCursor }) => {
  await store.removeAllRows({
    tables: ['my_table'],
    where: `block_number > {n:UInt32}`,
    params: { n: safeCursor.number },
  })
},
```

**State table.** Each row stores the cursor, the last finalized block, and the unfinalized block history used for fork recovery. Rows beyond `maxRows` are pruned every 25 saves. Set `maxRows` to cover your network's worst-case reorg depth — see [rollback depth](./fork-handling#3-rollback-depth-and-history-limits).

## Drizzle target

`drizzleTarget` saves the cursor inside the same PostgreSQL transaction as the data write — fully atomic, no crash-recovery pass needed.

```typescript theme={"system"}
drizzleTarget({
  db: drizzle(DB_URL),
  tables: [transfersTable],   // every table onData writes to — required
  settings: {
    state: {
      id: 'my-stream',
      schema: 'public',
      table: 'sync',
      unfinalizedBlocksRetention: 1000,  // cursor rows to keep (default: 1,000)
    },
    transaction: { isolationLevel: 'serializable' },  // default
  },
  onData: async ({ tx, data }) => {
    await tx.insert(transfersTable).values(...)
  },
})
```

**`tables` is required** for every table written in `onData`. At startup the target installs a PostgreSQL trigger on each listed table; the trigger copies the pre-change row into a `<name>__snapshots` table (keyed by block number and primary key). On a fork the target replays these snapshots in reverse, restoring pre-fork state automatically. Writing to a table not in `tables` raises a runtime error.

Snapshotting only fires for blocks at or above the current finalized head — historical blocks can never be reorged.

**Advisory lock.** Every batch acquires `pg_try_advisory_xact_lock(hashtext(id))` inside the transaction, preventing concurrent writers on the same stream. Two `drizzleTarget` instances sharing the same `id` will serialize correctly; two with different `id`s run independently.

**Retention.** Snapshot rows below `min(current, finalizedHead) - unfinalizedBlocksRetention` are deleted every 25 batches. Set this to cover your network's worst-case reorg depth.

**Rollback hooks.** `onBeforeRollback` and `onAfterRollback` receive `{ tx, cursor }` and run inside the fork transaction. Use them to perform additional cleanup that the snapshot mechanism cannot cover (e.g., rows in tables not tracked by `tables`).

## Async iterator

When consuming a pipeline with `for await...of` instead of `pipeTo`, the native `[Symbol.asyncIterator]()` always calls `read()` with no cursor — it has no way to accept one. The stream therefore starts from `range.from` on every run.

**Finalized streams.** If the stream only consumes already-finalized blocks (no forks possible), rebuilding the stream with `range.from` set to the stored cursor is sufficient:

```typescript theme={"system"}
let cursor = loadCursor()  // BlockCursor | undefined

const stream = evmPortalStream({
  id: 'my-pipeline',
  portal: '...',
  outputs: evmDecoder({
    // cursor.number is the last processed block; resume from the next one
    range: { from: cursor ? cursor.number + 1 : 0 },
  }),
})

for await (const { data, ctx } of stream) {
  await processData(data)
  saveCursor(ctx.stream.state.current)  // { number, hash, timestamp }
}
```

Save `ctx.stream.state.current` — the full `BlockCursor` of the batch's last block — not just the number. The `hash` is needed if you later switch to real-time or need the cursor as a fork anchor.

**Real-time streams.** Setting `range.from` to a stored number loses the block hash. On restart the first request carries no `parentBlockHash`, so fork detection is silently disabled for that request. For real-time streams, use the `pipeToIterator` helper from the [async iteration tab of the fork handling guide](./fork-handling), which accepts an `initialCursor` and passes it directly to `read()` inside `pipeTo`:

```typescript theme={"system"}
const stream = pipeToIterator(
  evmPortalStream({ id: 'my-pipeline', portal: '...', outputs: evmDecoder({ range: { from: 'latest' } }) }),
  loadCursor(),   // full BlockCursor with hash — passed to read(), not range.from
  onFork,
)

for await (const { data, ctx } of stream) {
  await processData(data)
  saveCursor(ctx.stream.state.current)
}
```

`pipeToIterator` preserves `parentBlockHash` across fork rounds because it uses `pipeTo` internally. On a fresh first run, pass `undefined` as `initialCursor` and the stream begins from `range.from` as normal.

## Custom cursor management

When using `createTarget` directly, you own the full cursor lifecycle.

At the start of `write`, fetch the stored cursor and pass it to `read`:

```typescript theme={"system"}
write: async ({ read }) => {
  const cursor = await db.getLatestCursor()
  for await (const { data, ctx } of read(cursor)) {
    // ...
  }
}
```

After processing each batch, persist the cursor together with the fork-recovery state:

```typescript theme={"system"}
await db.transaction(async (tx) => {
  await writeData(tx, data)
  await tx.saveCursor({
    cursor:        ctx.stream.state.current,
    rollbackChain: ctx.stream.state.rollbackChain,
    finalized:     ctx.stream.head.finalized,
  })
})
```

For **transactional stores** (Postgres): save all three fields in the same transaction as the data write. For **non-transactional stores** (ClickHouse): write data first, cursor last, and implement a startup check that detects and corrects any data written after the last cursor save. See [state rollback atomicity](./fork-handling#4-state-rollback-atomicity).

The `fork` callback and the algorithm for resolving rollback cursors from stored history are covered in detail in the [fork handling guide](./fork-handling).

<Card title="Example: cursor management with createTarget" icon="github" href="https://github.com/subsquid-labs/pipes-sdk-docs/blob/master/src/basics/06-cursor-from-target.ts">
  A minimal example showing manual cursor passing in createTarget
</Card>

<Card title="Example: ClickHouse target" icon="github" href="https://github.com/subsquid-labs/pipes-sdk-docs/blob/master/src/basics/08-clickhouse-target.ts">
  Full pipeline with onRollback and onData
</Card>

<Card title="Example: Drizzle / PostgreSQL target" icon="github" href="https://github.com/subsquid-labs/pipes-sdk/blob/main/docs/examples/evm/08.drizzle.example.ts">
  Full pipeline including GraphQL API
</Card>


# Fork handling
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/evm/guides/architecture-deep-dives/fork-handling

Handle blockchain forks and rollbacks in real-time streams

When consuming a real-time stream near the chain head, the portal can detect that the client's view of the chain has diverged from the canonical chain — a situation known as a fork or reorg. The portal signals this with an HTTP 409 response containing a sample of blocks from the new canonical chain. Your code must find the highest block that both chains agree on, roll back any state written after that point, and replay from there.

<Note>
  Fork handling is only needed for real-time streams (`range.from: 'latest'`). Historical streams consume already-finalized data and never produce forks. See [Fork detection scope](#7-fork-detection-scope-real-time-streams-only) below.
</Note>

The SDK provides two patterns for consuming a stream. Both use the same state-tracking logic; they differ in how the fork signal is delivered.

<Note>
  If your pipeline includes a [stateful transformer](../advanced-topics/stateful-transforms#fork-callbacks-in-stateful-transformers), it must also implement a `fork` callback to roll back its own state in lockstep with the target.
</Note>

<Tabs>
  <Tab title="Via pipeTo / targets">
    The `pipeTo(createTarget({write, fork}))` pattern keeps fork handling completely separate from batch processing. The SDK catches the 409 internally and calls `fork()` with the portal's consensus block sample; `write()` never sees the interruption and continues iterating batches without restarting.

    <Steps>
      <Step title="Declare state">
        Two variables span the lifetime of the stream:

        ```typescript theme={"system"}
        let recentUnfinalizedBlocks: BlockCursor[] = []
        let finalizedHighWatermark: BlockCursor | undefined
        ```

        `recentUnfinalizedBlocks` is the local history of unfinalized blocks used to find the common ancestor during a fork. `finalizedHighWatermark` tracks the highest finalized block ever seen — stored as a full `BlockCursor` (number **and** hash) so it can double as a rollback cursor when needed. Both must be declared outside `pipeTo` so `fork()` can access them.
      </Step>

      <Step title="Collect rollback history">
        Inside `write()`, append each batch's unfinalized blocks to the local history:

        ```typescript theme={"system"}
        ctx.stream.state.rollbackChain.forEach((bc) => {
          recentUnfinalizedBlocks.push(bc)
        })
        ```

        `ctx.stream.state.rollbackChain` contains only the blocks from **this batch** that are above the current finalized head — it is a per-batch delta, not a full snapshot. Always append to the end; never replace or reorder.
      </Step>

      <Step title="Prune finalized blocks">
        After collecting history, prune blocks that are now finalized and cap the queue:

        ```typescript theme={"system"}
        if (ctx.stream.head.finalized) {
          if (!finalizedHighWatermark || ctx.stream.head.finalized.number > finalizedHighWatermark.number) {
            finalizedHighWatermark = ctx.stream.head.finalized
          }
          recentUnfinalizedBlocks = recentUnfinalizedBlocks.filter(b => b.number >= finalizedHighWatermark!.number)
        }
        recentUnfinalizedBlocks = recentUnfinalizedBlocks.slice(recentUnfinalizedBlocks.length - 1000)
        ```

        Portal instances behind a load balancer can report different finalized heads. Using the **maximum** seen so far (the high-water mark) prevents the pruning threshold from moving backwards when the stream reconnects to a lagging instance. See [consideration 6](#6-load-balanced-portals-and-a-non-monotonic-finalized-head) for details.
      </Step>

      <Step title="Implement fork()">
        `fork()` receives `previousBlocks` — the portal's current-chain sample — and must return the last good block cursor, or `null` if recovery is impossible:

        ```typescript theme={"system"}
        fork: async (newConsensusBlocks) => {
          const rollbackIndex = findRollbackIndex(recentUnfinalizedBlocks, newConsensusBlocks)
          if (rollbackIndex >= 0) {
            recentUnfinalizedBlocks.length = rollbackIndex + 1
            return recentUnfinalizedBlocks[rollbackIndex]
          }
          if (finalizedHighWatermark &&
              newConsensusBlocks.every(b => b.number < finalizedHighWatermark!.number)) {
            recentUnfinalizedBlocks = recentUnfinalizedBlocks.filter(b => b.number <= finalizedHighWatermark!.number)
            return finalizedHighWatermark
          }
          return null
        }
        ```

        Three cases: (1) a common ancestor is found in local history — truncate and return it; (2) all `previousBlocks` fall below the finalized high-water mark, meaning the portal's sample doesn't reach local history — return the high-water mark cursor; (3) no recovery possible — return `null`, which surfaces a `ForkCursorMissingError`.
      </Step>
    </Steps>

    <Expandable title="Complete example">
      ```typescript theme={"system"}

      import { BlockCursor, createTarget } from '@subsquid/pipes'
      import { evmPortalStream, evmDecoder, commonAbis } from '@subsquid/pipes/evm'

      async function main() {
        let recentUnfinalizedBlocks: BlockCursor[] = []
        let finalizedHighWatermark: BlockCursor | undefined

        await evmPortalStream({
          id: 'forks',
          portal: 'https://portal.sqd.dev/datasets/ethereum-mainnet',
          outputs: evmDecoder({
            contracts: ['0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48'], // USDC
            events: { transfer: commonAbis.erc20.events.Transfer },
            range: { from: 'latest' }
          }),
        })
          .pipeTo(createTarget({
            write: async ({read}) => {
              for await (const {data, ctx} of read(recentUnfinalizedBlocks[recentUnfinalizedBlocks.length-1])) {
                console.log(`Got ${data.transfer.length} transfers`)
                ctx.stream.state.rollbackChain.forEach((bc) => { recentUnfinalizedBlocks.push(bc) })
                if (ctx.stream.head.finalized) {
                  if (!finalizedHighWatermark || ctx.stream.head.finalized.number > finalizedHighWatermark.number) {
                    finalizedHighWatermark = ctx.stream.head.finalized
                  }
                  recentUnfinalizedBlocks = recentUnfinalizedBlocks.filter(b => b.number >= finalizedHighWatermark!.number)
                }
                recentUnfinalizedBlocks = recentUnfinalizedBlocks.slice(recentUnfinalizedBlocks.length - 1000)
              }
            },
            fork: async (newConsensusBlocks) => {
              const rollbackIndex = findRollbackIndex(recentUnfinalizedBlocks, newConsensusBlocks)
              if (rollbackIndex >= 0) {
                recentUnfinalizedBlocks.length = rollbackIndex + 1
                return recentUnfinalizedBlocks[rollbackIndex]
              }
              if (finalizedHighWatermark &&
                  newConsensusBlocks.every(b => b.number < finalizedHighWatermark!.number)) {
                recentUnfinalizedBlocks = recentUnfinalizedBlocks.filter(b => b.number <= finalizedHighWatermark!.number)
                return finalizedHighWatermark
              }
              return null
            }
          }))
      }

      main().then(() => { console.log('\ndone') })



      function findRollbackIndex(chainA: BlockCursor[], chainB: BlockCursor[]): number {
        let aIndex = 0, bIndex = 0, lastCommonIndex = -1
        while (aIndex < chainA.length && bIndex < chainB.length) {
          const a = chainA[aIndex], b = chainB[bIndex]
          if (a.number < b.number) { aIndex++; continue }
          if (a.number > b.number) { bIndex++; continue }
          if (a.hash !== b.hash) return lastCommonIndex
          lastCommonIndex = aIndex; aIndex++; bIndex++
        }
        return lastCommonIndex
      }
      ```
    </Expandable>
  </Tab>

  <Tab title="Via async iteration (workaround)">
    The native `[Symbol.asyncIterator]()` on a `PortalSource` cannot handle forks that require multiple 409 rounds. After a fork, the only option with native iteration is to re-create the stream — but the re-created stream's first request carries no `parentBlockHash`, so the portal cannot detect whether the client is still on the wrong chain and will not send the second 409.

    The root cause: `pipeTo`'s internal `read()` generator maintains a `cursor` variable across fork rounds. After `target.fork()` returns a rollback cursor it sets `cursor = forkedCursor` before re-entering `self.read(cursor)`, keeping `parentBlockHash` populated on every subsequent request. The native async iterator calls `this.read()` with no cursor and has no equivalent mechanism.

    **Workaround:** wrap `pipeTo` in a helper called `pipeToIterator` that bridges its push-based `write()` into a pull-based iterator via a single-item queue with producer acknowledgement. This preserves the `for await...of` interface while using `pipeTo`'s cursor-tracking machinery internally.

    <Steps>
      <Step title="Declare state">
        Same two variables as the `pipeTo` approach — no extra `resumeCursor` needed, since `pipeTo` handles cursor updates internally:

        ```typescript theme={"system"}
        let recentUnfinalizedBlocks: BlockCursor[] = []
        let finalizedHighWatermark: BlockCursor | undefined
        ```
      </Step>

      <Step title="Write the fork callback">
        The fork callback passed to `pipeToIterator` is identical to `fork()` in the `pipeTo` example — the same three-case logic, the same state mutations:

        ```typescript theme={"system"}
        async (newConsensusBlocks) => {
          const rollbackIndex = findRollbackIndex(recentUnfinalizedBlocks, newConsensusBlocks)
          if (rollbackIndex >= 0) {
            recentUnfinalizedBlocks.length = rollbackIndex + 1
            return recentUnfinalizedBlocks[rollbackIndex]
          }
          if (finalizedHighWatermark &&
              newConsensusBlocks.every(b => b.number < finalizedHighWatermark!.number)) {
            recentUnfinalizedBlocks = recentUnfinalizedBlocks.filter(b => b.number <= finalizedHighWatermark!.number)
            return finalizedHighWatermark
          }
          return null
        }
        ```

        The SDK awaits this callback before resuming the stream, so `recentUnfinalizedBlocks` is safe to mutate here without additional locking.
      </Step>

      <Step title="Wrap the source and iterate">
        Pass the source, the initial cursor, and the fork callback to `pipeToIterator`, then iterate normally:

        ```typescript theme={"system"}
        const stream = pipeToIterator(source, recentUnfinalizedBlocks.at(-1), onFork)

        for await (const {data, ctx} of stream) {
          // batch processing — identical to the pipeTo example
        }
        ```
      </Step>
    </Steps>

    <Expandable title="pipeToIterator implementation">
      ```typescript theme={"system"}
      // WORKAROUND — see explanation above the tab
      function pipeToIterator<T>(
        source: { pipeTo(t: ReturnType<typeof createTarget<T>>): Promise<void> },
        initialCursor: BlockCursor | undefined,
        onFork: (previousBlocks: BlockCursor[]) => Promise<BlockCursor | null>
      ): AsyncIterableIterator<{ data: T; ctx: any }> {

        type Slot =
          | { k: 'batch'; v: { data: T; ctx: any } }
          | { k: 'end' }
          | { k: 'error'; err: unknown }

        const queue: Slot[] = []
        let consumerWake: (() => void) | null = null
        let producerAck:  (() => void) | null = null
        const wake = () => { consumerWake?.(); consumerWake = null }

        ;(source.pipeTo as any)(createTarget({
          write: async ({ read }: any) => {
            for await (const batch of read(initialCursor)) {
              queue.push({ k: 'batch', v: batch })
              wake()
              await new Promise<void>(r => { producerAck = r })
            }
            queue.push({ k: 'end' })
            wake()
          },
          fork: onFork,
        })).catch((err: unknown) => { queue.push({ k: 'error', err }); wake() })

        return {
          async next(): Promise<IteratorResult<{ data: T; ctx: any }>> {
            if (!queue.length) await new Promise<void>(r => { consumerWake = r })
            const slot = queue.shift()!
            if (slot.k === 'end')   return { done: true,  value: undefined as any }
            if (slot.k === 'error') throw slot.err
            producerAck?.(); producerAck = null
            return { done: false, value: slot.v }
          },
          [Symbol.asyncIterator]() { return this },
        }
      }
      ```
    </Expandable>

    <Expandable title="Complete example">
      ```typescript theme={"system"}

      import { BlockCursor, createTarget } from '@subsquid/pipes'
      import { evmPortalStream, evmDecoder, commonAbis } from '@subsquid/pipes/evm'

      // WORKAROUND — pipeToIterator defined above (see implementation expandable)

      async function main() {
        let recentUnfinalizedBlocks: BlockCursor[] = []
        let finalizedHighWatermark: BlockCursor | undefined

        const stream = pipeToIterator(
          evmPortalStream({
            id: 'forks-async',
            portal: 'https://portal.sqd.dev/datasets/ethereum-mainnet',
            outputs: evmDecoder({
              contracts: ['0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48'], // USDC
              events: { transfer: commonAbis.erc20.events.Transfer },
              range: { from: 'latest' }
            }),
          }),
          recentUnfinalizedBlocks.at(-1),
          async (newConsensusBlocks) => {
            const rollbackIndex = findRollbackIndex(recentUnfinalizedBlocks, newConsensusBlocks)
            if (rollbackIndex >= 0) {
              recentUnfinalizedBlocks.length = rollbackIndex + 1
              return recentUnfinalizedBlocks[rollbackIndex]
            }
            if (finalizedHighWatermark &&
                newConsensusBlocks.every(b => b.number < finalizedHighWatermark!.number)) {
              recentUnfinalizedBlocks = recentUnfinalizedBlocks.filter(b => b.number <= finalizedHighWatermark!.number)
              return finalizedHighWatermark
            }
            recentUnfinalizedBlocks.length = 0
            return null
          }
        )

        for await (const {data, ctx} of stream) {
          console.log(`Got ${data.transfer.length} transfers`)
          ctx.stream.state.rollbackChain.forEach((bc: BlockCursor) => { recentUnfinalizedBlocks.push(bc) })
          if (ctx.stream.head.finalized) {
            if (!finalizedHighWatermark || ctx.stream.head.finalized.number > finalizedHighWatermark.number) {
              finalizedHighWatermark = ctx.stream.head.finalized
            }
            recentUnfinalizedBlocks = recentUnfinalizedBlocks.filter(b => b.number >= finalizedHighWatermark!.number)
          }
          recentUnfinalizedBlocks = recentUnfinalizedBlocks.slice(recentUnfinalizedBlocks.length - 1000)
        }
      }

      main().then(() => { console.log('\ndone') })



      function findRollbackIndex(chainA: BlockCursor[], chainB: BlockCursor[]): number {
        let aIndex = 0, bIndex = 0, lastCommonIndex = -1
        while (aIndex < chainA.length && bIndex < chainB.length) {
          const a = chainA[aIndex], b = chainB[bIndex]
          if (a.number < b.number) { aIndex++; continue }
          if (a.number > b.number) { bIndex++; continue }
          if (a.hash !== b.hash) return lastCommonIndex
          lastCommonIndex = aIndex; aIndex++; bIndex++
        }
        return lastCommonIndex
      }
      ```
    </Expandable>
  </Tab>
</Tabs>

## The common-ancestor search

Both approaches use the same merge-sort scan. Given two ascending-sorted arrays of `BlockCursor` — local history and the portal's `previousBlocks` — `findRollbackIndex` returns the index in local history of the last entry that both chains agree on (same block number **and** hash):

```typescript theme={"system"}
function findRollbackIndex(chainA: BlockCursor[], chainB: BlockCursor[]): number {
  let aIndex = 0, bIndex = 0, lastCommonIndex = -1
  while (aIndex < chainA.length && bIndex < chainB.length) {
    const a = chainA[aIndex], b = chainB[bIndex]
    if (a.number < b.number) { aIndex++; continue }
    if (a.number > b.number) { bIndex++; continue }
    if (a.hash !== b.hash) return lastCommonIndex   // chains diverged here
    lastCommonIndex = aIndex; aIndex++; bIndex++
  }
  return lastCommonIndex
}
```

The scan advances the pointer for the lower-numbered entry until both point to the same block number. A hash mismatch means the chains diverged at this number; `lastCommonIndex` holds the last agreement point. Returning `-1` means no common ancestor was found in the sample.

## Edge cases and considerations

<AccordionGroup>
  <Accordion title="1. Rollback history bootstrap">
    **Empty history at stream start.** The rollback chain is built batch-by-batch from `ctx.stream.state.rollbackChain`. Until the first batch arrives the history is empty. A fork arriving before any batch has been processed means `fork()` will find no common ancestor and must return `null`, which the SDK turns into a fatal error. For a long-running process this window is typically acceptable, but it matters for freshly started consumers.

    **History gaps from fast-moving finalization.** `rollbackChain` in each batch contains only the blocks from *that batch* that are strictly above the current finalized head. A block that was already at or below the finalized head when its batch was fetched will never appear in any rollback chain and will therefore be absent from history. This can leave gaps in the number sequence. Algorithms that assume a contiguous history will fail; always match by both number *and* hash.

    **No finalized-head info in a batch.** When `batch.head.finalized` is absent, no history is accumulated. On networks or portal deployments that do not yet surface finality data, the rollback chain stays empty indefinitely. On such networks fork recovery is impossible unless unfinalized blocks are tracked through another mechanism.
  </Accordion>

  <Accordion title="2. The previousBlocks payload from the 409">
    **Ascending order, match by hash *and* number.** The API spec requires matching on both. Matching only by number is wrong — different chains can have the same block number. The array is ordered ascending (lowest number first); the last entry is the most recent block the portal knows about.

    **`previousBlocks` may have no overlap with local history.** The portal sends a bounded sample. If `findRollbackIndex` finds no agreement point at all (returns -1) and no HWM fallback applies, fork recovery is impossible — return `null`. The SDK will surface a `ForkCursorMissingError`. Do not silently roll back to block 0 or crash.

    **Multiple consecutive 409s converge to the common ancestor.** These two cases are distinct from each other: when `findRollbackIndex` *does* find an overlap point, the stream rolls back there and resumes. If the true common ancestor is deeper still — because the `previousBlocks` sample only reached partway — the portal detects another mismatch and sends a fresh 409 with an older window, this time closer to the true ancestor. The stream converges over several rounds. `fork()` must be idempotent across these calls; truncating the history array in place handles this correctly, since each call receives a shorter local history. Database-backed approaches must also handle re-entrant rollback calls.
  </Accordion>

  <Accordion title="3. Rollback depth and history limits">
    **Fork deeper than your history.** If you cap rollback history (e.g. to 1000 blocks), a reorg deeper than the cap is unrecoverable. Choose the cap based on the worst-case reorg depth for your target network. Ethereum mainnet finalizes within \~64 blocks (\~2 epochs), but PoW or pre-finality networks can reorg much deeper. Fail loudly rather than silently replaying from block 0.

    **The finalized block as the last-resort anchor.** Keep the current finalized block *in* your rollback history even though it is technically not unfinalized. It is the guaranteed safe floor: the portal will never ask you to roll back past it. Having it available means `fork()` can always return a valid cursor for the deepest possible reorg. Pruning with `number > finalized` instead of `number >= finalized` removes this anchor and makes very deep reorgs unrecoverable.

    **History that never gets pruned.** If the portal never sends a finalized head, rollback history will grow without bound. Apply a block-count cap as a secondary safeguard.
  </Accordion>

  <Accordion title="4. State rollback atomicity">
    **Business state and rollback-chain history must be rolled back atomically.** For databases with transactions (Postgres), both must be updated in the same transaction — a crash between the two leaves `fork()` computing the wrong rollback point.

    **For non-transactional databases (ClickHouse), atomicity is not achievable; use a crash-recovery callback instead.** Write application data first, write the rollback-chain checkpoint second. A crash after data but before the checkpoint save leaves the checkpoint pointing to the previous batch. On every restart, before the stream resumes, the checkpoint cursor should be read and used to purge any rows written after it — this closes the gap. This is how the Pipes SDK ClickHouse target works: `onRollback` is invoked with `type: 'offset_check'` on every startup so user code can delete the partial batch. Because ClickHouse `DELETE`s are asynchronous and unsafe under concurrent writes, the SDK inserts tombstone rows (`sign = -1`) via `CollapsingMergeTree` instead of issuing true deletes; queries that need to see only live rows must use the `FINAL` modifier.

    **Rolling back spans multiple batches.** A single reorg can invalidate data written across many batches. Your rollback mechanism must undo *all* rows/documents written after the rollback point, not just the last batch.

    **Idempotency of re-processing.** After a rollback the stream replays blocks from the rollback cursor forward. Write logic that is not idempotent (e.g. unconditional INSERT instead of UPSERT, incrementing a counter instead of setting it) will corrupt state on replay. Design writes so they are safe to run more than once for the same block.

    **Side effects that cannot be rolled back.** Database writes can be undone; emails, webhook calls, and Kafka publishes cannot. Either defer all external side effects until the block is finalized, or build a separate reconciliation layer. Treating unfinalized state as permanent is the most common source of production incidents in real-time blockchain consumers.
  </Accordion>

  <Accordion title="5. Cursor semantics">
    **The cursor returned from `fork()` is inclusive.** Return the last block you consider good; the SDK resumes from `cursor.number + 1`. Off-by-one errors cause either duplicate re-processing or skipped blocks.

    **The cursor hash must be set.** The SDK sends `parentBlockHash = cursor.hash` in the next request so the portal can detect the next fork. A cursor with a missing hash silently disables fork detection for that request.

    **The cursor in `write()`'s `read()` call is only the initial startup cursor.** `pipeTo()` handles post-fork cursor updates inside the `read()` generator; `write()` runs continuously through forks and is never restarted by the SDK. The cursor you pass to `read()` is only relevant if `write()` is re-invoked by an external retry mechanism. For in-memory implementations the cursor is effectively always `undefined`.

    **Process restart loses in-memory rollback history.** An in-memory rollback chain survives forks but not process restarts. After a restart you have no history. For services that must survive restarts, persist the rollback chain alongside application state and restore it on startup. See [Cursor management](./cursor-management) for patterns.
  </Accordion>

  <Accordion title="6. Load-balanced portals and a non-monotonic finalized head">
    **The `X-Sqd-Finalized-Head-Number` header can go backwards.** Portal instances behind a load balancer can be at different heights. When a reconnected stream lands on a lagging instance, the `finalized` value in `batch.head.finalized` may be lower than what was previously reported. Do not use the current batch's finalized number as a pruning threshold directly.

    **Treat the finalized head as a high-water mark.** Maintain the highest finalized number seen across all batches and key all pruning on that value. For database-backed implementations this is critical: a DELETE keyed on the current (possibly lower) finalized number will over-retain rows on some batches, and under-retain them if the logic is structured the other way.

    **A 409 from a lagging instance may have `previousBlocks` entirely below the high-water mark.** Two cases:

    * *All* of `previousBlocks` are strictly below the high-water mark. The lagging instance's sample doesn't reach local history. Because the high-water mark is truly final, every correct instance agrees on it: the fork is somewhere *above* it. Return the high-water mark cursor. This requires storing the finalized head as a full `BlockCursor` (number **and** hash), not just a number — the hash is needed for the next request's `parentBlockHash`.

    * Some of `previousBlocks` are at or above the high-water mark but no hash match is found. This is a genuine inconsistency at a height the client already considers final. Return `null` and surface the error.
  </Accordion>

  <Accordion title="7. Fork detection scope (real-time streams only)">
    **Forks only occur in the real-time (unfinalized) portion of the stream.** The `/finalized-stream` endpoint never returns a 409. Fork handling is only needed when consuming the `/stream` endpoint with `fromBlock` near or at the chain head. If your range is bounded and entirely in the past, you will never see a fork.

    **`parentBlockHash` is the tripwire.** Every request to the portal includes the hash of the last block the client has seen. A mismatch triggers a 409. Anything that disrupts this — starting from a cursor with a wrong or missing hash, replaying from a checkpoint that has drifted from the chain — will produce spurious fork events.
  </Accordion>

  <Accordion title="8. The rollbackChain field contract">
    **`rollbackChain` is per-batch, not cumulative.** It contains only the blocks in *this batch* that are above the current finalized head. Treat it as a delta to append to running history, not as a full snapshot of the current unfinalized chain.

    **Blocks near the finality boundary move between finalized and unfinalized.** A block that appears in one batch's `rollbackChain` may be at or below the finalized head in the next batch. The pruning filter must remove these once they are finalized, or rollback history will slowly fill with blocks that can never be the subject of a reorg.

    **Empty `rollbackChain` is valid.** It means either (a) the batch contained no blocks above the finalized head, or (b) the finalized head was unknown. Do not treat an empty rollback chain as an error.
  </Accordion>

  <Accordion title="9. Algorithm correctness for common-ancestor search">
    **Both arrays must be in ascending order.** The merge-sort scan breaks silently if either array is unsorted. Local history is ascending if you always append to the end; `previousBlocks` from the portal is ascending by protocol convention. After a rollback, the truncated history remains ascending.

    **Gaps in block numbers do not break correctness, only efficiency.** A gap (e.g. blocks 100, 101, 103 — 102 missing because it was already finalized) means a fork at 102 resolves by rolling back to 101. The extra re-processing of 102 is harmless because finalized blocks are immutable.

    **Duplicate entries break the scan.** If the same block number appears more than once with different hashes in your history, the scan may report the wrong common ancestor. UPSERT rather than INSERT when persisting rollback chain entries to a store.

    **Hash comparison requires both sides to be non-null.** `BlockCursor.hash` is optional in the type system. If either side is `undefined`, `undefined !== "0x..."` evaluates to `true`, which looks like a fork on a block that may be fine. Always verify hashes are present before comparing.
  </Accordion>

  <Accordion title="10. Concurrency and ordering invariants">
    **`fork()` is called synchronously relative to the batch stream.** The SDK awaits `fork()` before resuming the stream. No new batches arrive while `fork()` is running. It is safe to mutate shared state inside `fork()` without additional locking.

    **`write()` and `fork()` share mutable state without synchronization.** This is safe only because the SDK never calls them concurrently. If you introduce background workers or async tasks that also read or write rollback state, you must add explicit synchronization.

    **The order in which you update rollback history and application state matters.** If you update application state first and crash before updating rollback history, the next restart will not know how far to roll back. Prefer database transactions that update both atomically, or update rollback history first so a crash leaves you conservative — you can always re-process a block you have already seen.
  </Accordion>
</AccordionGroup>


# Pipe anatomy
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/evm/guides/basic-development/anatomy

How a pipe is put together

<Frame>
  <img alt="Pipe anatomy" />
</Frame>

An EVM pipe made with SQD's Pipes SDK consists of:

* A **source** - typically made with `evmPortalSource()`. Can have one or more outputs.

* **Queries** - tell the source which data has to be retrieved to compute each output. A query is defined by a chain call terminated by `.build()`. Here's an example:

  ```ts theme={"system"}
  evmQuery()
    .addFields({
      block: { timestamp: true },
      log: { address: true, transactionHash: true },
    })
    .addLog({
      topic0: [ TRANSFER_TOPIC ]
    })
    .build()
  ```

* **Per-query transforms** (optional) - you can pass data from each query through a chain of simple transforms:
  ```ts theme={"system"}
  query
    .pipe(data => data.map(item => ({
      funkyNumber: item.header.timestamp + item.header.number,
      ...item
    })))
    .pipe(someOtherSimpleTransformCallback)
  ```
  Source object streams the data you get out of each chain of transforms as the value of the corresponding output field.

* Making utils that return reusable **query-transform combos** is a very useful pattern. In particular, on EVM it is often convenient to keep retrieval and decoding of event logs in a single module. You can easily make such combos with the `evmDecoder()` function - see the [Handling events](./handling-events) guide.

* **Whole pipe transformers** (optional) - use this if you need to compute something based on data originating from multiple queries, or if you need access to per-batch context (cursor, logger, profiler, fork callbacks). Use `createTransformer()` so the SDK can thread cursor and rollback information ([1](../architecture-deep-dives/cursor-management), [2](../architecture-deep-dives/fork-handling)) through your transform:
  ```ts theme={"system"}
  import { createTransformer } from '@subsquid/pipes'

  const enrichTransfers = createTransformer<
    { transfers: Transfer[]; approvals: Approval[] },
    { events: EnrichedEvent[] }
  >({
    transform: ({ transfers, approvals }, ctx) => {
      ctx.logger.info({ batch: ctx.stream.state.current?.number }, 'enriching')
      return {
        events: [
          ...transfers.map((t) => ({ kind: 'transfer' as const, ...t })),
          ...approvals.map((a) => ({ kind: 'approval' as const, ...a })),
        ],
      }
    },
  })

  evmPortalStream({ /* ... */ }).pipe(enrichTransfers)
  ```

* Pipe termination: a plain async iterator or a **target**.
  * If you use the pipe as an async iterator it will throw exceptions if the underlying chain is experiencing reorgs, see [Fork handling](../architecture-deep-dives/fork-handling).
  * We offer two targets out of the box:

    * Postgres via Drizzle
    * ClickHouse

    You can make your own using [`createTarget()`](../../reference/basic-components/target/create-target).


# Developing pipes
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/evm/guides/basic-development/flow

Typical development workflow for pipes

Begin by making sure you know

* which on-chain data items (transations, event logs, traces etc) you need;
* how you are going to transform these items into usable data, based on your business logic;
* what is your preferred mode of consuming the transformed data.

Use [Pipes CLI](../../quickstart) to quickly generate a starter project. The process of getting from here to a useful data pipeline follows directly from [Pipe anatomy](./anatomy).

## Adding queries and developing transforms

Here are some things to keep in mind as you're developing the heart of your pipeline.

### Writing maintainable pipes

Recall that there are two kinds of transforms in Pipes SDK:

* **Per-query transforms** work on subsets of raw data. They can be bundled with their queries, making them logically self-contained and easily reusable.
* **Whole pipe transforms** process outputs of all per-query transforms at the same time. This unlocks arbitrary data combinations, but it also means that that each such transform might need to be changed whenever any of the upstream transforms changes.

For maximum mainatainability you'll have to balance the following two objectives:

1. Aim to push as much of your business logic into per-query transforms. Make the code of any whole pipe transforms as simple as possible.

2. Use ready-made, validated query-transform combos. For example, [evmDecoder()](./handling-events) fetches and decodes contract event logs, supports factory-discovered contracts, and indexed parameter filtering. It'll often be preferable to build your pipeline out of such modules, even if that happens to make whole-pipe transforms slightly more complicated.

### Stateful transforms

It's often the case that you need access to some part of the previously processed data to do the transform. For example, to compute a running ERC20 balance from transfers you need to know its value preceeding the current transfer. There are multiple ways to accomplish this in Pipes SDK, each with its advantages and disadvantages. Consult the [Stateful transforms](../advanced-topics/stateful-transforms) guide.

## Writing data

### Postgres and ClickHouse

If you need your transformed data in Postgres or ClickHouse, you should already have a basic configuration generated by [Pipes CLI](../../quickstart).

If you're working with real-time data, **it is very important to**

* **on Postgres** when adding or removing any relevant tables: update the list of tables in the target configuration;
* **on ClickHouse** when any data dependencies or structure of the stored data changes: update the `onRollback()` callback.

Consult the [Postgres via Drizzle](./targets/postgres-drizzle) and [ClickHouse](./targets/clickhouse) guides.

### Plain iterator

A complete pipeline without a `.pipeTo` is a valid async iterator.

* The pipeline will produce some logs by default. Disable them by setting `logger: false` when creating the data source. If you're looking to convert an existing standalone pipe into a module in a larger program and wish to get rid of any side effects, consult the [Running bare bones](./running-bare-bones) guide.

* If you're working with unfinalized data (default setting of the source), the iterator will throw `ForkException`s on blockchain reorgs. You should catch these and process them correctly. Consult the [fork handling guide](../architecture-deep-dives/fork-handling) for details.

  Alternatively, configure the data source to use final data only:

  ```ts theme={"system"}
  const source = evmPortalSource({
    portal: {
      url: '<portal_dataset_url>',
      finalized: true,
    },
    ...
  })
  ```

* By default, the pipeline is stateless: when re-created it'll restart from the earliest block relevant to any of the queries. If you want the pipeline to persist its sync state between restarts, you'll have to manage the state by yourself. See [Cursor management](../architecture-deep-dives/cursor-management).

### Developing your own target

Use the [createTarget() function](../../reference/basic-components/target/create-target).

* If you're working with unfinalized data (default setting of the source), you must define a fork handler callback. Consult the [fork handling guide](../architecture-deep-dives/fork-handling) for details.

  Alternatively, configure the data source to use final data only:

  ```ts theme={"system"}
  const source = evmPortalSource({
    portal: {
      url: '<portal_dataset_url>',
      finalized: true,
    },
    ...
  })
  ```

* If you want your pipeline to preserve its sync state between restarts, you'll have to manage this state in your `write` callback. See [Cursor management](../architecture-deep-dives/cursor-management).


# Handling contract events
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/evm/guides/basic-development/handling-events

Fetching and decoding EVM event logs with evmDecoder

`evmDecoder()` bundles an EVM log query with a decoding transform into a single reusable module. Pass the result as an output to `evmPortalStream`:

```ts theme={"system"}
import { commonAbis, evmDecoder, evmPortalStream } from '@subsquid/pipes/evm'

const stream = evmPortalStream({
  portal: 'https://portal.sqd.dev/datasets/ethereum-mainnet',
  outputs: {
    transfers: evmDecoder({
      range: { from: '0' },
      contracts: ['0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48'],
      events: { transfer: commonAbis.erc20.events.Transfer },
    }),
  },
})
```

`evmDecoder()` can:

* Fetch from **specific contracts** — pass an array of addresses to `contracts`. Omit it entirely to receive matching events from every contract on-chain.
* Filter by **indexed parameters** — instead of a bare event, supply `{ event, params }` to select only logs where specific indexed arguments match.
* Dynamically discover contracts via **factories** — pass a `contractFactory()` to `contracts` instead of a static list. See the [Factory guide](../advanced-topics/factory-transformers).
* Handle decode errors with a custom **`onError` callback** instead of letting them propagate.

See the [evmDecoder() reference](../../reference/utility-components/evm-decoder) for all parameters.

## Specifying events

The `events` parameter maps output field names to event specifications. There are three ways to obtain an event specification.

### `commonAbis`

`commonAbis` is a built-in collection of ABI modules for common token standards. Currently it ships one module, `erc20`:

```ts theme={"system"}
import { commonAbis, evmDecoder } from '@subsquid/pipes/evm'

evmDecoder({
  range: { from: 'latest' },
  contracts: ['0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48'],
  events: {
    transfers: commonAbis.erc20.events.Transfer,
    approvals: commonAbis.erc20.events.Approval,
  },
})
```

See the [`commonAbis` reference](../../reference/utility-components/evm-decoder#commonabis) for the full list of available events and functions.

### Typegen modules

`@subsquid/evm-typegen` generates TypeScript ABI modules from JSON ABIs. Each generated module exports typed `events` and `functions` objects, translating Solidity types to TypeScript — event argument types are statically known at compile time, so you get precise type checking and IDE autocompletion across the entire pipeline.

**Install the tool:**

```bash theme={"system"}
npm install -D @subsquid/evm-typegen
```

**Generate a module from a local JSON ABI file:**

```bash theme={"system"}
npx squid-evm-typegen src/abi your-contract.json
```

This creates `src/abi/your-contract.ts`. The tool also accepts a contract address (requires specifying `--chain-id`) or an arbitrary URL.

Use events from a generated module exactly as with `commonAbis`:

```ts theme={"system"}
import * as usdcAbi from './abi/usdc'

evmDecoder({
  range: { from: 'latest' },
  contracts: ['0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48'],
  events: {
    transfers: usdcAbi.events.Transfer,
    approvals: usdcAbi.events.Approval,
  },
})
```

### Raw JSON via `defineAbi()`

`defineAbi()` converts a JSON ABI array to a subsquid ABI module at runtime, with no code generation step. This is the quickest route — useful for one-off scripts or prototypes — but it comes at a cost: when the ABI is loaded from an external JSON file, event argument fields are typed as `any`, since TypeScript cannot inspect the runtime JSON value at compile time.

```ts theme={"system"}
import { defineAbi, evmDecoder } from '@subsquid/pipes/evm'
import erc20Json from './erc20.json'

const erc20 = defineAbi(erc20Json) // event args are `any`

evmDecoder({
  range: { from: 'latest' },
  contracts: ['0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48'],
  events: { transfers: erc20.events.Transfer },
})
```

`defineAbi()` also accepts Hardhat and Foundry artifact objects — it reads the `abi` field automatically:

```ts theme={"system"}
import artifact from './artifacts/MyContract.json'

const myContract = defineAbi(artifact) // reads artifact.abi
```

If you define the ABI inline with `as const`, TypeScript can infer the exact decoded types for scalar fields:

```ts theme={"system"}
const erc20 = defineAbi([
  {
    type: 'event',
    name: 'Transfer',
    inputs: [
      { indexed: true,  name: 'from',  type: 'address' },
      { indexed: true,  name: 'to',    type: 'address' },
      { indexed: false, name: 'value', type: 'uint256' },
    ],
  },
] as const)

// erc20.events.Transfer.decode() returns { from: string, to: string, value: bigint }
```

For projects where full type safety matters end-to-end, prefer the [typegen route](#typegen-modules) instead.


# Pipes UI
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/evm/guides/basic-development/pipes-ui

Live dashboard for monitoring a running pipe

Pipes UI is a local web dashboard that connects to a running pipe and visualises its progress, speed, portal query, and profiler breakdown. It reads the metrics server that the SDK exposes on the pipe process — nothing needs to be deployed or hosted.

<Frame>
  <video />
</Frame>

## Expose metrics on the pipe

Attach [`metricsServer()`](../advanced-topics/metrics) to the source. Listens on `localhost:9090` by default.

```ts theme={"system"}
import { evmPortalStream, evmDecoder, commonAbis } from '@subsquid/pipes/evm'
import { metricsServer } from '@subsquid/pipes/metrics/node'

evmPortalStream({
  id: 'my-pipe',                       // shows up in the dashboard as the pipe name
  portal: 'https://portal.sqd.dev/datasets/ethereum-mainnet',
  outputs: evmDecoder({
    profiler: { name: 'transfers' },   // labels this span in the profiler tree
    range: { from: 'latest' },
    events: { transfers: commonAbis.erc20.events.Transfer },
  }),
  metrics: metricsServer(),            // exposes /metrics, /stats, /profiler, /health on :9090
})
```

Start the pipe as usual (`ts-node`, `bun`, compiled JS, etc.).

## Run the dashboard

In a second terminal:

```bash theme={"system"}
npx @subsquid/pipes-ui@alpha
```

The UI serves on `http://localhost:3000` and polls the metrics server at `http://localhost:9090`. Open the URL in a browser — the page auto-refreshes once the pipe starts producing batches.

## What it shows

Per pipe (keyed by the `id` passed to the source):

* chain / dataset, with the inferred chain kind (EVM, Solana, …)
* progress: current block, target block, percent complete, ETA
* throughput: blocks/s and bytes/s over the last 30 samples
* the serialised portal query (helpful for reviewing what your decoder actually asked for)
* memory usage of the pipe process and SDK version

When [profiling is on](../advanced-topics/profiling) (the default in non-production environments), the UI also renders the per-batch span tree — useful for seeing which stage (`fetch data`, `apply transformers`, a named decoder, your own `ctx.profiler.start('…')` spans) is dominating batch time. Decorate spans you want to track with `profiler: { name: '…' }` on transformers and decoders.

Any [custom metrics](../advanced-topics/metrics) you register via `ctx.metrics.counter()`, `.gauge()`, `.histogram()`, or `.summary()` show up on the pipe's `/metrics` endpoint (as Prometheus text). The dashboard does not render arbitrary custom series — if you need charts for your own metrics, scrape `/metrics` with Prometheus and graph with Grafana.

The full list of HTTP endpoints served by the metrics process (useful for ad-hoc `curl` inspection) is in the [metricsServer reference](../../reference/utility-components/metrics-server#endpoints).

## Troubleshooting

* **"Failed to reach metrics server"** on the UI — the pipe is not running, or `metricsServer()` is not attached to the source, or it listens on a non-default port. Start the pipe first, then reload the dashboard.
* **UI shows no pipes** — the source config is missing an `id`. Add `id: 'my-pipe'` to the source options.
* **Profiler tab is empty** — the pipe has `profiler: false` set on the source, or `NODE_ENV=production` (the default is to enable profiling only outside production). Set `profiler: true` on the source to force it on. See [Profiling](../advanced-topics/profiling).


# Running bare bones
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/evm/guides/basic-development/running-bare-bones

Using a pipe as a plain async iterator without extra services

By default, `evmPortalSource` activates a console logger. Passing `metrics` and `progress` enables those services. To embed a pipe into external code with no side effects, disable them all:

```ts theme={"system"}
import { commonAbis, evmDecoder, evmPortalStream } from '@subsquid/pipes/evm'

const stream = evmPortalStream({
  portal: 'https://portal.sqd.dev/datasets/ethereum-mainnet',
  outputs: evmDecoder({
    range: { from: 0 },
    events: { transfers: commonAbis.erc20.events.Transfer },
  }),
  logger: false,   // disable all log output
  profiler: false, // profiler disabled under all circumstances
  // omit `metrics` — no metrics server
  // omit `progress` — no progress reporting
})

for await (const { data } of stream) {
  // data.transfers is available here
  console.log(data.transfers.length)
}
```

The source becomes a plain async iterable that yields `{ data, ctx }` per batch. `ctx.logger` is a no-op when `logger: false`.


# ClickHouse
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/evm/guides/basic-development/targets/clickhouse

Store pipe output in ClickHouse

Install the ClickHouse Node.js client:

```bash theme={"system"}
npm install @clickhouse/client
```

At a glance, the pipeline looks like this:

```ts theme={"system"}
import { createClient } from '@clickhouse/client'
import { clickhouseTarget } from '@subsquid/pipes/targets/clickhouse'

await evmPortalSource({ ... }).pipeTo(
  clickhouseTarget({
    client: createClient({ url: 'http://localhost:8123' }),
    onData: async ({ store, data }) => {
      store.insert({ table: 'transfers', values: data.transfers.map(...), format: 'JSONEachRow' })
    },
    onRollback: async ({ store, safeCursor }) => {
      await store.removeAllRows({ tables: ['transfers'], where: `block_number > ${safeCursor.number}` })
    },
  }),
)
```

## Table design

Use `CollapsingMergeTree` with a `sign Int8 DEFAULT 1` column. This engine enables efficient fork rollbacks: to cancel rows, the target re-inserts them with `sign = -1` and ClickHouse merges the pair during background processing.

```sql theme={"system"}
CREATE TABLE IF NOT EXISTS transfers (
  block_number     UInt32  CODEC(DoubleDelta, ZSTD),
  transaction_hash String,
  log_index        UInt16,
  from_address     LowCardinality(FixedString(42)),
  to_address       LowCardinality(FixedString(42)),
  value            UInt256,
  sign             Int8 DEFAULT 1
) ENGINE = CollapsingMergeTree(sign)
  ORDER BY (block_number, transaction_hash, log_index);
```

Design notes:

* Apply `DoubleDelta + ZSTD` codecs to monotonically increasing columns such as block numbers and timestamps.
* Use `LowCardinality` for columns with low cardinality like addresses to reduce storage and speed up filtering.
* Store 256-bit integers as `UInt256`; serialize JavaScript `BigInt` values to strings before insertion.

Create the table in `onStart` using `store.command()`:

```ts theme={"system"}
onStart: async ({ store }) => {
  await store.command({ query: `CREATE TABLE IF NOT EXISTS transfers ( ... )` })
}
```

## `onData`

Call `store.insert()` to queue an insert. The call is non-blocking — inserts fire concurrently and are fully flushed when the target closes:

```ts theme={"system"}
onData: async ({ store, data }) => {
  store.insert({
    table: 'transfers',
    values: data.transfers.map((t) => ({
      block_number:     t.block.number,
      transaction_hash: t.rawEvent.transactionHash,
      log_index:        t.rawEvent.logIndex,
      from_address:     t.event.from,
      to_address:       t.event.to,
      value:            t.event.value.toString(),
    })),
    format: 'JSONEachRow',
  })
}
```

## `onRollback`

Implement `onRollback` to handle blockchain forks. It is invoked in two situations:

* `type: 'offset_check'` — on startup, when a saved cursor is found, to discard writes from a previous crashed or partial run
* `type: 'blockchain_fork'` — when the stream detects a chain reorganisation

Use `store.removeAllRows()` to cancel rows past the safe point. For `CollapsingMergeTree` tables this re-inserts matching rows with `sign = -1`:

```ts theme={"system"}
onRollback: async ({ store, safeCursor }) => {
  await store.removeAllRows({
    tables: ['transfers'],
    where: `block_number > ${safeCursor.number}`,
  })
}
```

## Complete example

```ts expandable theme={"system"}
import { commonAbis, evmDecoder, evmPortalSource } from '@subsquid/pipes/evm'
import { clickhouseTarget } from '@subsquid/pipes/targets/clickhouse'
import { createClient } from '@clickhouse/client'

const client = createClient({ url: 'http://localhost:8123' })

await evmPortalSource({
  portal: 'https://portal.sqd.dev/datasets/ethereum-mainnet',
  outputs: evmDecoder({
    range: { from: 'latest' },
    events: { transfers: commonAbis.erc20.events.Transfer },
  }),
}).pipeTo(
  clickhouseTarget({
    client,
    onStart: async ({ store }) => {
      await store.command({
        query: `
          CREATE TABLE IF NOT EXISTS transfers (
            block_number     UInt32  CODEC(DoubleDelta, ZSTD),
            transaction_hash String,
            log_index        UInt16,
            from_address     LowCardinality(FixedString(42)),
            to_address       LowCardinality(FixedString(42)),
            value            UInt256,
            sign             Int8 DEFAULT 1
          ) ENGINE = CollapsingMergeTree(sign)
            ORDER BY (block_number, transaction_hash, log_index)
        `,
      })
    },
    onData: async ({ store, data }) => {
      store.insert({
        table: 'transfers',
        values: data.transfers.map((t) => ({
          block_number:     t.block.number,
          transaction_hash: t.rawEvent.transactionHash,
          log_index:        t.rawEvent.logIndex,
          from_address:     t.event.from,
          to_address:       t.event.to,
          value:            t.event.value.toString(),
        })),
        format: 'JSONEachRow',
      })
    },
    onRollback: async ({ store, safeCursor }) => {
      await store.removeAllRows({
        tables: ['transfers'],
        where: `block_number > ${safeCursor.number}`,
      })
    },
  }),
)
```

## Docker setup

```yaml docker-compose.yml theme={"system"}
services:
  clickhouse:
    image: clickhouse/clickhouse-server:latest
    ports:
      - "8123:8123"
      - "9000:9000"
    environment:
      CLICKHOUSE_DB: default
      CLICKHOUSE_USER: default
      CLICKHOUSE_PASSWORD: default
    volumes:
      - clickhouse-data:/var/lib/clickhouse

volumes:
  clickhouse-data:
```

```bash theme={"system"}
docker compose up -d
```

See the [clickhouseTarget reference](../../../reference/basic-components/target/clickhouse) for the full API.


# Postgres via Drizzle
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/evm/guides/basic-development/targets/postgres-drizzle

Store pipe output in PostgreSQL using Drizzle ORM

Install Drizzle ORM and the PostgreSQL driver:

```bash theme={"system"}
npm install drizzle-orm pg
npm install -D drizzle-kit @types/pg
```

At a glance, the pipeline looks like this:

```ts theme={"system"}
await evmPortalSource({ ... }).pipeTo(
  drizzleTarget({
    db: drizzle('postgresql://...'),
    tables: [transfersTable],
    onData: async ({ tx, data }) => {
      for (const batch of batchForInsert(data.transfers)) {
        await tx.insert(transfersTable).values(batch.map(...))
      }
    },
  }),
)
```

## Schema

Define your tables with Drizzle ORM. Every table needs a primary key, and every table written to in [`onData`](#ondata) must appear in [`tables`](#tables).

```ts theme={"system"}
import { integer, numeric, pgTable, primaryKey, varchar } from 'drizzle-orm/pg-core'

const transfersTable = pgTable('transfers', {
  blockNumber: integer().notNull(),
  logIndex:    integer().notNull(),
  from:        varchar({ length: 42 }).notNull(),
  to:          varchar({ length: 42 }).notNull(),
  value:       numeric({ mode: 'bigint' }).notNull(),
}, (t) => [primaryKey({ columns: [t.blockNumber, t.logIndex] })])
```

## `onData` and `batchForInsert`

`onData` runs inside a serializable transaction. Use `batchForInsert` to split data arrays into chunks that fit within PostgreSQL's 32,767-parameter limit — chunk size is calculated automatically from the number of columns:

```ts theme={"system"}
import { batchForInsert, drizzleTarget } from '@subsquid/pipes/targets/drizzle/node-postgres'

onData: async ({ tx, data }) => {
  for (const batch of batchForInsert(data.transfers)) {
    await tx.insert(transfersTable).values(
      batch.map((d) => ({
        blockNumber: d.block.number,
        logIndex:    d.rawEvent.logIndex,
        from:        d.event.from,
        to:          d.event.to,
        value:       d.event.value,
      })),
    )
  }
}
```

Pass an explicit second argument to `batchForInsert` to cap chunk size:

```ts theme={"system"}
for (const batch of batchForInsert(data.transfers, 100)) { ... }
```

## `tables`

Every table written to in `onData` must be listed in `tables`. At startup, the target installs PostgreSQL trigger functions on these tables to track row-level changes for automatic fork handling. Inserting into an unlisted table throws at runtime.

## Schema migrations

Use [Drizzle Kit](https://orm.drizzle.team/docs/kit-overview) to generate and apply migrations:

```bash theme={"system"}
npx drizzle-kit generate
npx drizzle-kit migrate
```

Alternatively, run migrations automatically on startup via `onStart`:

```ts theme={"system"}
import { migrate } from 'drizzle-orm/node-postgres/migrator'

drizzleTarget({
  db,
  tables: [...],
  onStart: async ({ db }) => {
    await migrate(db, { migrationsFolder: './drizzle' })
  },
  onData: async ({ tx, data }) => { ... },
})
```

## Rollback handling

Fork handling is fully automatic. Each batch runs inside a transaction that snapshots row-level changes. When the stream detects a fork, the target replays those snapshots in reverse to restore the pre-fork state.

Use `onBeforeRollback` and `onAfterRollback` to run custom logic around a rollback. Both callbacks receive the Drizzle transaction and the `cursor` (`BlockCursor`) to which state was rolled back:

```ts theme={"system"}
drizzleTarget({
  db,
  tables: [...],
  onBeforeRollback: async ({ tx, cursor }) => { /* e.g. log or acquire an external lock */ },
  onAfterRollback:  async ({ tx, cursor }) => { /* e.g. invalidate a cache */ },
  onData: async ({ tx, data }) => { ... },
})
```

## Complete example

```ts expandable theme={"system"}
import { commonAbis, evmDecoder, evmPortalSource } from '@subsquid/pipes/evm'
import { batchForInsert, drizzleTarget } from '@subsquid/pipes/targets/drizzle/node-postgres'
import { drizzle } from 'drizzle-orm/node-postgres'
import { integer, numeric, pgTable, primaryKey, varchar } from 'drizzle-orm/pg-core'

const transfersTable = pgTable('transfers', {
  blockNumber: integer().notNull(),
  logIndex:    integer().notNull(),
  from:        varchar({ length: 42 }).notNull(),
  to:          varchar({ length: 42 }).notNull(),
  value:       numeric({ mode: 'bigint' }).notNull(),
}, (t) => [primaryKey({ columns: [t.blockNumber, t.logIndex] })])

await evmPortalSource({
  portal: 'https://portal.sqd.dev/datasets/ethereum-mainnet',
  outputs: evmDecoder({
    range: { from: '0' },
    events: { transfers: commonAbis.erc20.events.Transfer },
  }),
}).pipeTo(
  drizzleTarget({
    db: drizzle('postgresql://postgres:postgres@localhost:5432/postgres'),
    tables: [transfersTable],
    onData: async ({ tx, data }) => {
      for (const batch of batchForInsert(data.transfers)) {
        await tx.insert(transfersTable).values(
          batch.map((d) => ({
            blockNumber: d.block.number,
            logIndex:    d.rawEvent.logIndex,
            from:        d.event.from,
            to:          d.event.to,
            value:       d.event.value,
          })),
        )
      }
    },
  }),
)
```

See the [drizzleTarget reference](../../../reference/basic-components/target/postgres-drizzle) for the full API.


# Quickstart
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/evm/quickstart

Bootstrap a Pipes SDK project

# Using with AI

The fastest way to get an AI coding agent productive on a Pipes SDK project is to install the official [Pipes SDK Agent Skill](/en/ai/agent-skills#pipes-sdk-skill):

```bash theme={"system"}
npx skills add subsquid-labs/skills/pipes-sdk
```

The skill activates automatically on tasks like *"create an indexer for Uniswap V3 swaps"* or *"my indexer is syncing slowly, help me optimize it"*. It covers scaffolding, runtime error diagnosis, sync tuning, and data-quality checks.

Pair the skill with one or both MCP servers so the agent can read live data and look things up:

* [Portal MCP server](/en/ai/mcp-server) — 29 tools for querying blocks, transactions, logs, instructions, and analytics across 225+ datasets. No API key.
* [Documentation MCP server](/en/ai/mcp-server-docs) — search and retrieve these docs from inside the agent.

If you'd rather feed docs into a model directly, the static [`llms.txt`](/llms.txt) (index) and [`llms-full.txt`](/llms-full.txt) (full content) files are kept in sync with the site. See the [AI Development overview](/en/ai/ai-development) for the full menu.

# Scaffolding with Pipes CLI

<Note>
  `pipes-cli` is a work in progress.
</Note>

In a few minutes, you'll have a running pipe that indexes USDC token transfers on Ethereum mainnet into a local PostgreSQL database.

## Prerequisites

* Node.js 22.15+
* `pnpm`
* Docker (for the bundled PostgreSQL container)

## Initialize the project

Run the CLI in the directory where you want the project folder to land:

```bash theme={"system"}
pnpx @subsquid/pipes-cli@1.0.0-alpha.4 init
```

The CLI prompts for the project folder name, package manager (please stick to `pnpm` for now), sink (please use `ClickHouse` or `Postgres`), network type, network, and template; then installs dependencies and writes a runnable project.

You can supply a JSON config instead of filling the prompts manually. Here's the configuration for USDC token transfers mentioned above:

```bash theme={"system"}
pnpx @subsquid/pipes-cli@1.0.0-alpha.4 init --config '{
  "projectFolder": "usdc-example",
  "packageManager": "pnpm",
  "sink": "postgresql",
  "networkType": "evm",
  "network": "ethereum-mainnet",
  "templates": [
    {
      "templateId": "erc20Transfers",
      "params": {
        "contractAddresses": ["0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48"],
        "range": { "from": "latest" }
      }
    }
  ]
}'
```

`--config` also accepts a path to a JSON file.

To inspect the full config schema run

```
pnpx @subsquid/pipes-cli@1.0.0-alpha.4 init --schema
```

## Run the pipeline

The generated project ships with a `docker-compose.yml` that brings up the sink database and the pipeline together:

```bash theme={"system"}
cd usdc-example
docker compose --profile with-pipeline up
```

For an iterative dev loop, run the database in Docker and the pipeline locally:

```bash theme={"system"}
docker compose up -d         # Postgres on :5432
pnpm run db:migrate          # apply the generated migration
pnpm run dev                 # tsx src/index.ts
```

Either way, rows start landing in the `erc20_transfers` table within a minute.

## What was generated

The project layout:

```
usdc-example/
├── src/
│   ├── index.ts        # the pipe — source, decoder, target
│   ├── schemas.ts      # Drizzle table definitions
│   └── utils/
├── migrations/         # SQL migrations generated by drizzle-kit
├── docker-compose.yml  # Postgres + optional pipeline service
├── Dockerfile
├── drizzle.config.ts
├── package.json
├── .env                # DB_CONNECTION_STR — points at local Postgres
└── README.md
```

The pipe lives in `src/index.ts`. The decoder block defines what to extract + a light transform:

```ts theme={"system"}
const erc20Transfers = evmDecoder({
  profiler: { name: 'erc20-transfers' },
  range: { from: 'latest' },
  contracts: ['0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48'],
  events: { transfers: commonAbis.erc20.events.Transfer },
}).pipe(({ transfers }) =>
  transfers.map((transfer) => ({
    blockNumber: transfer.block.number,
    txHash: transfer.rawEvent.transactionHash,
    logIndex: transfer.rawEvent.logIndex,
    timestamp: transfer.timestamp.getTime(),
    from: transfer.event.from,
    to: transfer.event.to,
    value: transfer.event.value,
    tokenAddress: transfer.contract,
  })),
)
```

This query-transform combo asks the Portal for ERC20 `Transfer` logs from the USDC contract, decodes them and (in the `.pipe` step) reshapes each one into a row matching the Drizzle table. See the [Pipe anatomy](./guides/basic-development/anatomy) and [Handling contract events](./guides/basic-development/handling-events) guides for more info on `evmDecoder()`.

The `main()` function wires the decoder to a [drizzleTarget](./reference/basic-components/target/postgres-drizzle):

```ts theme={"system"}
export async function main() {
  await evmPortalSource({
    id: 'ethereum-usdc-pipe',
    portal: 'https://portal.sqd.dev/datasets/ethereum-mainnet',
    outputs: { erc20Transfers },
  }).pipeTo(
    drizzleTarget({
      db: drizzle(env.DB_CONNECTION_STR),
      tables: [erc20TransfersTable],
      onData: async ({ tx, data }) => {
        for (const values of chunk(data.erc20Transfers)) {
          await tx.insert(erc20TransfersTable).values(values)
        }
      },
    }),
  )
}
```

The `id` is a per-pipeline identifier — keep it stable so the [target's cursor](./guides/architecture-deep-dives/cursor-management) survives restarts. See [evmPortalSource](./reference/basic-components/source) for the full source API and [Pipe anatomy](./guides/basic-development/anatomy) for how the pieces fit together.

## Other examples

<AccordionGroup>
  <Accordion title="Uniswap V3 swaps via factory">
    Tracks every pool created by the Uniswap V3 factory and indexes its `Swap` events. The generated decoder uses [factory transformers](./guides/advanced-topics/factory-transformers) with a SQLite-backed pool registry.

    ```bash theme={"system"}
    pnpx @subsquid/pipes-cli@1.0.0-alpha.4 init --config '{
      "projectFolder": "uniswapv3-swaps",
      "packageManager": "pnpm",
      "sink": "postgresql",
      "networkType": "evm",
      "network": "ethereum-mainnet",
      "templates": [
        {
          "templateId": "uniswapV3Swaps",
          "params": {
            "factoryAddress": "0x1f98431c8ad98523631ae4a59f267346ea31f984",
            "range": { "from": "latest" }
          }
        }
      ]
    }'
    ```
  </Accordion>

  <Accordion title="Aave V3 supply / withdraw (custom template)">
    The `custom` template generates ABI bindings and decoder wiring from an event list you provide. Drop in any contract and event set.

    ```bash theme={"system"}
    pnpx @subsquid/pipes-cli@1.0.0-alpha.4 init --config '{
      "projectFolder": "aave-supply-withdraw",
      "packageManager": "pnpm",
      "sink": "postgresql",
      "networkType": "evm",
      "network": "ethereum-mainnet",
      "templates": [
        {
          "templateId": "custom",
          "params": {
            "contracts": [
              {
                "contractAddress": "0x87870Bca3F3fD6335C3F4ce8392D69350B4fA4E2",
                "contractName": "AaveV3Pool",
                "contractEvents": [
                  {
                    "anonymous": false,
                    "inputs": [
                      { "indexed": true, "name": "reserve", "type": "address" },
                      { "indexed": false, "name": "user", "type": "address" },
                      { "indexed": true, "name": "onBehalfOf", "type": "address" },
                      { "indexed": false, "name": "amount", "type": "uint256" },
                      { "indexed": true, "name": "referralCode", "type": "uint16" }
                    ],
                    "name": "Supply",
                    "type": "event"
                  },
                  {
                    "anonymous": false,
                    "inputs": [
                      { "indexed": true, "name": "reserve", "type": "address" },
                      { "indexed": true, "name": "user", "type": "address" },
                      { "indexed": true, "name": "to", "type": "address" },
                      { "indexed": false, "name": "amount", "type": "uint256" }
                    ],
                    "name": "Withdraw",
                    "type": "event"
                  }
                ],
                "range": { "from": "latest" }
              }
            ]
          }
        }
      ]
    }'
    ```
  </Accordion>
</AccordionGroup>

The CLI ships two built-in EVM templates — `erc20Transfers` and `uniswapV3Swaps` — plus the open-ended `custom` template.


# Query builder
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/evm/reference/basic-components/query-builder

API reference for EvmQueryBuilder

`EvmQueryBuilder` assembles a typed portal query from a field selection and one or more data-request clauses. Pass `.build()` to `outputs` (or to `.pipe()`) on an EVM source. The resulting object is consumed by [`evmPortalStream()`](./source) and by the [evmDecoder](../utility-components/evm-decoder) which composes on top of it.

## `evmQuery()`

Returns a fresh `EvmQueryBuilder`.

```ts theme={"system"}
import { evmQuery } from '@subsquid/pipes/evm'

const query = evmQuery()
  .addFields({
    block: { timestamp: true },
    log: { address: true, topics: true, data: true },
  })
  .addLog({
    range: { from: 20_000_000 },
    request: { topic0: ['0xddf252ad…'] },
  })
  .build()
```

## `EvmQueryBuilder<F>`

```ts theme={"system"}
class EvmQueryBuilder<F extends FieldSelection = {}> {
  addFields<T>(fields: Subset<T, FieldSelection>): EvmQueryBuilder<F & T>
  addLog(options: RequestOptions<LogRequest>): this
  addTransaction(options: RequestOptions<TransactionRequest>): this
  addTrace(options: RequestOptions<TraceRequest>): this
  addStateDiff(options: RequestOptions<StateDiffRequest>): this
  addRange(range: PortalRange): this
  merge(other?: EvmQueryBuilder<F>): this
  build(opts?: { setupQuery?: SetupQueryFn<EvmQueryBuilder<F>> }): QueryAwareTransformer
}
```

The generic parameter `F` narrows the block type produced by the stream — only fields explicitly selected with `.addFields()` appear on the decoded records, at both compile and runtime.

### `RequestOptions<R>`

```ts theme={"system"}
type RequestOptions<R> = { range: PortalRange; request: R }
type PortalRange = { from?: number | string | 'latest' | Date; to?: number | string | Date }
```

`PortalRange.from` defaults to `0`; a `Date` or numeric timestamp is resolved to a block number via the portal at query start. `'latest'` is resolved to the current head.

***

## `.addFields(fields)`

Add to the field selection. Repeated calls are merged recursively. Block hash and number are returned regardless of selection.

### `block`

| Field              | Type                       |
| ------------------ | -------------------------- |
| `number`           | `number` (always returned) |
| `hash`             | `string` (always returned) |
| `parentHash`       | `string`                   |
| `timestamp`        | `number` (Unix seconds)    |
| `transactionsRoot` | `string`                   |
| `receiptsRoot`     | `string`                   |
| `stateRoot`        | `string`                   |
| `logsBloom`        | `string`                   |
| `sha3Uncles`       | `string`                   |
| `extraData`        | `string`                   |
| `miner`            | `string`                   |
| `nonce`            | `string`                   |
| `mixHash`          | `string`                   |
| `size`             | `number`                   |
| `gasLimit`         | `bigint`                   |
| `gasUsed`          | `bigint`                   |
| `difficulty`       | `bigint`                   |
| `totalDifficulty`  | `bigint?`                  |
| `baseFeePerGas`    | `bigint`                   |
| `blobGasUsed`      | `bigint`                   |
| `excessBlobGas`    | `bigint`                   |
| `l1BlockNumber`    | `number?` (L2 only)        |

### `transaction`

| Field                                                                                                        | Type                                 |
| ------------------------------------------------------------------------------------------------------------ | ------------------------------------ |
| `transactionIndex`                                                                                           | `number`                             |
| `hash`                                                                                                       | `string`                             |
| `nonce`                                                                                                      | `bigint`                             |
| `from`                                                                                                       | `string`                             |
| `to`                                                                                                         | `string?`                            |
| `input`                                                                                                      | `string`                             |
| `value`                                                                                                      | `bigint`                             |
| `gas`                                                                                                        | `bigint`                             |
| `gasPrice`                                                                                                   | `bigint`                             |
| `maxFeePerGas`                                                                                               | `bigint?`                            |
| `maxPriorityFeePerGas`                                                                                       | `bigint?`                            |
| `v`, `r`, `s`, `yParity`                                                                                     | `bigint`/`string`/`string`/`number?` |
| `chainId`                                                                                                    | `number?`                            |
| `sighash`                                                                                                    | `string?` (first 4 bytes of `input`) |
| `contractAddress`                                                                                            | `string?` (for create transactions)  |
| `gasUsed`                                                                                                    | `bigint`                             |
| `cumulativeGasUsed`                                                                                          | `bigint`                             |
| `effectiveGasPrice`                                                                                          | `bigint`                             |
| `type`                                                                                                       | `number`                             |
| `status`                                                                                                     | `number` (0 = fail, 1 = success)     |
| `blobVersionedHashes`                                                                                        | `string[]?`                          |
| `l1Fee`, `l1FeeScalar`, `l1GasPrice`, `l1GasUsed`, `l1BlobBaseFee`, `l1BlobBaseFeeScalar`, `l1BaseFeeScalar` | L2 fee metadata                      |

### `log`

| Field              | Type       |
| ------------------ | ---------- |
| `logIndex`         | `number`   |
| `transactionIndex` | `number`   |
| `transactionHash`  | `string`   |
| `address`          | `string`   |
| `data`             | `string`   |
| `topics`           | `string[]` |

### `trace`

Trace records are a tagged union over `type`. Each type has its own action and (optionally) result sub-objects; the field selection is flat with `create…`/`call…`/`suicide…`/`reward…` prefixes. Shared:

| Field              | Type                                          |
| ------------------ | --------------------------------------------- |
| `type`             | `'create' \| 'call' \| 'suicide' \| 'reward'` |
| `transactionIndex` | `number`                                      |
| `traceAddress`     | `number[]`                                    |
| `subtraces`        | `number`                                      |
| `error`            | `string \| null`                              |
| `revertReason`     | `string?`                                     |

Type-specific (appears only on that trace `type`):

| Field                                                                                    | Action/result  | Type                                                                         |
| ---------------------------------------------------------------------------------------- | -------------- | ---------------------------------------------------------------------------- |
| `createFrom`, `createValue`, `createGas`, `createInit`                                   | create action  | `string` / `bigint` / `bigint` / `string`                                    |
| `createResultGasUsed`, `createResultCode`, `createResultAddress`                         | create result  | `bigint` / `string?` / `string`                                              |
| `callCallType`, `callFrom`, `callTo`, `callValue`, `callGas`, `callInput`, `callSighash` | call action    | `string` / `string` / `string` / `bigint?` / `bigint` / `string` / `string?` |
| `callResultGasUsed`, `callResultOutput`                                                  | call result    | `bigint` / `string?`                                                         |
| `suicideAddress`, `suicideRefundAddress`, `suicideBalance`                               | suicide action | `string` / `string` / `bigint`                                               |
| `rewardAuthor`, `rewardValue`, `rewardType`                                              | reward action  | `string` / `bigint` / `string`                                               |

### `stateDiff`

State diffs are a tagged union over `kind` (`'+'` = add, `'-'` = delete, `'*'` = change, `'='` = no-change).

| Field              | Type                                                      |
| ------------------ | --------------------------------------------------------- |
| `transactionIndex` | `number`                                                  |
| `address`          | `string`                                                  |
| `key`              | `'balance' \| 'code' \| 'nonce' \| string` (storage slot) |
| `kind`             | `'+' \| '-' \| '*' \| '='`                                |
| `prev`             | `string` (present for `-`, `*`)                           |
| `next`             | `string` (present for `+`, `*`)                           |

***

## `.addLog(options)`

Filter logs. All list filters AND across fields but OR within a field. Request options:

| Field                        | Type       | Meaning                                                 |
| ---------------------------- | ---------- | ------------------------------------------------------- |
| `address`                    | `string[]` | Emitting contract addresses (lowercase).                |
| `topic0`                     | `string[]` | First topic (typically the event signature hash).       |
| `topic1`, `topic2`, `topic3` | `string[]` | Remaining indexed topics.                               |
| `transaction`                | `boolean`  | Also fetch the parent transaction of each matching log. |
| `transactionTraces`          | `boolean`  | Also fetch traces of parent transactions.               |
| `transactionLogs`            | `boolean`  | Also fetch all logs from parent transactions.           |
| `transactionStateDiffs`      | `boolean`  | Also fetch state diffs caused by parent transactions.   |

An empty `request: {}` matches every log in the range.

```ts theme={"system"}
evmQuery().addLog({
  range: { from: 20_000_000, to: 20_000_100 },
  request: {
    address: ['0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48'],
    topic0: ['0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef'],
    transaction: true,
  },
})
```

## `.addTransaction(options)`

Filter transactions.

| Field        | Type       | Meaning                                                             |
| ------------ | ---------- | ------------------------------------------------------------------- |
| `from`       | `string[]` | Sender addresses (lowercase).                                       |
| `to`         | `string[]` | Recipient addresses (lowercase).                                    |
| `sighash`    | `string[]` | First 4 bytes of `input` (e.g. `0xa9059cbb` for ERC-20 `transfer`). |
| `type`       | `number[]` | Transaction type (0 legacy, 2 EIP-1559, 3 blob, etc.).              |
| `logs`       | `boolean`  | Also fetch emitted logs.                                            |
| `traces`     | `boolean`  | Also fetch execution traces.                                        |
| `stateDiffs` | `boolean`  | Also fetch state diffs.                                             |

## `.addTrace(options)`

Filter execution traces.

| Field                  | Type                                              | Meaning                                        |
| ---------------------- | ------------------------------------------------- | ---------------------------------------------- |
| `type`                 | `('create' \| 'call' \| 'suicide' \| 'reward')[]` | Restrict to given trace types.                 |
| `createFrom`           | `string[]`                                        | Creator address for `create` traces.           |
| `callFrom`             | `string[]`                                        | Caller address for `call` traces.              |
| `callTo`               | `string[]`                                        | Callee address for `call` traces.              |
| `callSighash`          | `string[]`                                        | `input` sighash for `call` traces.             |
| `suicideRefundAddress` | `string[]`                                        | Refund address for `suicide` traces.           |
| `rewardAuthor`         | `string[]`                                        | Author of `reward` traces.                     |
| `transaction`          | `boolean`                                         | Fetch parent transactions of matching traces.  |
| `transactionLogs`      | `boolean`                                         | Fetch all logs emitted by parent transactions. |
| `subtraces`            | `boolean`                                         | Fetch all subtraces of matching traces.        |
| `parents`              | `boolean`                                         | Fetch parent traces of matching traces.        |

## `.addStateDiff(options)`

Filter storage diffs.

| Field         | Type                           | Meaning                                                   |
| ------------- | ------------------------------ | --------------------------------------------------------- |
| `address`     | `string[]`                     | Contract or account addresses (lowercase).                |
| `key`         | `string[]`                     | Storage keys or pseudo-keys (`balance`, `code`, `nonce`). |
| `kind`        | `('+' \| '-' \| '*' \| '=')[]` | Type of change.                                           |
| `transaction` | `boolean`                      | Fetch parent transactions.                                |

## `.addRange(range)`

Push a range-only request with no filters. Mostly useful to bound the stream or in combination with `includeAllBlocks` set elsewhere.

```ts theme={"system"}
evmQuery().addRange({ from: 20_000_000, to: 20_001_000 })
```

## `.merge(other)`

Merge another builder's requests and fields in-place. Overlapping ranges are reconciled at build time.

## `.build(opts?)`

Return a `QueryAwareTransformer` suitable for use as a source output.

```ts theme={"system"}
evmPortalStream({
  portal: 'https://portal.sqd.dev/datasets/ethereum-mainnet',
  outputs: { transfers: evmQuery().addLog({/*…*/}).build() },
})
```

`opts.setupQuery` is an advanced hook called when the query is finalized: it receives `{ query, logger }` and can mutate `query` (e.g. merge additional requests from runtime data). Default behaviour is to merge `this` into the stream's root query.

***

## See also

* [Source](./source) — how queries attach to the stream.
* [evmDecoder](../utility-components/evm-decoder) — typed wrapper that emits a pre-built query plus event/function decoding.
* [Handling contract events](../../guides/basic-development/handling-events) — higher-level guide.
* [Portal API (EVM) OpenAPI](/en/portal/evm/api) — raw wire protocol this builder serialises to.


# Source
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/evm/reference/basic-components/source

API reference for EVM Portal source

The source component connects to SQD Portal and streams blockchain data to your pipeline. It's the starting point for all Pipes SDK data flows.

## evmPortalSource

Create a Portal source for EVM chains.

```ts theme={"system"}
evmPortalSource(config: EvmPortalSourceConfig): Source
```

**Parameters:**

* `id`: (required) Pipeline ID. Must be unique within any infra shared with other pipelines (DB, logging sinks etc).

* `portal`: (required) Portal API URL or config object.
  * String: `"https://portal.sqd.dev/datasets/ethereum-mainnet"`
  * Object: `{ url: string, finalized?: boolean }`. When `finalized: true` is set the stream will consist of finalized blocks only and none of the [fork handling machinery](../../guides/architecture-deep-dives/fork-handling) will be required.

* `outputs`: (required) A single query-transformers chain combo or record of named outputs.

* `cache`: (optional) Portal cache instance. If supplied, saves portal responses locally and reuses them when the pipeline re-runs.

* `logger`: (optional) A pino-compatible `Logger` instance or a log level string. Accepted level values: `'fatal'`, `'error'`, `'warn'`, `'info'`, `'debug'`, `'trace'`, `'silent'`, `false`, `null`. Passing `false` or `null` silences all log output. When omitted, a default console logger is used.

* `metrics`: (optional) `metricsServer()` instance for exposing Prometheus metrics.

* `progress`: (optional) Options for progress tracking.

* `profiler`: (optional) Enable the built-in per-batch profiler. See [Profiling](../../guides/advanced-topics/profiling).

**Example:**

```ts theme={"system"}
import { evmPortalSource } from "@subsquid/pipes/evm";
import { portalSqliteCache } from "@subsquid/pipes/portal-cache/node";

const source = evmPortalSource({
  id: "ethereum-transfers",
  portal: "https://portal.sqd.dev/datasets/ethereum-mainnet",
  outputs: evmDecoder({ 
    range: { from: 20000000 },
    events: { transfers: commonAbis.erc20.events.Transfer },
  }),
  cache: portalSqliteCache({ path: "./cache.sqlite" }),
});

```

### Finalized Blocks

You can configure the source to only receive finalized blocks:

```ts theme={"system"}
const source = evmPortalSource({
  portal: {
    finalized: true,
    url: 'https://portal.sqd.dev/datasets/ethereum-mainnet'
  }
});
```

Using finalized blocks eliminates the need for rollback handlers in your targets, simplifying the logic of your pipeline.

## Pipe methods

### pipe()

Chain a single [whole-pipe transformer](./transformer) to the source.

```ts theme={"system"}
source.pipe(transformer)
```

The returned value behaves exactly as the source.

See also: [Stateful transformers](../../guides/advanced-topics/stateful-transforms).

### pipeTo()

Connect the pipeline to a [target](./target).

```ts theme={"system"}
source.pipeTo(target)
```

This is a terminal operation: you cannot continue piping after calling this method.

If you want your stream to resume on restarts and properly handle unfinalized data, make sure that the target [manages cursors](../../guides/architecture-deep-dives/cursor-management) and [handles forks](../../guides/architecture-deep-dives/fork-handling) correctly.

### \*[Symbol.asyncIterator]()

Use the pipeline as an async iterator:

```ts theme={"system"}
for await (const { data } of stream) {
  // ... do something with data ...
}
```

On blockchain forks this will throw `ForkException`s - see [Fork handling](../../guides/architecture-deep-dives/fork-handling).


# clickhouseTarget
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/evm/reference/basic-components/target/clickhouse

ClickHouse target for Pipes SDK

See the [ClickHouse guide](../../../guides/basic-development/targets/clickhouse) for usage examples, table design, and setup instructions.

```ts theme={"system"}
import { clickhouseTarget } from '@subsquid/pipes/targets/clickhouse'
```

## `clickhouseTarget`

```ts theme={"system"}
clickhouseTarget<T>({
  client: ClickHouseClient,
  onStart?: (ctx: { store: ClickhouseStore; logger: Logger }) => unknown | Promise<unknown>,
  onData: (ctx: { store: ClickhouseStore; data: T; ctx: Ctx }) => unknown | Promise<unknown>,
  onRollback?: (ctx: {
    type: 'offset_check' | 'blockchain_fork'
    store: ClickhouseStore
    safeCursor: BlockCursor
    /** @deprecated Use `safeCursor` instead. */
    cursor: BlockCursor
  }) => unknown | Promise<unknown>,
  settings?: Settings,
})
```

| Parameter    | Required | Description                                                                  |
| ------------ | -------- | ---------------------------------------------------------------------------- |
| `client`     | Yes      | Client from `@clickhouse/client`.                                            |
| `onStart`    | No       | Runs once before processing starts. Use for table creation or other setup.   |
| `onData`     | Yes      | Called for each batch.                                                       |
| `onRollback` | No       | Called on startup (`'offset_check'`) and on each fork (`'blockchain_fork'`). |
| `settings`   | No       | Configuration for the internal cursor state table. See `Settings` below.     |

**`Settings`:**

| Field      | Default                      | Description                                             |
| ---------- | ---------------------------- | ------------------------------------------------------- |
| `database` | Client's configured database | ClickHouse database for the state table.                |
| `table`    | `'sync'`                     | Name of the state table.                                |
| `id`       | `'stream'`                   | Stream identifier within the state table.               |
| `maxRows`  | `10000`                      | Maximum rows retained per stream id in the state table. |

## `ClickhouseStore` methods

| Method                                                  | Description                                                                                                                               |
| ------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| `store.insert(params)`                                  | Queues an insert. Non-blocking — returns a `Promise` but need not be awaited inside `onData`; inserts are flushed when the target closes. |
| `store.query(params)`                                   | Passthrough to `client.query()`.                                                                                                          |
| `store.command(params)`                                 | Passthrough to `client.command()`.                                                                                                        |
| `store.removeAllRows({ tables, where, params? })`       | Cancels rows matching `where` by re-inserting them with `sign = -1`. Requires `CollapsingMergeTree`.                                      |
| `store.removeAllRowsByQuery({ table, query, params? })` | Like `removeAllRows`, but uses a custom `SELECT` to identify the rows to cancel.                                                          |
| `store.executeFiles(dir)`                               | Executes all `.sql` files found in `dir`.                                                                                                 |


# createTarget
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/evm/reference/basic-components/target/create-target

API reference for createTarget

Build a custom data sink. A target drains batches from the pipe and is responsible for persisting them.

```ts theme={"system"}
createTarget<In>(config: Target<In>): Target<In>
```

**Config fields:**

* `write`: (required) Async function `({ read, logger }) => Promise<void>`. Iterate the stream by calling `read()` and consuming `{ data, ctx }` batches. The function returns when the stream ends.
* `fork`: (optional) `(previousBlocks: BlockCursor[]) => Promise<BlockCursor | null>`. Called when the source detects a chain reorg. Return the last safe cursor to roll back to, or `null` if no common ancestor can be determined (the stream will throw). See [Fork handling](../../../guides/architecture-deep-dives/fork-handling). You don't need this callback when the source is configured to [read only finalized blocks](../source#finalized-blocks).

## The `write` context

```ts theme={"system"}
type WriteCtx<In> = {
  read: (cursor?: BlockCursor) => AsyncIterableIterator<PortalBatch<In>>
  logger: Logger
}
```

| Field    | Description                                                                                                            |
| -------- | ---------------------------------------------------------------------------------------------------------------------- |
| `read`   | Opens an async iterator over pipeline batches. Pass `cursor` to resume from a specific block the target has persisted. |
| `logger` | Pino-compatible logger scoped to this target.                                                                          |

## Per-batch context (`ctx`)

Each `{ data, ctx }` yielded by `read()` carries the same `BatchContext` that transformers receive. Fields:

| Field                        | Type                        | Description                                                                                                 |
| ---------------------------- | --------------------------- | ----------------------------------------------------------------------------------------------------------- |
| `id`                         | `string`                    | Pipeline ID — the `id` passed to `evmPortalStream()`.                                                       |
| `logger`                     | `Logger`                    | Batch-scoped logger.                                                                                        |
| `metrics`                    | `Metrics`                   | Prometheus metrics registry. See [Metrics](../../../guides/advanced-topics/metrics).                        |
| `profiler`                   | `Profiler`                  | Open a span with `ctx.profiler.start('label')`. See [Profiling](../../../guides/advanced-topics/profiling). |
| `stream.dataset`             | `ApiDataset`                | Dataset metadata.                                                                                           |
| `stream.head.finalized`      | `BlockCursor \| undefined`  | Current finalized head.                                                                                     |
| `stream.head.latest`         | `BlockCursor \| undefined`  | Current unfinalized head.                                                                                   |
| `stream.state.initial`       | `number`                    | First block number the stream was configured to read.                                                       |
| `stream.state.last`          | `number`                    | Last block number the stream intends to read.                                                               |
| `stream.state.current`       | `BlockCursor`               | Latest block in this batch.                                                                                 |
| `stream.state.rollbackChain` | `BlockCursor[]`             | Tail of unfinalized cursors subject to rollback.                                                            |
| `stream.progress`            | `ProgressEvent['progress']` | Progress metrics when `progress` is enabled.                                                                |
| `stream.query`               | `{ url, hash, raw }`        | Portal query details for the batch.                                                                         |
| `batch.blocksCount`          | `number`                    | Number of blocks in this batch.                                                                             |
| `batch.bytesSize`            | `number`                    | Compressed payload size received from the portal.                                                           |
| `batch.requests`             | `Record<number, number>`    | Map of HTTP status code → count of responses that produced this batch.                                      |
| `batch.lastBlockReceivedAt`  | `Date`                      | Wall-clock time the last block was received.                                                                |

## Example

```ts theme={"system"}
const target = createTarget({
  write: async ({ read, logger }) => {
    for await (const { data, ctx } of read()) {
      const span = ctx.profiler.start('save')
      await database.save(data)
      span.end()
      logger.info(
        { block: ctx.stream.state.current.number, rows: ctx.batch.blocksCount },
        'saved batch',
      )
    }
  },
  fork: async (previousBlocks) => {
    // Return a cursor from your persisted state; null to fail hard.
    return previousBlocks[previousBlocks.length - 1] ?? null
  },
})
```

## Resuming from a persisted cursor

Stateful targets typically persist a cursor and resume from it on restart:

```ts theme={"system"}
createTarget({
  write: async ({ read, logger }) => {
    const lastSaved = await database.getCursor() // BlockCursor | undefined
    for await (const { data, ctx } of read(lastSaved)) {
      await database.save(data)
      await database.saveCursor(ctx.stream.state.current)
    }
  },
})
```


# drizzleTarget
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/evm/reference/basic-components/target/postgres-drizzle

PostgreSQL target for Pipes SDK via Drizzle ORM

See the [Postgres via Drizzle guide](../../../guides/basic-development/targets/postgres-drizzle) for usage examples and setup instructions.

```ts theme={"system"}
import { drizzleTarget } from '@subsquid/pipes/targets/drizzle/node-postgres'
```

## `drizzleTarget`

```ts theme={"system"}
drizzleTarget<T>({
  db: NodePgDatabase,
  tables: Table[] | Record<string, Table>,
  onStart?: (ctx: { db: NodePgDatabase }) => Promise<unknown>,
  onData: (ctx: { tx: Transaction; data: T; ctx: Ctx }) => Promise<unknown>,
  onBeforeRollback?: (ctx: { tx: Transaction; cursor: BlockCursor }) => Promise<unknown> | unknown,
  onAfterRollback?: (ctx: { tx: Transaction; cursor: BlockCursor }) => Promise<unknown> | unknown,
  settings?: {
    state?: StateOptions
    transaction?: {
      isolationLevel?: 'read uncommitted' | 'read committed' | 'repeatable read' | 'serializable'
    }
  },
})
```

| Parameter                             | Required | Description                                                                                     |
| ------------------------------------- | -------- | ----------------------------------------------------------------------------------------------- |
| `db`                                  | Yes      | Drizzle `NodePgDatabase` instance. Must expose `$client` (a `pg` Pool or Client).               |
| `tables`                              | Yes      | Tables tracked for automatic fork rollback. All tables written to in `onData` must appear here. |
| `onStart`                             | No       | Runs once before processing starts. Receives `{ db }`.                                          |
| `onData`                              | Yes      | Called for each batch inside a serializable transaction.                                        |
| `onBeforeRollback`                    | No       | Called inside the rollback transaction before snapshots are replayed.                           |
| `onAfterRollback`                     | No       | Called inside the rollback transaction after snapshots are replayed.                            |
| `settings.state`                      | No       | Configuration for the internal cursor state table. See `StateOptions` below.                    |
| `settings.transaction.isolationLevel` | No       | Transaction isolation level. Defaults to `'serializable'`.                                      |

**`StateOptions`:**

| Field                        | Default    | Description                                                           |
| ---------------------------- | ---------- | --------------------------------------------------------------------- |
| `schema`                     | `'public'` | PostgreSQL schema for the state table.                                |
| `table`                      | `'sync'`   | Name of the state table.                                              |
| `id`                         | `'stream'` | Stream identifier within the state table.                             |
| `unfinalizedBlocksRetention` | `1000`     | Number of unfinalized blocks retained in state for rollback purposes. |

## `batchForInsert`

```ts theme={"system"}
import { batchForInsert } from '@subsquid/pipes/targets/drizzle/node-postgres'
```

```ts theme={"system"}
function batchForInsert<T>(data: readonly T[], size?: number): Generator<T[]>
```

Splits an array into chunks that fit within PostgreSQL's 32,767-parameter limit. Chunk size is `Math.floor(32767 / columnsPerRecord)` by default. Pass `size` to set a smaller cap; values exceeding the computed maximum are silently clamped.

`chunk` is a deprecated alias for `batchForInsert`.


# Transformer
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/evm/reference/basic-components/transformer

API reference for createTransformer and pipe transforms

## createTransformer

Construct a whole-pipe transformer.

```ts theme={"system"}
createTransformer<I, O>(config: TransformerOptions<I, O>): Transformer<I, O>
```

**Config fields:**

* `transform`: (required) `(data: I, ctx: BatchContext) => O | Promise<O>`. Called once per batch.
* `start`: (optional) `(ctx: StartCtx) => void | Promise<void>`. Called once when the pipe starts. Use this to load state, warm up caches, or query the portal for historical data before the main stream begins.
* `stop`: (optional) `(ctx: StopCtx) => void | Promise<void>`. Called once when the pipe stops.
* `fork`: (optional) `(cursor: BlockCursor, ctx: Ctx) => void | Promise<void>`. Called before the next batch whenever the source detects a chain reorg. `cursor` identifies the last safe block. See [Fork handling](../../guides/architecture-deep-dives/fork-handling).
* `profiler`: (optional) `{ name: string; hidden?: boolean }`. Overrides the transformer's node name in the [profiler](../../guides/advanced-topics/profiling) tree.

**Example:**

```ts theme={"system"}
const transformer = createTransformer({
  transform: async (data, ctx) => {
    ctx.logger.info({ block: ctx.stream.state.current.number }, 'batch')
    return data.map((b) => b.logs)
  },
})
```

## Context variables

Each callback receives a context object. The fields differ by callback.

### `transform(data, ctx: BatchContext)`

`ctx` is the full per-batch context. Fields:

| Field      | Type                 | Description                                                                                                                                                                                    |
| ---------- | -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `id`       | `string`             | Pipeline ID — the `id` passed to `evmPortalStream()`.                                                                                                                                          |
| `logger`   | `Logger`             | Pino-compatible logger scoped to this batch. Defaults to the source-level logger.                                                                                                              |
| `metrics`  | `Metrics`            | Prometheus metrics registry. Use `ctx.metrics.counter()`, `.gauge()`, `.histogram()`, `.summary()` to register and update custom metrics. See [Metrics](../../guides/advanced-topics/metrics). |
| `profiler` | `Profiler`           | Open a span with `ctx.profiler.start('label')`. See [Profiling](../../guides/advanced-topics/profiling).                                                                                       |
| `stream`   | `BatchStreamContext` | Per-stream state (see below).                                                                                                                                                                  |
| `batch`    | `BatchMetadata`      | Per-batch volume info (see below).                                                                                                                                                             |

#### `ctx.stream: BatchStreamContext`

| Field                 | Type                        | Description                                                                               |
| --------------------- | --------------------------- | ----------------------------------------------------------------------------------------- |
| `dataset`             | `ApiDataset`                | Dataset metadata returned by the portal (chain name, genesis, tier).                      |
| `head.finalized`      | `BlockCursor \| undefined`  | Current finalized head known to the portal, if advertised.                                |
| `head.latest`         | `BlockCursor \| undefined`  | Current unfinalized head.                                                                 |
| `state.initial`       | `number`                    | First block number the stream was configured to read.                                     |
| `state.last`          | `number`                    | Last block number the stream intends to read (often `Infinity`).                          |
| `state.current`       | `BlockCursor`               | Latest block in this batch. Cursor has `{ number, hash?, timestamp? }`.                   |
| `state.rollbackChain` | `BlockCursor[]`             | Unfinalized-chain tail — cursors the stream will need to roll back if a fork is detected. |
| `progress`            | `ProgressEvent['progress']` | Progress metrics when `progress` is configured on the source; otherwise undefined.        |
| `query`               | `{ url, hash, raw }`        | Debug info for the portal query feeding this batch.                                       |

#### `ctx.batch: BatchMetadata`

| Field                 | Type                     | Description                                                             |
| --------------------- | ------------------------ | ----------------------------------------------------------------------- |
| `blocksCount`         | `number`                 | Number of blocks in this batch.                                         |
| `bytesSize`           | `number`                 | Compressed payload size received from the portal.                       |
| `requests`            | `Record<number, number>` | Map of HTTP status code → number of responses that produced this batch. |
| `lastBlockReceivedAt` | `Date`                   | Wall-clock time the last block was received.                            |

### `start(ctx: StartCtx)`

Fired once, before any batch. Use to warm up caches or run one-off queries.

| Field           | Type                       | Description                                                                 |
| --------------- | -------------------------- | --------------------------------------------------------------------------- |
| `id`            | `string`                   | Pipeline ID.                                                                |
| `logger`        | `Logger`                   | Same as in `BatchContext`.                                                  |
| `metrics`       | `Metrics`                  | Same as in `BatchContext`.                                                  |
| `portal`        | `PortalClient`             | Live portal client. Use `portal.getStream(query)` for warm-up reads.        |
| `state.initial` | `number`                   | First block the stream was configured to read.                              |
| `state.current` | `BlockCursor \| undefined` | Cursor persisted by the previous run, if any. `undefined` on a fresh start. |

### `fork(cursor, ctx: Ctx)`

Fired before the next batch whenever a reorg is detected. `cursor` is the last block to keep; drop state produced for anything after it.

| Field      | Type       | Description                |
| ---------- | ---------- | -------------------------- |
| `logger`   | `Logger`   | Same as in `BatchContext`. |
| `profiler` | `Profiler` | Same as in `BatchContext`. |

### `stop(ctx: StopCtx)`

Fired once when the pipe stops.

| Field    | Type     | Description                |
| -------- | -------- | -------------------------- |
| `logger` | `Logger` | Same as in `BatchContext`. |


# evmDecoder
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/evm/reference/utility-components/evm-decoder

Decode smart contract events as a pipe

See the [Handling contract events](../../guides/basic-development/handling-events) guide for usage examples and event specification routes.

## evmDecoder

Returns a query-transformer combo that instructs the [source](../basic-components/source) to fetch and decode smart contract event logs.

```ts theme={"system"}
evmDecoder<T>(config: EvmDecoderConfig<T>): Transformer
```

**Parameters:**

* `range`: Block range `{ from: number | 'latest', to?: number }` (required)
* `contracts`: Array of contract addresses or a [factory](./factory) (optional — omit to receive events from all contracts)
* `events`: Map of event names to ABI event objects or `{ event, params }` filter objects (required)
* `profiler`: Profiler config shard for labeling the transformer in profiling data `{ name: string }` (optional)
* `onError`: Error handler (optional)

**Example:**

```ts theme={"system"}
import { evmDecoder, commonAbis } from "@subsquid/pipes/evm";

const decoder = evmDecoder({
  range: { from: 20000000, to: 20100000 },
  contracts: ["0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48"],
  events: {
    transfer: commonAbis.erc20.events.Transfer,
  },
});

await evmPortalSource({
  portal: "https://portal.sqd.dev/datasets/ethereum-mainnet",
  outputs: decoder,
}).pipeTo(target);
```

## Decoded event structure

Each entry in the output arrays is a `DecodedEvent` object:

| Field       | Type                               | Description                                                                                                        |
| ----------- | ---------------------------------- | ------------------------------------------------------------------------------------------------------------------ |
| `event`     | decoded event type                 | Decoded event data fields                                                                                          |
| `contract`  | `string`                           | Address of the contract that emitted the event                                                                     |
| `block`     | `{ number: number, hash: string }` | Block number and hash                                                                                              |
| `timestamp` | `Date`                             | Block timestamp                                                                                                    |
| `rawEvent`  | `Log`                              | Raw log with `address`, `topics`, `data`, `transactionHash`, `logIndex`, `transactionIndex`                        |
| `factory`   | `{ contract, blockNumber, event }` | Present only when using a [factory](./factory); carries the factory deployment event that discovered this contract |

## commonAbis

```ts theme={"system"}
import { commonAbis } from '@subsquid/pipes/evm'
```

`commonAbis` is a built-in collection of typed ABI modules. See the [Handling contract events guide](../../guides/basic-development/handling-events#commonabis) for usage examples.

### commonAbis.erc20

**Events:**

|                                    | Signature                                                                 |
| ---------------------------------- | ------------------------------------------------------------------------- |
| `commonAbis.erc20.events.Transfer` | `Transfer(address indexed from, address indexed to, uint256 value)`       |
| `commonAbis.erc20.events.Approval` | `Approval(address indexed owner, address indexed spender, uint256 value)` |

**Functions:**

|                                           | Signature                                                         |
| ----------------------------------------- | ----------------------------------------------------------------- |
| `commonAbis.erc20.functions.name`         | `name() → string`                                                 |
| `commonAbis.erc20.functions.symbol`       | `symbol() → string`                                               |
| `commonAbis.erc20.functions.decimals`     | `decimals() → uint8`                                              |
| `commonAbis.erc20.functions.totalSupply`  | `totalSupply() → uint256`                                         |
| `commonAbis.erc20.functions.balanceOf`    | `balanceOf(address _owner) → uint256`                             |
| `commonAbis.erc20.functions.allowance`    | `allowance(address _owner, address _spender) → uint256`           |
| `commonAbis.erc20.functions.transfer`     | `transfer(address _to, uint256 _value) → bool`                    |
| `commonAbis.erc20.functions.approve`      | `approve(address _spender, uint256 _value) → bool`                |
| `commonAbis.erc20.functions.transferFrom` | `transferFrom(address _from, address _to, uint256 _value) → bool` |


# evmRpcLatencyWatcher
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/evm/reference/utility-components/evm-rpc-latency-watcher

Compare block arrival at Portal vs RPC endpoints

Subscribe to RPC endpoints via WebSocket and measure when blocks arrive at the Portal versus when they appear at the RPC. Use this transformer to monitor relative latency.

```ts theme={"system"}
import { evmPortalSource, evmRpcLatencyWatcher } from "@subsquid/pipes/evm";

const stream = evmPortalSource({
  portal: "https://portal.sqd.dev/datasets/base-mainnet",
  outputs: evmDecoder({ range: { from: 'latest' }, events: {} }),
}).pipe(
  evmRpcLatencyWatcher({
    rpcUrl: ["https://base.drpc.org", "https://base-rpc.publicnode.com"],
  })
);

for await (const { data } of stream) {
  if (!data) continue;
  console.table(data.rpc); // url, receivedAt, portalDelayMs
}
```

**Parameters:**

* `rpcUrl`: Array of RPC WebSocket or HTTP URLs to compare against Portal

**Output:** Each batch includes `rpc` array with `url`, `receivedAt`, and `portalDelayMs` per endpoint.

<Warning>
  Measured values include client-side network latency. Results are end-to-end delays as seen by the client, not pure Portal or RPC processing performance.
</Warning>


# factory
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/evm/reference/utility-components/factory

Track dynamically created contracts

Track dynamically created contracts with [evmDecoder()](./evm-decoder).

```ts theme={"system"}
factory(config: FactoryConfig): Factory
```

**Parameters:**

* `address`: Factory contract address or array of addresses (required)
* `event`: Factory creation event ABI or filtered event object (required)

  * **Simple format**: `AbiEvent<T>` - Capture all factory events
  * **Filtered format**: `{ event: AbiEvent<T>, params: {...} }` - Filter by indexed parameters

  Events should be specified using [the same approach as `evmDecoder()` itself uses](../../guides/basic-development/handling-events#specifying-events).
* `parameter`: Extract child address `(event) => string` (required)
* `database`: A [factory store](#contractfactorystore) that persists the list of known child contracts (required)

**Example:**

```ts theme={"system"}
import { factory, contractFactoryStore } from "@subsquid/pipes/evm";

const factoryInstance = factory({
  address: "0x1f98431c8ad98523631ae4a59f267346ea31f984",
  event: factoryAbi.events.PoolCreated,
  parameter: "pool",
  database: contractFactoryStore({ path: "./pools.sqlite" }),
});
```

**Filtered factory events:**

```ts theme={"system"}
factory({
  address: "0x1f98431c8ad98523631ae4a59f267346ea31f984",
  event: {
    event: factoryAbi.events.PoolCreated,
    params: {
      token0: "0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2", // WETH
    },
  },
  parameter: "pool",
  database: contractFactoryStore({ path: "./weth-pools.sqlite" }),
});
```

<Warning>
  Only **indexed event parameters** can be used in the `params` object. Another way to look at it is that parameter values should be available as event topics. [Reference](https://docs.soliditylang.org/en/latest/contracts.html#events).
</Warning>

## contractFactoryStore

Create an SQLite factory database: an object used to persist the list of child contracts in a fork-aware way. For now, only SQLite-based factory databases are supported.

```ts theme={"system"}
contractFactoryStore(config: { path: string }): FactoryDatabase
```


# metricsServer
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/evm/reference/utility-components/metrics-server

Expose Prometheus metrics and the live-stats API from your pipe

Start a metrics server on the pipe process. Required by [Pipes UI](../../guides/basic-development/pipes-ui) and by anything that scrapes Prometheus (Grafana, Alertmanager, etc.).

```ts theme={"system"}
import { metricsServer } from "@subsquid/pipes/metrics/node";
import { evmPortalStream } from "@subsquid/pipes/evm";

evmPortalStream({
  // ...
  metrics: metricsServer({ port: 9090 }),
  // ...
});
```

**Parameters:**

* `port`: HTTP port for the server (default: `9090`).

## Endpoints

`metricsServer()` serves four HTTP endpoints on the configured port. They are all also useful for ad-hoc inspection with `curl`.

| Path        | Content                                                                                                                             |
| ----------- | ----------------------------------------------------------------------------------------------------------------------------------- |
| `/stats`    | JSON — per-pipe progress, speed, portal query, SDK version. This is what [Pipes UI](../../guides/basic-development/pipes-ui) polls. |
| `/metrics`  | Prometheus text — built-in `sqd_*` series plus any custom metrics you registered. Scrape this from Prometheus.                      |
| `/profiler` | JSON — recent per-batch span trees. See [Profiling](../../guides/advanced-topics/profiling). Empty when profiling is disabled.      |
| `/health`   | Responds with `ok`.                                                                                                                 |

## Custom metrics

Register counters, gauges, histograms, and summaries via `ctx.metrics` in [whole-pipe transformers](../basic-components/transformer), [targets](../basic-components/target), or when consuming the pipe as an async iterator. See the [Metrics guide](../../guides/advanced-topics/metrics).

## See also

* [Pipes UI](../../guides/basic-development/pipes-ui) — visual dashboard that consumes `/stats` and `/profiler`.
* [Metrics](../../guides/advanced-topics/metrics) — walkthrough for exposing Prometheus metrics and adding custom series.
* [Profiling](../../guides/advanced-topics/profiling) — interpreting the `/profiler` span tree.


# portalSqliteCache
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/evm/reference/utility-components/sqlite-cache

SQLite cache for Portal responses

Create SQLite cache for Portal responses. Use with `evmPortalSource` to cache Portal API responses locally.

```ts theme={"system"}
portalSqliteCache(config: { path: string }): PortalCache
```

**Example:**

```ts theme={"system"}
import { portalSqliteCache } from "@subsquid/pipes/portal-cache/node";
import { evmPortalSource } from "@subsquid/pipes/evm";

const source = evmPortalSource({
  portal: "https://portal.sqd.dev/datasets/ethereum-mainnet",
  outputs: evmDecoder({ range: { from: 0 }, events: {} }),
  cache: portalSqliteCache({ path: "./cache.sqlite" }),
});
```

<Note>
  Import from `@subsquid/pipes/portal-cache/node` instead of `@subsquid/pipes/portal-cache`.
</Note>

### When to Use

* Development iteration
* Testing pipelines
* Repeated processing of same block ranges.


# Why Pipes SDK?
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/evm/why-pipes-sdk

And when you might want to use it

<CardGroup>
  <Card title="Solana version" icon="book" href="/en/sdk/pipes-sdk/solana/quickstart">
    Pipes SDK is also available for Solana
  </Card>
</CardGroup>

Pipes SDK is a TypeScript library for retrieving blockchain data from SQD Portals and transforming it. It features:

* **All features of the SQD Portal API:**
  * Data is downloaded in big chunks and at a high speed.
  * It is filtered on the server side - you only download what you need.
  * Real-time data is supported.
  * Information on blockchain reorganizations and finality is available; in standard modules these are handled automatically.
* **Being a library**: although Pipes SDK can be used to build full-featured blockchain indexers, it is easy to embed it into larger applications, microservices, or data processing workflows.
* **Reusable modules:** data filters can be bundled with transformation logic, and the resulting modules can be mixed and matched. For example you can create modules to get you decoded Uniswap-stype swaps and ERC-20 transfers, then just plug them into your pipe when you need either.
* **Simplicity of extension:** we've made adding new modules as simple as possible.
  * This includes data sinks: adding support for your database, data lake or message queue is no longer a hassle.

## When to Use Pipes SDK

1. You want common protocols (Uniswap, ERC20, ERC721/1155 etc) handled for you. If all of your data is like that, you can start uploading it into your database in minutes.

2. You need deep customization of any part of the data pipeline.

## When to Use Alternatives

1. If you want to use Portal data in an non-JS app, consider using
   * [Raw Portal API](/en/portal/evm/api) for all languages.

2. Consider using [Squid SDK](/en/sdk/squid-sdk/evm) if:
   * You're making a self-contained Web3 data service such as a GraphQL API.
   * You're looking for an indexing framework similar to TheGraph, Ponder or Envio.


# sqd CLI cheatsheet
Source: https://docs.sqd.dev/en/sdk/squid-sdk/how-to-start/cli-cheatsheet

Cheatsheet of commonly used sqd CLI commands — init, deploy, logs, secrets, gateways — with examples for everyday Squid SDK development workflows.

# Squid CLI cheatsheet

The [`sqd` CLI tool](/en/sdk/squid-sdk/squid-cli) has [built-in aliasing](/en/sdk/squid-sdk/squid-cli/commands-json) that picks up the commands defined in `commands.json` in the project root. In all [squid templates](/en/sdk/squid-sdk/how-to-start/squid-development#templates) this file is pre-populated with some handy scripts briefly described below.

One can always inspect the available commands defined in `commands.json` with

```
sqd --help
```

The commands defined by `commands.json` will appear in the `SQUID COMMANDS` help sections.

Before using the `sqd` CLI tool, make sure all the project dependencies are installed:

```sh theme={"system"}
npm i
```

### Building the squid

```sh theme={"system"}
sqd build             Build the squid project
sqd clean             Delete all build artifacts
```

### Running the squid

<Info>
  Both `sqd up` and `sqd down` assume that the `docker compose` command is supported and the `docker` deamon is running. Modify the definitions in `commands.json` accordingly if `docker-compose` should be used instead.
</Info>

```
sqd up                  Start a local PG database
sqd down                Drop the local PG database
sqd run [PATH]          Run all the services defined in squid.yaml locally
sqd serve               Start the GraphQL server
sqd serve:prod          Start the GraphQL API server with caching and limits
```

### DB migrations

Read [TypeORM Migration generation](/en/sdk/squid-sdk/resources/tools/migrations-gen) for details.

```
sqd migration:apply             apply pending migrations
sqd migration:generate          generate the migration for the schema defined in schema.graphql
sqd migration:clean             clean the db/migrations folder 
```

### Code generation

Consult [TypeORM Model generation](/en/sdk/squid-sdk/resources/tools/model-gen) for TypeORM model generation details, and [Type-safe decoding](https://docs.subsquid.io/sdk/resources/tools/typegen/) for type generation.

<Info>
  Depending on the template, `sqd typegen` is aliased to a different typegen tool specific to the chain type and thus has different usage. Consult `sqd typegen --help` for details.
</Info>

```
sqd codegen        Generate TypeORM entities from schema.graphql
sqd typegen        Generate data access classes for an ABI file(s) in the ./abi folder
```


# Environment set up
Source: https://docs.sqd.dev/en/sdk/squid-sdk/how-to-start/development-environment-set-up

Prepare your development environment for Squid SDK — install Node, Docker, sqd CLI, and required tools for building TypeScript blockchain indexers.

### Node.js

To install Node.js:

<Tabs>
  <Tab title="Linux">
    Use the package manager of your distro or the [official binaries](https://nodejs.org/en/download).
  </Tab>

  <Tab title="Mac">
    Use the [official binaries](https://nodejs.org/en/download) or [Homebrew](https://nodejs.org/en/download/package-manager/#alternatives-2).
  </Tab>

  <Tab title="Windows">
    The best bet is to leverage WSL2 and follow [this guide](https://docs.microsoft.com/en-us/windows/dev-environment/javascript/nodejs-on-wsl). [Official installer](https://nodejs.org/en/download) also works.
  </Tab>
</Tabs>

Make sure that your Node.js installation is v16 or newer. To check an existing installation, run:

```bash theme={"system"}
node --version
```

### Squid CLI

Follow [these instructions](/en/sdk/squid-sdk/squid-cli/installation).

### Git

Squid CLI uses [Git](https://git-scm.com) to retrieve templates. To install it:

<Tabs>
  <Tab title="Linux">
    Use the package manager of your distro.
  </Tab>

  <Tab title="Mac">
    Use the [official installer](https://sourceforge.net/projects/git-osx-installer/) or any one of the [alternative approaches](https://git-scm.com/download/mac).
  </Tab>

  <Tab title="Windows">
    Use [Git for Windows](https://git-scm.com/download/win).
  </Tab>
</Tabs>

### Docker

Most squids [use a database](/en/sdk/squid-sdk/resources/persisting-data/typeorm) to store the processed data. Install Docker to manage local squid databases with convenience.

<Tabs>
  <Tab title="Linux">
    * [Here’s an instruction](https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository) on how to install Docker on Ubuntu
    * On other distros consider using their package managers
  </Tab>

  <Tab title="Mac">
    Install the [Desktop version](https://docs.docker.com/desktop/mac/install/).
  </Tab>

  <Tab title="Windows">
    Install the [Desktop version](https://docs.docker.com/desktop/windows/install/).
  </Tab>
</Tabs>


# Project structure
Source: https://docs.sqd.dev/en/sdk/squid-sdk/how-to-start/layout

Squid SDK project folder layout — entry points, schema file, processor source, migrations, manifest, environment, and conventions for clean structure.

# Squid project structure

All files and folders except `package.json` are optional.

* `package.json` -- Configuration file for dependencies and the build script (invoked with `npm run build`). Hard requirement for deploying to [SQD Cloud](/en/cloud).
* `package-lock.json` OR `yarn.lock` OR `pnpm-lock.yaml` -- Dependencies shrinkwrap. Required for [Cloud](/en/cloud) deployment, except those that [override the dependencies installation command](/en/cloud/reference/manifest#cmd).
* `tsconfig.json` -- Configuration of `tsc`. Required for most squids.
* [Deployment manifest](/en/cloud/reference/manifest) (`squid.yaml` by default) -- Definitions of squid services used for running it locally with [`sqd run`](/en/sdk/squid-sdk/squid-cli/run) and deploying to [SQD Cloud](/en/cloud).
* `.squidignore` -- Files and patterns to be excluded when sending the squid code to the [Cloud](/en/cloud). When not supplied, some files will still be omitted: see the [reference page](/en/cloud/reference/squidignore) for details.
* `schema.graphql` -- [The schema definition file](/en/sdk/squid-sdk/reference/schema-file). Required if your squid [stores its data in PostgreSQL](/en/sdk/squid-sdk/resources/persisting-data/typeorm).
* `/src` -- The TypeScript source code folder for the squid processor.
  * `/src/main.ts` -- The entry point of the squid processor process. Typically, contains a `processor.run()` call.
  * `/src/processor.ts` -- Processor object ([EVM](/en/sdk/squid-sdk/reference/processors/evm-batch) or [Substrate](/en/sdk/squid-sdk/reference/processors/substrate-batch)) definition and configuration.
  * `/src/model/generated` -- The folder for the TypeORM entities generated from `schema.graphql`.
  * `/src/model` -- The module exporting the entity classes.
  * `/src/server-extension/resolvers` -- A folder for [user-defined GraphQL resolvers](/en/sdk/squid-sdk/reference/openreader-server/configuration/custom-resolvers) used by [OpenReader](/en/sdk/squid-sdk/reference/openreader-server).
  * `/src/types` -- A folder for types generated by the Substrate [typegen](/en/sdk/squid-sdk/resources/tools/typegen) tool for use in data decoding.
  * `/src/abi` -- A folder for modules generated by the EVM [typegen](/en/sdk/squid-sdk/resources/tools/typegen) tool containing type definitions and data decoding boilerplate code.
* `/db` -- The designated folder with the [database migrations](/en/sdk/squid-sdk/resources/persisting-data/typeorm).
* `/lib` -- The output folder for the compiled squid code.
* `/assets` -- A designated folder for custom user-provided files (e.g. static data files to seed the squid processor with).
* `/abi` -- A designated folder for JSON ABI files used as input by the EVM [typegen](/en/sdk/squid-sdk/resources/tools/typegen).
* `docker-compose.yml` -- A Docker compose file for local runs. Has a Postgres service definition by default.
* `.env` -- Defines environment variables used by `docker-compose.yml` and when the squid is run locally.
* `typegen.json` -- The config file for the Substrate [typegen](/en/sdk/squid-sdk/resources/tools/typegen) tool.
* `commands.json` -- [User-defined scripts](/en/sdk/squid-sdk/squid-cli/commands-json) picked up by [Squid CLI](/en/sdk/squid-sdk/squid-cli/commands-json). See also the [CLI cheatsheet](/en/sdk/squid-sdk/how-to-start/cli-cheatsheet).


# Development flow
Source: https://docs.sqd.dev/en/sdk/squid-sdk/how-to-start/squid-development

Recommended Squid SDK development workflow — model the schema, write the processor, run migrations, iterate locally, and deploy to SQD Cloud.

This page is a definitive end-to-end guide into practical squid development. It uses templates to simplify the process. Check out [Squid from scratch](/en/sdk/squid-sdk/how-to-start/squid-from-scratch) for a more educational barebones approach.

<Info>
  Feel free to also use the template-specific `sqd` scripts defined in [`commands.json`](/en/sdk/squid-sdk/squid-cli/commands-json) to simplify your workflow. See [sqd CLI cheatsheet](/en/sdk/squid-sdk/how-to-start/cli-cheatsheet) for a short intro.
</Info>

## Prepare the environment

* Node v16.x or newer
* Git
* [Squid CLI](/en/sdk/squid-sdk/squid-cli/installation)
* Docker (if your squid will store its data to PostgreSQL)

See also the [Environment set up](/en/sdk/squid-sdk/how-to-start/development-environment-set-up) page.

## Understand your technical requirements

Consider your business requirements and find out

1. How the data should be delivered. Options:
   * [PostgreSQL](/en/sdk/squid-sdk/resources/persisting-data/typeorm) with an optional [GraphQL API](/en/sdk/squid-sdk/resources/serving-graphql) - can be real-time
   * [file-based dataset](/en/sdk/squid-sdk/resources/persisting-data/file) - local or on S3
   * [Google BigQuery](/en/sdk/squid-sdk/resources/persisting-data/bigquery)
2. What data should be delivered
3. What are the technologies powering the blockchain(s) in question. Supported options:

   * Ethereum Virtual Machine (EVM) chains like [Ethereum](https://ethereum.org) - [supported networks](/en/data/evm)
   * [Substrate](https://substrate.io)-powered chains like [Polkadot](https://polkadot.network) and [Kusama](https://kusama.network) - [supported networks](/en/data/substrate)

   Note that you can use SQD via [RPC ingestion](/en/sdk/squid-sdk/resources/unfinalized-blocks) even if your network is not listed.
4. What exact data should be retrieved from blockchain(s)
5. Whether you need to mix in any [off-chain data](/en/sdk/squid-sdk/resources/external-api)

#### Example requirements

<AccordionGroup>
  <Accordion title="DEX analytics on Polygon">
    Suppose you want to train a prototype ML model on all trades done on Uniswap Polygon since the v3 upgrade.

    1. A delay of a few hours typically won't matter for training, so you may want to deliver the data as files for easier handling.
    2. The output could be a simple list of swaps, listing pair, direction and token amounts for each.
    3. Polygon is an EVM chain.
    4. All the required data is contained within `Swap` events emitted by the pair pool contracts. Uniswap deploys these [dynamically](/en/sdk/squid-sdk/resources/evm/factory-contracts), so you will also have to capture `PoolCreated` events from the factory contract to know which `Swap` events are coming from Uniswap and map them to pairs.
    5. No off-chain data will be necessary for this task.
  </Accordion>

  <Accordion title="NFT ownership on Ethereum">
    Suppose you want to make a website that shows the image and ownership history for ERC721 NFTs from a certain Ethereum contract.

    1. For this application it makes sense to deliver a GraphQL API.
    2. Output data might have `Token`, `Owner` and `Transfer` database tables / [entities](/en/sdk/squid-sdk/reference/schema-file/entities), with e.g. `Token` supplying all the fields necessary to show ownership history and the image.
    3. Ethereum is an EVM chain.
    4. Data on token mints and ownership history can be derived from `Transfer(address,address,uint256)` EVM event logs emitted by the contract. To render images, you will also need token metadata URLs that are only available by [querying the contract state](/en/sdk/squid-sdk/resources/tools/typegen/state-queries) with the `tokenURI(uint256)` function.
    5. You'll need to retrieve the off-chain token metadata (usually from IPFS).
  </Accordion>

  <Accordion title="Kusama transfers BigQuery dataset">
    Suppose you want to create a BigQuery dataset with Kusama native tokens transfers.

    1. The delivery format is BigQuery.
    2. A single table with `from`, `to` and `amount` columns may suffice.
    3. Kusama is a Substrate chain.
    4. The required data is available from `Transfer` events emitted by the `Balances` pallet. Take a look at our [Substrate data sourcing miniguide](/en/sdk/squid-sdk/resources/substrate/data-sourcing-miniguide) for more info on how to figure out which pallets, events and calls are necessary for your task.
    5. No off-chain data will be necessary for this task.
  </Accordion>
</AccordionGroup>

## Start from a template

Although it is possible to [compose a squid from individual packages](/en/sdk/squid-sdk/how-to-start/squid-from-scratch), in practice it is usually easier to start from a template.

<Tabs>
  <Tab title="EVM">
    <AccordionGroup>
      <Accordion title="Templates for the PostgreSQL+GraphQL data destination">
        * A minimal template intended for developing EVM squids. Indexes ETH burns.
          ```bash theme={"system"}
          sqd init my-squid-name -t evm
          ```
        * A starter squid for indexing ERC20 transfers.
          ```bash theme={"system"}
          sqd init my-squid-name -t https://github.com/subsquid-labs/squid-erc20-template
          ```
        * Classic [example Subgraph](https://github.com/graphprotocol/example-subgraph) after a [migration](/en/sdk/squid-sdk/resources/migrate/migrate-subgraph) to SQD.
          ```bash theme={"system"}
          sqd init my-squid-name -t gravatar
          ```
        * A template showing how to [combine data from multiple chains](/en/sdk/squid-sdk/resources/multichain). Indexes USDC transfers on Ethereum and Binance.
          ```bash theme={"system"}
          sqd init my-squid-name -t multichain
          ```
      </Accordion>

      <Accordion title="Templates for storing data in files">
        * USDC transfers -> local CSV
          ```bash theme={"system"}
          sqd init my-squid-name -t https://github.com/subsquid-labs/file-store-csv-example
          ```
        * USDC transfers -> local Parquet
          ```bash theme={"system"}
          sqd init my-squid-name -t https://github.com/subsquid-labs/file-store-parquet-example
          ```
        * USDC transfers -> CSV on S3
          ```bash theme={"system"}
          sqd init my-squid-name -t https://github.com/subsquid-labs/file-store-s3-example
          ```
      </Accordion>

      <Accordion title="Templates for the Google BigQuery data destination">
        * USDC transfers -> BigQuery dataset
          ```bash theme={"system"}
          sqd init my-squid-name -t https://github.com/subsquid-labs/squid-bigquery-example
          ```
      </Accordion>
    </AccordionGroup>
  </Tab>

  <Tab title="Substrate">
    <Accordion title="Templates for the PostgreSQL+GraphQL data destination">
      * Native events emitted by Substrate-based chains
        ```bash theme={"system"}
        sqd init my-squid-name -t substrate
        ```
      * ink! smart contracts
        ```bash theme={"system"}
        sqd init my-squid-name -t ink
        ```
      * Frontier EVM contracts on Astar and Moonbeam
        ```bash theme={"system"}
        sqd init my-squid-name -t frontier-evm
        ```
    </Accordion>
  </Tab>
</Tabs>

After retrieving the template of choice install its dependencies:

```bash theme={"system"}
cd my-squid-name
npm i
```

Test the template locally. The procedure varies depending on the data sink:

<Tabs>
  <Tab title="PostgreSQL+GraphQL">
    1. Launch a PostgreSQL container with
       ```bash theme={"system"}
       docker compose up -d
       ```

    2. Build the squid with
       ```bash theme={"system"}
       npm run build
       ```

    3. Apply the DB migrations with
       ```bash theme={"system"}
       npx squid-typeorm-migration apply
       ```

    4. Start the squid processor with
       ```bash theme={"system"}
       node -r dotenv/config lib/main.js
       ```
       You should see output that contains lines like these ones:
       ```bash theme={"system"}
       04:11:24 INFO  sqd:processor processing blocks from 6000000
       04:11:24 INFO  sqd:processor using archive data source
       04:11:24 INFO  sqd:processor prometheus metrics are served at port 45829
       04:11:27 INFO  sqd:processor 6051219 / 18079056, rate: 16781 blocks/sec, mapping: 770 blocks/sec, 544 items/sec, eta: 12m
       ```

    5. Start the GraphQL server by running
       ```bash theme={"system"}
       npx squid-graphql-server
       ```
       in a separate terminal, then visit the [GraphiQL console](http://localhost:4350/graphql) to verify that the GraphQL API is up.

    When done, shut down and erase your database with `docker compose down`.
  </Tab>

  <Tab title="filesystem dataset">
    1. (for the S3 template only) Set the credentials and prepare a bucket for your data as described in the [template README](https://github.com/subsquid-labs/file-store-s3-example/blob/main/README.md).

    2. Build the squid with
       ```bash theme={"system"}
       npm run build
       ```

    3. Start the squid processor with
       ```bash theme={"system"}
       node -r dotenv/config lib/main.js
       ```
       The output should contain lines like these ones:
       ```bash theme={"system"}
       04:11:24 INFO  sqd:processor processing blocks from 6000000
       04:11:24 INFO  sqd:processor using archive data source
       04:11:24 INFO  sqd:processor prometheus metrics are served at port 45829
       04:11:27 INFO  sqd:processor 6051219 / 18079056, rate: 16781 blocks/sec, mapping: 770 blocks/sec, 544 items/sec, eta: 12m
       ```
       You should see a `./data` folder populated with indexer data appear in a bit. A local folder looks like this:
       ```bash theme={"system"}
       $ tree ./data/
       ./data/
       ├── 0000000000-0007242369
       │   └── transfers.tsv
       ├── 0007242370-0007638609
       │   └── transfers.tsv
       ...
       └── status.txt
       ```
  </Tab>

  <Tab title="BigQuery">
    Create a dataset with your BigQuery account, then follow the [template README](https://github.com/subsquid-labs/squid-bigquery-example/blob/master/README.md).
  </Tab>
</Tabs>

## The bottom-up development cycle

The advantage of this approach is that the code remains buildable at all times, making it easier to catch issues early.

### I. Regenerate the task-specific utilities

<Tabs>
  <Tab title="EVM">
    Retrieve JSON ABIs for all contracts of interest (e.g. from Etherscan), taking care to get ABIs for implementation contracts and not [proxies](/en/sdk/squid-sdk/resources/evm/proxy-contracts) where appropriate. Assuming that you saved the ABI files to `./abi`, you can then regenerate the utilities with

    ```bash theme={"system"}
    npx squid-evm-typegen ./src/abi ./abi/*.json --multicall
    ```

    Or if you would like the tool to retrieve the ABI from Etherscan in your stead, you can run e.g.

    ```bash theme={"system"}
    npx squid-evm-typegen \
      src/abi \
      0xdAC17F958D2ee523a2206206994597C13D831ec7#usdt
    ```

    The utility classes will become available at `src/abi`.

    See also [EVM typegen code generation](/en/sdk/squid-sdk/resources/tools/typegen/generation).
  </Tab>

  <Tab title="Substrate">
    Follow the respective reference configuration pages of each typegen tool:

    * [Substrate typegen configuration](/en/sdk/squid-sdk/resources/tools/typegen/generation)
    * [ink! typegen configuration](/en/sdk/squid-sdk/resources/tools/typegen/generation)

    <Accordion title="A tip for `frontier-evm` squids">
      These squids use both Substrate typegen *and* EVM typegen. To generate all the required utilities, [configure the Substrate part](/en/sdk/squid-sdk/resources/tools/typegen/generation), then save all relevant JSON ABIs to `./abi`, then run

      ```bash theme={"system"}
      npx squid-evm-typegen ./src/abi ./abi/*.json --multicall
      ```

      followed by

      ```bash theme={"system"}
      npx squid-substrate-typegen ./typegen.json
      ```
    </Accordion>
  </Tab>
</Tabs>

### II. Configure the data requests

Data requests are [customarily](/en/sdk/squid-sdk/how-to-start/layout) defined at `src/processor.ts`. The details depend on the network type:

<Tabs>
  <Tab title="EVM">
    Edit the definition of `const processor` to

    1. Use a data source appropriate for your chain and task.
       * It is possible to [use RPC](/en/sdk/squid-sdk/reference/processors/evm-batch/general#set-rpc-endpoint) as the only data source, but [adding](/en/sdk/squid-sdk/reference/processors/evm-batch/general#set-gateway) a [SQD Network](/en/data/evm) data source will make your squid sync much faster.
       * RPC is a hard requirement if you're building a real-time API.
       * If you're using RPC as one of your data sources, make sure to [set the number of finality confirmations](/en/sdk/squid-sdk/reference/processors/evm-batch/general#set-finality-confirmation) so that [hot blocks ingestion](/en/sdk/squid-sdk/resources/unfinalized-blocks) works properly.

    2. Request all [event logs](/en/sdk/squid-sdk/reference/processors/evm-batch/logs), [transactions](/en/sdk/squid-sdk/reference/processors/evm-batch/transactions), [execution traces](/en/sdk/squid-sdk/reference/processors/evm-batch/traces) and [state diffs](/en/sdk/squid-sdk/reference/processors/evm-batch/state-diffs) that your task requires, with any necessary related data (e.g. parent transactions for event logs).

    3. [Select all data fields](/en/sdk/squid-sdk/reference/processors/evm-batch/field-selection) necessary for your task (e.g. `gasUsed` for transactions).

    See [reference documentation](/en/sdk/squid-sdk/reference/processors/evm-batch) for more info and [processor configuration showcase](/en/sdk/squid-sdk/examples) for a representative set of examples.
  </Tab>

  <Tab title="Substrate">
    Edit the definition of `const processor` to

    1. Use a data source appropriate for your chain and task
       * [Use](/en/sdk/squid-sdk/reference/processors/substrate-batch/general#set-gateway) a [SQD Network gateway](/en/data/substrate) whenever it is available. [RPC](/en/sdk/squid-sdk/reference/processors/evm-batch/general#set-rpc-endpoint) is still required in this case.
       * For networks without a gateway use just the RPC.

    2. Request all [events](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#events) and [calls](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#calls) that your task requires, with any necessary related data (e.g. parent extrinsics).

    3. If your squid indexes any of the following:

       * an [ink! contract](/en/sdk/squid-sdk/resources/substrate/ink)
       * an EVM contract running on the [Frontier EVM pallet](/en/sdk/squid-sdk/resources/substrate/frontier-evm)
       * [Gear messages](/en/sdk/squid-sdk/resources/substrate/gear)

       then you can use some of the [specialized data requesting methods](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#specialized-setters) to retrieve data more selectively.

    4. [Select all data fields](/en/sdk/squid-sdk/reference/processors/substrate-batch/field-selection) necessary for your task (e.g. `fee` for extrinsics).

    See [reference documentation](/en/sdk/squid-sdk/reference/processors/substrate-batch) for more info. Processor config examples can be found in the tutorials:

    * [general Substrate](/en/sdk/squid-sdk/tutorials/substrate)

    * [ink!](/en/sdk/squid-sdk/tutorials/ink)

    * [Frontier EVM](/en/sdk/squid-sdk/tutorials/frontier-evm)
  </Tab>
</Tabs>

### III. Decode and normalize the data

Next, change the batch handler to decode and normalize your data.

In templates, the batch handler is defined at the [`processor.run()`](/en/sdk/squid-sdk/reference/processors/architecture#processorrun) call in `src/main.ts` as an inline function. Its sole argument `ctx` contains:

* at `ctx.blocks`: all the requested data for a batch of blocks
* at `ctx.store`: the means to save the processed data
* at `ctx.log`: a [`Logger`](/en/sdk/squid-sdk/reference/logger)
* at `ctx.isHead`: a boolean indicating whether the batch is at the current chain head
* at `ctx._chain`: the means to access RPC for [state calls](#external-data)

This structure ([reference](/en/sdk/squid-sdk/reference/processors/architecture#batch-context)) is common for all processors; the structure of `ctx.blocks` items varies.

<Tabs>
  <Tab title="EVM">
    Each item in `ctx.blocks` contains the data for the requested logs, transactions, traces and state diffs for a particular block, plus some info on the block itself. See [EVM batch context reference](/en/sdk/squid-sdk/reference/processors/evm-batch/context-interfaces).

    Use the `.decode` methods from the [contract ABI utilities](#typegen) to decode events and transactions, e.g.

    ```ts theme={"system"}
    import * as erc20abi from './abi/erc20'

    processor.run(db, async ctx => {
      for (let block of ctx.blocks) {
        for (let log of block.logs) {
          if (log.topics[0]===erc20abi.events.Transfer.topic) {
            let {from, to, value} = erc20.events.Transfer.decode(log)
          }
        }
      }
    })
    ```

    See also the [EVM data decoding](/en/sdk/squid-sdk/resources/tools/typegen/decoding).
  </Tab>

  <Tab title="Substrate">
    Each item in `ctx.blocks` contains the data for the requested events, calls and, if requested, any related extrinsics; it also has some info on the block itself. See [Substrate batch context reference](/en/sdk/squid-sdk/reference/processors/substrate-batch/context-interfaces).

    Use the `.is()` and `.decode()` functions to decode the data for each runtime version, e.g. like this:

    ```ts theme={"system"}
    import {events} from './types'

    processor.run(db, async ctx => {
      for (let block of ctx.blocks) {
        for (let event of block.events) {
          if (event.name == events.balances.transfer.name) {
            let rec: {from: string; to: string; amount: bigint}
            if (events.balances.transfer.v1020.is(event)) {
              let [from, to, amount] = events.balances.transfer.v1020.decode(event)
              rec = {from, to, amount}
            }
            else if (events.balances.transfer.v1050.is(event)) {
              let [from, to, amount] = events.balances.transfer.v1050.decode(event)
              rec = {from, to, amount}
            }
            else if (events.balances.transfer.v9130.is(event)) {
              rec = events.balances.transfer.v9130.decode(event)
            }
            else {
              throw new Error('Unsupported spec')
            }
          }
        }
      }
    })

    ```

    See also the [Substrate data decoding](/en/sdk/squid-sdk/resources/tools/typegen/decoding).

    You can also decode the data of certain pallet-specific events and transactions with specialized tools:

    * use the utility classes made with `@substrate/squid-ink-typegen` to [decode events emitted by ink! contracts](/en/sdk/squid-sdk/resources/tools/typegen/decoding)
    * use the [`@subsquid/frontier` utils](/en/sdk/squid-sdk/reference/frontier) and the [EVM typegen](/en/sdk/squid-sdk/resources/tools/typegen/decoding) to decode event logs and transactions of EVM contracts
  </Tab>
</Tabs>

### (Optional) IV. Mix in external data and chain state calls output

If you need external (i.e. non-blockchain) data in your transformation, take a look at the [External APIs and IPFS](/en/sdk/squid-sdk/resources/external-api) page.

If any of the on-chain data you need is unavalable from the processor or incovenient to retrieve with it, you have an option to get it via [direct chain queries](/en/sdk/squid-sdk/resources/tools/typegen/state-queries).

### V. Prepare the store

At `src/main.ts`, change the [`Database`](/en/sdk/squid-sdk/resources/persisting-data/overview) object definition to accept your output data. The methods for saving data will be exposed by `ctx.store` within the [batch handler](/en/sdk/squid-sdk/reference/processors/architecture).

<Tabs>
  <Tab title="PostgreSQL+GraphQL">
    1. Define the schema of the database (and the [core schema of the OpenReader GraphQL API](/en/sdk/squid-sdk/reference/openreader-server/api) if it is used) at [`schema.graphql`](/en/sdk/squid-sdk/reference/schema-file).

    2. Regenerate the TypeORM model classes with
       ```bash theme={"system"}
       npx squid-typeorm-codegen
       ```
       The classes will become available at `src/model`.

    3. Compile the models code with
       ```bash theme={"system"}
       npm run build
       ```

    4. Ensure that the squid has access to a blank database. The easiest way to do so is to start PostgreSQL in a Docker container with

       ```bash theme={"system"}
       docker compose up -d
       ```

       If the container is running, stop it and erase the database with

       ```bash theme={"system"}
       docker compose down
       ```

       before issuing an `docker compose up -d`.

       The alternative is to connect to an external database. See [this section](/en/sdk/squid-sdk/reference/store/typeorm#database-connection-parameters) to learn how to specify the connection parameters.

    5. Regenerate a migration with
       ```bash theme={"system"}
       rm -r db/migrations
       ```
       ```bash theme={"system"}
       npx squid-typeorm-migration generate
       ```

    You can now use the async functions [`ctx.store.upsert()`](/en/sdk/squid-sdk/reference/store/typeorm#upsert) and [`ctx.store.insert()`](/en/sdk/squid-sdk/reference/store/typeorm#insert), as well as various [TypeORM lookup methods](/en/sdk/squid-sdk/reference/store/typeorm#typeorm-methods) to access the database.

    See the `typeorm-store` [guide](/en/sdk/squid-sdk/resources/persisting-data/typeorm) and [reference](/en/sdk/squid-sdk/reference/store/typeorm) for more info.
  </Tab>

  <Tab title="filesystem dataset">
    Filesystem dataset writing, as performed by the `@subsquid/file-store` package and its extensions, stores the data into one or more flat tables. The exact table definition format depends on the output file format.

    1. Decide on the file format you're going to use:

       * [Parquet](/en/sdk/squid-sdk/reference/store/file/parquet)
       * [CSV](/en/sdk/squid-sdk/reference/store/file/csv)
       * [JSON/JSONL](/en/sdk/squid-sdk/reference/store/file/json)

       If your template does not have any of the necessary packages, install them.

    2. Define any tables you need at the `tables` field of the `Database` constructor argument:
       ```ts theme={"system"}
       import { Database } from '@subsquid/file-store'

       const dbOptions = {
         tables: {
           FirstTable: new Table(/* ... */),
           SecondTable: new Table(/* ... */),
           // ...
         },
         // ...
       }

       processor.run(new Database(dbOptions), async ctx => { // ...
       ```

    3. Define the destination filesystem via the `dest` field of the `Database` constructor argument. Options:
       * local folder - use `LocalDest` from `@subsquid/file-store`
       * S3-compatible file storage service - install `@subsquid/file-store-s3` and use [`S3Dest`](/en/sdk/squid-sdk/reference/store/file/s3-dest)

    Once you're done you'll be able to enqueue data rows for saving using the `write()` and `writeMany()` methods of the context store-provided table objects:

    ```ts theme={"system"}
    ctx.store.FirstTable.writeMany(/* ... */)
    ctx.store.SecondTable.write(/* ... */)
    ```

    The store will write the files automatically as soon as the buffer reaches the size set by the `chunkSizeMb` field of the `Database` constructor argument, or at the end of the batch if a call to [`setForceFlush()`](/en/sdk/squid-sdk/resources/persisting-data/file#setforceflush) was made anywhere in the batch handler.

    See the `file-store` [guide](/en/sdk/squid-sdk/resources/persisting-data/file) and the [reference pages of its extensions](/en/sdk/squid-sdk/reference/store/file).
  </Tab>

  <Tab title="BigQuery">
    Follow the [guide](/en/sdk/squid-sdk/resources/persisting-data/bigquery).
  </Tab>
</Tabs>

### VI. Persist the transformed data to your data sink

Once your data is [decoded](#batch-handler-decoding), optionally [enriched with external data](#external-data) and transformed the way you need it to be, it is time to save it.

<Tabs>
  <Tab title="PostgreSQL+GraphQL">
    For each batch, create all the instances of all TypeORM model classes at once, then save them with the minimal number of calls to `upsert()` or `insert()`, e.g.:

    ```ts theme={"system"}
    import { EntityA, EntityB } from './model'

    processor.run(new TypeormDatabase(), async ctx => {
      const aEntities: Map<string, EntityA> = new Map() // id -> entity instance
      const bEntities: EntityB = []

      for (let block of ctx.blocks) {
        // fill the containets aEntities and bEntities
      }

      await ctx.store.upsert([...aEntities.values()])
      await ctx.store.insert(bEntities)
    })
    ```

    It will often make sense to keep the entity instances in maps rather than arrays to make it easier to reuse them when defining instances of other entities with [relations](/en/sdk/squid-sdk/reference/schema-file/entity-relations) to the previous ones. The process is described in more detail in the [step 2 of the BAYC tutorial](/en/sdk/squid-sdk/tutorials/bayc/step-two-deriving-owners-and-tokens).

    If you perform any [database lookups](/en/sdk/squid-sdk/reference/store/typeorm#typeorm-methods), try to do so in batches and make sure that the entity fields that you're searching over are [indexed](/en/sdk/squid-sdk/reference/schema-file/indexes-and-constraints).

    See also the [patterns](/en/sdk/squid-sdk/resources/batch-processing#patterns) and [anti-pattens](/en/sdk/squid-sdk/resources/batch-processing#anti-patterns) sections of the Batch processing guide.
  </Tab>

  <Tab title="filesystem dataset">
    You can enqueue the transformed data for writing whenever convenient without any sizeable impact on performance.

    At low output data rates (e.g. if your entire dataset is in tens of Mbytes or under) take care to call [`ctx.store.setForceFlush()`](/en/sdk/squid-sdk/resources/persisting-data/file#setforceflush) when appropriate to make sure your data actually gets written.
  </Tab>

  <Tab title="BigQuery">
    You can enqueue the transformed data for writing whenever convenient without any sizeable impact on performance. The actual data writing will happen automatically at the end of each batch.
  </Tab>
</Tabs>

## The top-down development cycle

The [bottom-up development cycle](#bottom-up-development) described above is convenient for inital squid development and for trying out new things, but it has the disadvantage of not having the means of saving the data ready at hand when initially writing the data decoding/transformation code. That makes it necessary to come back to that code later, which is somewhat inconvenient e.g. when adding new squid features incrementally.

The alternative is to do the same steps in a different order:

1. [Update the store](#store)
2. If necessary, [regenerate the utility classes](#typegen)
3. [Update the processor configuration](#processor-config)
4. [Decode and normalize the added data](#batch-handler-decoding)
5. [Retrieve any external data](#external-data) if necessary
6. [Add the persistence code for the transformed data](#batch-handler-persistence)

## GraphQL options

[Store your data to PostgreSQL](/en/sdk/squid-sdk/resources/persisting-data/typeorm), then consult [Serving GraphQL](/en/sdk/squid-sdk/resources/serving-graphql) for options.

## Scaling up

If you're developing a large squid, make sure to use [batch processing](/en/sdk/squid-sdk/resources/batch-processing) throughout your code.

A common mistake is to make handlers for individual event logs or transactions; for updates that require data retrieval that results in lots of small database lookups and ultimately in poor syncing performance. Collect all the relevant data and process it at once. A simple architecture of that type is discussed in the [BAYC tutorial](/en/sdk/squid-sdk/tutorials/bayc).

You should also check the [Cloud best practices page](/en/cloud/resources/best-practices) even if you're not planning to deploy to [SQD Cloud](/en/cloud) - it contains valuable performance-related tips.

Many issues commonly arising when developing larger squids are addressed by the third party [`@belopash/typeorm-store` package](https://github.com/belopash/squid-typeorm-store). Consider using it.

For complete examples of complex squids take a look at the [Giant Squid Explorer](https://github.com/subsquid-labs/giant-squid-explorer) and [Thena Squid](https://github.com/subsquid-labs/thena-squid) repos.

## Next steps

* Learn about [batch processing](/en/sdk/squid-sdk/resources/batch-processing).
* Learn how squid deal with [unfinalized blocks](/en/sdk/squid-sdk/resources/unfinalized-blocks).
* [Use external APIs and IPFS](/en/sdk/squid-sdk/resources/external-api) in your squid.
* See how squid should be set up for the [multichain setting](/en/sdk/squid-sdk/resources/multichain).
* Deploy your squid [on own infrastructure](/en/sdk/squid-sdk/resources/self-hosting) or to [SQD Cloud](/en/cloud).


# Indexer from scratch
Source: https://docs.sqd.dev/en/sdk/squid-sdk/how-to-start/squid-from-scratch

Build a Squid SDK indexer from scratch — learn how to compose evm-processor, typeorm-store, and graphql-server NPM packages without a starter template.

Here's an example of how SDK packages can be combined into a working indexer (called *squid*).

This page goes through all the technical details to make the squid architecture easier to understand. If you would like to get to a working indexer ASAP, [bootstrap from a template](/en/sdk/squid-sdk/how-to-start/squid-development#templates).

## USDT transfers API

<Info>
  **Pre-requisites**: NodeJS 20.x or newer, Docker.
</Info>

Suppose the task is to track transfers of USDT on Ethereum, then save the resulting data to PostgreSQL and serve it as a GraphQL API. From this description we can immediately put together a list of [packages](/en/sdk/squid-sdk/overview):

* `@subsquid/evm-processor` - for retrieving Ethereum data
* the triad of `@subsquid/typeorm-store`, `@subsquid/typeorm-codegen` and `@subsquid/typeorm-migration` - for saving data to PostgreSQL

We also assume the following choice of *optional* packages:

* `@subsquid/evm-typegen` - for decoding Ethereum data and useful constants such as event topic0 values
* `@subsquid/evm-abi` - as a peer dependency for the code generated by `@subsquid/evm-typegen`
* `@subsquid/graphql-server` / [OpenReader](/en/sdk/squid-sdk/reference/openreader-server)

To make the indexer, follow these steps:

<Steps>
  <Step title="Create a new folder and initialise a new project">
    * create `package.json`
      ```bash theme={"system"}
      npm init
      ```
    * add `.gitignore`
      ```bash title=".gitignore" theme={"system"}
      node_modules
      lib
      ```
  </Step>

  <Step title="Install the packages">
    ```bash theme={"system"}
    npm i dotenv typeorm @subsquid/evm-processor @subsquid/typeorm-store @subsquid/typeorm-migration @subsquid/graphql-server @subsquid/evm-abi
    ```

    ```bash theme={"system"}
    npm i typescript @subsquid/typeorm-codegen @subsquid/evm-typegen --save-dev
    ```
  </Step>

  <Step title="Add a minimal `tsconfig.json`">
    ```json title="tsconfig.json" theme={"system"}
    {
      "compilerOptions": {
        "rootDir": "src",
        "outDir": "lib",
        "module": "commonjs",
        "target": "es2020",
        "esModuleInterop": true,
        "skipLibCheck": true,
        "experimentalDecorators": true,
        "emitDecoratorMetadata": true
      }
    }
    ```
  </Step>

  <Step title="Define the schema for both the database and the core GraphQL API">
    Define the schema for both the database and the core GraphQL API in [`schema.graphql`](/en/sdk/squid-sdk/reference/schema-file):

    ```graphql title="schema.graphql" theme={"system"}
    type Transfer @entity {
      id: ID!
      from: String! @index
      to: String! @index
      value: BigInt!
    }
    ```
  </Step>

  <Step title="Generate TypeORM classes based on the schema">
    ```bash theme={"system"}
    npx squid-typeorm-codegen
    ```

    The TypeORM classes are now available at `src/model/index.ts`.
  </Step>

  <Step title="Prepare the database">
    * create `.env` and `docker-compose.yaml` files
      ```bash title=".env" theme={"system"}
      DB_NAME=squid
      DB_PORT=23798
      RPC_ETH_HTTP=https://rpc.ankr.com/eth
      ```
      ```yaml title="docker-compose.yaml" theme={"system"}
      services:
        db:
          image: postgres:15
          environment:
            POSTGRES_DB: "${DB_NAME}"
            POSTGRES_PASSWORD: postgres
          ports:
            - "${DB_PORT}:5432"
      ```
    * start the database container
      ```bash theme={"system"}
      docker compose up -d
      ```
    * compile the TypeORM classes
      ```bash theme={"system"}
      npx tsc
      ```
    * generate the migration file
      ```bash theme={"system"}
      npx squid-typeorm-migration generate
      ```
    * apply the migration with
      ```bash theme={"system"}
      npx squid-typeorm-migration apply
      ```
  </Step>

  <Step title="Generate utility classes for decoding USDT contract data">
    [Generate utility classes](/en/sdk/squid-sdk/resources/tools/typegen/generation) for decoding [USDT contract](https://etherscan.io/address/0xdac17f958d2ee523a2206206994597c13d831ec7) data based on its ABI.

    * Create an `./abi` folder:
      ```bash theme={"system"}
      mkdir abi
      ```

    * Find the ABI at the ["Contract" tab of the contract page on Etherscan](https://etherscan.io/address/0xdAC17F958D2ee523a2206206994597C13D831ec7#code). Scroll down a bit:

    * Copy the ABI, then paste to a new file at `./abi/usdt.json`.

    * Run the utility classes generator:
      ```bash theme={"system"}
      npx squid-evm-typegen src/abi ./abi/*
      ```

    The utility classes are now available at `src/abi/usdt.ts`
  </Step>

  <Step title="Tie all the generated code together with `src/main.ts`">
    Tie all the generated code together with a `src/main.ts` executable with the following code blocks:

    * Imports
      ```ts theme={"system"}
      import { EvmBatchProcessor } from '@subsquid/evm-processor'
      import { TypeormDatabase } from '@subsquid/typeorm-store'
      import * as usdtAbi from './abi/usdt'
      import { Transfer } from './model'
      ```
    * [`EvmBatchProcessor`](/en/sdk/squid-sdk/reference/processors/evm-batch) object definition
      ```ts theme={"system"}
      const processor = new EvmBatchProcessor()
        .setGateway('https://v2.archive.subsquid.io/network/ethereum-mainnet')
        .setRpcEndpoint({
          url: process.env.RPC_ETH_HTTP,
          rateLimit: 10
        })
        .setFinalityConfirmation(75) // 15 mins to finality
        .addLog({
          address: [ '0xdAC17F958D2ee523a2206206994597C13D831ec7' ],
          topic0: [ usdtAbi.events.Transfer.topic ]
        })
      ```
    * [`TypeormDatabase`](/en/sdk/squid-sdk/reference/store/typeorm) object definition
      ```ts theme={"system"}
      const db = new TypeormDatabase()
      ```
    * A call to [`processor.run()`](/en/sdk/squid-sdk/reference/processors/architecture#processorrun) with an inline definition of the [batch handler](/en/sdk/squid-sdk/reference/processors/architecture#batch-context)
      ```ts theme={"system"}
      processor.run(db, async ctx => {
        const transfers: Transfer[] = []
        for (let block of ctx.blocks) {
          for (let log of block.logs) {
            let {from, to, value} = usdtAbi.events.Transfer.decode(log)
            transfers.push(new Transfer({
              id: log.id,
              from, to, value
            }))
          }
        }
        await ctx.store.insert(transfers)
      })
      ```
      Note how supplying a `TypeormDatabase` to the function caused `ctx.store` to be a [PostgreSQL-compatible `Store` object](/en/sdk/squid-sdk/reference/store/typeorm#store-interface).
  </Step>

  <Step title="Compile the project and start the processor process">
    Compile the project and start the [processor process](/en/sdk/squid-sdk/overview#processor)

    ```bash theme={"system"}
    npx tsc
    ```

    ```bash theme={"system"}
    node -r dotenv/config lib/main.js
    ```
  </Step>

  <Step title="Start the GraphQL server">
    In a separate terminal, configure the GraphQL port and start the GraphQL server:

    ```diff title=".env" theme={"system"}
     DB_NAME=squid
     DB_PORT=23798
     RPC_ETH_HTTP=https://rpc.ankr.com/eth
    +GRAPHQL_SERVER_PORT=4350
    ```

    ```bash theme={"system"}
    npx squid-graphql-server
    ```
  </Step>
</Steps>

The finished GraphQL API with GraphiQL is available at [localhost:4350/graphql](http://localhost:4350/graphql).

Final code for this mini-tutorial is available in [this repo](https://github.com/subsquid-labs/squid-from-scratch).

<Tip>
  The commands listed here are often abbreviated as [custom `sqd` commands](/en/sdk/squid-sdk/squid-cli/commands-json) in squids. If you'd like to do that too you can use the [`commands.json` file of the EVM template](https://github.com/subsquid-labs/squid-evm-template/blob/main/commands.json) as a starter.
</Tip>


# Squid SDK Overview
Source: https://docs.sqd.dev/en/sdk/squid-sdk/overview

Squid SDK architecture overview — processor, batch handlers, schema, and Store roles in building TypeScript blockchain indexers with GraphQL APIs.

A *squid* is an indexing project built with [Squid SDK](https://github.com/subsquid/squid-sdk) to retrieve and process blockchain data from the [SQD Network](/en/network/overview)
(either permissioned or decentralized instance). The Squid SDK is a set of open source Typescript libraries that retrieve, decode, transform and persist the data. It can also make the transformed data available via an API. All stages of the indexing pipeline, from the data extraction to transformation to persistence are performed on [batches of blocks](/en/sdk/squid-sdk/resources/batch-processing) to maximize the indexing speed. Modular architecture of the SDK makes it possible to extend indexing projects (squids) with custom plugins and data targets.

## Required squid components

### Processor

*Processor* is the word used for

1. The main NodeJS process of the squid.
2. The main object (`processor`) of this process: its method call `processor.run()` is the entry point.

`processor` objects handle data retrieval and transformation; data persistence is handled by a separate object called [Store](#store). Squid SDK offers two processor classes:

* [`EvmBatchProcessor`](/en/sdk/squid-sdk/reference/processors/evm-batch) via the [`@subsquid/evm-processor`](https://www.npmjs.com/package/@subsquid/evm-processor) NPM package - for Ethereum-compatible [networks](/en/data/evm)
* [`SubstrateBatchProcessor`](/en/sdk/squid-sdk/reference/processors/substrate-batch) via [`@subsquid/substrate-processor`](https://www.npmjs.com/package/@subsquid/substrate-processor) - for [networks](/en/data/substrate) based on [Substrate](https://substrate.io) such as [Polkadot](https://polkadot.network)

### Store

A *store* is an object that processors use to persist their data. SQD offers three store classes:

* [`TypeormStore`](/en/sdk/squid-sdk/resources/persisting-data/typeorm) for saving data to PostgreSQL, via

  * [`@subsquid/typeorm-store`](https://www.npmjs.com/package/@subsquid/typeorm-store)
  * [`@subsquid/typeorm-codegen`](https://www.npmjs.com/package/@subsquid/typeorm-codegen) (a code generator, install with `--save-dev`)
  * [`@subsquid/typeorm-migration`](https://www.npmjs.com/package/@subsquid/typeorm-migration)

  Install all three packages to use this store.
* [`file-store`](/en/sdk/squid-sdk/resources/persisting-data/file) via [`@subsquid/file-store`](https://www.npmjs.com/package/@subsquid/file-store) - for saving data to filesystems. It is a modular system with a variety of [extensions](/en/sdk/squid-sdk/reference/store/file) for various formats and destinations.
* [`bigquery-store`](/en/sdk/squid-sdk/resources/persisting-data/bigquery) via [`@subsquid/bigquery-store`](https://www.npmjs.com/package/@subsquid/bigquery-store) - for saving data to [Google BigQuery](https://cloud.google.com/bigquery).

You can mix and match any store class with any processor class.

## Optional squid components

### Typegen

A *typegen* is a tool for generating utility code for technology-specific operations such as decoding. Here are the typegens available:

<Tabs>
  <Tab title="On EVM">
    * [`squid-evm-typegen`](/en/sdk/squid-sdk/resources/tools/typegen/generation) via [`@subsquid/evm-typegen`](https://www.npmjs.com/package/@subsquid/evm-typegen):
      * decodes smart contract data
      * handles direct calls to contract methods
      * exposes useful constants such as event topics and function signature hashes
        The generated code depends on [`@subsquid/evm-abi`](https://www.npmjs.com/package/@subsquid/evm-abi), SQD's own high performance, open source EVM codec.
  </Tab>

  <Tab title="On Substrate">
    * [`squid-substrate-typegen`](/en/sdk/squid-sdk/resources/tools/typegen/generation) via [`@subsquid/substrate-typegen`](https://www.npmjs.com/package/@subsquid/substrate-typegen):
      * general purpose pallet data decoding (aware of runtime versions)
      * handles direct storage queries
    * [`squid-ink-typegen`](/en/sdk/squid-sdk/resources/tools/typegen/generation) via [`@subsquid/ink-typegen`](https://www.npmjs.com/package/@subsquid/ink-typegen) for decoding the data of [ink!](https://use.ink) contracts
  </Tab>
</Tabs>

Install these with `--save-dev`.

### GraphQL server

Squids that store their data in PostgreSQL can subsequently make it available as a GraphQL API via a variety of supported servers. See [Serving GraphQL](/en/sdk/squid-sdk/resources/serving-graphql).

Among other alternatives, SQD provides its own server called [OpenReader](/en/sdk/squid-sdk/reference/openreader-server) via the [`@subsquid/graphql-server`](https://www.npmjs.com/package/@subsquid/graphql-server) package. The server runs as a separate process. [Core API](/en/sdk/squid-sdk/reference/openreader-server/api) is automatically derived from the schema file; it is possible to extend it with [custom queries](/en/sdk/squid-sdk/reference/openreader-server/configuration/custom-resolvers) and [basic access control](/en/sdk/squid-sdk/reference/openreader-server/configuration/authorization).

### Misc utilities

<Tabs>
  <Tab title="On EVM">
    * [Squid CLI](/en/sdk/squid-sdk/squid-cli/installation) is a utility for [retrieving squid templates](/en/sdk/squid-sdk/squid-cli/init), managing [chains of commands](https://github.com/subsquid/squid-sdk/tree/master/util/commands) commonly used in development and running [all squid processes at once](/en/sdk/squid-sdk/squid-cli/run). It can also be used for deploying to [SQD Cloud](/en/cloud).
  </Tab>

  <Tab title="On Substrate">
    * [Squid CLI](/en/sdk/squid-sdk/squid-cli/installation) is a utility for [retrieving squid templates](/en/sdk/squid-sdk/squid-cli/init), managing [chains of commands](https://github.com/subsquid/squid-sdk/tree/master/util/commands) commonly used in development and running [all squid processes at once](/en/sdk/squid-sdk/squid-cli/run). It can also be used for deploying to [SQD Cloud](/en/cloud).
    * [`@subsquid/ss58`](https://www.npmjs.com/package/@subsquid/ss58) handles encoding and decoding of SS58 addresses
    * [`@subsquid/frontier`](https://www.npmjs.com/package/@subsquid/frontier) decodes events and calls of the [Frontier EVM pallet](https://paritytech.github.io/frontier/frame/evm.html) to make them decodable with [`squid-evm-typegen`](/en/sdk/squid-sdk/resources/tools/typegen/generation)
  </Tab>
</Tabs>


# Squid SDK Quickstart
Source: https://docs.sqd.dev/en/sdk/squid-sdk/quickstart

5-minute Squid SDK quickstart — build a TypeScript indexer that streams USDC transfers from Ethereum into local Postgres for GraphQL queries.

This is a 5 min quickstart on how to build an indexer using Squid SDK.

The indexer (squid) will:

* Fetch all historical USDC transfers on Ethereum from the SQD Network
* Decode it
* Save it to a local Postgres database
* Start a GraphQL server with a rich API to query the historical USDC transfers.

## Prerequisites

* (On Windows) [WSL](https://learn.microsoft.com/en-us/windows/wsl/install)
* Node.js v18+
* Git
* Docker (for running Postgres)

<Steps>
  <Step title="Step 1">
    Install Squid CLI:

    ```bash theme={"system"}
    npm i -g @subsquid/cli
    ```

    Squid CLI is a multi-purpose utility tool for scaffolding and managing the indexers, both locally and in SQD Cloud.
  </Step>

  <Step title="Step 2">
    Scaffold the indexer project (squid) from an example repo using Squid CLI:

    ```bash theme={"system"}
    sqd init hello-squid -t https://github.com/subsquid-labs/showcase01-all-usdc-transfers
    cd hello-squid
    ```

    The example is just a public GitHub repo with a squid project pre-configured to index USDC.
  </Step>

  <Step title="Step 3">
    Inspect the `./src` folder:

    <Tree>
      <Tree.Folder name="src">
        <Tree.Folder name="abi">
          <Tree.File name="usdc.ts" />
        </Tree.Folder>

        <Tree.File name="main.ts" />

        <Tree.Folder name="model">
          <Tree.Folder name="generated">
            <Tree.File name="index.ts" />

            <Tree.File name="marshal.ts" />

            <Tree.File name="usdcTransfer.model.ts" />
          </Tree.Folder>

          <Tree.File name="index.ts" />
        </Tree.Folder>
      </Tree.Folder>
    </Tree>

    Here,

    * `src/abi/usdc` is a utility module generated from the JSON ABI of the USDC contract. It contains methods for event decoding, direct RPC queries and some useful constants.
    * `src/model` contains TypeORM model classes autogenerated from `schema.graphql`. Squids use them to populate Postgres.

    `main.ts` is the main executable. In this example, it also contains all the data retrieval configuration:

    ```ts theme={"system"}
    const processor = new EvmBatchProcessor()
      // SQD Network gateways are the primary source of blockchain data in
      // squids, providing pre-filtered data in chunks of roughly 1-10k blocks.
      // Set this for a fast sync.
      .setGateway('https://v2.archive.subsquid.io/network/ethereum-mainnet')
      // Another data source squid processors can use is chain RPC.
      // In this particular squid it is used to retrieve the very latest chain data
      // (including unfinalized blocks) in real time. It can also be used to
      //   - make direct RPC queries to get extra data during indexing
      //   - sync a squid without a gateway (slow)
      .setRpcEndpoint('https://rpc.ankr.com/eth')
      // The processor needs to know how many newest blocks it should mark as "hot".
      // If it detects a blockchain fork, it will roll back any changes to the
      // database made due to orphaned blocks, then re-run the processing for the
      // main chain blocks.
      .setFinalityConfirmation(75)
      // .addXXX() methods request data items. In this case we're asking for
      // Transfer(address,address,uint256) event logs emitted by the USDC contract.
      //
      // We could have omitted the "address" filter to get Transfer events from
      // all contracts, or the "topic0" filter to get all events from the USDC
      // contract, or both to get all event logs chainwide. We also could have
      // requested some related data, such as the parent transaction or its traces.
      //
      // Other .addXXX() methods (.addTransaction(), .addTrace(), .addStateDiff()
      // on EVM) are similarly feature-rich.
      .addLog({
        range: { from: 6_082_465 },
        address: [USDC_CONTRACT_ADDRESS],
        topic0: [usdcAbi.events.Transfer.topic],
      })
      // .setFields() is for choosing data fields for the selected data items.
      // Here we're requesting hashes of parent transaction for all event logs.
      .setFields({
        log: {
          transactionHash: true,
        },
      })
    ```

    The rest of the file is about data processing and storage:

    ```ts theme={"system"}
    // TypeormDatabase objects store the data to Postgres. They are capable of
    // handling the rollbacks that occur due to blockchain forks.
    //
    // There are also Database classes for storing data to files and BigQuery
    // datasets.
    const db = new TypeormDatabase({supportHotBlocks: true})

    // The processor.run() call executes the data processing. Its second argument is
    // the handler function that is executed once on each batch of data. Processor
    // object provides the data via "ctx.blocks". However, the handler can contain
    // arbitrary TypeScript code, so it's OK to bring in extra data from IPFS,
    // direct RPC calls, external APIs etc.
    processor.run(db, async (ctx) => {
      // Making the container to hold that which will become the rows of the
      // usdc_transfer database table while processing the batch. We'll insert them
      // all at once at the end, massively saving IO bandwidth.
      const transfers: UsdcTransfer[] = []

      // The data retrieved from the SQD Network gatewat and/or the RPC endpoint
      // is supplied via ctx.blocks
      for (let block of ctx.blocks) {
        // On EVM, each block has four iterables - logs, transactions, traces,
        // stateDiffs
        for (let log of block.logs) {
          if (log.address === USDC_CONTRACT_ADDRESS &&
              log.topics[0] === usdcAbi.events.Transfer.topic) {
            // SQD's very own EVM codec at work - about 20 times faster than ethers
            let {from, to, value} = usdcAbi.events.Transfer.decode(log)
            transfers.push(new UsdcTransfer({
              id: log.id,
              block: block.header.height,
              from,
              to,
              value,
              txnHash: log.transactionHash
            }))
          }
        }
      }

      // Just one insert per batch!
      await ctx.store.insert(transfers)
    })
    ```
  </Step>

  <Step title="Step 4">
    Install the dependencies and build

    ```bash theme={"system"}
    npm i
    npm run build
    ```
  </Step>

  <Step title="Step 5">
    The processor is a background process that continously fetches the data, decodes it and stores it in a local Postgres. All the logic is defined in `main.ts` and is fully customizable.

    To run the processor, we first start a local Postgres where the decoded data is persisted (the template comes with a Docker compose file):

    ```bash theme={"system"}
    docker compose up -d
    ```

    Processor will connect to Postgres using the connection parameters from `.env`.

    Apply database migrations with

    ```bash theme={"system"}
    npx squid-typeorm-migration apply
    ```

    then start the processor with

    ```bash theme={"system"}
    node -r dotenv/config lib/main.js
    ```

    The indexer is now running.
  </Step>

  <Step title="Step 6">
    Start the GraphQL API serving the transfers data from Postgres:

    ```bash theme={"system"}
    npx squid-graphql-server
    ```

    The server comes with a GraphQL playground available at [`localhost:4350/graphql`](http://localhost:4350/graphql).
  </Step>

  <Step title="Step 7">
    Query the data!
  </Step>
</Steps>


# AND/OR filters
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/openreader-server/api/and-or-filters

Combine GraphQL query filters with AND/OR operators in OpenReader — basic logic combinators for refining entity queries served by Squid SDK.

## Overview

Our GraphQL implementation offers a vast selection of tools to filter and section results. One of these is the `where` clause, very common in most database query languages and [explained here](/en/sdk/squid-sdk/reference/openreader-server/api/queries#filter-query-results--search-queries) in detail.

In our GraphQL server implementation, we included logical operators to be used in the `where` clause, allowing to group multiple parameters in the same `where` argument using the `AND` and `OR` operators to filter results based on more than one criteria.

Note that the [newer](/en/sdk/squid-sdk/reference/openreader-server/overview#supported-queries) and [more advanced](/en/sdk/squid-sdk/reference/openreader-server/api/paginate-query-results) `{entityName}sConnection` queries support exactly the same format of the `where` argument as the older `{entityName}s` queries used in the examples provided here.

### Example of an `OR` clause:

Fetch a list of `accounts` that either have a balance bigger than a certain amount, or have a specific id.

```graphql theme={"system"}
query {
  accounts(
    orderBy: balance_DESC, 
    where: {
      OR: [
        {balance_gte: "240000000000000000"}
        {id_eq: "CksmaBx9rKUG9a7eXwc5c965cJ3QiiC8ELFsLtJMYZYuRWs"}
      ]
    }
  ) {
    balance
    id
  }
}

```

### Example of `AND` clause:

Fetch a list of `accounts` that have a balance between two specific amounts:

```graphql theme={"system"}
query {
  accounts(
    orderBy: balance_DESC, 
    where: {
      AND: [
        {balance_lte: "240000000000000000"}
        {balance_gte: "100000000000000"}
      ]
    }
  ) {
    balance
    id
  }
}

```


# Cross-relation queries
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/openreader-server/api/cross-relation-field-queries

Filter Squid SDK GraphQL queries by fields of related entities — cross-relation predicates in OpenReader for traversing nested data in queries.

# Cross-relation field queries

## Introduction

The [previous section](/en/sdk/squid-sdk/reference/openreader-server/api/nested-field-queries) has already demonstrated that queries can return not just scalars such as a String, but also fields that refer to object or entity types. What's even more interesting is that queries can leverage fields of related objects to filter results.

Let's take this sample schema with two entity types and a one-to-many relationship between them:

```graphql title="schema.graphql" theme={"system"}
type Account @entity {
    id: ID!
    wallet: String!
    balance: Int!
    history: [HistoricalBalance!] @derivedFrom(field: "account")
}

type HistoricalBalance @entity {
    "Unique identifier"
    id: ID!
    
    "Related account"
    account: Account!
    
    "Balance"
    balance: Int!
}
```

With the functionality offered by cross-relation field queries, we could ask for `Account`s that have at least some `historicalBalance`s with a `balance` smaller than a certain threshold:

```graphql theme={"system"}
query MyQuery {
  accounts(where: {historicalBalances_some: {balance_lt: "10000000000"}}) {
    id
  }
}
```

This allows to query based not just on the entity itself, but on the related entities as well, which is intuitively a very powerful feature.

`*_some` is not the only operator available for making cross-relation field queries. A short description of each such operator is provided in the sections below.

## The `*_every` filter

Returns entities for which **all** of the nested entities linked via the related field satisfy the condition. Example:

```graphql title="schema.graphql" theme={"system"}
query MyQuery {
  accounts(where: {historicalBalances_every: {balance_lt: "10000000000"}}) {
    id
  }
}
```

This query will return all `Account`s where **each and every one** of the `HistoricalBalance` entities related to them have a `balance` smaller than the threshold. It is sufficient for a single `HistoricalBalance` to have a `balance` larger than the set value to make sure that the related `Account` is not returned in the query.

## The `*_none`  filter

Returns entities for which **none** of the nested entities linked via the related field satisfy the condition. Example:

```graphql theme={"system"}
query MyQuery {
  accounts(where: {historicalBalances_none: {balance_lt: "10000000000"}}) {
    id
  }
}
```

The query will return all `Account`s in which not a single related `HistoricalBalance` has a `balance` smaller than the set threshold.

## The `*_some` filter

Returns entities for which **at least one** of the nested entities linked via the related field satisfies the condition. Example:

```graphql theme={"system"}
query MyQuery {
  accounts(where: {historicalBalances_some: {balance_lt: "10000000000"}}) {
    id
  }
}
```

All `Account`s that have at least some `historicalBalance`s with a `balance` smaller than `10000000000` will be returned. This means that a single `HistoricalBalance` satisfying the condition is sufficient for the related `Account` to become a part of the results.

## `{entityName}sConnection` queries

Same as always, the `where` argument works for these queries in exactly the same way as it does for `{entityName}s` queries used in examples above. For example this query

```graphql theme={"system"}
query MyQuery {
  accountsConnection(orderBy: id_ASC, where: {historicalBalances_some: {balance_lt: "10000000000"}}) {
    edges {
      node {
        id
      }
    }
  }
}
```

will return (in an appropriately shaped response) IDs for all `Accounts` that have at least some `historicalBalance`s with a `balance` smaller than `10000000000`.


# OpenReader GraphQL Intro
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/openreader-server/api/intro

Introduction to GraphQL queries served by OpenReader, the Squid SDK GraphQL server — entity queries, filters, sorting, and pagination basics.

<Info>
  At the moment, [Squid SDK GraphQL server](/en/sdk/squid-sdk/reference/openreader-server) can only be used with squids that use Postgres as their target database.
</Info>

GraphQL is an API query language, and a server-side runtime for executing queries using a custom type system. Head over to the [official documentation website](https://graphql.org/learn/) for more info.

A GraphQL API served by the [GraphQL server](/en/sdk/squid-sdk/reference/openreader-server) has two components:

1. Core API is defined by the [schema file](/en/sdk/squid-sdk/reference/schema-file).
2. Extensions added via [custom resolvers](/en/sdk/squid-sdk/reference/openreader-server/configuration/custom-resolvers).

In this section we cover the core GraphQL API, with short explanations on how to perform GraphQL queries, how to paginate and sort results. This functionality is supported via [OpenReader](https://github.com/subsquid/squid-sdk/tree/master/graphql/openreader), SQD's own implementation of [OpenCRUD](https://www.opencrud.org).


# JSON queries
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/openreader-server/api/json-queries

Query entities with object-typed and JSON fields in OpenReader — Squid SDK GraphQL syntax for filtering and selecting nested JSON content.

The possibility of defining JSON objects as fields of a type in a GraphQL schema has been explained in the [schema reference](/en/sdk/squid-sdk/reference/schema-file).

This guide is focusing on how to query such objects and how to fully leverage their potential. Let's take the example of this (non-crypto related, for once😁) schema:

```graphql title="schema.graphql" theme={"system"}
type Entity @entity {
    id: ID!
    a: A
}

type A {
    a: String
    b: B
}

type B {
    a: A
    b: String
    e: Entity
}
```

It's composed of one entity and two JSON objects definitions, used in a "nested" way.

Let's now look at a simple query:

```graphql theme={"system"}
query {
    entities(orderBy: id_ASC) {
        id
        a { a }
    }
}
```

This will return a result such as this one (imagining this data exists in the database):

```graphql theme={"system"}
{
  entities: [
    {id: '1', a: {a: 'a'}},
    {id: '2', a: {a: 'A'}},
    {id: '3', a: {a: null}},
    {id: '4', a: null}
  ]
}
```

Simply enough, the first two objects have an object of type `A` with some content inside, the third one has an object, but its `a` field is `null` and the fourth one simply does not have an `A` object at all.


# Nested field queries
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/openreader-server/api/nested-field-queries

Query Squid SDK entities related to other entities via nested fields in OpenReader — traverse one-to-many and many-to-many relations in GraphQL.

With OpenReader, fields of an Entity that contain fields themselves are shown as nested fields and it is possible to filter these as well. GraphQL queries can traverse related objects and their fields, letting clients fetch lots of related data in one request, instead of making several roundtrips as one would need in a classic REST architecture.

As an example, this query searches for all `accounts` whose balance is bigger than a threshold value, fetching the `id` and `balance` simple fields, as well as the `historicalBalances` **nested field**.

```graphql theme={"system"}
query {
  accounts(orderBy: balance_ASC, where: {balance_gte: "250000000000000000"}) {
    id
    balance
    historicalBalances {
      balance
      date
      id
    }
  }
}

```

A nested field is a list (one account can have multiple `historicalBalances`) of objects with fields of their own. These objects can be filtered, too.

In the following query the `historicalBalances` are filtered in order to only return the balances created after a certain date:

```graphql theme={"system"}
query {
  accounts(orderBy: balance_ASC, where: {balance_gte: "250000000000000000"}) {
    id
    balance
    historicalBalances(where: {date_lte: "2020-10-31T11:59:59.000Z"}, orderBy: balance_DESC) {
      balance
      date
      id
    }
  }
}

```

Note that the [newer](/en/sdk/squid-sdk/reference/openreader-server/overview#supported-queries) and [more advanced](/en/sdk/squid-sdk/reference/openreader-server/api/paginate-query-results) `{entityName}sConnection` queries support exactly the same format of the `where` argument as the older `{entityName}s` queries used in the examples provided here.


# OpenReader Pagination
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/openreader-server/api/paginate-query-results

Paginate large GraphQL query results in OpenReader — first/after, last/before cursor pagination, and offset-based syntax for Squid SDK APIs.

# Paginate query results

There are multiple ways to obtain this behavior, let's take a look at a couple of them.

## Cursor based pagination

Cursors are used to traverse across entities of an entity set. They work by returning a pointer ("cursor") to a specific entity which can then be used to fetch the next batch. The batch will start with the entity after the one the cursor points to. For cursor-based pagination, OpenReader follows the [Relay Cursor Connections spec](https://relay.dev/graphql/connections.htm).

Currently, only forward pagination is supported. If your use case requires bidirectional pagination please let us know at our [Telegram channel](https://t.me/HydraDevs).

In SQD GraphQL server, cursor based pagination is implemented with `{entityName}sConnection` queries available for every entity in the input schema. These queries require an explicitly supplied [`orderBy` argument](/en/sdk/squid-sdk/reference/openreader-server/api/sorting), and *the field that is used for ordering must also be requested by the query itself*. Check out [this section](/en/sdk/squid-sdk/reference/openreader-server/api/paginate-query-results#important-note-on-orderby) for a valid query template.

Example: this query fetches a list of videos where `isExplicit` is true and gets their count.

```graphql theme={"system"}
query {
  videosConnection(orderBy: id_ASC, where: { isExplicit_eq: true }) {
    totalCount
    edges {
      node {
        id
        title
      }
    }
  }
}
```

### **Operator `first`**

The `first` operator is used to fetch a specified number of entities from the beginning of the output.

Example: Fetch the first 5 videos:

```graphql theme={"system"}
query Query1 {
  videosConnection(orderBy: id_ASC, first: 5) {
    edges {
      node {
        id
        title
      }
    }
  }
}
```

### **PageInfo object**

`PageInfo` is a "virtual" entity that can be requested from any `{entityName}sConnection` query (see below). It returns the relevant cursors and some page information:

```graphql theme={"system"}
pageInfo {
  startCursor
  endCursor
  hasNextPage
  hasPreviousPage
}
```

### **Operator `after`**

Example: Fetch the first 10 channels, ordered by `createdAt`. Then, in a second query, fetch the next 10 channels:

```graphql theme={"system"}
query FirstBatchQ {
  channelsConnection(first: 10, orderBy: createdAt_ASC) {
    pageInfo {
      endCursor
      hasNextPage
    }
    edges {
      node {
        id
        handle
        createdAt
      }
    }
  }
}

query SecondBatchQ {
  channelsConnection(after: <endCursor>, orderBy: createdAt_ASC) {
    pageInfo {
      endCursor
      hasNextPage
    }
    edges {
      node {
        id
        handle
        createdAt
      }
    }
  }
}
```

### **Important Note on `orderBy`**

The field chosen to `orderBy` needs to be present in the query itself. For example, any `after` query must follow this template:

```graphql theme={"system"}
query QueryName {
  <entityName>sConnection(after: <endCursor>, orderBy: <fieldNameToOrderBy>_ASC) {
    pageInfo {
      endCursor
      hasNextPage
      ...<any other page info fields>...
    }
    edges {
      node {
        <fieldNameToOrderBy>
        ...<any other fields of interest>...
      }
    }
  }
}
```

Otherwise, the returned result wouldn't be ordered correctly.

### Examples

An interactive example of using cursor-based pagination can be found in [this repo](https://github.com/subsquid-labs/cursor-pagination-client-example).

## Paginating with `{entityName}s` queries

### Arguments `limit` and `offset`

In a list of entities returned by a query, the `limit` argument specifies how many should be retained, while the `offset` argument specifies how many should be skipped first. Default values are `50` for `limit` and `0` for `offset`.

### **Limit results**

Example: Fetch the first 5 channels:

```graphql theme={"system"}
query {
  channels(limit: 5) {
    id
    handle
  }
}
```

### **Limit results from an offset**

Example: Fetch 5 channels from the list of all channels, starting with the 6th one:

```graphql theme={"system"}
query {
  channels(limit: 5, offset: 5) {
    id
    handle
  }
}
```


# Entity queries
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/openreader-server/api/queries

Run basic GraphQL entity queries against a Squid SDK GraphQL API — single-entity, list, and connection-style queries with filtering and pagination.

# Entity Queries

## Introduction

OpenReader auto-generates queries from the `schema.graphql` file. All entities defined in the schema can be queried over the GraphQL endpoint.

## Exploring queries

Let’s take a look at the different queries you can run using the GraphQL server. We’ll use examples based on a typical channel/video schema.

### Simple entity queries

#### **Fetch list of entities**

Fetch a list of channels:

```graphql theme={"system"}
query {
  channels {
    id
    handle
  }
}
```

or, using a [newer](/en/sdk/squid-sdk/reference/openreader-server/overview#supported-queries) and [more advanced](/en/sdk/squid-sdk/reference/openreader-server/api/paginate-query-results) `{entityName}sConnection` query

```graphql theme={"system"}
query {
  channelsConnection(orderBy: id_ASC) {
    edges {
      node {
        id
        handle
      }
    }
  }
}
```

#### **Fetch an entity using its unique fields**

Fetch a channel by a unique id or handle:

```graphql theme={"system"}
query Query1 {
  channelByUniqueInput(where: { id: "1" }) {
    id
    handle
  }
}

query Query2 {
  channelByUniqueInput(where: { handle: "Joy Channel" }) {
    id
    handle
  }
}
```

### Filter query results / search queries

#### **The `where` argument**

You can use the `where` argument in your queries to filter results based on some field’s values. You can even use multiple filters in the same `where` clause using the `AND` or the `OR` operators.

For example, to fetch data for a channel named `Joy Channel`:

```graphql theme={"system"}
query {
  channels(where: { handle_eq: "Joy Channel" }) {
    id
    handle
  }
}
```

Note that `{entityName}sConnection` queries support exactly the same format of the `where` argument:

```graphql theme={"system"}
query {
  channelsConnection(orderBy: id_ASC, where: { handle_eq: "Joy Channel"}) {
    edges {
      node {
        id
        handle
      }
    }
  }
}
```

#### **Supported Scalar Types**

SQD supports the following scalar types:

* String
* Int
* Float
* BigInt
* Boolean
* Bytes
* DateTime

#### **Equality Operators (`_eq`)**

`_eq` is supported by all the scalar types.

The following are examples of using this operator on different types:

* Fetch a list of videos where `title` is "Bitcoin"
* Fetch a list of videos where `isExplicit` is "true"
* Fetch a list of videos `publishedOn` is "2021-01-05"

```graphql theme={"system"}
query Query1 {
  videos(where: { title_eq: "Bitcoin" }) {
    id
    title
  }
}

query Query2 {
  videos(where: { isExplicit_eq: true }) {
    id
    title
  }
}

query Query3 {
  videos(where: { publishedOn_eq: "2021-01-05" }) {
    id
    title
  }
}
```

#### **Greater than or less than operators (`gt`, `lt`, `gte`, `lte`)**

The `_gt` (greater than), `_lt` (less than), `_gte` (greater than or equal to), `_lte` (less than or equal to) operators are available on `Int, BigInt, Float, DateTime` types.

The following are examples of using these operators on different types:

* Fetch a list of videos published before "2021-01-05"
* Fetch a list of channels before block "999"

```graphql theme={"system"}
query Query1 {
  videos(where: { publishedOn_gte: "2021-01-05" }) {
    id
    title
  }
}

query Query2 {
  channels(where: { block_lte: "999" }) {
    id
    handle
  }
}
```

#### **Text search or pattern matching operators (`_contains`, `_startsWith`, `_endsWith`)**

The `_contains`, `_startsWith`, `_endsWith` operators are used for pattern matching on string fields.

Example:

```graphql theme={"system"}
query Query1 {
  videos(where: { title_contains: "Bitcoin" }) {
    id
    title
  }
}

query Query2 {
  videos(where: { title_endsWith: "cryptocurrency" }) {
    id
    title
  }
}
```


# Union type resolution
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/openreader-server/api/resolve-union-types-interfaces

Resolve GraphQL union types and interfaces in Squid SDK OpenReader — use the __typename meta field to discriminate between concrete entity types.

Use cases for [Union types](/en/sdk/squid-sdk/reference/schema-file/unions-and-typed-json) have been discussed in the [schema reference](/en/sdk/squid-sdk/reference/schema-file). Here, we discuss how to query union types.

Let's take this modified schema from the [Substrate tutorial](/en/sdk/squid-sdk/tutorials/substrate):

```graphql title="schema.graphql" theme={"system"}
type Account @entity {
  id: ID! #Account address
  events: [Event]
}

type WorkReport {
  id: ID! #event id
  addedFiles: [[String]]
  deletedFiles: [[String]]
  extrinsicId: String
  blockHash: String!
}

type JoinGroup {
  id: ID!
  owner: String!
  extrinsicId: String
  blockHash: String!
}

type StorageOrder {
  id: ID!
  fileCid: String!
  extrinsicId: String
  blockHash: String!
}

union Event = WorkReport | JoinGroup | StorageOrder
```

Here, an `Event` will have different fields depending on the underlying type. This query demonstrates how to request different fields for each of these types:

```graphql theme={"system"}
query MyQuery {
  accounts {
    events {
      __typename
      ... on WorkReport {
        id
        blockHash
        extrinsicId
        deletedFiles
      }
      ... on JoinGroup {
        id
        blockHash
        extrinsicId
      }
      ... on StorageOrder {
        id
        blockHash
        extrinsicId
      }
    }
    id
  }
}
```

The special `__typename` field allows users to discern the returned object type without relying on comparing the sets of regular fields. For example, in the output of the query above `JoinGroup` and `StorageOrder` events can only be distingushed by looking at the `__typename` field. Here is a possible output to illustrate:

```json theme={"system"}
{
  "data": {
    "accounts": [
      {
        "events": [
          {
            "__typename": "WorkReport",
            "id": "0000584321-000001-01cdb",
            "blockHash": "0x01cdb3cb6fa00f62fd20220104f1d740a53518b63517419da8a89325d065562b",
            "extrinsicId": "0000584321-000001-01cdb",
            "deletedFiles": []
          }
        ],
        "id": "cTKmzHG3RHa1yhujyZpPnNL17p8a48Av3JFwDjpttLcxeSo26"
      },
      {
        "events": [
          {
            "__typename": "JoinGroup",
            "id": "0000584598-000010-d06ec",
            "blockHash": "0xd06ec6716e96108e24987ef03d23c857ef3b467dd057d7a32c4e123fe5a8df36",
            "extrinsicId": "0000584598-000004-d06ec"
          }
        ],
        "id": "cTKqevWRdvbNNAQ3hLxhsNYhQ8pf5YGkYnnVjgjLNiVr4kd7a"
      },
      {
        "events": [
          {
            "__typename": "StorageOrder",
            "id": "0000584627-000013-1fa19",
            "blockHash": "0x1fa19ae98731afad853ffd491fcbc0c3dcda6b8b7f5a2d56ac6c4c1eb9e4f95e",
            "extrinsicId": "0000584627-000005-1fa19"
          }
        ],
        "id": "cTGYF8jvcpnRmgNopqT4nVs5rWHEviAAdRdfNrZE8NFz2Av7B"
      }
    ]
  }
}
```


# Sorting
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/openreader-server/api/sorting

Sort GraphQL query results in OpenReader with the orderBy argument — sort by entity fields, nested fields, and apply multiple sort keys at once.

## Sort order

The sort order (ascending vs. descending) is set by specifying an `ASC` or `DESC` suffix for the column name in the `orderBy` input object, e.g. `title_DESC`.

### **Sorting entities**

Example: Fetch a list of videos sorted by their titles in an ascending order:

```graphql theme={"system"}
query {
  videos(orderBy: title_ASC) {
    id
    title
  }
}
```

or

```graphql theme={"system"}
query {
  videos(orderBy: [title_ASC]) {
    id
    title
  }
}
```

### **Sorting entities by multiple fields**

The `orderBy` argument takes an array of fields to allow sorting by multiple columns.

Example: Fetch a list of videos that is sorted by their titles (ascending) and then on their published date (descending):

```graphql theme={"system"}
query {
  videos(orderBy: [title_ASC, publishedOn_DESC]) {
    id
    title
    publishedOn
  }
}
```


# Access control
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/openreader-server/configuration/authorization

Configure authentication and authorization in OpenReader — protect Squid SDK GraphQL APIs with API keys, signed JWTs, and per-entity rules.

To implement access control, define the following function in the designated `src/server-extension/check` module:

```typescript theme={"system"}
import {
  RequestCheckContext
} from '../../node_modules/@subsquid/graphql-server/src/check'

export async function requestCheck(
  req: RequestCheckContext
): Promise<boolean | string> {
  ...
}
```

Once defined, this function will be called every time a request arrives. Then,

* if the function returns `true`, the request is processed as usual;
* if the function returns `false`, the server responds with `'{"errors":[{"message":"not allowed"}]}'`;
* if the function returns an `errorString`, the server responds with `` `{{"errors":[{"message":"${errorString}"}]}` ``.

The request information such as HTTP headers and GraphQL selections is available in the context. This makes it possible to authenticate the user that sent the query and either allow or deny access. The decision may take the query contents into account, allowing for some authorization granularity.

## `RequestCheckContext`

The context type has the following interface:

```typescript theme={"system"}
RequestCheckContext {
  http: {uri: string, method: string, headers: HttpHeaders}
  operation: OperationDefinitionNode
  operationName: string | null
  schema: GraphQLSchema
  context: Record<string, any>
  model: Model
}
```

Here,

* `http` field contains the low level HTTP info. Information on headers is stored in a `Map` from lowercase header names to values. For example, `req.http.headers.get('authorization')` is the value of the authorization header.
* `operation` is the root [`OperationDefinitionNode`](https://graphql-js.org/api/interface/OperationDefinitionNode) of the tree describing the query. Useful if the authorization decision depends on the query contents.
* `operationName` is the query name.
* `schema` is a [`GraphQLSchema`](https://graphql-js.org/api/class/GraphQLSchema) object.
* `context` holds a [`PoolOpenreaderContext`](https://github.com/subsquid/squid-sdk/blob/master/graphql/openreader/src/db.ts) at `context.openreader`. It can be used to access the database, though this is highly discouraged: the interfaces involved are considered to be internal and are subject to change without notice.
* `model` is an Openreader data [`Model`](https://github.com/subsquid/squid-sdk/blob/master/graphql/openreader/src/model.ts).

## Sending user data to resolvers

Authentication data such as user name can be passed from `requestCheck()` to a [custom resolver](/en/sdk/squid-sdk/reference/openreader-server/configuration/custom-resolvers) through Openreader context:

```typescript theme={"system"}
export async function requestCheck(req: RequestCheckContext): Promise<boolean | string> {
  ...
  // obtain user name e.g. by decoding the authentication header
  let user = ...
  // save user name to Openreader context
  req.context.openreader.user = user
  ...
}
```

A custom resolver that retrieves it may look like this:

```typescript theme={"system"}
@Resolver()
export class UserCommentResolver {
  constructor(private tx: () => Promise<EntityManager>) {}

  @Query(() => [UserCommentCountQueryResult])
  async countUserComments(
    @Ctx() ctx: any
  ): Promise<UserCommentCountQueryResult[]> {
    let user = ctx.openreader.user
    let manager = await this.tx()
    let result: UserCommentCountQueryResult[] =
      await manager
        .getRepository(UserComment)
        .query(`
          SELECT COUNT(*) as total
          FROM user_comment
          WHERE "user" = '${user}'
        `)
    return result
  }

  @Mutation(() => Boolean)
  async addComment(
    @Arg('text') comment: string,
    @Ctx() ctx: any
  ): Promise<Boolean> {
    let user = ctx.openreader.user
    let manager = await this.tx()
    await manager.save(new UserComment({
      id: `${user}-${comment}`,
      user,
      comment
    }))
    return true
  }
}
```

See full code in [this branch](https://github.com/subsquid-labs/access-control-example/tree/interacting-with-resolver).

This approach does not work with [subscriptions](/en/sdk/squid-sdk/reference/openreader-server/configuration/subscriptions).

## Examples

A simple strategy that authorizes anyone with a `12345` token to perform any query can be implemented with

```typescript title="src/server-extension/check.ts" theme={"system"}
import {
  RequestCheckContext
} from '../../node_modules/@subsquid/graphql-server/src/check'

export async function requestCheck(
  req: RequestCheckContext
): Promise<boolean | string> {
  return req.http.headers.get('authorization')==='Bearer 12345'
}
```

A more elaborate example with two users authorized to perform different query sets is available in [this repo](https://github.com/subsquid-labs/access-control-example). Another great example of using `requestCheck()` for authorization can be [spotted in the wild](https://github.com/reef-defi/reef-subsquid-processor/tree/master/src/server-extension) in the code of a squid used by [Reef](https://reef.io).


# Caching
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/openreader-server/configuration/caching

Enable response caching in OpenReader for faster Squid SDK GraphQL queries — TTL-based cache with cache-busting on schema or data updates.

The GraphQL API server provided by `@subsquid/graphql-server` supports caching via additional flags. It is done on a per-query basis. The whole response is cached for a specified amount of time (`maxAge`).

To enable caching when deploying to SQD Cloud, add the caching flags to the `serve:prod` command definition at [`commands.json`](/en/sdk/squid-sdk/squid-cli/commands-json), then use that command to run the server in the [deployment manifest](/en/cloud/reference/manifest#deploy). Cloud currently supports only in-memory cache.
For example, snippets below will deploy a GraphQL API server with a `100Mb` in-memory cache and invalidation time of `5` seconds:

```json title="commands.json" theme={"system"}
...
  "serve:prod": {
    "description": "Start the GraphQL API server with caching and limits",
    "cmd": [ "squid-graphql-server",
      "--dumb-cache", "in-memory",
      "--dumb-cache-ttl", "5000",
      "--dumb-cache-size", "100",
      "--dumb-cache-max-age", "5000" ]
  }
...
```

```yaml title="squid.yaml" theme={"system"}
# ...
deploy:
  # other services ...
  api:
    cmd: [ "sqd", "serve:prod" ]
```

Caching flags list is available via `npx squid-graphql-server --help`. Here are some more details on them:

### `--dumb-cache <cache-type>`

Enables cache, either `in-memory` or `redis`. For `redis`, a Redis connection string must be set by a variable `REDIS_URL`. SQD Cloud deployments currently support only `in-memory` cache.

### `--dumb-cache-size <mb>`

Cache max size. Applies only to in-memory cache.

### `--dumb-cache-max-age <ms>`

A globally set max age in milliseconds. The cached queries are invalidated after that period of time.

### `--dumb-cache-ttl <ms>`

Time-to-live for in-memory cache entries. Applies only to in-memory cache. The entries are eligible for eviction from the cache if not updated for longer than the time-to-live time.


# Custom API extensions
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/openreader-server/configuration/custom-resolvers

Extend a Squid SDK GraphQL API with custom resolvers — add aggregations, computed fields, and ad-hoc queries beyond the schema-derived entities.

# Custom GraphQL resolvers

One can extend the GraphQL API generated by OpenReader with custom queries. To do that, one can define GraphQL [query resolvers](https://www.apollographql.com/docs/apollo-server/data/resolvers/) in the designated module `src/server-extension/resolvers`. **Note that all resolver classes (including any additional types) must be exported by `src/server-extension/resolvers/index.ts`.**

A custom resolver should import [TypeGraphQL](https://typegraphql.com) types and [use annotations](https://typegraphql.com/docs/resolvers.html) provided by the library to define query arguments and return types. If your squid lacks a `type-graphql` dependency, add it with:

```bash theme={"system"}
npm i type-graphql
```

Custom resolvers are normally used in combination with [TypeORM EntityManager](https://typeorm.io/entity-manager-api) for accessing the API server target database. It is automatically injected when defined as a single constructor argument of the resolver.

## Examples

#### Simple entity counter

```typescript theme={"system"}
import { Query, Resolver } from 'type-graphql'
import type { EntityManager } from 'typeorm'
import { Burn } from '../model'

@Resolver()
export class CountResolver {
  constructor(private tx: () => Promise<EntityManager>) {}

  @Query(() => Number)
  async totalBurns(): Promise<number> {
    const manager = await this.tx()
    return await manager.getRepository(Burn).count()
  }
}
```

This example is designed to work with the `evm` template:

1. grab a test squid as described [here](/en/sdk/squid-sdk/how-to-start/squid-development);
2. install `type-graphql`;
3. save the example code to `src/server-extension/resolver.ts`;
4. re-export `CountResolver` at `src/server-extension/resolvers/index.ts`:
   ```ts theme={"system"}
   export { CountResolver } from '../resolver'
   ```
5. rebuild the squid with `npm run build`;
6. (re)start the GraphQL server with `npx squid-graphql-server`.

`totalBurns` selection will appear in the [GraphiQL playground](http://localhost:4350/graphql).

#### Custom SQL query

```typescript theme={"system"}
import { Arg, Field, ObjectType, Query, Resolver } from 'type-graphql'
import type { EntityManager } from 'typeorm'
import { MyEntity } from '../model'

// Define custom GraphQL ObjectType of the query result
@ObjectType()
export class MyQueryResult {
  @Field(() => Number, { nullable: false })
  total!: number

  @Field(() => Number, { nullable: false })
  max!: number

  constructor(props: Partial<MyQueryResult>) {
    Object.assign(this, props);
  }
}

@Resolver()
export class MyResolver {
  // Set by depenency injection
  constructor(private tx: () => Promise<EntityManager>) {}

  @Query(() => [MyQueryResult])
  async myQuery(): Promise<MyQueryResult[]> {
    const manager = await this.tx()
    // execute custom SQL query
    const result: = await manager.getRepository(MyEntity).query(
      `SELECT 
        COUNT(x) as total, 
        MAX(y) as max
      FROM my_entity 
      GROUP BY month`)
    return result
  }
}
```

#### More examples

Some great examples of `@subsquid/graphql-server`-based custom resolvers can be spotted in the wild in the [Rubick repo](https://github.com/kodadot/rubick/tree/main/src/server-extension/resolvers) by [KodaDot](https://github.com/kodadot).

For more examples of resolvers, see [TypeGraphQL examples repo](https://github.com/MichalLytek/type-graphql/tree/master/examples).

## Logging

To keep logging consistent across the entire GraphQL server, use `@subsquid/logger`:

```ts theme={"system"}
import {createLogger} from '@subsquid/logger'

// using a custom namespace ':my-resolver' for resolver logs
const LOG = createLogger('sqd:graphql-server:my-resolver')
LOG.info('created a dedicated logger for my-resolver')
```

`LOG` here is a [logger object](/en/sdk/squid-sdk/reference/logger) identical to `ctx.log` interface-wise.

## Interaction with global settings

* `--max-response-size` used for [DoS protection](/en/sdk/squid-sdk/reference/openreader-server/configuration/dos-protection) is ignored in custom resolvers.
* [Caching](/en/sdk/squid-sdk/reference/openreader-server/configuration/caching) works on custom queries in exactly the same way as it does on the schema-derived queries.

## Troubleshooting

#### `Reflect.getMetadata is not a function`

Add `import 'reflect-metadata'` on top of your custom resolver module and install the package if necessary.


# OpenReader Server Overview
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/openreader-server/overview

OpenReader is the open source GraphQL server bundled with Squid SDK — generates an OpenCRUD-style API from your schema.graphql with caching and DoS protection.

<Info>
  OpenReader is no longer recommended for use in new squid projects [relying on PostgreSQL](/en/sdk/squid-sdk/resources/persisting-data/typeorm). See [Serving GraphQL](/en/sdk/squid-sdk/resources/serving-graphql) to learn about the new options and the [Limitations](#limitations) section to understand our motivation.
</Info>

OpenReader is a server that presents data of PostgreSQL-powered squids as a GraphQL API. It relies on the [eponymous library](https://github.com/subsquid/squid-sdk/tree/master/graphql/openreader) lib of the Squid SDK for schema generation. [Schema file](/en/sdk/squid-sdk/reference/schema-file) is used as an input; the resulting API supports [OpenCRUD](https://www.opencrud.org/) queries for the entities defined in the schema.

To start the API server based on `schema.graphql` install `@subsquid/graphql-server` and run the following in the squid project root:

```bash theme={"system"}
npx squid-graphql-server
```

The `squid-graphql-server` executable supports multiple optional flags to enable [caching](/en/sdk/squid-sdk/reference/openreader-server/configuration/caching), [subscriptions](/en/sdk/squid-sdk/reference/openreader-server/configuration/subscriptions), [DoS protection](/en/sdk/squid-sdk/reference/openreader-server/configuration/dos-protection) etc. Its features are covered in the next sections.

The API server listens at port defined by the `GQL_PORT` environment variable (defaults to `4350`). The database connection is configured with the env variables `DB_NAME`, `DB_USER`, `DB_PASS`, `DB_HOST`, `DB_PORT`.

In [SQD Cloud](/en/cloud), OpenReader is usually ran as the `api:` service in the `deploy:` section of the [Deployment manifest](/en/cloud/reference/manifest).

## Supported queries

The details of the supported OpenReader queries can be found in a separate section [Core API](/en/sdk/squid-sdk/reference/openreader-server/api). Here is a brief overview of the queries generated by OpenReader for each entity defined in the schema file:

* the squid last processed block is available with `squidStatus { height }` query
* a "get one by ID" query with the name `{entityName}ById` for each [entity](/en/sdk/squid-sdk/reference/schema-file/entities) defined in the schema file
* a "get one" query for [`@unique` fields](/en/sdk/squid-sdk/reference/schema-file/indexes-and-constraints), with the name `{entityName}ByUniqueInput`
* Entity queries named `{entityName}sConnection`. Each query supports rich filtering support, including [field-level filters](/en/sdk/squid-sdk/reference/openreader-server/api/queries), composite [`AND` and `OR` filters](/en/sdk/squid-sdk/reference/openreader-server/api/and-or-filters), [nested queries](/en/sdk/squid-sdk/reference/openreader-server/api/nested-field-queries), [cross-relation queries](/en/sdk/squid-sdk/reference/openreader-server/api/cross-relation-field-queries) and [Relay-compatible](https://relay.dev/graphql/connections.htm) cursor-based [pagination](/en/sdk/squid-sdk/reference/openreader-server/api/paginate-query-results).
* [Subsriptions](/en/sdk/squid-sdk/reference/openreader-server/configuration/subscriptions) via live queries
* (Deprecated in favor of Relay connections) Lookup queries with the name `{entityName}s`.

[Union and typed JSON types](/en/sdk/squid-sdk/reference/schema-file/unions-and-typed-json) are mapped into [GraphQL Union Types](https://graphql.org/learn/schema/#union-types) with a [proper type resolution](/en/sdk/squid-sdk/reference/openreader-server/api/resolve-union-types-interfaces) with `__typename`.

## Built-in custom scalar types

The OpenReader GraphQL API defines the following custom scalar types:

* `DateTime` entity field values are presented in the ISO format
* `Bytes` entity field values are presented as hex-encoded strings prefixed with `0x`
* `BigInt` entity field values are presented as strings

## Limitations

* RAM usage of [subscriptions](/en/sdk/squid-sdk/reference/openreader-server/configuration/subscriptions) scales poorly under high load, making the feature unsuitable for most production uses. There are currently no plans to fix this issue.
* Setting up custom resolvers for subscriptions is unreasonably hard.
* `@subsquid/graphql-server` depends on the deprecated Apollo Server v3.


# Processor architecture
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/processors/architecture

Anatomy of Squid SDK processors — data source, query builder, batch handler, and Store roles in the processor lifecycle and the SQD Network protocol.

The processor service is a Node.js process responsible for data ingestion, transformation and data persisting into the target database. By [convention](/en/sdk/squid-sdk/how-to-start/layout), the processor entry point is at `src/main.ts`. It is run as

```bash theme={"system"}
node lib/main.js
```

For local runs, one normally additionally exports environment variables from `.env` using `dotenv`:

```bash theme={"system"}
node -r dotenv/config lib/main.js
```

## Processor choice

The Squid SDK currently offers specialized processor classes for EVM (`EvmBatchProcessor`) and Substrate networks (`SubstrateBatchProcessor`). More networks will be supported in the future. By convention, the processor object is defined at `src/processor.ts`.

Navigate to a dedicated section for each processor class:

* [`EvmBatchProcessor`](/en/sdk/squid-sdk/reference/processors/evm-batch)
* [`SubstrateBatchProcessor`](/en/sdk/squid-sdk/reference/processors/substrate-batch)

## Configuration

A processor instance should be configured to define the block range to be indexed, and the selectors of data to be fetched from [SQD Network](/en/network) and/or a node RPC endpoint.

## `processor.run()`

The actual data processing is done by the `run()` method called on a processor instance (typically at `src/main.ts`). The method has the following signature:

```ts theme={"system"}
run<Store>(
  db: Database<Store>,
  batchHander: (
    context: DataHandlerContext<Store, F extends FieldSelection>
  ) => Promise<void>
): void
```

The `db` parameter defines the target [data sink](/en/sdk/squid-sdk/resources/persisting-data), and `batchHandler` is an `async` `void` function defining the data transformation and persistence logic. It repeatedly receives batches of SQD Network data stored in `context.blocks`, transforms them and persists the results to the target database using the `context.store` interface (more on `context` in the next section).

To jump straight to examples, see [EVM Processor in action](/en/sdk/squid-sdk/tutorials/batch-processor-in-action) and [Substrate Processor in action](/en/sdk/squid-sdk/tutorials/batch-processor-in-action).

## Batch context

Batch handler takes a single argument of `DataHandlerContext` type:

```ts theme={"system"}
export interface DataHandlerContext<Store, F extends FieldSelection> {
    _chain: Chain
    log: Logger
    store: Store
    blocks: BlockData<F>[]
    isHead: boolean
}
```

Here, `F` is the type of the argument of the `setFields()` ([EVM](/en/sdk/squid-sdk/reference/processors/evm-batch/field-selection), [Substrate](/en/sdk/squid-sdk/reference/processors/substrate-batch/field-selection)) processor configuration method. `Store` type is inferred from the `Database` instance passed into the `run()` method.

#### `ctx._chain`

Internal handle for direct access to the underlying chain state via RPC calls. Rarely used directly, but rather by the facade access classes generated by the [typegen tools](/en/sdk/squid-sdk/glossary#typegen).

#### `ctx.log`

The native logger handle. See [Logging](/en/sdk/squid-sdk/reference/logger).

#### `ctx.store`

Interface for the target data sink. See [Persisting data](/en/sdk/squid-sdk/resources/persisting-data).

#### `ctx.blocks`

On-chain data items are grouped into blocks, with each block containing a header and iterables for all supported data item types. Boundary blocks are always included into the `ctx.blocks` iterable with valid headers, even when they do not contain any requested data. It follows that batch context *always* contains at least one block.

The set of iterables depends on the processor type (docs for [EVM](/en/sdk/squid-sdk/reference/processors/evm-batch/context-interfaces)/[Substrate](/en/sdk/squid-sdk/reference/processors/substrate-batch/context-interfaces)). Depending on the data item type, items within the iterables can be canonically ordered by how the data is recorded on-chain (e.g. transactions are ordered but traces are not). The shape of item objects is determined by the processor configuration done via the `.setFields()` method.

An idiomatic use of the context API is to iterate first over blocks and then over each iterable of each block:

<Tabs>
  <Tab title="EVM">
    ```ts theme={"system"}
    processor.run(new TypeormDatabase(), async (ctx) => {
      for (let block of ctx.blocks) {
        for (let log of block.logs) {
          // filter and process logs
        }
        for (let txn of block.transactions) {
          // filter and process transactions
        }
        for (let stDiff of block.stateDiffs) {
          // filter and process state diffs
        }
        for (let traces of block.traces) {
          // filter and process execution traces
        }
      }
    })
    ```
  </Tab>

  <Tab title="Substrate">
    ```ts theme={"system"}
    processor.run(new TypeormDatabase(), async (ctx) => {
      for (let block of ctx.blocks) {
        for (let event of block.events) {
          // filter and process events
        }
        for (let call of block.calls) {
          // filter and process calls
        }
        for (let extrinsic of block.extrinsics) {
          // filter and process extrinsics
        }
      }
    })
    ```
  </Tab>
</Tabs>

The canonical ordering of `ctx.blocks` enables efficient in-memory data processing. For example, multiple updates of the same entity can be compressed into a single database transaction.

Please be aware that the processor cannot ensure that data not meeting its filters will be excluded from iterables. It only guarantees the inclusion of data that matches the filters. Therefore, it is necessary to filter the data in the batch handler prior to processing.

#### `ctx.isHead`

Is `true` if the processor has reached the chain head. The last block `ctx.blocks` is then the current chain tip.


# Block data for EVM
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/processors/evm-batch/context-interfaces

Block data interfaces for EVM indexing with Squid SDK — type definitions for blocks, transactions, logs, traces, and state diffs in the processor context.

`EvmBatchProcessor` follows the common [squid processor architecture](/en/sdk/squid-sdk/overview), in which data processing happens within the [batch handler](/en/sdk/squid-sdk/reference/processors/architecture#processorrun), a function repeatedly called on batches of on-chain data. The function takes a single argument called "batch context". Its structure follows the [common batch context layout](/en/sdk/squid-sdk/reference/processors/architecture#batch-context), with `ctx.blocks` being an array of `BlockData` objects containing the data to be processed, aligned at the block level.

For `EvmBatchProcessor` the `BlockData` interface is defined as follows:

```ts theme={"system"}
export type BlockData<F extends FieldSelection = {}> = {
  header: BlockHeader<F>
  transactions: Transaction<F>[]
  logs: Log<F>[]
  traces: Trace<F>[]
  stateDiffs: StateDiff<F>[]
}
```

`F` here is the type of the argument of the [`setFields()`](/en/sdk/squid-sdk/reference/processors/evm-batch/field-selection) processor method.

`BlockData.header` contains the block header data. The rest of the fields are iterables containing four kinds of blockchain data. The canonical ordering within each iterable depends on the data kind:

* `transactions` and `logs` are ordered in the same way as they are within blocks;
* [`stateDiffs`](/en/sdk/squid-sdk/reference/processors/evm-batch/state-diffs) follow the order of transactions that gave rise to them;
* `traces` are ordered in a deterministic but otherwise unspecified way.

The exact fields available in each data item type are inferred from the `setFields()` call argument. They are documented on the [field selection](/en/sdk/squid-sdk/reference/processors/evm-batch/field-selection) page:

* [transactions section](/en/sdk/squid-sdk/reference/processors/evm-batch/field-selection#transactions);
* [logs section](/en/sdk/squid-sdk/reference/processors/evm-batch/field-selection#logs);
* [traces section](/en/sdk/squid-sdk/reference/processors/evm-batch/field-selection#traces);
* [state diffs section](/en/sdk/squid-sdk/reference/processors/evm-batch/field-selection#state-diffs);
* [block header section](/en/sdk/squid-sdk/reference/processors/evm-batch/field-selection#block-headers).

## Example

The handler below simply outputs all the log items emitted by the contract `0x2E645469f354BB4F5c8a05B3b30A929361cf77eC` in [real time](/en/sdk/squid-sdk/resources/unfinalized-blocks):

```ts theme={"system"}
import { TypeormDatabase } from '@subsquid/typeorm-store'
import { EvmBatchProcessor } from '@subsquid/evm-processor'

const CONTRACT_ADDRESS = '0x2E645469f354BB4F5c8a05B3b30A929361cf77eC'.toLowerCase()

const processor = new EvmBatchProcessor()
  .setGateway('https://v2.archive.subsquid.io/network/ethereum-mainnet')
  .setRpcEndpoint('<my_eth_rpc_url>')
  .setFinalityConfirmation(75)
  .setBlockRange({ from: 17000000 })
  .addLog({
    address: [CONTRACT_ADDRESS]
  })
  .setFields({ // could be omitted: this call does not change the defaults
    log: {
      topics: true,
      data: true
    }
  })

processor.run(new TypeormDatabase(), async (ctx) => {
  for (let c of ctx.blocks) {
    for (let log of c.logs) {
      if (log.address === CONTRACT_ADDRESS) {
        ctx.log.info(log, `Log:`)
      }
    }
  }
})
```

One can experiment with the [`setFields()`](/en/sdk/squid-sdk/reference/processors/evm-batch/field-selection) argument and see how the output changes.

For more elaborate examples, check [EVM Examples](/en/sdk/squid-sdk/examples).


# EVM Field Selection
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/processors/evm-batch/field-selection

Fine-tune EVM data requests with setFields() in Squid SDK — choose only the block, transaction, log, trace, and state diff fields you need to index.

#### `setFields(options)`

Set the fields to be retrieved for data items of each supported type. The `options` object has the following structure:

```ts theme={"system"}
{
  log?:         // field selector for logs
  transaction?: // field selector for transactions
  stateDiff?:   // field selector for state diffs
  trace?:       // field selector for traces
  block?:       // field selector for block headers
}
```

Every field selector is a collection of boolean fields, typically (with a notable exception of [trace field selectors](#traces)) mapping one-to-one to the fields of data items within the batch context [iterables](/en/sdk/squid-sdk/reference/processors/evm-batch/context-interfaces). Defining a field of a field selector of a given type and setting it to true will cause the processor to populate the corresponding field of all data items of that type. Here is a definition of a processor that requests `gas` and `value` fields for transactions:

```ts theme={"system"}
let processor = new EvmBatchProcessor()
  .setFields({
    transaction: {
      gas: true,
      value: true
    }
  })
```

Same fields will be available for all data items of any given type, including nested items. Suppose we used the processor defined above to subscribe to some transactions as well as some logs, and for each log we requested a parent transaction:

```ts theme={"system"}
processor
  .addLog({
    // some log data requests
    transaction: true
  })
  .addTransaction({
    // some transaction data requests
  })
```

As a result, `gas` and `value` fields would be available both within the transaction items of the `transactions` iterable of [block data](/en/sdk/squid-sdk/reference/processors/evm-batch/context-interfaces) and within the transaction items that provide parent transaction information for the logs:

```ts theme={"system"}
processor.run(db, async ctx => {
  for (let block of ctx.blocks) {
    for (let txn of block.transactions) {
      let txnGas = txn.gas // OK
    }
    for (let log of block.logs) {
      let parentTxnGas = log.transaction.gas // also OK!
    }
  }
})
```

Some data fields, like `hash` for transactions, are enabled by default but can be disabled by setting a field of a field selector to `false`. For example, this code will not compile:

```ts theme={"system"}
let processor = new EvmBatchProcessor()
  .setFields({
    transaction: {
      hash: false
    }
  })
  .addTransaction({
    // some transaction data requests
  })

processor.run(db, async ctx => {
  for (let block of ctx.blocks) {
    for (let txn of block.transactions) {
      let txnHash = txn.hash // ERROR: no such field
    }
  }
})
```

Disabling unused fields will improve sync performance, as the disabled fields will not be fetched from the SQD Network gateway.

## Data item types and field selectors

<Tip>
  Most IDEs support smart suggestions to show the possible field selectors. For VS Code, press `Ctrl+Space`.
</Tip>

Here we describe the data item types as functions of the field selectors. Unless otherwise mentioned, each data item type field maps to the eponymous field of its corresponding field selector. Item fields are divided into three categories:

* Fields that are added independently of the `setFields()` call. These are either fixed or depend on the related data retrieval flags (e.g. `transaction` for logs).
* Fields that can be disabled by `setFields()`. E.g. a `topics` field will be fetched for logs by default, but can be disabled by setting `topics: false` within the `log` field selector.
* Fields that can be requested by `setFields()`. E.g. a `transactionHash` field will only be available in logs if the `log` field selector sets `transactionHash: true`.

### Logs

`Log` data items may have the following fields:

```ts theme={"system"}
Log {
  // independent of field selectors
  id: string
  logIndex: number
  transactionIndex: number
  block: BlockHeader
  transaction?: Transaction

  // can be disabled with field selectors
  address: string
  data: string
  topics: string[]

  // can be requested with field selectors
  transactionHash: string
}
```

See the [block headers section](#block-headers) for the definition of `BlockHeader` and the [transactions section](#transactions) for the definition of `Transaction`.

### Transactions

`Transaction` data items may have the following fields:

```ts theme={"system"}
Transaction {
  // independent of field selectors
  id: string
  transactionIndex: number
  block: BlockHeader

  // can be disabled with field selectors
  from: string
  to?: string
  hash: string

  // can be requested with field selectors
  gas: bigint
  gasPrice: bigint
  maxFeePerGas?: bigint
  maxPriorityFeePerGas?: bigint
  input: string
  nonce: number
  value: bigint
  v?: bigint
  r?: string
  s?: string
  yParity?: number
  chainId?: number
  gasUsed?: bigint
  cumulativeGasUsed?: bigint
  effectiveGasPrice?: bigint
  contractAddress?: string
  type?: number
  status?: number
  sighash: string
  // limited availability (see below)
  l1Fee?: bigint
  l1FeeScalar?: number
  l1GasPrice?: bigint
  l1GasUsed?: bigint
  l1BlobBaseFee?: bigint
  l1BlobBaseFeeScalar?: number
  l1BaseFeeScalar?: number
}
```

`status` field contains the value returned by [`eth_getTransactionReceipt`](https://geth.ethereum.org/docs/interacting-with-geth/rpc/batch): `1` for successful transactions, `0` for failed ones and `undefined` for chains and block ranges not compliant with the post-Byzantinum hard fork EVM specification (e.g. 0-4,369,999 on Ethereum).

`type` field is populated similarly. For example, on Ethereum `0` is returned for Legacy txs, `1` for EIP-2930 and `2` for EIP-1559. Other networks may have a different set of types.

See the [block headers section](#block-headers) for the definition of `BlockHeader`.

`l1*` fields can only be requested for networks from [this list](/en/data/evm). Requesting them for other networks may cause HTTP 500 responses.

### State diffs

`StateDiff` data items may have the following fields:

```ts theme={"system"}
StateDiff {
  // independent of field selectors
  transactionIndex: number
  block: BlockHeader
  transaction?: Transaction
  address: string
  key: 'balance' | 'code' | 'nonce' | string

  // can be disabled with field selectors
  kind: '=' | '+' | '*' | '-'
  prev?: string | null
  next?: string | null
}
```

The meaning of the `kind` field values is as follows:

* `'='`: no change has occurred;
* `'+'`: a value was added;
* `'*'`: a value was changed;
* `'-'`: a value was removed.

The values of the `key` field are regular hexadecimal contract storage key strings or one of the special keys `'balance' | 'code' | 'nonce'` denoting ETH balance, contract code and nonce value associated with the state diff.

See the [block headers section](#block-headers) for the definition of `BlockHeader` and the [transactions section](#transactions) for the definition of `Transaction`.

### Traces

Field selection for trace data items is somewhat more involved because its fixed fields `action` and `result` may contain different fields depending on the value of the `type` field. The retrieval of each one of these subfields is configured independently. For example, to ensure that all traces of `'call'` type contain the `.action.gas` field, the processor must be configured as follows:

```ts theme={"system"}
processor.setFields({
  trace: {
    callGas: true
  }
})
```

The full `Trace` type with all its possible (sub)fields looks like this:

```ts theme={"system"}
Trace {
  // independent of field selectors
  transactionIndex: number
  block: BlockHeader
  transaction?: Transaction
  traceAddress: number[]
  type: 'create' | 'call' | 'suicide' | 'reward'
  subtraces: number

  // can be disabled with field selectors
  error: string | null

  // can be requested with field selectors
  // if (type==='create')
  action: {
                  // request the subfields with
    from: string  // createFrom: true
    value: bigint // createValue: true
    gas: bigint   // createGas: true
    init: string  // createInit: true
  }
  result?: {
    gasUsed: bigint  // createResultGasUsed: true
    code: string     // createResultCode: true
    address?: string // createResultAddress: true
  }
  // if (type==='call')
  action: {
    from: string    // callFrom: true
    to: string      // callTo: true
    value: bigint   // callValue: true
    gas: bigint     // callGas: true
    sighash: string // callSighash: true
    input: string   // callInput: true
  }
  result?: {
    gasUsed: bigint // callResultGasUsed: true
    output: string  // callResultOutput: true
  }
  // if (type==='suicide')
  action: {
    address: string        // suicideAddress: true
    refundAddress: string  // suicideRefundAddress: true
    balance: bigint        // suicideBalance: true
  }
  // if (type==='reward')
  action: {
    author: string // rewardAuthor: true
    value: bigint  // rewardValue: true
    type: string   // rewardType: true
  }
}
```

### Block headers

`BlockHeader` data items may have the following fields:

```ts theme={"system"}
BlockHeader{
  // independent of field selectors
  hash: string
  height: number
  id: string
  parentHash: string

  // can be disabled with field selectors
  timestamp: number

  // can be requested with field selectors
  nonce?: string
  sha3Uncles: string
  logsBloom: string
  transactionsRoot: string
  stateRoot: string
  receiptsRoot: string
  mixHash?: string
  miner: string
  difficulty?: bigint
  totalDifficulty?: bigint
  extraData: string
  size: bigint
  gasLimit: bigint
  gasUsed: bigint
  baseFeePerGas?: bigint
  // limited availability (see below)
  l1BlockNumber: number
}
```

The `l1BlockNumber` field can only be requested for networks from [this list](/en/data/evm). Requesting it for other networks may cause HTTP 500 responses.

## A complete example

```ts theme={"system"}
import {EvmBatchProcessor} from '@subsquid/evm-processor'
import * as gravatarAbi from './abi/gravatar'
import * as erc721abi from './abi/erc721'
import {TypeormDatabase} from '@subsquid/typeorm-store'

const gravatarRegistryContract = '0x2e645469f354bb4f5c8a05b3b30a929361cf77ec'
const gravatarTokenContract = '0xac5c7493036de60e63eb81c5e9a440b42f47ebf5'

const processor = new EvmBatchProcessor()
  .setGateway('https://v2.archive.subsquid.io/network/ethereum-mainnet')
  .setRpcEndpoint('<my_eth_rpc_url>')
  .setFinalityConfirmation(75)
  .setBlockRange({ from: 6_000_000 })
  .addLog({
    address: [
      gravatarRegistryContract
    ],
    topic0: [
      gravatarAbi.events.NewGravatar.topic,
      gravatarAbi.events.UpdatedGravatar.topic,
    ]
  })
  .addTransaction({
    to: [
      gravatarTokenContract
    ],
    range: { from: 15_500_000 },
    sighash: [
      erc721abi.functions.setApprovalForAll.sighash
    ]
  })
  .setFields({
    log: {
      topics: true,
      data: true,
    },
    transaction: {
      from: true,
      input: true,
      to: true
    }
  })

processor.run(new TypeormDatabase(), async (ctx) => {
  // Simply output all the items in the batch.
  // It is guaranteed to have all the data matching the data requests,
  // but not guaranteed to not have any other data.
  ctx.log.info(ctx.blocks, "Got blocks")
})
```


# EVM Batch General Settings
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/processors/evm-batch/general

General settings for Squid SDK EvmBatchProcessor — configure RPC, archive endpoint, batch size, finality, prometheus metrics, and runtime options.

<Tip>
  The method documentation is also available inline and can be accessed via suggestions in most IDEs.
</Tip>

The following setters configure the global settings of `EvmBatchProcessor`. They return the modified instance and can be chained.

Certain configuration methods are required:

* one or both of [`setGateway()`](#set-gateway) and [`setRpcEndpoint()`](#set-rpc-endpoint)
* [`setFinalityConfirmation()`](#set-finality-confirmation) whenever [RPC ingestion](/en/sdk/squid-sdk/resources/unfinalized-blocks) is enabled, namely when
  * a RPC endpoint was configured with [`setRpcEndpoint()`](#set-rpc-endpoint)
  * RPC ingestion has **NOT** been explicitly disabled by calling [`setRpcDataIngestionSettings({ disabled: true })`](#set-rpc-data-ingestion-settings)

Here's how to choose the data sources depending on your use case:

* If you need real-time data and your network [has a SQD Network gateway](/en/data/evm), use both [`setGateway()`](#set-gateway) and [`setRpcEndpoint()`](#set-rpc-endpoint). The processor will obtain as much data as is currently available from the network, then switch to ingesting recent data from the RPC endpoint.
* If you can tolerate your data being several thousands of blocks behind the chain head, you do not want to use a RPC endpoint and your network [has a SQD Network gateway](/en/data/evm), use [`setGateway()`](#set-gateway) only.
* If your EVM network does not have a SQD Network gateway, use [`setRpcEndpoint()`](#set-rpc-endpoint) only. You can use this regime to [work with local development nodes](/en/sdk/squid-sdk/tutorials/evm-local).
* If your squid uses [direct RPC queries](/en/sdk/squid-sdk/resources/tools/typegen/state-queries) then [`setRpcEndpoint()`](#set-rpc-endpoint) is a hard requirement. You can reduce the RPC usage by adding a Network data source with [`setGateway()`](#set-gateway). Further, if you can tolerate a latency of a few thousands of blocks, you can disable RPC ingestion with [`setRpcDataIngestionSettings({ disabled: true })`](#set-rpc-data-ingestion-settings). In this scenario RPC will only be used for the queries you explicitly make in your code.

### `setGateway(url: string | GatewaySettings)`

Adds a [SQD Network](/en/network) data source. The argument is either a string URL of a SQD Network gateway or

```ts theme={"system"}
{
  url: string // gateway URL
  requestTimeout?: number // in milliseconds
}
```

See [EVM gateways](/en/data/evm).

### `setRpcEndpoint(rpc: ChainRpc)`

Adds a RPC data source. If added, it will be used for

* [RPC ingestion](/en/sdk/squid-sdk/resources/unfinalized-blocks) (unless explicitly disabled with [`setRpcDataIngestionSettings()`](#set-rpc-data-ingestion-settings))
* any [direct RPC queries](/en/sdk/squid-sdk/resources/tools/typegen/state-queries) you make in your squid code

A node RPC endpoint can be specified as a string URL or as an object:

```ts theme={"system"}
type ChainRpc = string | {
  url: string // http, https, ws and wss are supported
  capacity?: number // num of concurrent connections, default 10
  maxBatchCallSize?: number // default 100
  rateLimit?: number // requests per second, default is no limit
  requestTimeout?: number // in milliseconds, default 30_000
  headers: Record<string, string> // http headers
}
```

Setting `maxBatchCallSize` to `1` disables batching completely.

<Tip>
  We recommend using private endpoints for better performance and stability of your squids. For SQD Cloud deployments you can use the [RPC addon](/en/cloud/resources/rpc-proxy). If you use an external private RPC, keep the endpoint URL in a [Cloud secret](/en/cloud/resources/env-variables#secrets).
</Tip>

### `setDataSource(ds: {archive?: string, chain?: ChainRpc})` (deprecated)

Replaced by [`setGateway()`](#set-gateway) and [`setRpcEndpoint()`](#set-rpc-endpoint).

### `setRpcDataIngestionSetting(settings: RpcDataIngestionSettings)`

Specify the [RPC ingestion](/en/sdk/squid-sdk/resources/unfinalized-blocks) settings.

```ts theme={"system"}
type RpcDataIngestionSettings = {
  disabled?: boolean
  preferTraceApi?: boolean
  useDebugApiForStateDiffs?: boolean
  debugTraceTimeout?: string
  headPollInterval?: number
  newHeadTimeout?: number
}
```

Here,

* `disabled`: Explicitly disables data ingestion from an RPC endpoint.
* `preferTraceApi`: By default, [`debug_traceBlockByHash`](https://geth.ethereum.org/docs/interacting-with-geth/rpc/ns-debug#debugtraceblockbyhash) is used to obtain [call traces](/en/sdk/squid-sdk/reference/processors/evm-batch/traces). This flag instructs the processor to utilize [`trace_` methods](https://openethereum.github.io/JSONRPC-trace-module) instead. This setting is only effective for finalized blocks.
* `useDebugApiForStateDiffs`: By default, [`trace_replayBlockTransactions`](https://openethereum.github.io/JSONRPC-trace-module#trace_replayblocktransactions) is used to obtain [state diffs](/en/sdk/squid-sdk/reference/processors/evm-batch/state-diffs) for finalized blocks. This flag instructs the processor to utilize [`debug_traceBlockByHash`](https://geth.ethereum.org/docs/interacting-with-geth/rpc/ns-debug#debugtraceblockbyhash) instead. This setting is only effective for finalized blocks. **WARNING:** this will significantly increase the amount of data retrieved from the RPC endpoint. Expect download rates in the megabytes per second range.
* `debugTraceTimeout`: If set, the processor will pass the `timeout` parameter to [debug trace config](https://geth.ethereum.org/docs/interacting-with-geth/rpc/ns-debug#traceconfig).
* `headPollInterval`: Poll interval for new blocks in milliseconds. Poll mechanism is used to get new blocks via HTTP connections. Default: 5000.
* `newHeadTimeout`: When ingesting from a websocket, this setting specifies the timeout in milliseconds after which the connection will be reset and subscription re-initiated if no new blocks were received. Default: no timeout.

### `setFinalityConfirmation(nBlocks: number)`

Sets the number of blocks after which the processor will consider the consensus data final. Use a value appropriate for your network. For example, for Ethereum mainnet a widely cited value is 15 minutes/75 blocks.

### `setBlockRange({from: number, to?: number})`

Limits the range of blocks to be processed. When the upper bound is specified, processor will terminate with exit code 0 once it reaches it.

Note that block ranges can also be specified separately for each data request. This method sets global bounds for all block ranges in the configuration.

### `includeAllBlocks(range?: {from: number, to?: number})`

By default, processor will fetch only blocks which contain requested items. This method modifies such behavior to fetch all chain blocks. Optionally a range of blocks can be specified for which the setting should be effective.

### `setPrometheusPort(port: string | number)`

Sets the port for a built-in prometheus health metrics server (serving at `http://localhost:${port}/metrics`). By default, the value of PROMETHEUS\_PORT environment variable is used. When it is not set, processor will pick an ephemeral port.


# Event logs
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/processors/evm-batch/logs

Subscribe to EVM event log data with addLog() in Squid SDK — filter by contract address, topic, and signature for efficient EVM event indexing.

#### `addLog(options)`

Get event logs emitted by some *or all* contracts in the network. `options` has the following structure:

```typescript theme={"system"}
{
  // data requests
  address?: string[]
  topic0?: string[]
  topic1?: string[]
  topic2?: string[]
  topic3?: string[]
  range?: {from: number, to?: number}

  // related data retrieval
  transaction?: boolean
  transactionLogs?: boolean
  transactionTraces?: boolean
}
```

Data requests:

* `address`: the set of addresses of contracts emitting the logs. Omit to subscribe to events from all contracts in the network.
* `topicN`: the set of values of topicN.
* `range`: the range of blocks to consider.

Related data retrieval:

* `transaction = true`: the processor will retrieve all parent transactions and add them to the `transactions` iterable within the [block data](/en/sdk/squid-sdk/reference/processors/evm-batch/context-interfaces). Additionally it will expose them via the `.transaction` field of each log item.
* `transactionLogs = true`: the processor will retrieve all "sibling" logs, that is, all logs emitted by transactions that emitted at least one matching log. The logs will be exposed through the regular `logs` block data iterable and via `.transaction.logs` for matching logs.
* `transactionTraces = true`: the processor will retrieve the traces for all transactions that emitted at least one matching log. The traces will be exposed through the regular `traces` block data iterable and via `.transaction.traces`.

Note that logs can also be requested by [`addTransaction()`](/en/sdk/squid-sdk/reference/processors/evm-batch/transactions) and [`addTrace()`](/en/sdk/squid-sdk/reference/processors/evm-batch/traces) method as related data.

Selection of the exact data to be retrieved for each log and its optional parent transaction is done with the `setFields()` method documented on the [Field selection](/en/sdk/squid-sdk/reference/processors/evm-batch/field-selection) page. Some examples are available below.

## Examples

1. Fetch `NewGravatar(uint256,address,string,string)` and `UpdateGravatar(uint256,address,string,string)` event logs emitted by `0x2E645469f354BB4F5c8a05B3b30A929361cf77eC`. For each log, fetch topic set, log data. Fetch parent transactions with their inputs.

```ts theme={"system"}
const processor = new EvmBatchProcessor()
  .setGateway('https://v2.archive.subsquid.io/network/ethereum-mainnet')
  .setRpcEndpoint('<my_eth_rpc_url>')
  .setFinalityConfirmation(75)
  .addLog({
    address: ['0x2e645469f354bb4f5c8a05b3b30a929361cf77ec'],
    topic0: [
      // topic: 'NewGravatar(uint256,address,string,string)'
      '0x9ab3aefb2ba6dc12910ac1bce4692cf5c3c0d06cff16327c64a3ef78228b130b',
      // topic: 'UpdatedGravatar(uint256,address,string,string)'
      '0x76571b7a897a1509c641587568218a290018fbdc8b9a724f17b77ff0eec22c0c',
    ],
    transaction: true
  })
  .setFields({
    log: {
      topics: true,
      data: true
    },
    transaction: {
      input: true
    }
  })
```

<Tip>
  Typescript ABI modules generated by [`squid-evm-typegen`](/en/sdk/squid-sdk/resources/tools/typegen/state-queries) provide event signatures/topic0 values as constants, e.g.

  ```ts theme={"system"}
    import * as gravatarAbi from './abi/gravatar'
    // ...
      topic0: [
        gravatarAbi.events.NewGravatar.topic,
        gravatarAbi.events.UpdatedGravatar.topic,
      ],
    // ...
  ```
</Tip>

2. Fetch every `Transfer(address,address,uint256)` event on Ethereum mainnet where *topic2* is set to the destination address (a common but [non-standard](https://eips.ethereum.org/EIPS/eip-20) practice) and the destination is `vitalik.eth` a.k.a. `0xd8dA6BF26964aF9D7eEd9e03E53415D37aA96045`. For each log, fetch transaction hash.

```ts theme={"system"}
const processor = new EvmBatchProcessor()
  .setGateway('https://v2.archive.subsquid.io/network/ethereum-mainnet')
  .setRpcEndpoint('<my_eth_rpc_url>')
  .setFinalityConfirmation(75)
  .addLog({
    topic0: [
      // topic0: 'Transfer(address,address,uint256)'
      '0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef'
    ],
    topic2: [
      // vitalik.eth
      '0x000000000000000000000000d8da6bf26964af9d7eed9e03e53415d37aa96045'
    ]
  })
  .setFields({
    log: {
      transactionHash: true
    }
  })
```

<Tip>
  As you may observe, the address in the `topic2` is a bit longer than usual (`0xd8dA6BF26964aF9D7eEd9e03E53415D37aA96045`, 42 chars).
  This is caused by the fact that Squid SDK expects `Bytes32[]`; therefore, the length has to be 66 chars long.
  The possible quick fix is to pad the original address with zeros and prepend `0x`.

  ```ts theme={"system"}
    const address = '0xd8dA6BF26964aF9D7eEd9e03E53415D37aA96045'
    const topic = '0x' + address.replace('x', '0').padStart(64, '0').toLowerCase()
  ```
</Tip>


# Storage state diffs
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/processors/evm-batch/state-diffs

Track EVM storage state changes with addStateDiff() in Squid SDK — subscribe to storage slot writes for any contract address with selective filtering.

<Tip>
  State diffs for historical blocks are [currently available](/en/data/evm) from [SQD Network](/en/network) on the same basis as all other data stored there: for free. If you deploy a squid that indexes traces [in real-time](/en/sdk/squid-sdk/resources/unfinalized-blocks) to SQD Cloud and use our [RPC addon](/en/cloud/resources/rpc-proxy), the necessary `trace_` or `debug_` RPC calls made will be counted alongside all other calls and [the price](/en/cloud/pricing#rpc-requests) will be computed for the total count. There are no surcharges for traces or state diffs.
</Tip>

#### `addStateDiff(options)`

Subscribe to changes in the [contract storage](https://coinsbench.com/solidity-layout-and-access-of-storage-variables-simply-explained-1ce964d7c738). This allows for tracking the contract state changes that are difficult to infer from events or transactions, such as the changes that take into account the output of internal calls. `options` has the following structure:

```typescript theme={"system"}
{
  // data requests
  address?: string[]
  key?: string[]
  kind?: ('=' | '+' | '*' | '-')[]
  range?: {from: number, to?: number}

  // related data retrieval
  transaction?: boolean
}
```

The data requests here are:

* `address`: the set of addresses of contracts to track. Leave undefined to subscribe to state changes of all contracts from the whole network.
* `key`: the set of storage keys that should be tracked. Regular hexadecimal contract storage keys and [special keys](/en/sdk/squid-sdk/reference/processors/evm-batch/field-selection#state-diffs) (`'balance'`, `'code'`, `'nonce'`) are allowed. Leave undefined to subscribe to all state changes.
* `kind`: the set of diff kinds that should be tracked. Refer to the [`StateDiff` section](/en/sdk/squid-sdk/reference/processors/evm-batch/field-selection#state-diffs) of data items documentation for an explanation of the meaning of the permitted values.
* `range`: the range of blocks within which the storage changes should be tracked.

Enabling the `transaction` flag will cause the processor to retrieve the transaction that gave rise to each state change and add it to the [`transactions` iterable of block data](/en/sdk/squid-sdk/reference/processors/evm-batch/context-interfaces).

Note that state diffs can also be requested by the [`addTransaction()`](/en/sdk/squid-sdk/reference/processors/evm-batch/transactions) method as related data.

Selection of the exact data to be retrieved for each state diff item and its optional parent transaction is done with the `setFields()` method documented on the [Field selection](/en/sdk/squid-sdk/reference/processors/evm-batch/field-selection) page. Unlike other data items, state diffs do not have any fields that can be enabled, but some can be disabled for improved sync performance.


# Traces
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/processors/evm-batch/traces

Retrieve EVM internal call traces with addTrace() in Squid SDK — subscribe to nested contract calls, reverts, and create operations during indexing.

<Tip>
  Traces for historical blocks are [currently available](/en/data/evm) from [SQD Network](/en/network) on the same basis as all other data stored there: for free. If you deploy a squid that indexes traces [in real-time](/en/sdk/squid-sdk/resources/unfinalized-blocks) to SQD Cloud and use our [RPC addon](/en/cloud/resources/rpc-proxy), the necessary `trace_` or `debug_` RPC calls made will be counted alongside all other calls and [the price](/en/cloud/pricing#rpc-requests) will be computed for the total count. There are no surcharges for traces or state diffs.
</Tip>

#### `addTrace(options)`

Subscribe to [execution traces](https://geth.ethereum.org/docs/interacting-with-geth/rpc/ns-debug#debugtraceblockbyhash). This allows for tracking internal calls. The `options` object has the following structure:

```typescript theme={"system"}
{
  // data requests
  callTo?: string[]
  callFrom?: string[]
  callSighash?: string[]
  createFrom?: string[]
  rewardAuthor?: string[]
  suicideRefundAddress?: string[]
  type?: string[]
  range?: {from: number, to?: number}

  // related data retrieval
  transaction?: boolean
  transactionLogs?: boolean
  subtraces?: boolean
  parents?: boolean
}
```

The data requests here are:

* `type`: get traces of types from this set. Allowed types are `'create' | 'call' | 'suicide' | 'reward'`.
* `callTo`: get `call` traces *to* the addresses in this set.
* `callFrom`: get `call` traces *from* the addresses in this set.
* `callSighash`: get `call` traces with signature hashes in this set.
* `createFrom`: get `create` traces *from* the addresses in this set.
* `rewardAuthor`: get `reward` traces where block authors are in this set.
* `suicideRefundAddress`: get `suicide` traces where refund addresses in this set.
* `range`: get traces from transactions from this range of blocks.

Related data retrieval:

* `transaction = true` will cause the processor to retrieve transactions that the traces belong to.
* `transactionLogs = true` will cause the processor to retrieve all logs emitted by transactions that the traces belong to.
* `subtraces = true` will cause the processor to retrieve downstream traces in addition to those that matched the data requests.
* `parents = true` will cause the processor to retrieve upstream traces in addition to those that matched the data requests.

These extra data items will be added to the appropriate iterables within the [block data](/en/sdk/squid-sdk/reference/processors/evm-batch/context-interfaces).

Note that traces can also be requested by [`addTransaction()`](/en/sdk/squid-sdk/reference/processors/evm-batch/transactions) and [`addLog()`](/en/sdk/squid-sdk/reference/processors/evm-batch/logs) method as related data.

Selection of the exact data to be retrieved for each trace item is done with the `setFields()` method documented on the [Field selection](/en/sdk/squid-sdk/reference/processors/evm-batch/field-selection) page. Be aware that field selectors for traces do not share their names with the fields of trace data items, unlike field selectors for other data item types. This is due to traces varying their structure depending on the value of the `type` field.

## Examples

### Exploring internal calls of a given transaction

For a [`mint` call to Uniswap V3 Positions NFT](https://etherscan.io/tx/0xf178718219151463aa773deaf7d9367b8408e35a624550af975e089ca6e015ca).

```ts theme={"system"}
import {EvmBatchProcessor} from '@subsquid/evm-processor'
import {TypeormDatabase} from '@subsquid/typeorm-store'

const TARGET_TRANSACTION = '0xf178718219151463aa773deaf7d9367b8408e35a624550af975e089ca6e015ca'
const TO_CONTRACT = '0xc36442b4a4522e871399cd717abdd847ab11fe88' // Uniswap v3 Positions NFT
const METHOD_SIGHASH = '0x88316456' // mint

const processor = new EvmBatchProcessor()
  .setGateway('https://v2.archive.subsquid.io/network/ethereum-mainnet')
  .setRpcEndpoint('<my_eth_rpc_url>')
  .setFinalityConfirmation(75)
  .setBlockRange({ from: 16962349, to: 16962349 })
  .addTransaction({
    to: [TO_CONTRACT],
    sighash: [METHOD_SIGHASH],
    traces: true
  })
  .setFields({ trace: { callTo: true } })

processor.run(new TypeormDatabase(), async ctx => {
  let involvedContracts = new Set<string>()
  let traceCount = 0

  for (let block of ctx.blocks) {
    for (let trc of block.traces) {
      if (trc.type === 'call' && trc.transaction?.hash === TARGET_TRANSACTION) {
        involvedContracts.add(trc.action.to)
        traceCount += 1
      }
    }
  }

  console.log(`txn ${TARGET_TRANSACTION} had ${traceCount-1} internal transactions`)
  console.log(`${involvedContracts.size} contracts were involved in txn ${TARGET_TRANSACTION}:`)
  involvedContracts.forEach(c => { console.log(c) })
})
```

### Grabbing addresses of all contracts ever created on Ethereum

Full code is available in [this branch](https://github.com/subsquid-labs/grab-all-contracts/tree/ascetic). WARNING: will contain addresses of some contracts that failed to deploy.

```ts theme={"system"}
import {EvmBatchProcessor} from '@subsquid/evm-processor'
import {TypeormDatabase} from '@subsquid/typeorm-store'
import {CreatedContract} from './model'

const processor = new EvmBatchProcessor()
  .setGateway('https://v2.archive.subsquid.io/network/ethereum-mainnet')
  .setFields({
    trace: {
      createResultAddress: true,
    },
  })
  .addTrace({
    type: ['create'],
    transaction: true,
  })

processor.run(new TypeormDatabase({supportHotBlocks: false}), async (ctx) => {
  const contracts: Map<string, CreatedContract> = new Map()
  const addresses: Set<string> = new Set()
  for (let c of ctx.blocks) {
    for (let trc of c.traces) {
      if (trc.type === 'create' &&
          trc.result?.address != null &&
          trc.transaction?.hash !== undefined) {
        contracts.set(trc.result.address, new CreatedContract({id: trc.result.address}))
      }
    }
  }
  await ctx.store.upsert([...contracts.values()])
})
```

Currently there is no convenient way to check whether a trace had effect on the chain state, but this feature will be added in future releases.


# EVM Transactions
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/processors/evm-batch/transactions

Subscribe to EVM transaction data with addTransaction() in Squid SDK — filter by from, to, signature, and contract address for efficient indexing.

#### `addTransaction(options)`

Get some *or all* transactions on the network. `options` has the following structure:

```typescript theme={"system"}
{
  // data requests
  from?: string[]
  to?: string[]
  sighash?: string[]
  range?: {from: number, to?: number}

  // related data retrieval
  logs?: boolean
  stateDiffs?: boolean
  traces?: boolean
}
```

Data requests:

* `from` and `to`: the sets of addresses of tx senders and receivers. Omit to subscribe to transactions from/to any address.
* `sighash`: [first four bytes](https://ethereum.org/en/developers/docs/transactions/#the-data-field) of the Keccak hash (SHA3) of the canonical representation of the function signature. Omit to subscribe to any transaction.
* `range`: the range of blocks to consider.

Enabling the `stateDiffs`, `traces` and/or `logs` flags will cause the processor to retrieve [state diffs](/en/sdk/squid-sdk/reference/processors/evm-batch/state-diffs), [traces](/en/sdk/squid-sdk/reference/processors/evm-batch/traces) and/or event logs that occurred as a result of each selected transaction. The data will be added to the appropriate iterables within the [block data](/en/sdk/squid-sdk/reference/processors/evm-batch/context-interfaces).

Note that transactions can also be requested by [`addLog()`](/en/sdk/squid-sdk/reference/processors/evm-batch/logs), [`addStateDiff()`](/en/sdk/squid-sdk/reference/processors/evm-batch/state-diffs) and [`addTrace()`](/en/sdk/squid-sdk/reference/processors/evm-batch/traces) as related data.

Selection of the exact data to be retrieved for each transaction and the optional related data items is done with the `setFields()` method documented on the [Field selection](/en/sdk/squid-sdk/reference/processors/evm-batch/field-selection) page. Some examples are available below.

<Tip>
  Typescript ABI modules generated by [`squid-evm-typegen`](/en/sdk/squid-sdk/resources/tools/typegen/state-queries) provide function sighashes as constants, e.g.

  ```ts theme={"system"}
    import * as erc20abi from './abi/erc20'
    // ...
      sighash: [erc20abi.functions.transfer.sighash],
    // ...
  ```
</Tip>

## Examples

1. Request all EVM calls to the contract `0x6a2d262D56735DbA19Dd70682B39F6bE9a931D98`:

```ts theme={"system"}
processor.addTransaction({to: ['0x6a2d262d56735dba19dd70682b39f6be9a931d98']})
```

2. Request all transactions matching sighash of `transfer(address,uint256)`:

```ts theme={"system"}
processor.addTransaction({sighash: ['0xa9059cbb']})
```

3. Request all `transfer(address,uint256)` calls to the specified addresses, from block `6_000_000` onwards and fetch their inputs. Also retrieve all logs emitted by these calls.

```ts theme={"system"}
processor
  .addTransaction({
    to: [
      '0x6a2d262d56735dba19dd70682b39f6be9a931d98',
      '0x3795c36e7d12a8c252a20c5a7b455f7c57b60283'
    ],
    sighash: [
      '0xa9059cbb'
    ],
    range: {
      from: 6_000_000
    },
    logs: true
  })
  .setFields({
    transaction: {
      input: true
    }
  })
```

4. Mine all transactions to and from Vitalik Buterin's address [`vitalik.eth`](https://etherscan.io/address/vitalik.eth). Fetch the involved addresses, ETH value and hash for each transaction. Get execution traces with the [default fields](/en/sdk/squid-sdk/reference/processors/evm-batch/field-selection#transactions) for outgoing transactions.

```ts theme={"system"}
const VITALIK_ETH = '0xd8dA6BF26964aF9D7eEd9e03E53415D37aA96045'.toLowerCase()

const processor = new EvmBatchProcessor()
  .setGateway('https://v2.archive.subsquid.io/network/ethereum-mainnet')
  .setRpcEndpoint('<my_eth_rpc_url>')
  .setFinalityConfirmation(75)
  .addTransaction({
    to: [VITALIK_ETH]
  })
  .addTransaction({
    from: [VITALIK_ETH],
    traces: true
  })
  .setFields({
    transaction: {
      from: true,
      to: true,
      value: true,
      hash: true
    }
  })

processor.run(new TypeormDatabase(), async (ctx) => {
  for (let c of ctx.blocks) {
    for (let txn of c.transactions) {
      if (txn.to === VITALIK_ETH || txn.from === VITALIK_ETH) {
        // just output the tx data to console
        ctx.log.info(txn, 'Tx:')
      }
    }
  }
})
```


# Entities
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/schema-file/entities

Define high-level Squid SDK entities in schema.graphql — entity types, scalar fields, derived fields, and indexes that drive code generation.

Entities are defined by root-level GraphQL types decorated with `@entity`. Names and properties of entities are expected to be camelCased. They are converted into snake\_case for use as the corresponding database table and column names. The primary key column is always mapped to the entity field of a special `ID` type mapped as string (`varchar`). Non-nullable fields are marked with an exclamation mark (`!`) and are nullable otherwise.

The following [scalar types](https://graphql.org/learn/schema/#scalar-types) are supported by the `schema.graphql` dialect:

* `String` (mapped to `text`)
* `Int` (mapped to `int4`)
* `Float` (mapped to `numeric`, ts type `number`)
* `Boolean` (mapped to `bool`)
* `DateTime` (mapped to `timestamptz`, ts type `Date`)
* `BigInt` (mapped to `numeric`, ts type `bigint`)
* `BigDecimal` (mapped to `numeric`, ts type `BigDecimal` of [`@subsquid/big-decimal`](https://www.npmjs.com/package/@subsquid/big-decimal))
* `Bytes` (mapped to `bytea`, ts type `UInt8Array`)
* `JSON` (mapped to `jsonb`, ts type `unknown`)
* Enums (mapped to `text`)
* User-defined scalars (non-entity types). Such properties are mapped as `jsonb` columns.

**Example**

```graphql theme={"system"}
type Scalar @entity {
  id: ID!
  boolean: Boolean
  string: String
  enum: Enum
  bigint: BigInt
  dateTime: DateTime
  bytes: Bytes
  json: JSON
  deep: DeepScalar
}
        
type DeepScalar {
  bigint: BigInt
  dateTime: DateTime
  bytes: Bytes
  boolean: Boolean
}
        
enum Enum {
  A B C
}
```

## Arrays

An entity field can be an array of any scalar type except `BigInt` and `BigDecimal`. It will be mapped to the corresponding Postgres array type. Array elements may be defined as nullable or non-nullable.

**Example**

```graphql theme={"system"}
type Lists @entity {
  id: ID!
  intArray: [Int!]!
  enumArray: [Enum!]
  datetimeArray: [DateTime!]
  bytesArray: [Bytes!]
  listOfListsOfInt: [[Int]]
  listOfJsonObjects: [Foo!]
}
        
enum Enum {
  A B C D E F
}
        
type Foo {
  foo: Int
  bar: Int
}
```


# Entity relations
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/schema-file/entity-relations

Define one-to-many and many-to-many entity relations in Squid SDK schema files — @derivedFrom, @relation, and TypeORM-friendly conventions.

# Entity relations and inverse lookups

The term "entity relation" refers to the situation when an entity instance contains an instance of another entity within one of its fields. Type-wise this means that some entity (called the *owning entity*) has a field of a type that is some other, *non-owning* entity. Within the database, this is implemented as an (automatically indexed) foreign key column within the table mapped to the owning entity. A `fieldName` entity-typed field will map to a column named `field_name_id`.

[One-to-one](https://github.com/typeorm/typeorm/blob/master/docs/one-to-one-relations.md) and [one-to-many](https://github.com/typeorm/typeorm/blob/master/docs/many-to-one-one-to-many-relations.md) relations are supported by Typeorm. The "many" side of the one-to-many relations is always the owning side. Many-to-many relations are modeled as [two one-to-many relations with an explicit join table](#many-to-many-relations).

An entity relation is always unidirectional, but it is possible to request the data on the owning entity from the non-owning one. To do so, define a field decorated `@derivedFrom` in the schema. Doing so will cause the Typeorm code generated by [`squid-typeorm-codegen`](/en/sdk/squid-sdk/resources/persisting-data/typeorm) and the GraphQL API served by [`squid-graphql-server`](/en/sdk/squid-sdk/reference/openreader-server/overview) to show a virtual (that is, **not mapping to a database column**) field populated via inverse lookup queries.

The following examples illustrate the concepts.

## One-to-one relations

```graphql theme={"system"}
type Account @entity {
  id: ID!
  balance: BigInt!
  user: User @derivedFrom(field: "account")
}

type User @entity {
  id: ID!
  account: Account! @unique
  username: String!
  creation: DateTime!
}
```

The `User` entity references `Account` and owns the one-to-one relation. This is implemented as follows:

* On the database side: the `account` property of the `User` entity maps to the `account_id` foreign key column of the `user` table referencing the `account` table.
* On the TypeORM side: the `account` property of the `User` entity gets decorated with `@OneToOne` and `@JoinColumn`.
* On the GraphQL side: sub-selection of the `account` property is made available in `user`-related queries. Sub-selection of the `user` property is made available in `account`-related queries.

<Info>
  Unlike for the many-to-one case, the codegen will not add a virtual reverse lookup property to the TypeORM code for one-to-one relations. You can add it manually:

  ```typescript title="src/model/generated/account.model.ts" theme={"system"}
  import {OneToOne as OneToOne_} from "typeorm"

  @Entity_()
  export class Account {
    // ...
    @OneToOne_(() => User, e => e.account)
    user: User
  }
  ```

  If you are using this feature, please let us know at [the SquidDevs Telegram channel](https://t.me/HydraDevs).
</Info>

## Many-to-one/One-to-many relations

```graphql theme={"system"}
type Account @entity {
  "Account address"
  id: ID!
  transfersTo: [Transfer!] @derivedFrom(field: "to")
  transfersFrom: [Transfer!] @derivedFrom(field: "from")
}

type Transfer @entity {
  id: ID!
  to: Account!
  from: Account!
  amount: BigInt! 
}
```

Here `Transfer` defines owns the two relations and `Account` defines the corresponding inverse lookup properties. This is implemented as follows:

* On the database side: the `from` and `to` properties of the `Transfer` entity map to `from_id` and `to_id` foreign key columns of the `transfer` table referencing the `account` table.
* On the TypeORM side: properties `to` and `from` of the `Transfer` entity class get decorated with `@ManyToOne`. Properties `transfersTo` and `transfersFrom` decorated with `@OneToMany` get added to the `Account` entity class.
* On the GraphQL side: sub-selection of all relation-defined properties is made available in the schema.

## Many-to-many relations

Many-to-many entity relations should be modeled as two one-to-many relations with an explicitly defined join table.
Here is an example:

```graphql theme={"system"}
# an explicit join table 
type TradeToken @entity {
  id: ID! # This is required, even if useless
  trade: Trade!
  token: Token! 
}

type Token @entity {
  id: ID!
  symbol: String!
  trades: [TradeToken!]! @derivedFrom(field: "token")    
}

type Trade @entity {
  id: ID!
  tokens: [TradeToken!]! @derivedFrom(field: "trade")
}
```


# Indexes and constraints
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/schema-file/indexes-and-constraints

Annotate Squid SDK schema fields with @index and @unique to speed up GraphQL queries — match indexes to your most common entity filter patterns.

# Indexes and unique constraints

<Warning>
  The lack of indices is the most common cause of slow API queries
</Warning>

It is crucial to add database indexes to the entity fields on which one expects filtering and ordering. To add an index to a column, the corresponding entity field must be decorated with `@index`. The corresponding entity field will be decorated with [TypeORM `@Index()`](https://typeorm.io/indices#column-indices).

One can additionally decorate the field with `@unique` to enforce uniqueness. It corresponds to the [`@Index({ unique: true })`](https://typeorm.io/indices#unique-indices) TypeORM decorator.

### Example

```graphql theme={"system"}
type Transfer @entity {
  id: ID!
  to: Account!
  amount: BigInt! @index
  fee: BigInt! @index @unique
}
```

## Multi-column indices

Multi-column indices are defined on the entity level, with an optional `unique` constraint.

### Example

```graphql theme={"system"}
type Foo @entity @index(fields: ["foo", "bar"]) @index(fields: ["bar", "baz"]) 
  {
  id: ID!
  bar: Int!
  baz: [Enum!]
  foo: String!

type Extrinsic @entity @index(fields: ["hash", "block"], unique: true)  {
  id: ID!
  hash: String! @unique
  block: String!
}
```


# Interfaces
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/schema-file/interfaces

Define queryable GraphQL interfaces in Squid SDK schema files — share fields across entity types and run polymorphic queries via OpenReader.

The schema file supports [GraphQL Interfaces](https://graphql.org/learn/schema/#interfaces) for modelling complex types sharing common traits. Interfaces are annotated with `@query` at the type level and do not affect the database schema, only enriching the GraphQL API queries with [inline fragments](https://graphql.org/learn/queries/#inline-fragments).

Currently, only [OpenReader](/en/sdk/squid-sdk/reference/openreader-server) supports GraphQL interfaces defined in the schema file.

### Examples

```graphql theme={"system"}
interface MyEntity @query {
  id: ID!
  name: String
  ref: Ref
}

type Ref @entity {
  id: ID!
  name: String
  foo: Foo! @unique
  bar: Bar! @unique
} 

type Foo implements MyEntity @entity {
  id: ID!
  name: String
  ref: Ref @derivedFrom(field: "foo")
  foo: Int
}

type Bar implements MyEntity @entity {
  id: ID!
  name: String
  ref: Ref @derivedFrom(field: "bar")
  bar: Int
}

type Baz implements MyEntity @entity {
  id: ID!
  name: String
  ref: Ref
  baz: Int
}
```

The `MyEntity` interface above enables `myEntities` and `myEntitiesConnection` [GraphQL API queries](/en/sdk/squid-sdk/reference/openreader-server/api) with inline fragments and the `_type`, `__typename` [meta fields](https://graphql.org/learn/queries/#meta-fields):

```graphql theme={"system"}
query {
  myEntities(orderBy: [_type_DESC, id_ASC]) {
    id
    name
    ref {
        id
        name
    }
    __typename
    ... on Foo { foo }
    ... on Bar { bar }
    ... on Baz { baz }
  }
}
```


# Schema file and codegen
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/schema-file/intro

Introduction to the Squid SDK schema.graphql file and codegen tool — define entities, run sqd codegen, and generate TypeORM models for storage.

The schema file `schema.graphql` uses a GraphQL dialect to model the target entities and entity relations. The tooling around the schema file is then used to:

* Generate TypeORM entities (with `squid-typeorm-codegen(1)`, see below)
* Generate the database schema from the TypeORM entities (see [db migrations](/en/sdk/squid-sdk/resources/persisting-data/typeorm))
* Optionally, the schema can be used to present the target data with a [GraphQL API](/en/sdk/squid-sdk/resources/serving-graphql).

The schema file format is loosely compatible with the [subgraph schema](https://thegraph.com/docs/en/developing/creating-a-subgraph/) file, see [Migrate from subgraph](/en/sdk/squid-sdk/resources/migrate/migrate-subgraph) section for details.

## TypeORM codegen

The [`squid-typeorm-codegen(1)`](https://github.com/subsquid/squid-sdk/tree/master/typeorm/typeorm-codegen) tool is used to generate [TypeORM entity](https://typeorm.io/) classes from the schema defined in `schema.graphql`. Invoke it with

```bash theme={"system"}
npx squid-typeorm-codegen
```

By default the entity classes are generated in `src/model/generated`.

### Example

A `Foo` entity defined in the schema file:

```graphql title="schema.graphql" theme={"system"}
type Foo @entity {
  id: ID!
  bar: String
  baz: BigInt!
}
```

The generated `Foo` entity with TypeORM decorators:

```ts title="src/model/generated/foo.ts" theme={"system"}
import {Entity as Entity_, Column as Column_, PrimaryColumn as PrimaryColumn_} from "typeorm"
import * as marshal from "./marshal"

@Entity_()
export class Foo {
    constructor(props?: Partial<Foo>) {
        Object.assign(this, props)
    }

    @PrimaryColumn_()
    id!: string

    @Column_("text", {nullable: true})
    bar!: string | undefined | null

    @Column_("numeric", {transformer: marshal.bigintTransformer, nullable: false})
    baz!: bigint
}
```


# Unions and typed JSON
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/schema-file/unions-and-typed-json

Model Squid SDK union types and typed JSON fields in schema.graphql — discriminate variants and store structured JSON in Postgres-backed entities.

Complex scalar types can be modelled using a typed JSON fields together with union types, making safe union types.

## Typed JSON

It is possible to define explicit types for JSON fields. The generated entity classes and the GraphQL API will respect the type definition of the field, enforcing the data integrity.

**Example**

```graphql theme={"system"}
type Entity @entity {
  a: A
}

type A {
  a: String
  b: B
  c: JSON
}

type B {
  a: A
  b: String
  e: Entity
}
```

## Union types

One can leverage union types supported both by [Typescript](https://www.typescriptlang.org/docs/handbook/2/everyday-types.html#union-types) and [GraphQL](https://graphql.org/learn/schema/#union-types).  The union operator for `schema.graphql` supports only non-entity types, including typed JSON types described above. JSON types, however, are allowed to reference an entity type.

**Example**

```graphql theme={"system"}
type User @entity {
  id: ID!
  login: String!
}
        
type Farmer {
  user: User!
  crop: Int
}

type Degen {
  user: User!
  bag: String
}
        
union Owner = Farmer | Degen
        
type NFT @entity {
  name: String!
  owner: Owner!
}
```


# bigquery-store
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/store/bigquery

Reference for the @subsquid/bigquery-store package — write Squid SDK indexed data into Google BigQuery datasets with batch loading and schema sync.

# `@subsquid/bigquery-store`

See also the [BigQuery guide](/en/sdk/squid-sdk/resources/persisting-data/bigquery).

## Column types

|           Column type          |             Value type            |                                                     Dataset column type                                                     |
| :----------------------------: | :-------------------------------: | :-------------------------------------------------------------------------------------------------------------------------: |
|           `String()`           |              `string`             |                [STRING](https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types#string_type)               |
|   `Numeric(precision, scale)`  | <code>number \&#124 bigint</code> |   [NUMERIC(P\[, S\])](https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types#parameterized_decimal_type)  |
| `BigNumeric(precision, scale)` | <code>number \&#124 bigint</code> | [BIGNUMERIC(P\[, S\])](https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types#parameterized_decimal_type) |
|            `Bool()`            |             `boolean`             |                [BOOL](https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types#boolean_type)                |
|          `Timestamp()`         |               `Date`              |             [TIMESTAMP](https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types#timestamp_type)            |
|           `Float64()`          |              `number`             |           [FLOAT64](https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types#floating_point_types)          |
|            `Int64()`           | <code>number \&#124 bigint</code> |               [INT64](https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types#integer_types)               |


# CSV support
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/store/file/csv

Write Squid SDK indexed data as CSV files via the Table class — schema, encoding, and per-batch flushing for analytics-friendly file output.

# CSV format support

## `Table` Implementation

The `@subsquid/file-store-csv` package provides a `Table` implementation for writing to CSV files. Use it by [supplying one or more of its instances via the `tables` field of the `Database` constructor argument](/en/sdk/squid-sdk/resources/persisting-data/file#database-options). Constructor of the `Table` implementation accepts the following arguments:

* **`fileName: string`**: the name of the output file in every dataset partition folder.
* **`schema: {[column: string]: ColumnData}`**: a mapping from CSV column names to [`ColumnData` objects](#columns). A mapping of the same keys to data values is the row type used by the [table writer](/en/sdk/squid-sdk/resources/persisting-data/file#table-writer-interface).
* **`options?: TableOptions`**: see [`Table` Options](#table-options).

## Columns

`ColumnData` objects determine how the in-memory data representation of each table column should be serialized. They are made with the `Column` factory function that accepts a column data type and an optional `{nullable?: boolean}` `options` object as arguments.

Column types can be obtained by making the function calls listed below from the `Types` submodule. They determine the type that the [table writer](/en/sdk/squid-sdk/resources/persisting-data/file#table-writer-interface) will expect to find at the corresponding field of data row objects.

|            Column type            | Type of the data row field |
| :-------------------------------: | :------------------------: |
|          `Types.String()`         |          `string`          |
|         `Types.Numeric()`         |    `number` or `bigint`    |
|         `Types.Boolean()`         |          `boolean`         |
| `Types.DateTime(format?: string)` |           `Date`           |
|         `Types.JSON<T>()`         |             `T`            |

`Types.DateTime` accepts an optional [strftime](https://pubs.opengroup.org/onlinepubs/009695399/functions/strftime.html)-compatible format string. If it is omitted, the dates will be serialized to [ISO strings](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date/toISOString).

The type `T` supplied to the `Types.JSON()` generic function must be an object with string keys (extend `{[k: string]: any}`).

## `Table` Options

As its optional final argument, the constructor of `Table` accepts an object that defines table options:

```typescript theme={"system"}
TableOptions {
  dialect?: Dialect
  header?: boolean
}
```

Here,

* **`dialect`** determines the details of the CSV formatting (see the details below, default: `dialects.excel`)
* **`header`** determines whether a CSV header should be added (default: `true`)

`Dialect` type is defined as follows:

```typescript theme={"system"}
Dialect {
  delimiter: string
  escapeChar?: string
  quoteChar: string
  quoting: Quote
  lineterminator: string
}
```

where

```typescript theme={"system"}
enum Quote {
  ALL,        // Put all values in quotes.
  MINIMAL,    // Only quote strings with special characters.
              // A special character is one of the following:
              // delimiter, lineterminator, quoteChar.
  NONNUMERIC, // Quote strings, booleans, DateTimes and JSONs.
  NONE        // Do not quote values.
}
```

is the enum determining how the formatted values should be quoted. The quote character is escaped for all values of `quoting`; `Quote.NONE` additionally escapes the rest of the special characters and the escape character.

Two dialect presets are available via the `dialects` object exported by `@subsquid/file-store-csv`:

```typescript theme={"system"}
export let dialects = {
  excel: {
    delimiter: ',',
    quoteChar: '"',
    quoting: Quote.MINIMAL,
    lineterminator: '\r\n'
  },
  excelTab: {
    delimiter: '\t',
    quoteChar: '"',
    quoting: Quote.MINIMAL,
    lineterminator: '\r\n'
  }
}
```

## Example

This saves ERC20 `Transfer` events captured by the processor to TSV (tab-separated values) files. Full squid code is available in [this repo](https://github.com/subsquid-labs/file-store-csv-example).

```typescript theme={"system"}
import {Database, LocalDest} from '@subsquid/file-store'
import {
  Column,
  Table,
  Types,
  dialects
} from '@subsquid/file-store-csv'
...

const dbOptions = {
  tables: {
    TransfersTable: new Table(
      'transfers.tsv',
      {
        from: Column(Types.String()),
        to: Column(Types.String()),
        value: Column(Types.Numeric())
      },
      {
        dialect: dialects.excelTab,
        header: true
      }
    )
  },
  dest: new LocalDest('./data'),
  chunkSizeMb: 10
}

processor.run(new Database(dbOptions), async (ctx) => {
  ...
  let from: string = ...
  let to: string = ...
  let value: bigint = ...
  ctx.store.TransfersTable.write({ from, to, value })
  ...
})
```


# JSON support
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/store/file/json

Write Squid SDK indexed data as JSON or JSONL files via the Table class — newline-delimited and array variants with schema validation.

# JSON format support

## `Table` Implementation

The `@subsquid/file-store-json` package provides a `Table` implementation for writing to JSON and [JSONL](https://jsonlines.org) files. Use it by [supplying one or more of its instances via the `tables` field of the `Database` constructor argument](/en/sdk/squid-sdk/resources/persisting-data/file#database-options). The `Table` uses a constructor with the following signature:

```typescript theme={"system"}
Table<S extends Record<string, any>>(fileName: string, options?: {lines?: boolean})
```

Here,

* **`S`** is a Typescript type describing the schema of the table data.
* **`fileName: string`** is the name of the output file in every dataset partition folder.
* **`options?: {lines?: boolean}`** are table options. At the moment the only available setting is whether to use JSONL instead of a plain JSON array (default: false).

## Example

This saves ERC20 `Transfer` events captured by the processor to a JSONL file where each line is a JSON serialization of a `{from: string, to: string, value: number}` object. Full squid code is available in [this repo](https://github.com/subsquid-labs/file-store-json-example).

```typescript theme={"system"}
import {Database} from '@subsquid/file-store'
import {Table} from '@subsquid/file-store-json'

...

const dbOptions = {
  tables: {
    TransfersTable: new Table<{
      from: string,
      to: string,
      value: bigint
    }>('transfers.jsonl', { lines: true })
  },
  dest: new LocalDest('./data'),
  chunkSizeMb: 10
}

processor.run(new Database(dbOptions), async (ctx) => {
  ...
  let from: string = ...
  let to: string = ...
  let value: bigint = ...
  ctx.store.TransfersTable.write({ from, to, value })
  ...
})
```


# Parquet support
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/store/file/parquet

Write Squid SDK indexed data as Apache Parquet files via the Table class — columnar storage, optional compression, and schema-driven file output.

# Parquet format support

<Info>
  Support for the Parquet format is currently experimental. Contact us at the [SquidDevs Telegram channel](https://t.me/HydraDevs) for support.
</Info>

## `Table` Implementation

[Apache Parquet](https://parquet.apache.org) is an advanced format for storing tabular data in files. It divides table columns into [column chunks](https://parquet.apache.org/docs/concepts/). Each column chunk is stored contiguously, allowing efficient partial reads of column subsets. Column chunks can also be compressed with row-specific compression algorithms, further enhancing the performance. Retrieval relies on metadata appended to the end of a Parquet file. [Metadata standard](https://parquet.apache.org/docs/file-format/metadata/) of Apache Parquet is extremely powerful, enabling all sorts of [extensions](https://parquet.apache.org/docs/file-format/extensibility/). Among other things, metadata contains the schema of the data, making the format self-describing.

The `@subsquid/file-store-parquet` package provides a `Table` implementation for writing to Parquet files. Use it by [supplying one or more of its instances via the `tables` field of the `Database` constructor argument](/en/sdk/squid-sdk/resources/persisting-data/file#database-options). Constructor of the `Table` implementation accepts the following arguments:

* **`fileName: string`**: the name of the output file in every dataset partition folder.
* **`schema: {[column: string]: ColumnData}`**: a mapping from Parquet column names to [`ColumnData` objects](#columns). A mapping of the same keys to data values is the row type used by the [table writer](/en/sdk/squid-sdk/resources/persisting-data/file#table-writer-interface).
* **`options?: TableOptions`**: see [`Table` Options](#table-options).

## Columns

`ColumnData` objects define storage options for each table column. They are made with the `Column` factory function that accepts a column data type and an optional `options: ColumnOptions` object.

Column types can be obtained by making the function calls listed below from the `Types` submodule. They determine the [Parquet type](https://parquet.apache.org/docs/file-format/types/) that will be used to store the data and the type that the [table writer](/en/sdk/squid-sdk/resources/persisting-data/file#table-writer-interface) will expect to find at the corresponding field of data row objects.

|                  Column type                 |                                     Logical type                                     |                  Primitive type                 |                                    Valid data row object field contents                                    |
| :------------------------------------------: | :----------------------------------------------------------------------------------: | :---------------------------------------------: | :--------------------------------------------------------------------------------------------------------: |
|               `Types.String()`               |                                variable length string                                |                   `BYTE_ARRAY`                  |                                           `string` of any length                                           |
|          `Types.Binary` `(length?)`          |                          variable or fixed length byte array                         |    `BYTE_ARRAY` or `FIXED_LEN_` `BYTE_ARRAY`    |              `Uint8Array` of length equal to `length` if it is set or of any length otherwise              |
|                `Types.Int8()`                |                                 8-bit signed integer                                 |                     `INT32`                     |                                          `number` from -128 to 127                                         |
|                `Types.Int16()`               |                                 16-bit signed integer                                |                     `INT32`                     |                                        `number` from -32768 to 32767                                       |
|                `Types.Int32()`               |                                 32-bit signed integer                                |                     `INT32`                     |                                   `number` from -2147483648 to 2147483647                                  |
|                `Types.Int64()`               |                                 64-bit signed integer                                |                     `INT64`                     |                    `bigint` or `number` from -9223372036854775808 to 9223372036854775807                   |
|                `Types.Uint8()`               |                                8-bit unsigned integer                                |                     `INT32`                     |                                           `number` from 0 to 255                                           |
|               `Types.Uint16()`               |                                16-bit unsigned integer                               |                     `INT32`                     |                                          `number` from 0 to 65535                                          |
|               `Types.Uint32()`               |                                32-bit unsigned integer                               |                     `INT32`                     |                                        `number` from 0 to 4294967295                                       |
|               `Types.Uint64()`               |                                64-bit unsigned integer                               |                     `INT64`                     |                             `bigint` or `number` from 0 to 18446744073709551615                            |
|                `Types.Float()`               |                             32-bit floating point number                             |                     `FLOAT`                     |                                             non-`Nan` `number`                                             |
|               `Types.Double()`               |                             64-bit floating point number                             |                     `DOUBLE`                    |                                             non-`Nan` `number`                                             |
|               `Types.Boolean()`              |                                     boolean value                                    |                    `BOOLEAN`                    |                                                  `boolean`                                                 |
|              `Types.Timestamp()`             |                            UNIX timestamp in milliseconds                            |                     `INT64`                     |                                                   `Date`                                                   |
|    `Types.Decimal` `(precision, scale=0)`    | decimal with `precision` digits and `scale` digits to the right of the decimal point | `INT32` or `INT64` or `FIXED_LEN_` `BYTE_ARRAY` | `number` or `bigint` or [`BigDecimal`](https://github.com/subsquid/squid-sdk/tree/master/util/big-decimal) |
|  `Types.List` `(itemType, {nullable=false})` |        a list filled with optionally nullable items of `itemType` column type        |                        -                        |                                   `Array` of items satisfying `itemType`                                   |
| `Types.JSON<T extends {[k: string]: any}>()` |                                JSON object of type `T`                               |                   `BYTE_ARRAY`                  |                                            `Object` of type `T`                                            |
| `Types.BSON<T extends {[k: string]: any}>()` |                                BSON object of type `T`                               |                   `BYTE_ARRAY`                  |                                            `Object` of type `T`                                            |

<Tip>
  The widest decimals that [PyArrow](https://arrow.apache.org/docs/python/index.html) can read are `Types.Decimal(76)`.
</Tip>

The following column options are available:

```typescript theme={"system"}
ColumnOptions {
  nullable?: boolean
  compression?: Compression
  encoding?: Encoding
}
```

See the [Encoding and Compression](#encoding-and-compression) section for details.

## `Table` Options

As its optional final argument, the constructor of `Table` accepts an object that defines table options:

```typescript theme={"system"}
TableOptions {
  compression?: Compression
  rowGroupSize?: number
  pageSize?: number
}
```

Here,

* **`compression`** determines the file-wide compression algorithm. Per-column settings override this. See [Encoding and Compression](#encoding-and-compression) for the list of available algorithms. Default: `Compression.UNCOMPRESSED`.
* **`rowGroupSize`** determines the approximate uncompressed size of the row group in bytes. Default: `32 * 1024 * 1024`.
* **`pageSize`** determines the approximate uncompressed page size in bytes. Default: `8 * 1024`.

When `pageSize` is less than `rowGroupSize` times the number of columns, the latter setting will be ignored. In this case each row group will contain exactly one roughly `pageSize`d page for each column.

## Encoding and Compression

[Encodings](https://parquet.apache.org/docs/file-format/data-pages/encodings/) are set at a per-column basis. At the moment the default and the only supported value is `'PLAIN'`.

[Compression](https://github.com/apache/parquet-format/blob/master/Compression.md) can be set at a per-file or a per-column basis. Available values are

* `'UNCOMPRESSED'` (default)
* `'GZIP'`
* `'LZO'`
* `'BROTLI'`
* `'LZ4'`

## Example

This saves ERC20 `Transfer` events captured by the processor to a Parquet file. All columns except for `from` are `GZIP`ped. Row groups are set to be roughly 30000 bytes in size each. Each row group contains roughly ten \~1000 bytes-long pages per column. Full squid code is available in [this repo](https://github.com/subsquid-labs/file-store-parquet-example).

```typescript theme={"system"}
import {Database, LocalDest} from '@subsquid/file-store'
import {
  Column,
  Table,
  Types
} from '@subsquid/file-store-parquet'

...

const dbOptions = {
  tables: {
    TransfersTable: new Table(
      'transfers.parquet',
      {
        from: Column(
          Types.String(),
          {
            compression: 'UNCOMPRESSED'
          }
        ),
        to: Column(Types.String()),
        value: Column(Types.Uint64())
      },
      {
        compression: 'GZIP',
        rowGroupSize: 300000,
        pageSize: 1000
      }
    )
  },
  dest: new LocalDest('./data'),
  chunkSizeMb: 10
}

processor.run(new Database(dbOptions), async (ctx) => {
  ...
  let from: string = ...
  let to: string = ...
  let value: bigint = ...
  ctx.store.TransfersTable.write({ from, to, value })
  ...
})
```


# S3 support
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/store/file/s3-dest

Upload Squid SDK file-store output to S3-compatible buckets via the Dest class — Squid CSV, JSON, and Parquet streaming to AWS S3 or MinIO.

# S3 destination support

## Overview

Writing to Amazon S3-compatible file storage services such as [AWS](https://aws.amazon.com) and [Filebase](https://filebase.com) is supported via the `S3Dest` class from the `@subsquid/file-store-s3` package. Use it by [setting the `dest` field of the `Database` constructor argument](/en/sdk/squid-sdk/resources/persisting-data/file#database-options) to its instance. Constructor of `S3Dest` accepts the following arguments:

* **`url: string`**: S3 URL in the `s3://bucket/path` format.
* **`optionsOrClient?: S3Client | S3ClientConfig`**: an optional [S3 client](https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/Package/-aws-sdk-client-s3/Class/S3Client/) or [client config](https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-s3/interfaces/s3clientconfig.html). By default, a simple config parameterized by environment variables is used:
  ```typescript theme={"system"}
  {
    region: process.env.S3_REGION,
    endpoint: process.env.S3_ENDPOINT,
    credentials: {
      accessKeyId: assertNotNull(process.env.S3_ACCESS_KEY_ID),
      secretAccessKey: assertNotNull(process.env.S3_SECRET_ACCESS_KEY),
    },
  }
  ```

## Example

This saves the processor data in the `transfers-data` folder of the `subsquid-testing-bucket` bucket at the [Filebase](https://filebase.com) service. The service only has one region and one endpoint, and here they are hardcoded to reduce the number of required envirionment variables and illustrate how connection parameters can be supplied programmatically. Full squid code is available in [this repo](https://github.com/subsquid-labs/file-store-s3-example).

```typescript theme={"system"}
import {Database} from '@subsquid/file-store'
import {S3Dest} from '@subsquid/file-store-s3'
import {assertNotNull} from '@subsquid/util-internal' // pulled by @subsquid/file-store-s3

...

const dbOptions = {
  ...
  dest: new S3Dest(
    's3://subsquid-testing-bucket/transfers-data',
    {
      region: 'us-east-1',
      endpoint: 'https://s3.filebase.com',
      credentials: {
        accessKeyId: assertNotNull(process.env.S3_ACCESS_KEY_ID),
        secretAccessKey: assertNotNull(process.env.S3_SECRET_ACCESS_KEY)
      }
    }
  ),
  ...
}

processor.run(new Database(dbOptions), async (ctx) => {
  ...
}
```


# typeorm-store
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/store/typeorm

Reference for @subsquid/typeorm-store — the optimized TypeORM-based Postgres store that powers Squid SDK indexers with batched upserts and inserts.

# `@subsquid/typeorm-store`

This page describes the interface of the classes from the `@subsquid/typeorm-store` NPM package. If you're looking for a guide on saving squid data to databases and the related workflows, check out the [Saving to PostgreSQL](/en/sdk/squid-sdk/resources/persisting-data/typeorm) page.

## `TypeormDatabase` constructor arguments

The argument of the `TypeormDatabase` class constructor may have the following fields:

* `stateSchema: string`: the name of the [database schema](https://www.postgresql.org/docs/current/sql-createschema.html) that the processor uses to persist its status (hash + height of the highest reached block). Useful for making sure that each processor uses its own state schema when running multiple processors against the same database (e.g. in a multichain setting). Default: `'squid_processor'`.
* `isolationLevel: 'SERIALIZABLE' | 'READ COMMITTED' | 'REPEATABLE READ'`: sets the [transaction isolation level](https://www.postgresql.org/docs/current/transaction-iso.html) of processor transactions. Default: `'SERIALIZABLE'`.
* `supportHotBlocks: boolean`: controls the support for hot blocks. Necessary in all squids that must be able to handle short-lived [blockchain forks](https://en.wikipedia.org/wiki/Fork_\(blockchain\)). That includes all squids that index chain data in near-real time using RPC endpoints. Default: `true`.
* `projectDir: string`: the folder where `TypeormDatabase` will look for the TypeORM model definition (at `lib/model`) and for migrations (at `db/migrations`). Default: `process.cwd()`.

## `Store` interface

### Batch access methods

#### **`upsert(e: E | E[])`**

Upsert a single or multiple entities to the database. **Does not cascade the upsert to the relations.**

```ts theme={"system"}
await ctx.store.upsert([new User({id: 'Bob'}), new User({id: 'Alice'}))])
```

#### **`save(e: E | E[])`**

Deprecated alias for [`upsert()`](#upsert).

#### **`insert(e: E | E[])`**

Inserts a given entity or entities into the database. Does not check if the entity(s) exist in the database and will fail if a duplicate is inserted. Executes a primitive INSERT operation **without cascading to the relations**.

```ts theme={"system"}
await ctx.store.insert([new User({ id: "Bob" }), new User({ id: "Alice" })]);
```

#### **`remove(e: E | E[] | EntityClass<E>, id?: string | string[])`**

Deletes a given entity or entities from the database. Accepts either an object or an entity ID(s). **Does not cascade the deletion**.

```ts theme={"system"}
await ctx.store.remove(User, ["Alice", "Bob"]);
```

### TypeORM methods

For details see [TypeORM EntityManager reference](https://typeorm.io/entity-manager-api).

#### **`get`**

Get an entity by ID.

```ts theme={"system"}
await ctx.store.get(User, "Bob");
```

#### **`count`**

Count the number of entities matching a where filter.

```ts theme={"system"}
await ctx.store.count(User, {
  where: {
    firstName: "Timber",
  },
});
```

#### **`countBy`**

Count the number of entities matching a filter.

```ts theme={"system"}
await ctx.store.countBy(User, { firstName: "Timber" })
```

#### **`find`**

Return a list matching a where filter.

```ts theme={"system"}
await ctx.store.find(User, {
  where: {
    firstName: "Timber",
  },
});
```

#### **`findBy`**

Return a list matching a filter.

```ts theme={"system"}
let accounts = await ctx.store.findBy(Account, {id: In([...accountIds])})
```

#### **`findOne`**

Return the first entity matching a where filter.

```ts theme={"system"}
const timber = await ctx.store.findOne(User, {
  where: {
    firstName: "Timber",
  },
});
```

#### **`findOneBy`**

Return the first entity matching a filter.

```ts theme={"system"}
const timber = await ctx.store.findOneBy(User, { firstName: "Timber" })
```

#### **`findOneOrFail`**

Throws if nothing is found.

```ts theme={"system"}
const timber = await ctx.store.findOneOrFail(User, {
  where: {
    firstName: "Timber",
  },
});
```

#### **`findOneByOrFail`**

Throws if nothing is found.

```ts theme={"system"}
const timber = await ctx.store.findOneByOrFail(User, { firstName: "Timber" })
```

### Find Operators

`find()` and `findXXX()` methods support the following operators:

* `In` (contains in array)
* `Not`
* `LessThan`
* `LessThanOrEqual`
* `MoreThan`
* `MoreThanOrEqual`
* `Like`
* `ILike`
* `Between`
* `Any`
* `IsNull`
* `Raw` (raw SQL fragments)

See the details and examples in the [TypeORM `FindOption` docs](https://typeorm.io/find-options#advanced-options).

#### Example

```ts theme={"system"}
let accounts = await ctx.store.findBy(Account, {id: In([...accountIds])})
```

### Joining relations

To load an entity with relations, use `relations` field on the `find` options and specify which relations should be joined:

```ts theme={"system"}
await ctx.store.find(User, {
  relations: {
    project: true,
  },
  where: {
    project: {
      name: "TypeORM",
      initials: "TORM",
    },
  },
});
```

See the [TypeORM docs](https://typeorm.io/find-options) sections for details.

## Database connection parameters

Database credentials must be supplied via the environment variables:

* `DB_HOST` (default `localhost`)
* `DB_PORT` (default `5432`)
* `DB_NAME` (default `postgres`)
* `DB_USER` (default `postgres`)
* `DB_PASS` (default `postgres`)
* `DB_SSL` (default `false`)
* `DB_SSL_REJECT_UNAUTHORIZED` (default `true`)
* `DB_URL` (default `undefined`, see the [DB\_URL section](#db_url))

<Info>
  When deploying to [Cloud](/en/cloud) with the [Postgres addon](/en/cloud/reference/pg) enabled in the [manifest](/en/cloud/reference/manifest), any user-supplied values are overwritten for most of these variables. See [Variable shadowing](/en/cloud/reference/pg#variable-shadowing).
</Info>

`typorm-store` also supports the following variables for connecting to databases that require client-side SSL:

* `DB_SSL_CA` - the root certificate in plain text
* `DB_SSL_CA_FILE` - path to a root certificate file
* `DB_SSL_CERT` - client certificate in plain text
* `DB_SSL_CERT_FILE` - path to client certificate in plain text
* `DB_SSL_KEY` - client key in plain text
* `DB_SSL_KEY_FILE` - path to client key in plain text

<Tip>
  In case you're deploying to [Cloud](/en/cloud) you can set [secrets](/en/cloud/resources/env-variables#secrets) to the contents of any given file via stdin:

  ```bash theme={"system"}
  sqd secrets set DB_SSL_CA < ca.crt
  ```
</Tip>

### `DB_URL`

When set, `DB_URL` takes precedence over all individual variables. Its format is as follows:

```
postgres[ql]://[username[:password]@][host[:port]]/database[?parameter_list]
```

where `parameter_list` is an `&`-separated list of assignments of SSL connection parameters:

* `ssl=(0|1|true|false)`
* `sslmode=(disabled|no-verify|prefer|require|verify-ca|verify-full)`
* `sslcert=<path_to_cert_file>`
* `sslkey=<path_to_key_file>`
* `sslrootcert=<path_to_root_cert_file>`

When any value is omitted from the URL, the value of the corresponding individual `DB_*` variable will be used instead. If that is not set, the default will be used.


# sqd auth
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/auth

Log in to SQD Cloud with the sqd auth CLI command — exchange your account credentials for an API token to authenticate sqd deploy and other commands.

Log in to the Cloud

* [`sqd auth`](#sqd-auth-1)

## `sqd auth`

Log in to the Cloud

```
USAGE
  $ sqd auth -k <value> [--interactive]

FLAGS
  -k, --key=<value>       (required) Cloud auth key. Log in to https://app.subsquid.io to create or update your key.
      --[no-]interactive  Disable interactive mode
```

*See code: [src/commands/auth.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/auth.ts)*


# sqd autocomplete
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/autocomplete

Display sqd CLI shell autocomplete installation instructions — supported shells (bash, zsh, fish, PowerShell) and how to enable command completion.

Display autocomplete installation instructions.


# commands.json
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/commands-json

Define custom sqd subcommands and aliases via commands.json in your Squid SDK project — extend the workflow with reusable scripts for build and deploy.

The `sqd` tool automatically discovers and loads any extra commands defined in the `commands.json` file. Here is a sample file demonstrating the available features:

```json theme={"system"}
{ // comments are ok
  "$schema": "https://subsquid.io/schemas/commands.json",
  "commands": {
    "clean": {
      "description": "delete all build artifacts",
      "cmd": ["rm", "-rf", "lib"]
    },
    "build": {
      "description": "build the project",
      "deps": ["clean"], // commands to execute before
      "cmd": ["tsc"]
    },
    "typegen": {
      "hidden": true, // Don't show in the overview listing
      "workdir": "abi", // change working dir
      "command": [
        "squid-evm-typegen", // node_modules/.bin is in the PATH
        "../src/abi",
        {"glob": "*.json"} // cross-platform glob expansion
      ],
      "env": { // additional environment variables
        "DEBUG": "*"
      }
    }
  }
}
```

This functionality is managed by the [`@subsquid/commands`](https://github.com/subsquid/squid-sdk/tree/master/util/commands) package.

All [squid templates](/en/sdk/squid-sdk/how-to-start/squid-development#templates) include such a file with a predefined set of useful shortcuts. See [Cheatsheet](/en/sdk/squid-sdk/how-to-start/cli-cheatsheet).


# sqd deploy
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/deploy

Deploy new or update an existing squid deployment in the Cloud. Squid name and also optionally slot and/or tag are taken from the provided deployment manifest.

Deploy new or update an existing squid deployment in the Cloud. Squid name and also optionally slot and/or tag are taken from the provided deployment manifest.

* [`sqd deploy SOURCE`](#sqd-deploy-source)

## `sqd deploy SOURCE`

Deploy new or update an existing squid in the Cloud

```
USAGE
  $ sqd deploy SOURCE [--interactive]
    [-r [<org>/]<name>(@<slot>|:<tag>) | -o <code> | -n <name> | [-s <slot>] | [-t <tag>]]
    [-m <manifest_path>] [--hard-reset] [--stream-logs] [--add-tag <value>]
    [--allow-update] [--allow-tag-reassign] [--allow-manifest-override]

ARGUMENTS
  SOURCE  [default: .] Squid source. Could be:
          - a relative or absolute path to a local folder (e.g. ".")
          - a URL to a .tar.gz archive
          - a github URL to a git repo with a branch or commit tag

FLAGS
  -m, --manifest=<manifest_path>  [default: squid.yaml] Specify the relative local path
                                  to a squid manifest file in the squid working directory
      --add-tag=<value>           Add a tag to the deployed squid
      --allow-manifest-override   Allow overriding the manifest during deployment
      --allow-tag-reassign        Allow reassigning an existing tag
      --allow-update              Allow updating an existing squid
      --hard-reset                Perform a hard reset before deploying. This will drop
                                  and re-create all squid resources, including the
                                  database, causing a short API downtime
      --[no-]interactive          Disable interactive mode
      --[no-]stream-logs          Attach and stream squid logs after the deployment

SQUID FLAGS
  -n, --name=<name>                               Name of the squid
  -r, --reference=[<org>/]<name>(@<slot>|:<tag>)  Fully qualified reference of the squid.
                                                  It can include the organization, name,
                                                  slot, or tag
  -s, --slot=<slot>                               Slot of the squid
  -t, --tag=<tag>                                 Tag of the squid

ORG FLAGS
  -o, --org=<code>  Code of the organization

DESCRIPTION
  Deploy new or update an existing squid in the Cloud

EXAMPLES

  // Create a new squid with name provided in the manifest file
  $ sqd deploy .

  // Create a new squid deployment and override it's name to "my-squid-override"
  $ sqd deploy . -n my-squid-override

  // Update the "my-squid" squid with slot "asmzf5"
  $ sqd deploy . -n my-squid -s asmzf5

  // Use a manifest file located in ./path-to-the-squid/squid.prod.yaml
  $ sqd deploy ./path-to-the-squid -m squid.prod.yaml

  // Full paths are also fine
  $ sqd deploy /Users/dev/path-to-the-squid -m /Users/dev/path-to-the-squid/squid.prod.yaml
```

*See code: [src/commands/deploy.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/deploy.ts)*


# sqd gateways
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/gateways

Explore SQD Network gateways and data sources for a squid with the sqd gateways CLI command — list endpoints, statuses, and chains supported.

Explore data sources for a squid

* [`sqd gateways list`](#sqd-gateways-list)

## `sqd gateways list`

List available gateways

```
USAGE
  $ sqd gateways list [--interactive] [-t <evm|substrate>] [-n <regex>] [-c <number>]

FLAGS
  -c, --chain=<number>        Filter by chain ID or SS58 prefix
  -n, --name=<regex>          Filter by network name
  -t, --type=<evm|substrate>  Filter by network type
      --[no-]interactive      Disable interactive mode

ALIASES
  $ sqd gateways ls
```

*See code: [src/commands/gateways/ls.ts](https://github.com/subsquid/squid-cli/blob/master/src/commands/gateways/ls.ts)*


# sqd init
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/init

Initialize a new Squid SDK project from a template or GitHub repo with sqd init — choose EVM, Substrate, Solana, Tron, or Fuel starters and scaffold files.

Setup a new squid project from a template or github repo

* [`sqd init NAME`](#sqd-init-name)

## `sqd init NAME`

Setup a new squid project from a template or github repo

```
USAGE
  $ sqd init NAME [--interactive] [-t <value>] [-d <value>] [-r]

ARGUMENTS
  NAME  The squid name. It must contain only alphanumeric or dash ("-") symbols and must not start with "-".

FLAGS
  -d, --dir=<value>
      The target location for the squid. If omitted, a new folder NAME is created.

  -r, --remove
      Clean up the target directory if it exists

  -t, --template=<value>
      A template for the squid. Accepts:
      - a github repository URL containing a valid squid.yaml manifest in the root folder
      or one of the pre-defined aliases:
      - evm  A minimal squid template for indexing EVM data.
      - abi  A template to auto-generate a squid indexing events and txs from a contract ABI
      - multichain  A template for indexing data from multiple chains
      - gravatar  A sample EVM squid indexing the Gravatar smart contract on Ethereum.
      - substrate  A template squid for indexing Substrate-based chains.
      - ink  A template for indexing Ink! smart contracts
      - ink-abi  A template to auto-generate a squid from an ink! contract ABI
      - frontier-evm  A template for indexing Frontier EVM chains, like Moonbeam and Astar.

  --[no-]interactive
      Disable interactive mode
```

*See code: [src/commands/init.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/init.ts)*


# Installation
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/installation

Install the sqd CLI in your local environment — npm install -g, standalone binaries for macOS/Linux/Windows, and shell autocomplete configuration.

Squid CLI is a command line tool for

* scaffolding new squids from templates
* running SDK tools and scripts defined in `commands.json` in a cross-platform way
* managing squid deployments in [SQD Cloud](/en/cloud) (former Aquarium)

The CLI is distributed as a [`npm` package](https://www.npmjs.com/package/@subsquid/cli).

To install Squid CLI, follow the steps below.

<Steps>
  <Step title="0. Install and setup Squid CLI">
    First, install the latest version of Squid CLI as a global `npm` package:

    ```bash theme={"system"}
    npm i -g @subsquid/cli@latest
    ```

    Check the version:

    ```bash theme={"system"}
    sqd --version
    ```

    Make sure the output looks like `@subsquid/cli@<version>`.

    <Info>
      The next steps are **optional** for building and running squids. A key is required to enable the CLI commands managing the [SQD Cloud](/en/cloud) deployments.
    </Info>
  </Step>

  <Step title="1. Obtain a SQD Cloud deployment key">
    Sign in to [Cloud](https://app.subsquid.io/), and obtain (or refresh) the deployment key page by clicking at the profile picture > "Deployment key":

    <Frame>
      <img alt="SQD Cloud deployment key page" />
    </Frame>
  </Step>

  <Step title="2. Authenticate Squid CLI">
    Open a terminal window and run

    ```bash theme={"system"}
    sqd auth -k <DEPLOYMENT_KEY>
    ```
  </Step>

  <Step title="3. Explore with `--help`">
    Use `sqd --help` to get a list of the available command and `sqd <command> --help` to get help on the available options for a specific command, e.g.

    ```bash theme={"system"}
    sqd deploy --help
    ```
  </Step>
</Steps>


# sqd list
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/list

List squids deployed to SQD Cloud with the sqd ls command — running, stopped, and archived deployments per organization with their slots and tags.

List squids deployed to the Cloud

* [`sqd list`](#sqd-list-1)

## `sqd list`

List squids deployed to the Cloud

```
USAGE
  $ sqd list [--interactive] [--truncate]
    [-r [<org>/]<name>(@<slot>|:<tag>) | -o <code> | -n <name> | [-s <slot>] | [-t <tag>]]

FLAGS
  --[no-]interactive  Disable interactive mode
  --[no-]truncate     Truncate data in columns: false by default

SQUID FLAGS
  -n, --name=<name>                               Name of the squid
  -r, --reference=[<org>/]<name>(@<slot>|:<tag>)  Fully qualified reference
                                                  of the squid. It can include
                                                  the organization, name, slot,
                                                  or tag
  -s, --slot=<slot>                               Slot of the squid
  -t, --tag=<tag>                                 Tag of the squid

ORG FLAGS
  -o, --org=<code>  Code of the organization

ALIASES
  $ sqd ls
```

*See code: [src/commands/ls.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/ls.ts)*


# sqd logs
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/logs

Fetch live and historical logs from a squid deployed to SQD Cloud with the sqd logs command — filter by container, time range, slot, and tag.

Fetch logs from a squid deployed to the Cloud

* [`sqd logs`](#sqd-logs-1)

## `sqd logs`

Fetch logs from a squid deployed to the Cloud

```
USAGE
  $ sqd logs [--interactive] [--since <value>] [--search <value>] [-f | -p <value>]
    [-r [<org>/]<name>(@<slot>|:<tag>) | -o <code> | [-s <slot> -n <name>] | [-t <tag> ]]
    [-c processor|query-node|api|db-migrate|db...]
    [-l error|debug|info|warning...] [--since <value>]

FLAGS
  -c, --container=<option>...  Container name
                               <options: processor|query-node|api|db-migrate|db>
  -f, --follow                 Follow
  -l, --level=<option>...      Log level
                               <options: error|debug|info|warning>
  -p, --pageSize=<value>       [default: 100] Logs page size
      --[no-]interactive       Disable interactive mode
      --search=<value>         Filter by content
      --since=<value>          [default: 1d] Filter by date/interval

SQUID FLAGS
  -n, --name=<name>                               Name of the squid
  -r, --reference=[<org>/]<name>(@<slot>|:<tag>)  Fully qualified reference of the squid.
                                                  It can include the organization, name,
                                                  slot, or tag
  -s, --slot=<slot>                               Slot of the squid
  -t, --tag=<tag>                                 Tag of the squid

ORG FLAGS
  -o, --org=<code>  Code of the organization
```

Notes:

* `--since` accepts the notation of the [`ms` library](https://www.npmjs.com/package/ms): 1d, 10h, 1m.
* With `--level=error` `sqd` will fetch logs emitted by three different calls to the [SDK Logger](/en/sdk/squid-sdk/reference/logger), namely
  * `ctx.log.error`
  * `ctx.log.fatal`
  * `ctx.log.trace`

*See code: [src/commands/logs.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/logs.ts)*


# sqd remove
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/remove

Remove a squid deployment from SQD Cloud with the sqd rm command — delete a specific slot or tag and free up resources from your organization quota.

Remove a squid deployed to the Cloud

* [`sqd remove`](#sqd-remove-1)

## `sqd remove`

Remove a squid deployed to the Cloud

```
USAGE
  $ sqd remove [--interactive] [-f]
    [-r [<org>/]<name>(@<slot>|:<tag>) | -o <code> | [-s <slot> -n <name>] | [-t <tag> ]]

FLAGS
  -f, --force             Does not prompt before removing a squid or its version
      --[no-]interactive  Disable interactive mode

SQUID FLAGS
  -n, --name=<name>                               Name of the squid
  -r, --reference=[<org>/]<name>(@<slot>|:<tag>)  Fully qualified reference of the squid. It can include the organization, name, slot, or tag
  -s, --slot=<slot>                               Slot of the squid
  -t, --tag=<tag>                                 Tag of the squid

ORG FLAGS
  -o, --org=<code>  Code of the organization

ALIASES
  $ sqd rm
```

*See code: [src/commands/rm.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/rm.ts)*


# sqd restart
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/restart

Restart a squid deployment on SQD Cloud with the sqd restart command — recycle the processor container without redeploying source code or migrations.

Restart a squid deployed to the Cloud

* [`sqd restart`](#sqd-restart-1)

## `sqd restart`

Restart a squid deployed to the Cloud

```
USAGE
  $ sqd restart [--interactive]
    [-r [<org>/]<name>(@<slot>|:<tag>) | -o <code> | [-s <slot> -n <name>] | [-t <tag> ]]

FLAGS
  --[no-]interactive  Disable interactive mode

SQUID FLAGS
  -n, --name=<name>                               Name of the squid
  -r, --reference=[<org>/]<name>(@<slot>|:<tag>)  Fully qualified reference of the squid.
                                                  It can include the organization, name,
                                                  slot, or tag
  -s, --slot=<slot>                               Slot of the squid
  -t, --tag=<tag>                                 Tag of the squid

ORG FLAGS
  -o, --org=<code>  Code of the organization
```

*See code: [src/commands/restart.ts](https://github.com/subsquid/squid-cli/blob/master/src/commands/restart.ts)*


# sqd run
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/run

Run a squid locally with the sqd run command — load environment variables and execute the processor and GraphQL server using the deployment manifest.

Run a squid locally according to the [deployment manifest](/en/cloud/reference/manifest).

* [sqd run PATH](#sqd-run-path)

Notes:

* The command is especially useful for running [multichain squids](/en/sdk/squid-sdk/resources/multichain), as it runs all services in the same terminal and handles failures gracefully.

## `sqd run PATH`

Run a squid project locally

```
USAGE
  $ sqd run PATH [--interactive] [-m <value>] [-f <value>] [-i <value>... | -e <value>...] [-r <value>]

FLAGS
  -e, --exclude=<value>...  Do not run specified services
  -f, --envFile=<value>     [default: .env] Relative path to an additional environment file
  -i, --include=<value>...  Run only specified services
  -m, --manifest=<value>    [default: squid.yaml] Relative path to a squid manifest file
  -r, --retries=<value>     [default: 5] Attempts to restart failed or stopped services
      --[no-]interactive    Disable interactive mode
```

*See code: [src/commands/run.ts](https://github.com/subsquid/squid-cli/blob/master/src/commands/run.ts)*


# sqd secrets
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/secrets

Manage SQD Cloud organization secrets with the sqd secrets CLI — set, list, and delete API keys, RPC endpoints, and database credentials.

Manage account secrets

The secrets are [exposed as a context](/en/cloud/resources/env-variables#secrets), and are accessible to all the squids deployed by the current SQD Cloud [organization](/en/cloud/resources/organizations).

* [`sqd secrets list`](#sqd-secrets-list)
* [`sqd secrets remove NAME`](#sqd-secrets-remove-name)
* [`sqd secrets set NAME VALUE`](#sqd-secrets-set-name-value)

## `sqd secrets list`

List organization secrets in the Cloud

```
USAGE
  $ sqd secrets list [--interactive] [-o <value>]

FLAGS
  -o, --org=<value>       Organization
      --[no-]interactive  Disable interactive mode

ALIASES
  $ sqd secrets ls
```

*See code: [src/commands/secrets/ls.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/secrets/ls.ts)*

## `sqd secrets remove NAME`

Delete an organization secret in the Cloud

```
USAGE
  $ sqd secrets remove NAME [--interactive] [-o <value>]

ARGUMENTS
  NAME  The secret name

FLAGS
  -o, --org=<value>       Organization
      --[no-]interactive  Disable interactive mode

ALIASES
  $ sqd secrets rm
```

*See code: [src/commands/secrets/rm.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/secrets/rm.ts)*

## `sqd secrets set NAME VALUE`

Add or update an organization secret in the Cloud. If value is not specified, it is read from standard input. The secret will be exposed as an environment variable with the given name to all the squids in the organization. NOTE: The changes take affect only after a squid is restarted or updated.

```
USAGE
  $ sqd secrets set NAME VALUE [--interactive] [-o <value>]

ARGUMENTS
  NAME   The secret name
  VALUE  The secret value

FLAGS
  -o, --org=<value>       Organization
      --[no-]interactive  Disable interactive mode
```

*See code: [src/commands/secrets/set.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/secrets/set.ts)*


# sqd tags
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/tags

Manage version tags for squids deployed to SQD Cloud with the sqd tags CLI — promote, alias, and roll back releases following the slots-and-tags model.

Manage squid deployements' tags. See the [slots and tags guide](/en/cloud/resources/slots-and-tags).

* [`sqd tags add`](#sqd-tags-add)
* [`sqd tags remove`](#sqd-tags-remove)

## `sqd tags add`

Add a tag to a squid deployment / slot

```
USAGE
  $ sqd tags add TAG [--interactive] [--allow-tag-reassign]
    [-r [<org>/]<name>(@<slot>|:<tag>) | -o <code> | [-s <slot> -n <name>] | [-t <tag> ]]

ARGUMENTS
  TAG  New tag to assign

FLAGS
  --allow-tag-reassign  Allow reassigning an existing tag
  --[no-]interactive    Disable interactive mode

SQUID FLAGS
  -n, --name=<name>                               Name of the squid
  -r, --reference=[<org>/]<name>(@<slot>|:<tag>)  Fully qualified reference of the squid.
                                                  It can include the organization, name,
                                                  slot, or tag
  -s, --slot=<slot>                               Slot of the squid
  -t, --tag=<tag>                                 Tag of the squid

ORG FLAGS
  -o, --org=<code>  Code of the organization
```

*See code: [src/commands/tags/add.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/tags/add.ts)*

## `sqd tags remove`

Remove a tag from a squid deployment / slot

```
USAGE
  $ sqd tags remove TAG [--interactive]
    [-r [<org>/]<name>(@<slot>|:<tag>) | -o <code> | [-s <slot> -n <name>] | [-t <tag> ]]

ARGUMENTS
  TAG  New tag to assign

FLAGS
  --[no-]interactive  Disable interactive mode

SQUID FLAGS
  -n, --name=<name>                               Name of the squid
  -r, --reference=[<org>/]<name>(@<slot>|:<tag>)  Fully qualified reference of the squid.
                                                  It can include the organization, name,
                                                  slot, or tag
  -s, --slot=<slot>                               Slot of the squid
  -t, --tag=<tag>                                 Tag of the squid

ORG FLAGS
  -o, --org=<code>  Code of the organization
```

*See code: [src/commands/tags/remove.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/tags/remove.ts)*


# sqd whoami
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/whoami

Show the SQD Cloud user account currently authenticated with the sqd CLI — display username, organization, and roles for the active session.

Show the user details for the current Cloud account

* [`sqd whoami`](#sqd-whoami-1)

## `sqd whoami`

Show the user details for the current Cloud account

```
USAGE
  $ sqd whoami [--interactive]

FLAGS
  --[no-]interactive  Disable interactive mode
```

*See code: [src/commands/whoami.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/whoami.ts)*


# Processor in action
Source: https://docs.sqd.dev/en/sdk/squid-sdk/tutorials/batch-processor-in-action

Walkthrough of a Squid SDK batch processor in action — see how blocks are batched, fed through handlers, and persisted in a real indexer example.

<Tabs>
  <Tab title="EVM">
    An end-to-end idiomatic usage of `EvmBatchProcessor` can be inspected in the [gravatar template repository](https://github.com/subsquid-labs/gravatar-squid) and also learned from more elaborate [examples](/en/sdk/squid-sdk/examples).

    In order to illustrate the concepts covered in the [development guide](/en/sdk/squid-sdk/how-to-start/squid-development), here we highlight the key steps, put together a processor configuration and a data handling definition.

    <Info>
      **Pre-requisites:** NodeJS, Git, Docker, [Squid CLI](/en/sdk/squid-sdk/squid-cli/installation), any of the [EVM templates](/en/sdk/squid-sdk/how-to-start/squid-development#templates).
    </Info>

    ## 1. Model the target schema and generate entity classes

    Create or edit `schema.graphql` to define the target entities and relations. Consult [the schema reference](/en/sdk/squid-sdk/reference/schema-file).

    Update the entity classes, start a fresh database and regenerate migrations:

    ```bash theme={"system"}
    npx squid-typeorm-codegen
    ```

    ```bash theme={"system"}
    docker compose down
    ```

    ```bash theme={"system"}
    docker compose up -d
    ```

    ```bash theme={"system"}
    rm -r db/migrations
    ```

    ```bash theme={"system"}
    npx squid-typeorm-migration generate
    ```

    Apply the migrations with

    ```bash theme={"system"}
    npx squid-typeorm-migration apply
    ```

    ## 2. Generate Typescript ABI modules

    Use [`evm-typegen`](/en/sdk/squid-sdk/resources/tools/typegen) to generate the facade classes, for example like this:

    ```bash theme={"system"}
    npx squid-evm-typegen src/abi 0x2E645469f354BB4F5c8a05B3b30A929361cf77eC#Gravity --clean
    ```

    ## 3. Configuration

    See [Configuration section](/en/sdk/squid-sdk/reference/processors/evm-batch) for more details.

    ```ts theme={"system"}
    const processor = new EvmBatchProcessor()
      .setGateway('https://v2.archive.subsquid.io/network/ethereum-mainnet')
      .setRpcEndpoint('<my_eth_rpc_url>')
      .setFinalityConfirmation(75)
      .setBlockRange({ from: 6175243 })
      // fetch logs emitted by '0x2E645469f354BB4F5c8a05B3b30A929361cf77eC'
      // matching either `NewGravatar` or `UpdatedGravatar`
      .addLog({
        address: ['0x2E645469f354BB4F5c8a05B3b30A929361cf77eC'],
        topic0: [
          events.NewGravatar.topic,
          events.UpdatedGravatar.topic,
        ]
      })
    ```

    ## 4. Iterate over the batch items and group events

    The following code snippet illustrates a typical data transformation in a batch. The strategy is to

    * Iterate over `ctx.blocks` and `block.logs`
    * Decode each log using a suitable facade class
    * Enrich and transform the data
    * Upsert arrays of entities in batches using `ctx.save()`

    The `processor.run()` method the looks as follows:

    ```ts theme={"system"}
    processor.run(new TypeormDatabase(), async (ctx) => {
      // storing the new/updated entities in
      // an in-memory identity map
      const gravatars: Map<string, Gravatar> = new Map()
      // iterate over the data batch stored in ctx.blocks
      for (const c of ctx.blocks) {
        for (const log of c.logs) {
          // decode the log data
          const { id, owner, displayName, imageUrl } = extractData(log)
          // transform and normalize to match the target entity (Gravatar)
          gravatars.set(id.toHexString(), new Gravatar({
            id: id.toHexString(),
            owner: decodeHex(owner),
            displayName,
            imageUrl
          }))
        }
      }
      // Upsert the entities that were updated.
      // Note that store.save() automatically updates
      // the existing entities and creates new ones.
      // It splits the data into suitable chunks to
      // guarantee an adequate performance.
      await ctx.store.save([...gravatars.values()])
    });
    ```

    In the snippet above, we decode both `NewGravatar` and `UpdatedGravatar` with a single helper function that uses the generated `events` facade class:

    ```ts theme={"system"}
    function extractData(evmLog: any): { id: ethers.BigNumber, owner: string, displayName: string, imageUrl: string} {
      if (evmLog.topics[0] === events.NewGravatar.topic) {
        return events.NewGravatar.decode(evmLog)
      }
      if (evmLog.topics[0] === events.UpdatedGravatar.topic) {
        return events.UpdatedGravatar.decode(evmLog)
      }
      throw new Error('Unsupported topic')
    }
    ```

    ## 5. Run the processor and store the transformed data into the target database

    Run the processor with

    ```bash theme={"system"}
    node -r dotenv/config lib/main.js
    ```

    The script will build the code automatically before running.

    In a separate terminal window, run

    ```bash theme={"system"}
    npx squid-graphql-server
    ```

    Inspect the GraphQL API at [`http://localhost:4350/graphql`](http://localhost:4350/graphql).
  </Tab>

  <Tab title="Substrate">
    An end-to-end idiomatic usage of `SubstrateBatchProcessor` can be inspected in the [squid-substrate-template](https://github.com/subsquid-labs/squid-substrate-template) and also learned from more elaborate [examples](/en/sdk/squid-sdk/examples).

    Here we highlight the key steps and put together a configuration and a data handling definition to illustrate the concepts covered in the [squid development guide](/en/sdk/squid-sdk/how-to-start/squid-development).

    <Info>
      **Pre-requisites:** NodeJS, Git, Docker, [Squid CLI](/en/sdk/squid-sdk/squid-cli/installation), any of the [Substrate templates](/en/sdk/squid-sdk/how-to-start/squid-development#templates).
    </Info>

    ## 1. Set up the processor

    See [Setup section](/en/sdk/squid-sdk/reference/processors/substrate-batch) for more details.

    ```ts title="src/processor.ts" theme={"system"}
    import {
      SubstrateBatchProcessor
    } from '@subsquid/substrate-processor'

    export const processor = new SubstrateBatchProcessor()
      // set the data source
      .setGateway('https://v2.archive.subsquid.io/network/kusama')
      .setRpcEndpoint('https://kusama-rpc.dwellir.com')
      // add data requests
      .addEvent({
        name: ['Balances.Transfer'],
        extrinsic: true
      })
      .addCall({
        name: ['Balances.transfer_keep_alive'],
        extrinsic: true
      })
      // define the data
      // to be fetched for each data item
      .setFields({
        extrinsic: {
          hash: true,
          fee: true
        },
        call: {
          success: true
        },
        block: {
          timestamp: true
        }
      })
    ```

    ## 2. Define a custom data facade and extract normalized data from `BatchContext`

    The following code snippet illustrates how the data is extracted and normalized into some user-specific facade interface `TransferData`. Note how the data items in each of the block data iterables (`events`, `calls`, `extrinsics`) are filtered in the batch handler to make sure they match the data requests.

    Type-safe access and decoding of the call and event data is done with the help of the classes generated by a suitable [typegen tool](/en/sdk/squid-sdk/glossary#typegen), in this case [squid-substrate-typegen](/en/sdk/squid-sdk/resources/tools/typegen).

    ```ts title="src/main.ts" theme={"system"}
    import {Store} from '@subsquid/typeorm-store'
    import * as ss58 from '@subsquid/ss58'

    import {events} from './types'
    import {ProcessorContext} from './processor'

    // some normalized user-define data
    interface TransferData {
      id: string
      timestamp: Date | undefined
      extrinsicHash: string | undefined
      from: string
      to: string
      amount: bigint
      fee?: bigint
    }

    // extract and normalize
    function getTransfers(ctx: ProcessorContext<Store>): TransferData[] {
      let transfers: TransferData[] = []
      for (let block of ctx.blocks) {
        for (let event of block.events) {
          if (event.name==='Balances.Transfer') {
            // decode the event data with runtime-versioned
            // classes generated by typegen
            let rec: {from: string, to: string, amount: bigint}
            if (events.balances.transfer.v1020.is(event)) {
              let [from, to, amount,] =
                events.balances.transfer.v1020.decode(event)
              rec = {from, to, amount}
            } else if (events.balances.transfer.v1050.is(event)) {
              let [from, to, amount] =
                events.balances.transfer.v1050.decode(event)
              rec = {from, to, amount}
            } else {
              rec = events.balances.transfer.v9130.decode(event)
            }
            // extract and normalize the `item.event` data
            let timestamp =
              block.header.timestamp == null ?
              undefined :
              new Date(block.header.timestamp)
            transfers.push({
              id: event.id,
              timestamp,
              extrinsicHash: event.extrinsic?.hash,
              from: ss58.codec('kusama').encode(rec.from),
              to: ss58.codec('kusama').encode(rec.to),
              amount: rec.amount,
              fee: event.extrinsic?.fee || 0n
            })
          }
        }
        for (let call of block.calls) {
          if (call.name==='Balances.transfer_keep_alive') {
            // extranct and normalize the call data
            transfer.push({
              /* some data manipulation on the call data */
            })
          }
        }
      }
      return transfers
    }
    ```

    ## 3. Run the processor and store the transformed data into the target database

    The snippet below assumes that we are using `TypeormDatabase` and that the database schema together with the model entity types has already been prepared. Consult the [Schema file](/en/sdk/squid-sdk/reference/schema-file) section for more details.

    ```ts title="src/main.ts" theme={"system"}
    import {TypeormDatabase} from '@subsquid/typeorm-store'
    // an entity type generated from the schema file
    import {Transfer} from './model'
    import {processor} from './processor'

    // ... `getTransfers(Ctx)` definition

    processor.run(new TypeormDatabase(), async ctx => {
      // get the normalized data from the context
      let transfersData = getTransfers(ctx)

      // an array of entity class instances
      let transfers: Transfer[] = []

      for (let t of transfersData) {
        let {id, blockNumber, timestamp, extrinsicHash, amount, fee} = t

        // join some extra data
        let from = getAccount(accounts, t.from)
        let to = getAccount(accounts, t.to)

        // populate entity class instances
        // with the extracted data
        transfers.push(new Transfer({
          id,
          blockNumber,
          timestamp,
          extrinsicHash,
          from,
          to,
          amount,
          fee
        }))
      }

      // persist an array of entities
      // in a single statement
      await ctx.store.insert(transfers)
    })
    ```
  </Tab>
</Tabs>


# Step 4: Optimization
Source: https://docs.sqd.dev/en/sdk/squid-sdk/tutorials/bayc/step-four-optimizations

BAYC indexer tutorial step 4: optimization — sync faster while tracking dynamic NFT metadata using Squid SDK batch handler tuning techniques.

This is the fourth part of the tutorial where we build a squid that indexes [Bored Ape Yacht Club](https://boredapeyachtclub.com) NFTs, their transfers, and owners from the [Ethereum blockchain](https://ethereum.org), fetches the metadata from [IPFS](https://ipfs.tech/) and regular HTTP URLs, stores all the data in a database, and serves it over a GraphQL API. In the first three parts ([1](/en/sdk/squid-sdk/tutorials/bayc/step-one-indexing-transfers), [2](/en/sdk/squid-sdk/tutorials/bayc/step-two-deriving-owners-and-tokens), [3](/en/sdk/squid-sdk/tutorials/bayc/step-three-adding-external-data)), we created a squid that does all the above but performs many IO operations sequentially, resulting in a long sync time. In this part, we discuss strategies for mitigating that shortcoming. We also discuss an alternative metadata fetching strategy that reduces redundant fetches and handles the changes in metadata of "cold" (i.e., not involved in any transfers) tokens more effectively.

Pre-requisites: Node.js, [Squid CLI](/en/sdk/squid-sdk/squid-cli/installation), Docker, a project folder with the code from the third part ([this commit](https://github.com/subsquid-labs/bayc-squid-1/tree/ab5f094ae34e8822dfb912f6e6116df2cfa800b5)).

## Using Multicall for aggregating state queries

We begin by introducing [batch processing](/en/sdk/squid-sdk/resources/batch-processing) wherever possible, and our first step is to replace individual contract state queries with [batch calls](/en/sdk/squid-sdk/resources/tools/typegen/state-queries#batch-state-queries) to a [MakerDAO multicall contract](https://github.com/mds1/multicall). Retrieve the multicall contract ABI by re-running `squid-evm-typegen` with `--multicall` option:

```bash theme={"system"}
npx squid-evm-typegen --multicall src/abi 0xbc4ca0eda7647a8ab7c2061c2e118a18a936f13d#bayc
```

This adds a Typescript ABI interface at `src/abi/multicall.ts`. Let us use it in a rewrite of `completeTokens()`:

```typescript title="src/main.ts" theme={"system"}
import {Multicall} from './abi/multicall'

const MULTICALL_ADDRESS = '0xeefba1e63905ef1d7acba5a8513c70307c1ce441'
const MULTICALL_BATCH_SIZE = 100

// ...

async function completeTokens(
    ctx: Context,
    partialTokensMap: Map<string, PartialToken>
): Promise<Map<string, Token>> {

    let partialTokens: PartialToken[] = [...partialTokensMap.values()]

    let tokens: Map<string, Token> = new Map()
    if (partialTokens.length === 0) return tokens

    let lastBatchBlockHeader = ctx.blocks[ctx.blocks.length-1].header
    let contract = new Multicall(ctx, lastBatchBlockHeader, MULTICALL_ADDRESS)

    let tokenURIs = await contract.aggregate(
        bayc.functions.tokenURI,
        CONTRACT_ADDRESS,
        partialTokens.map(t => [t.tokenId]),
        MULTICALL_BATCH_SIZE // paginating to avoid RPC timeouts
    )

    for (let [i, ptoken] of partialTokens.entries()) {
        let uri = tokenURIs[i]
        let metadata: TokenMetadata | undefined = await fetchTokenMetadata(ctx, uri)
        tokens.set(ptoken.id, new Token({
            ...ptoken,
            uri,
            ...metadata
        }))
    }

    return tokens
}

// ...
```

Here we replaced the direct calls to `tokenURI()` of the BAYC token contract with aggreaged batches of 100 state calls and called `aggregate()` of the multicall contract for each batch. `Multicall.aggregate()` takes care of splitting the set of state calls into chunks and merging the results together. The number 100 for the batch size was chosen to be as large as possible while not triggering any response size limits on the public RPC endpoint we use. With a private RPC endpoint one would try to further increase the batch size.

Note that we used the old Multicall V1 contract that was deployed before the BAYC contract. This is required because we're making queries to the historical state, and some of them will be made at block heights just a few hundred blocks above the BAYC contract deployment. Those queries would fail we used a multicall contract that did not yet exist back then.

You can find the full code after this optimization at [this commit](https://github.com/subsquid-labs/bayc-squid-1/tree/fca1918bf9f307071fe867916f989627bfc73893). In our test this optimization reduced the time required for retrieving the contract state for all 10000 tokens once from 114 minutes to 94 seconds (figures out of date).

## Retrieving metadata from HTTPS concurrently

Next, we make our metadata retrieval requests concurrent.

Many metadata URIs point to the same HTTPS server or to IPFS that we are accessing through a single gateway. When retrieving data from just a few servers, it is a common courtesy to not send all the requests at once; often, this is enforced by rate limits. Our code takes that into account, limiting the rate of the outgoing requests.

We implement HTTPS batching at `src/metadata.ts`:

```typescript theme={"system"}
import { asyncSleep, splitIntoBatches } from './util'

const MAX_REQ_SEC = 10

export async function fetchTokenMetadatasConcurrently(
    ctx: Context,
    uris: string[]
): Promise<(TokenMetadata | undefined)[]> {

    let metadatas: (TokenMetadata | undefined)[] = []
    for (let batch of splitIntoBatches(uris, MAX_REQ_SEC)) {
        let m = await Promise.all(batch.map((uri, index) => {
            // spread out the requests evenly within a second interval
            let sleepMs = Math.ceil(1000*(index+1)/MAX_REQ_SEC)
            return asyncSleep(sleepMs).then(() => fetchTokenMetadata(ctx, uri))
        }))
        metadatas.push(...m)
    }
    return metadatas
}
```

then we call the function from [`completeTokens()`](https://github.com/subsquid-labs/bayc-squid-1/blob/9e9c93859632f193f30aca0ff1362f408f493e62/src/main.ts#L110) and use its output to populate metadata fields of `Token` entity instances. Utility functions `asyncSleep()` and `splitIntoBathches()` are implemented [here](https://github.com/subsquid-labs/bayc-squid-1/blob/9e9c93859632f193f30aca0ff1362f408f493e62/src/util.ts). Full code is available at [this commit](https://github.com/subsquid-labs/bayc-squid-1/tree/9e9c93859632f193f30aca0ff1362f408f493e62).

On the first processor batch, when we retrieve metadata from `us-central1-bayc-metadata.cloudfunctions.net` for all 10000 tokens, this optimization reduced the required time from 45 to 21 minutes (figures out of date).

## Retrieving immutable metadata once

Another glaring inefficiency in our current code is the fact that we often retrieve metadata from each URI more than once. This is understandable when we work with HTTPS links that point to mutable data; however, the data IPFS links point to is immutable, so there is no need to retrieve it more than once.

To avoid these repeated retrievals we find the already known tokens in the database. If a token metadata URI is an immutable IPFS link (that is, does not [point to MFS](https://gateway.ipfs.io/ipfs/QmTkzDwWqPbnAh5YiV5VwcTLnGdwSNsNTn2aDxdXBFca7D/example#/ipfs/QmThrNbvLj7afQZhxH72m5Nn1qiVn3eMKWFYV49Zp2mv9B/ipns/readme.md)) and metadata is already available, we skip the retrieval. The implementation is at `src/metadata.ts`:

```typescript theme={"system"}
import { Token } from './model'
import { In } from 'typeorm'

export async function selectivelyUpdateMetadata(
    ctx: Context,
    tokens: Map<string, Token>
): Promise<Map<string, Token>> {

    let knownTokens: Map<string, Token> = await ctx.store.findBy(
            Token,
            {id: In([...tokens.keys()])}
        )
        .then(ts => new Map(ts.map(t => [t.id, t])))

    let updatedTokens: Map<string, Token> = new Map()
    let tokensToBeUpdated: Token[] = []
    for (let [id, t] of tokens) {
        let ktoken: Token | undefined = knownTokens.get(id)
        if (ktoken != null &&
            ktoken.image != null && ktoken.attributes != null &&
            ktoken.uri === t.uri && uriPointsToImmutable(t.uri)) {

            ctx.log.info(`Repeated retrieval from ${t.uri} skipped`)
            updatedTokens.set(id, ktoken)
        }
        else {
            ctx.log.info(`Re-retrieving from ${t.uri}`)
            tokensToBeUpdated.push(t)
        }
    }

    let metadatas: (TokenMetadata | undefined)[] = await fetchTokenMetadatasConcurrently(
        ctx,
        tokensToBeUpdated.map(t => t.uri)
    )

    for (let [i, t] of tokensToBeUpdated.entries()) {
        let m = metadatas[i]
        if (m != null) {
            t.image = m.image
            t.attributes = m.attributes
        }
        updatedTokens.set(t.id, t)
    }

    return updatedTokens
}
```

`uriPointsToImmutable()` used to classify metadata URIs rejects any non-IPFS links and IPFS links that may point to MFS:

```typescript theme={"system"}
export function uriPointsToImmutable(uri: string): boolean {
    return uri.startsWith('ipfs://') && !uri.includes('ipns')
}
```

Full code is available at [this commit](https://github.com/subsquid-labs/bayc-squid-1/tree/714aee44d9eb9f43902ae55bf41cedad0384000e). In our test this optimization reduced the total sync time from about 4.1 to 1.5 hours (figures out of date).

## Alternative: Post-sync retrieval of metadata

Despite all the optimizations, our squid still takes 1.5 hours to sync instead of 11 minutes it needed [before we introduced metadata retrieval](https://github.com/subsquid-labs/bayc-squid-1/tree/3568f70b7ac1802fbbf80079e435a3348b0d3a62) (figures out of date). This is the cost of maintaining a fully populated database at all stages of the sync, which is rarely a requirement. An alternative is to begin retrieving metadata only after the rest of the data has been fully synced, and the squid has caught up with the blockchain. We do that by reading `Token` entities from the database after the initial sync, retrieving their metadata and persisting them.

This approach has another advantage: metadata URIs can change without notice, and our old retrieval strategy would only pick up the changes when the token has been transferred. If our goal is to keep token metadata as up-to-date as possible, we have to constantly renew it even for tokens that are not involved in any recent transfers. This is easy to implement with our new metadata retrieval strategy: simply add a field to the `Token` entity that tracks the block height of the most recent metadata update and select `Token`s that were updated some fixed number of blocks ago for metadata updates.

Here is how the new metadata retrieval strategy reflects in the batch handler code:

```diff theme={"system"}
processor.run(new TypeormDatabase(), async (ctx) => {
     let rawTransfers: RawTransfer[] = getRawTransfers(ctx)
 
     let owners: Map<string, Owner> = createOwners(rawTransfers)
     let tokens: Map<string, Token> = await createTokens(ctx, rawTransfers, owners)
     let transfers: Transfer[] = createTransfers(rawTransfers, owners, tokens)
 
     await ctx.store.upsert([...owners.values()])
     await ctx.store.upsert([...tokens.values()])
     await ctx.store.insert(transfers)
+
+    if (ctx.isHead) {
+        let updatedTokens = await updateTokensWithOutdatedMetadata(ctx)
+        await ctx.store.upsert(updatedTokens)
+        ctx.log.info(`Updated metadata for ${updatedTokens.length} tokens`)
+    }
 })
```

Full implementation requires changes to the schema, replacement of `completeTokens()` with `updateTokensWithOutdatedMetadata()` and rewrites of `createTokens()` and `selectivelyUpdateMetadata()`. It is available in [this branch](https://github.com/subsquid-labs/bayc-squid-1/tree/lazy-metadata). The resulting squid took 10 minutes for the initial sync, then 50 minutes more to retrieve metadata at least once for every token (figures out of date).

## Extra: Using Multicall for metadata exploration

In part 3 of this tutorial we [explored metadata URIs](/en/sdk/squid-sdk/tutorials/bayc/step-three-adding-external-data#exploring-token-metadata) by running `tokenURI()` directly on the BAYC token contract. This process took several hours. Replacing the exploration code with its \[multicall]-based equivalent, we can reduce that time to about 17 minutes (figure out of date). Starting with the code [as it was at the end of part two](https://github.com/subsquid-labs/bayc-squid-1/tree/3568f70b7ac1802fbbf80079e435a3348b0d3a62), get `src/abi/multicall.ts` by running

```bash theme={"system"}
npx squid-evm-typegen --multicall src/abi 0xbc4ca0eda7647a8ab7c2061c2e118a18a936f13d#bayc
```

then add the code for batch URI retrieval to the batch handler:

```diff title="src/main.ts" theme={"system"}
+import {Multicall} from './abi/multicall'

+const MULTICALL_ADDRESS = '0xeefba1e63905ef1d7acba5a8513c70307c1ce441'
+const MUTLTICALL_BATCH_SIZE = 100
 
 processor.run(new TypeormDatabase(), async (ctx) => {
     let tokens: Map<string, Token> = createTokens(rawTransfers, owners)
     let transfers: Transfer[] = createTransfers(rawTransfers, owners, tokens)
 
+    let lastBatchBlockHeader = ctx.blocks[ctx.blocks.length-1].header
+    let contract = new Multicall(ctx, lastBatchBlockHeader, MULTICALL_ADDRESS)
+    let tokenURIs = await contract.aggregate(
+        bayc.functions.tokenURI,
+        CONTRACT_ADDRESS,
+        [...tokens.values()].map(t => [t.tokenId]),
+        MUTLTICALL_BATCH_SIZE
+    )
+    for (let uri of tokenURIs) {
+        ctx.log.info(`Retrieved a metadata URI: ${uri}`)
+    }
+
     await ctx.store.upsert([...owners.values()])
     await ctx.store.upsert([...tokens.values()])
     await ctx.store.insert(transfers)
})
```


# Step 1: Transfer events
Source: https://docs.sqd.dev/en/sdk/squid-sdk/tutorials/bayc/step-one-indexing-transfers

BAYC indexer tutorial step 1: index Transfer events from the BAYC ERC-721 contract on Ethereum using Squid SDK and the EvmBatchProcessor.

# Step 1: Indexing Transfer events

In this step-by-step tutorial we will build a squid that gets data about [Bored Ape Yacht Club](https://boredapeyachtclub.com) NFTs, their transfers and owners from the [Ethereum blockchain](https://ethereum.org), indexes the NFT metadata from [IPFS](https://ipfs.tech/) and regular HTTP URLs, stores all the data in a database and serves it over a GraphQL API. Here we do the first step: build a squid that indexes only the `Transfer` events emitted by the [BAYC token contract](https://etherscan.io/address/0xbc4ca0eda7647a8ab7c2061c2e118a18a936f13d).

Pre-requisites: Node.js, [Squid CLI](/en/sdk/squid-sdk/squid-cli/installation), Docker.

## Starting with a template

Begin by retrieving the `evm` template and installing the dependencies:

```bash theme={"system"}
sqd init my-bayc-squid -t evm
cd my-bayc-squid
npm i
```

The resulting code can be found at [this commit](https://github.com/subsquid-labs/bayc-squid-1/tree/4ad7bdfed0771aa6b6e9cf0368ac7f15f09cd629).

## Interfacing with the contract ABI

First, we inspect which data is available for indexing. For EVM contracts, the metadata descrbing the shape of the smart contract logs, transactions and contract state methods is distributed as an [Application Binary Interface](https://docs.soliditylang.org/en/latest/abi-spec.html) (ABI) JSON file. For many popular contracts ABI files are published on Etherscan (as in the case of the BAYC NFT contract). SQD provides a [tool](/en/sdk/squid-sdk/resources/tools/typegen/state-queries) for retrieving contract ABIs from Etherscan-like APIs and generating the boilerplate for retrieving and decoding the data. For the contract of interest, this can be done with

```bash theme={"system"}
npx squid-evm-typegen src/abi 0xbc4ca0eda7647a8ab7c2061c2e118a18a936f13d#bayc
```

Here, `src/abi` is the destination folder and the `bayc` suffix sets the base name for the generated file.

Checking out the generated `src/abi/bayc.ts` file we see all events and contract functions listed in the ABI. Among them there is the `Transfer` event:

```typescript theme={"system"}
export const events = {
    ...
    Transfer: new LogEvent<([from: string, to: string, tokenId: bigint] & {from: string, to: string, tokenId: bigint})>(
        abi, '0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef'
    ),
}
```

Reading about [elsewhere](https://eips.ethereum.org/EIPS/eip-721) we learn that it is emitted every time an NFT changes hand and that its [logs](https://medium.com/mycrypto/understanding-event-logs-on-the-ethereum-blockchain-f4ae7ba50378) contain the addresses of both involved parties, as well as the unique ID of the token. This is the data we need, so we proceed to configure our squid to retrieve it.

## Configuring the data filters

A "squid processor" is the Node.js process and the [object that powers it](/en/sdk/squid-sdk/overview). Together they are responsible for retrieving filtered blockchain data from a specialized data lake ([SQD Network](/en/network/overview)), transforming it and saving the result to a destination of choice. To configure the processor (object) to retrieve the `Transfer` events of the BAYC token contract, we initialize it like this:

```typescript title="src/processor.ts" theme={"system"}
// ...
import * as bayc from './abi/bayc'

export const CONTRACT_ADDRESS = '0xbc4ca0eda7647a8ab7c2061c2e118a18a936f13d'

export const processor = new EvmBatchProcessor()
    .setGateway('https://v2.archive.subsquid.io/network/ethereum-mainnet')
    .setRpcEndpoint({
        url: '<my_eth_rpc_url>',
        rateLimit: 10
    })
    .setFinalityConfirmation(75)
    .setBlockRange({
        from: 12_287_507,
    })
    .addLog({
        address: [CONTRACT_ADDRESS],
        topic0: [bayc.events.Transfer.topic]
    })
    .setFields({
        log: {
            transactionHash: true
        }
    })
// ...
```

Here,

* `'https://v2.archive.subsquid.io/network/ethereum-mainnet'` is the URL the public SQD Network gateway for Ethereum mainnet. Check out the exhaustive [SQD Network gateways list](/en/data/evm).
* `'<eth_rpc_endpoint_url>'` is a public RPC endpoint we chose to use in this example. When an endpoint is available, the processor will begin ingesting data from it once it reaches the highest block available within SQD Network. Please use a private endpoint or SQD Cloud's [RPC addon](/en/cloud/resources/rpc-proxy) in production.
* `setFinalityConfirmation(75)` call instructs the processor to consider blocks final after 75 confirmations when ingesting data from an RPC endpoint.
* `12_287_507` is the block at which the BAYC token contract was deployed. Can be found on the [contract's Etherscan page](https://etherscan.io/address/0xbc4ca0eda7647a8ab7c2061c2e118a18a936f13d).
* The argument of [`addLog()`](/en/sdk/squid-sdk/reference/processors/evm-batch/logs) is a set of filters that tells the processor to retrieve all event logs emitted by the BAYC contract with topic0 matching the hash of the full signature of the `Transfer` event. The hash is taken from the previously generated Typescript ABI.
* The argument of [`setFields()`](/en/sdk/squid-sdk/reference/processors/evm-batch/field-selection) specifies the exact data we need on every event to be retrieved. In addition to the data that is [provided by default](/en/sdk/squid-sdk/reference/processors/evm-batch/field-selection#logs) we are requesting hashes of parent transactions for all event logs.

See [configuration](/en/sdk/squid-sdk/reference/processors/evm-batch) for more options.

## Decoding the event data

The other part of processor configuration is the callback function used to process batches of the filtered data, the [batch handler](/en/sdk/squid-sdk/reference/processors/architecture#processorrun). It is typically defined at the `processor.run()` call at `src/main.ts`, like this:

```typescript theme={"system"}
processor.run(db, async (ctx) => {
    // data transformation and persistence code here
})
```

Here,

* `db` is a [`Database` implementation](/en/sdk/squid-sdk/resources/persisting-data/overview) specific to the target data sink. We want to store the data in a PostgreSQL database and present with a GraphQL API, so we provide a [`TypeormDatabase`](/en/sdk/squid-sdk/resources/persisting-data/typeorm) object here.
* `ctx` is a [batch context](/en/sdk/squid-sdk/reference/processors/architecture#batch-context) object that exposes a batch of data retrieved from SQD Network or a RPC endpoint (at `ctx.blocks`) and any data persistence facilities derived from `db` (at `ctx.store`).

Batch handler is where the raw on-chain data is decoded, transformed and persisted. This is the part we'll be concerned with for the rest of the tutorial.

We begin by defining a batch handler decoding the `Transfer` event:

```typescript title="src/main.ts" theme={"system"}
import {TypeormDatabase} from '@subsquid/typeorm-store'
import {processor, CONTRACT_ADDRESS} from './processor'
import * as bayc from './abi/bayc'

processor.run(new TypeormDatabase(), async (ctx) => {
    for (let block of ctx.blocks) {
        for (let log of block.logs) {
            if (log.address === CONTRACT_ADDRESS && log.topics[0] === bayc.events.Transfer.topic) {
                let {from, to, tokenId} = bayc.events.Transfer.decode(log)
                ctx.log.info(`Parsed a Transfer of token ${tokenId} from ${from} to ${to}`)
            }
        }
    }
})
```

This goes through all the log items in the batch, verifies that they indeed are `Transfer` events emitted by the BAYC contract and decodes the data of each log item, then logs the results to the terminal. The verification step is required because the processor does not guarantee that it won't supply any extra data, only that it *will* supply the data matching the filters. The decoding is done with the `bayc.events.Transfer.decode()` function from the Typescript ABI we previously generated.

At this point the squid is ready for its first test run. Execute

```bash theme={"system"}
docker compose up -d
npm run build
npx squid-typeorm-migration apply
node -r dotenv/config lib/main.js
```

and you should see lots of lines like these in the output:

```
03:56:02 INFO  sqd:processor Parsed a Transfer of token 6325 from 0x0000000000000000000000000000000000000000 to 0xb136c6A1Eb83d0b4B8e4574F28e622A57F8EF01A
03:56:02 INFO  sqd:processor Parsed a Transfer of token 6326 from 0x0000000000000000000000000000000000000000 to 0xb136c6A1Eb83d0b4B8e4574F28e622A57F8EF01A
03:56:02 INFO  sqd:processor Parsed a Transfer of token 4407 from 0x082C99d47E020a00C95460D50a83338d509A0e3a to 0x5cf0E6da6Ec2bd7165edcD52D3d31f2528dCf007
```

The full code can be found at [this commit](https://github.com/subsquid-labs/bayc-squid-1/tree/b7ccfc06b283a885878cddcb18483c4bf571e97c).

## Extending and persisting the data

`TypeormDatabase` requires us to define a TypeORM data model to actually send the data to the database. In SQD, the same data model is also used by the GraphQL server to generate the API schema. To avoid any potential discrepancies, processor and GraphQL server rely on a shared data model description defined at `schema.graphql` in a GraphQL schema dialect fully documented [here](/en/sdk/squid-sdk/reference/schema-file).

TypeORM code is generated from `schema.graphql` with the [`squid-typeorm-codegen`](/en/sdk/squid-sdk/resources/persisting-data/typeorm) tool and must be regenerated every time the schema is changed. This is usually accompanied by regenerating the [database migrations](/en/sdk/squid-sdk/resources/persisting-data/typeorm) and recreating the database itself. The migrations are applied before every run of the processor, ensuring that whenever any TypeORM code within the processor attempts to access the database, the database is in a state that allows it to succeed.

The main unit of data in `schema.graphql` is [entity](/en/sdk/squid-sdk/reference/schema-file/entities). These map onto [TypeORM entites](https://typeorm.io/entities) that in turn map onto database tables. We define one for `Transfer` events by replacing the file contents with

```graphql title="schema.graphql" theme={"system"}
type Transfer @entity {
    id: ID!
    tokenId: BigInt! @index
    from: String! @index
    to: String! @index
    timestamp: DateTime!
    blockNumber: Int!
    txHash: String! @index
}
```

Here,

* The `id` is a required field and the `ID` type is an alias for `String`. Entity IDs must be unique.
* `@index` decorators tell the codegen tool that the corresponding database columns should be indexed.
* Extra fields `timestamp` and `blockNumber` are added to make the resulting GraphQL API more convenient. We will fill them using the block metadata available in `ctx`.

Once we're done editing the schema, we regenerate the TypeORM code, recreate the database and regenerate the migrations:

```bash theme={"system"}
npx squid-typeorm-codegen
npm run build
docker compose down
docker compose up -d
rm -r db/migrations
npx squid-typeorm-migration generate
```

The generated code is in `src/model`. We can now import a `Transfer` entity class from there and use it to perform [various operations](/en/sdk/squid-sdk/resources/persisting-data/typeorm) on the corresponding database table. Let us rewrite our batch handler to save the parsed `Transfer` events data to the database:

```typescript title="src/main.ts" theme={"system"}
import {TypeormDatabase} from '@subsquid/typeorm-store'
import {processor, CONTRACT_ADDRESS} from './processor'
import * as bayc from './abi/bayc'
import {Transfer} from './model'

processor.run(new TypeormDatabase(), async (ctx) => {
    let transfers: Transfer[] = []

    for (let block of ctx.blocks) {
        for (let log of block.logs) {
            if (log.address === CONTRACT_ADDRESS && log.topics[0] === bayc.events.Transfer.topic) {
                let {from, to, tokenId} = bayc.events.Transfer.decode(log)
                transfers.push(new Transfer({
                    id: log.id,
                    tokenId,
                    from,
                    to,
                    timestamp: new Date(block.header.timestamp),
                    blockNumber: block.header.height,
                    txHash: log.transactionHash,
                }))
            }
        }
    }

    await ctx.store.insert(transfers)
})
```

Note a few things here:

* A unique event log ID is available at `log.id` - no need to generate your own!
* `tokenId` returned from the decoder is an `ethers.BigNumber`, so it has to be explicitly converted to `number`. The conversion is valid only because we know that BAYC NFT IDs run from 0 to 9999; in most cases we would use `BigInt` for the entity field type and convert with `tokenId.toBigInt()`.
* `block.header` contains block metadata that we use to fill the extra fields.
* Accumulating the `Transfer` entity instances before using `ctx.store.insert()` on the whole array of them in the end allows us to get away with just one database transaction per batch. This is [crucial for achieving a good syncing performance](/en/sdk/squid-sdk/resources/batch-processing).

At this point we have a squid that indexes the data on BAYC token transfers and is capable of serving it over a GraphQL API. Full code is available at [this commit](https://github.com/subsquid-labs/bayc-squid-1/tree/aeb6268168385cc605ce04fe09d0159f708efe47). Test it by running

```bash theme={"system"}
npm run build
npx squid-typeorm-migration apply
node -r dotenv/config lib/main.js
```

then

```bash theme={"system"}
npx squid-graphql-server
```

in a separate terminal. If all is well, a GraphiQL playground should become available at [localhost:4350/graphql](http://localhost:4350/graphql):


# Step 3: External data
Source: https://docs.sqd.dev/en/sdk/squid-sdk/tutorials/bayc/step-three-adding-external-data

BAYC indexer tutorial step 3: enrich on-chain data with state calls, IPFS metadata, and HTTP requests using Squid SDK and the @subsquid packages.

# Step 3: Adding external data

This is the third part of the tutorial in which we build a squid that indexes [Bored Ape Yacht Club](https://boredapeyachtclub.com) NFTs, their transfers and owners from the [Ethereum blockchain](https://ethereum.org), fetches the metadata from [IPFS](https://ipfs.tech/) and regular HTTP URLs, stores all the data in a database and serves it over a GraphQL API. In the first two parts of the tutorial ([1](/en/sdk/squid-sdk/tutorials/bayc/step-one-indexing-transfers), [2](/en/sdk/squid-sdk/tutorials/bayc/step-two-deriving-owners-and-tokens)) we created a squid that scraped `Transfer` events emitted by the [BAYC token contract](https://etherscan.io/address/0xbc4ca0eda7647a8ab7c2061c2e118a18a936f13d) and derived some information on tokens and their owners from that data. In this part we enrich token data with information obtained from contract state calls, IPFS and regular HTTP URLs.

Prerequisites: Node.js, [Squid CLI](/en/sdk/squid-sdk/squid-cli/installation), Docker, a project folder with the code from the second part ([this commit](https://github.com/subsquid-labs/bayc-squid-1/tree/3568f70b7ac1802fbbf80079e435a3348b0d3a62)).

## Exploring token metadata

Now that we have a record for each BAYC NFT, let's explore how we can retrieve more data for each token.

[EIP-721](https://eips.ethereum.org/EIPS/eip-721) suggests that token metadata contracts may make token data available in a JSON referred to by the output of the `tokenURI()` contract function. Upon examining `src/abi/bayc.ts`, we find that the BAYC token contract implements this function. Also, the public ABI has no obvious contract methods that may set token URI or events that may be emitted on its change. In other words, it appears that the only way to retrieve this data is by [querying the contract state](/en/sdk/squid-sdk/resources/tools/typegen/state-queries).

This requires a RPC endpoint of an archival Ethereum node, but we do not need to add one here: processor will reuse the endpoint we [supplied in part one](/en/sdk/squid-sdk/tutorials/bayc/step-one-indexing-transfers#configuring-the-data-filters) of the tutorial for use in [RPC ingestion](/en/sdk/squid-sdk/resources/unfinalized-blocks).

The next step is to prepare for retrieving and parsing the metadata proper. For this, we need to understand the protocols used in the URIs and the structure of metadata JSONs. To learn that, we retrieve and inspect some URIs ahead of the main squid sync. The most straightforward way to achieve this is by adding the following to the batch handler:

```diff title="src/main.ts" theme={"system"}
processor.run(new TypeormDatabase(), async (ctx) => {
     let tokens: Map<string, Token> = createTokens(rawTransfers, owners)
     let transfers: Transfer[] = createTransfers(rawTransfers, owners, tokens)
+
+    let lastBatchBlockHeader = ctx.blocks[ctx.blocks.length-1].header
+    let contract = new bayc.Contract(ctx, lastBatchBlockHeader, CONTRACT_ADDRESS)
+    for (let t of tokens.values()) {
+        const uri = await contract.tokenURI(t.tokenId)
+        ctx.log.info(`Token ${t.id} has metadata at "${uri}"`)
+    }

     await ctx.store.upsert([...owners.values()])
     await ctx.store.upsert([...tokens.values()])
     await ctx.store.insert(transfers)
})
```

Here, we utilize an instance of the `Contract` class provided by the `src/abi/bayc.ts` module. It uses an RPC endpoint supplied by the processor via `ctx` to call methods of contract `CONTRACT_ADDRESS` at the height corresponding to the last block of each batch. Once we have the `Contract` instance, we call `tokenURI()` for each token mentioned in the batch and print the retrieved URI.

This simple approach is rather slow: the modified squid needs about three to eight hours to get a reasonably sized sample of URIs (figure out of date). A faster yet more complex [alternative](/en/sdk/squid-sdk/tutorials/bayc/step-four-optimizations#extra-using-multicall-for-metadata-exploration) will be discussed in the next part of the tutorial.

Running the modified squid reveals that some metadata URIs point to HTTPS and some point to IPFS. Here is [one](<https://us-central1-bayc-metadata.cloudfunctions.net/api/tokens/ 0>) of the metadata JSONs:

```json theme={"system"}
{
    "image": "https://ipfs.io/ipfs/QmRRPWG96cmgTn2qSzjwr2qvfNEuhunv6FNeMFGa9bx6mQ",
    "attributes": [
        {
            "trait_type": "Fur",
            "value": "Robot"
        },
        {
            "trait_type": "Eyes",
            "value": "X Eyes"
        },
        {
            "trait_type": "Background",
            "value": "Orange"
        },
        {
            "trait_type": "Earring",
            "value": "Silver Hoop"
        },
        {
            "trait_type": "Mouth",
            "value": "Discomfort"
        },
        {
            "trait_type": "Clothes",
            "value": "Striped Tee"
        }
    ]
}
```

Note how it does not conform to the [ERC721 Metadata JSON Schema](https://eips.ethereum.org/EIPS/eip-721).

Summary of our findings:

* BAYC metadata URIs can point to HTTPS or IPFS -- we need to be able to retrieve both.
* Metadata JSONs have two fields: `"image"`, a string, and `"attributes"`, an array of pairs `{"trait_type": string, "value": string}`.

Once finished, roll back the exploratory code:

```diff theme={"system"}
processor.run(new TypeormDatabase(), async (ctx) => {
     let tokens: Map<string, Token> = createTokens(rawTransfers, owners)
     let transfers: Transfer[] = createTransfers(rawTransfers, owners, tokens)
-
-    let lastBatchBlockHeader = ctx.blocks[ctx.blocks.length-1].header
-    let contract = new bayc.Contract(ctx, lastBatchBlockHeader, CONTRACT_ADDRESS)
-    for (let t of tokens.values()) {
-        const uri = await contract.tokenURI(t.tokenId)
-        ctx.log.info(`Token ${t.id} has metadata at "${uri}"`)
-    }
```

## Extending the `Token` entity

We will save both `image` and `attributes` metadata fields and the metadata URI to the database. To do this, we need to add some new fields to the existing `Token` entity:

```diff theme={"system"}
 type Token @entity {
     id: ID! # string form of tokenId
     tokenId: Int!
     owner: Owner!
+    uri: String!
+    image: String
+    attributes: [Attribute!]
     transfers: [Transfer!]! @derivedFrom(field: "token")
 }
+
+type Attribute {
+    traitType: String!
+    value: String!
+}
```

Here, `Attribute` is a [non-entity type](/en/sdk/squid-sdk/reference/schema-file/unions-and-typed-json#typed-json) that we use to type the `attributes` field.

Once `schema.graphql` is updated, we regenerate the TypeORM data model code::

```bash theme={"system"}
npx squid-typeorm-codegen
```

To populate the new fields, let us add an extra step at the end of `createTokens()`:

```typescript theme={"system"}
async function createTokens(
    ctx: Context,
    rawTransfers: RawTransfer[],
    owners: Map<string, Owner>
): Promise<Map<string, Token>> {

    let tokens: Map<string, PartialToken> = new Map()
    for (let t of rawTransfers) {
        let tokenIdString = `${t.tokenId}`
        let ptoken: PartialToken = {
            id: tokenIdString,
            tokenId: t.tokenId,
            owner: owners.get(t.to)!
        }
        tokens.set(tokenIdString, ptoken)
    }
    return await completeTokens(ctx, tokens)
}

interface PartialToken {
    id: string
    tokenId: bigint
    owner: Owner
}
```

Here, `PartialToken` stores the incomplete `Token` information obtained purely from blockchain events and function calls, before any [state queries](/en/sdk/squid-sdk/resources/tools/typegen/state-queries) or enrichment with [external data](/en/sdk/squid-sdk/resources/external-api).
The function `completeTokens()` is responsible for filling `Token` fields that are missing in `PartialToken`s. This involves IO operations, so both the function and its caller `createTokens()` have to be asynchronous. The functions also require a batch context for state queries and logging. We modify the `createTokens()` call in the batch handler to accommodate these changes:

```diff theme={"system"}
 processor.run(new TypeormDatabase(), async (ctx) => {
     let rawTransfers: RawTransfer[] = getRawTransfers(ctx)
 
     let owners: Map<string, Owner> = createOwners(rawTransfers)
-    let tokens: Map<string, Token> = createTokens(rawTransfers, owners)
+    let tokens: Map<string, Token> = await createTokens(ctx, rawTransfers, owners)
     let transfers: Transfer[] = createTransfers(rawTransfers, owners, tokens)
```

Next, we implement `completeTokens()`:

```typescript theme={"system"}
async function completeTokens(
    ctx: Context,
    partialTokens: Map<string, PartialToken>
): Promise<Map<string, Token>> {

    let tokens: Map<string, Token> = new Map()
    if (partialTokens.size === 0) return tokens

    let lastBatchBlockHeader = ctx.blocks[ctx.blocks.length-1].header
    let contract = new bayc.Contract(ctx, lastBatchBlockHeader, CONTRACT_ADDRESS)

    for (let [id, ptoken] of partialTokens) {
        let uri = await contract.tokenURI(ptoken.tokenId)
        ctx.log.info(`Retrieved metadata URI ${uri}`)
        let metadata: TokenMetadata | undefined = await fetchTokenMetadata(ctx, uri)
        tokens.set(id, new Token({
            ...ptoken,
            uri,
            ...metadata
        }))
    }

    return tokens
}
```

URI retrieval here is similar to what we did in the exploration step: we create a `Contract` object and use it to call the `tokenURI()` method of the BAYC token contract. The retrieved URIs are then used by the `fetchTokenMetadata()` function, which is responsible for HTTPS/IPFS metadata retrieval and parsing. Once we have its output, we can create and return the final `Token` entity instances.

## Retrieving external resources

In the `fetchTokenMetadata()` implementation we first classify the URIs depending on the protocol. For IPFS links we replace `'ipfs://'` with an address of an IPFS gateway, then retrieve the metadata from all links using a regular HTTPS client. Here for the demonstration purposes we use the public [ipfs.io](https://ipfs.io) gateway, which is slow and prone to dropping requests due to rate-limiting. For production squids we recommend using a dedicated gateway, e.g. from [Filebase](https://filebase.com/dedicated-gateways/).

```typescript theme={"system"}
export async function fetchTokenMetadata(
  ctx: Context,
  uri: string
): Promise<TokenMetadata | undefined> {

    try {
        if (uri.startsWith('ipfs://')) {
            const gatewayURL = path.posix.join(IPFS_GATEWAY, ipfsRegExp.exec(uri)![1])
            let res = await client.get(gatewayURL)
            ctx.log.info(`Successfully fetched metadata from ${gatewayURL}`)
            return res.data
        } else if (uri.startsWith('http://') || uri.startsWith('https://')) {
            let res = await client.get(uri)
            ctx.log.info(`Successfully fetched metadata from ${uri}`)
            return res.data
        } else {
            ctx.log.warn(`Unexpected metadata URL protocol: ${uri}`)
            return undefined
        }
    } catch (e) {
        throw new Error(`Failed to fetch metadata at ${uri}. Error: ${e}`)
    }
}

const ipfsRegExp = /^ipfs:\/\/(.+)$/
```

We use [Axios](https://axios-http.com) for HTTPS retrieval. Install it with

```bash theme={"system"}
npm i axios
```

To avoid reinitializing the HTTPS client every time we call the function we bind it to a module-scope constant:

```typescript theme={"system"}
const client = axios.create({
    headers: {'Content-Type': 'application/json'},
    httpsAgent: new https.Agent({keepAlive: true}),
    transformResponse(res: string): TokenMetadata {
        let data: {image: string; attributes: {trait_type: string; value: string}[]} = JSON.parse(res)
        return {
            image: data.image,
            attributes: data.attributes.map((a) => new Attribute({traitType: a.trait_type, value: a.value})),
        }
    },
})
```

We move all the code related to metadata retrieval to a separate module `src/metadata.ts`. Examine its full contents [here](https://github.com/subsquid-labs/bayc-squid-1/blob/ab5f094ae34e8822dfb912f6e6116df2cfa800b5/src/metadata.ts).

Then all that is left is to import the relevant parts in `src/main.ts`:

```diff title="src/main.ts" theme={"system"}
+import {TokenMetadata, fetchTokenMetadata} from './metadata'
```

and we are done with the processor code for this part of the tutorial. Full squid code at this point is available at [this commit](https://github.com/subsquid-labs/bayc-squid-1/tree/ab5f094ae34e8822dfb912f6e6116df2cfa800b5).

Recreate the database, rebuild the code and refresh the migrations with

```bash theme={"system"}
docker compose down
docker compose up -d
npm run build
rm -r db/migrations
npx squid-typeorm-migration generate
npx squid-typeorm-migration apply
```

and test the processor by running

```bash theme={"system"}
node -r dotenv/config lib/main.js
```

It runs much slower than before, requiring about three hours to get through the first batch and more than a day to sync (figures out of date). This is something we will address in the next part of the tutorial.

Nevertheless, the squid is already fully capable of scraping token metadata and serving it over GraphQL. Verify that by running `npx squid-graphql-server` and visiting the [local GraphiQL playground](http://localhost:4350/graphql). It is now possible to retrieve image URLs and attributes for each token:


# Step 2: Owners & tokens
Source: https://docs.sqd.dev/en/sdk/squid-sdk/tutorials/bayc/step-two-deriving-owners-and-tokens

BAYC indexer tutorial step 2: derive Owner and Token entities from BAYC Transfer events with Squid SDK and persist them in Postgres via TypeORM.

# Step 2: Deriving owners and tokens

This is the second part of the tutorial in which we build a squid that indexes [Bored Ape Yacht Club](https://boredapeyachtclub.com) NFTs, their transfers, and owners from the [Ethereum blockchain](https://ethereum.org), fetches the metadata from [IPFS](https://ipfs.tech/) and regular HTTP URLs, stores all the data in a database, and serves it over a GraphQL API. In the [first part](/en/sdk/squid-sdk/tutorials/bayc/step-one-indexing-transfers) we created a simple squid that scraped Transfer events emitted by the [BAYC token contract](https://etherscan.io/address/0xbc4ca0eda7647a8ab7c2061c2e118a18a936f13d). Here, we go a step further and derive separate entities for the NFTs and their owners from the transfers. The new entities will reference the corresponding `Transfer` entities. It will be automatically translated into primary key-foreign key references in the new database schema, and enable efficient cross-entity GraphQL queries.

Prerequisites: Node.js, [Squid CLI](/en/sdk/squid-sdk/squid-cli/installation), Docker, a project folder with the code from the first part ([this commit](https://github.com/subsquid-labs/bayc-squid-1/tree/aeb6268168385cc605ce04fe09d0159f708efe47)).

## Writing `schema.graphql`

Start the process by adding new [entities](/en/sdk/squid-sdk/reference/schema-file/entities) to the `schema.graphql` file:

```graphql theme={"system"}
# any unique string can be used as id
type Owner @entity {
    id: ID! # owner address
}

type Token @entity {
    id: ID! # string form of tokenId
    tokenId: BigInt!
}
```

Next, add [entity relations](/en/sdk/squid-sdk/reference/schema-file/entity-relations). Let us begin with adding a simple relation linking tokens to their owners:

```diff theme={"system"}
 type Token @entity {
     id: ID! # string form of tokenId
     tokenId: BigInt!
+    owner: Owner!
 }
```

Now, `Token` is considered an *owning entity* in relation to `Owner`. As a result,

* On the database side: the `token` table that maps to the `Token` entity gains a foreign key column `owner_id` holding primary keys of the `owner` table. The column is automatically indexed - no need to add `@index`.
* On the Typeorm side: the `Token` entity gains an `owner` field decorated with [`@ManyToOne`](https://github.com/typeorm/typeorm/blob/master/docs/many-to-one-one-to-many-relations.md). To create a well-formed `Token` entity instance in processor code, we now will have to first get a hold of an appropriate `Owner` entity instance and populate the `owner` field of a new `Token` with a reference to it:
  ```typescript theme={"system"}
  let newOwner: Owner = new Owner({
      id: '0xd8dA6BF26964aF9D7eEd9e03E53415D37aA96045'
  })
  let newToken: Token = new Token({
      id: '1',
      tokenId: 1,
      owner: newOwner // the whole entity instance
  })
  ```
* On the GraphQL side: queries to `token` can now select `owner` and any of its subfields (`id` is the only one available now).

Introduce more entity relations by replacing the `from`, `to` and `tokenId` fields of the `Transfer` entity with fields from the new entity types:

```diff theme={"system"}
 type Transfer @entity {
     id: ID!
-    tokenId: BigInt! @index
-    from: String! @index
-    to: String! @index
+    token: Token!
+    from: Owner!
+    to: Owner!
     timestamp: DateTime!
     blockNumber: Int!
     txHash: String! @index
 }
```

Lastly, include the virtual (i.e., not mapped to a column in the database schema) [reverse lookup fields](/en/sdk/squid-sdk/reference/schema-file/entity-relations):

```diff theme={"system"}
 type Owner @entity {
     id: ID! # owner address
+    ownedTokens: [Token!]! @derivedFrom(field: "owner")
 }

 type Token @entity {
     id: ID! # string form of tokenId
     tokenId: BigInt!
+    transfers: [Transfer!]! @derivedFrom(field: "token")
 }
```

This addition doesn't create any new database columns, but it makes the `ownedTokens` and `transfers` fields accessible through GraphQL and Typeorm.

You can find the final version of schema.graphql [here](https://github.com/subsquid-labs/bayc-squid-1/blob/6a148b1345f6c26acc2678e10b60aadcd4bbbcee/schema.graphql). Once you're finished, regenerate the Typeorm entity code with the following command:

```bash theme={"system"}
npx squid-typeorm-codegen
```

We also need to regenerate the database migrations to match the new schema. However, we'll postpone this step for now: it requires recompiling the squid code and it is not possible until we fix the entity creation code.

## Creating the entities

Note how the entities we define form an acyclic dependency graph:

* `Owner` entity instances can be made straight from the raw events data;
* `Token`s require the raw data plus the `Owner`s;
* `Transfer` entities require all the above.

As a consequence, the creation of entity instances must proceed in a [particular order](https://en.wikipedia.org/wiki/Topological_sorting). Squids usually use small graphs like this one, and in these the order can be easily found manually (e.g. `Owner`s then `Token`s then `Transfer`s in this case). We will assume that it can be hardcoded by the programmer.

Further, at each step we will [process the data for the whole batch](/en/sdk/squid-sdk/resources/batch-processing) instead of handling the items individually. This is crucial for achieving a good syncing performance.

With all that in mind, let's create a batch processor that generates and persists all of our entities:

```typescript title="src/main.ts" theme={"system"}
import {Owner, Token} from './model'

processor.run(new TypeormDatabase(), async (ctx) => {
    let rawTransfers: RawTransfer[] = getRawTransfers(ctx)

    let owners: Map<string, Owner> = createOwners(rawTransfers)
    let tokens: Map<string, Token> = createTokens(rawTransfers, owners)
    let transfers: Transfer[] = createTransfers(rawTransfers, owners, tokens)

    await ctx.store.upsert([...owners.values()])
    await ctx.store.upsert([...tokens.values()])
    await ctx.store.insert(transfers)
})
```

where

```typescript theme={"system"}
interface RawTransfer {
    id: string
    tokenId: bigint
    from: string
    to: string
    timestamp: Date
    blockNumber: number
    txHash: string
}
```

is an interface very similar to that of the `Transfer` entity as it was at the beginning of this part of the tutorial. This allows us to reuse most of the code of the old batch handler in `getRawTransfers()`:

```typescript title="src/main.ts" theme={"system"}
import {Context} from './processor'

function getRawTransfers(ctx: Context): RawTransfer[] {
    let transfers: RawTransfer[] = []

    for (let block of ctx.blocks) {
        for (let log of block.logs) {
            if (log.address === CONTRACT_ADDRESS && log.topics[0] === bayc.events.Transfer.topic) {
                let {from, to, tokenId} = bayc.events.Transfer.decode(log)
                transfers.push({
                    id: log.id,
                    tokenId,
                    from,
                    to,
                    timestamp: new Date(block.header.timestamp),
                    blockNumber: block.header.height,
                    txHash: log.transactionHash,
                })
            }
        }
    }

    return transfers
}
```

The next step involves creating `Owner` entity instances. We will need these to create both `Tokens` and `Transfers`. In both scenarios, we'll have the IDs of the owners (i.e., their addresses) prepared. To simplify future lookups, we choose to return the `Owner` instances as a `Map<string, Owner>`:

```typescript title="src/main.ts" theme={"system"}
function createOwners(rawTransfers: RawTransfer[]): Map<string, Owner> {
    let owners: Map<string, Owner> = new Map()
    for (let t of rawTransfers) {
        owners.set(t.from, new Owner({id: t.from}))
        owners.set(t.to, new Owner({id: t.to}))
    }
    return owners
}
```

Similarly, `Token` instances will also need to be looked up later, so we return them as a `Map<string, Token>`. To identify the most recent owner of each token, we traverse all the transfers in the order they appear on the blockchain and assign the owner of any involved tokens to their recipient:

```typescript title="src/main.ts" theme={"system"}
function createTokens(
    rawTransfers: RawTransfer[],
    owners: Map<string, Owner>
): Map<string, Token> {

    let tokens: Map<string, Token> = new Map()
    for (let t of rawTransfers) {
        let tokenIdString = `${t.tokenId}`
        tokens.set(tokenIdString, new Token({
            id: tokenIdString,
            tokenId: t.tokenId,
            owner: owners.get(t.to)
        }))
    }
    return tokens
}
```

Some `Token` and `Owner` instances might have been created in previous batches, so we use `ctx.store.upsert()` to store these instances while updating any existing ones.

<Info>
  In some circumstances we might have had to retrieve the old entity instances from the database before updating, but here we have all the required fields populated, so we simply overwrite the whole entity with `ctx.store.upsert()`.
</Info>

Finally, we create an array of `Transfer` entity instances through a simple mapping:

```typescript title="src/main.ts" theme={"system"}
function createTransfers(
    rawTransfers: RawTransfer[],
    owners: Map<string, Owner>,
    tokens: Map<string, Token>
): Transfer[] {

    return rawTransfers.map(t => new Transfer({
        id: t.id,
        token: tokens.get(`${t.tokenId}`),
        from: owners.get(t.from),
        to: owners.get(t.to),
        timestamp: t.timestamp,
        blockNumber: t.blockNumber,
        txHash: t.txHash
    }))
}
```

Since `Transfer`s are unique, we can safely use `ctx.store.insert()` to persist them.

At this point, the squid has accomplished everything planned for this part of the tutorial. The only remaining task is to drop and recreate the database (if it's running), then regenerate and apply the migrations:

```bash theme={"system"}
npm run build
docker compose down
docker compose up -d
rm -r db/migrations
npx squid-typeorm-migration generate
npx squid-typeorm-migration apply
```

Full code can be found at [this commit](https://github.com/subsquid-labs/bayc-squid-1/tree/3568f70b7ac1802fbbf80079e435a3348b0d3a62).

To test it, start the processor and the GraphQL server by running

```bash theme={"system"}
node -r dotenv/config lib/main.js
```

and

```bash theme={"system"}
npx squid-graphql-server
```

in separate terminals. Then, visit the [GraphiQL playground](http://localhost:4350/graphql):

The new entities should be displayed in the query schema.

Thanks to the added entity relations, we can now execute more complex nested queries. For example, the one displayed in the screenshot selects a transfer, retrieves its token, looks up its owner, and finds out which tokens are currently owned by them.


# Squid SDK Case Studies
Source: https://docs.sqd.dev/en/sdk/squid-sdk/tutorials/case-studies

Deep dives into larger Squid SDK projects — multichain indexers, NFT analytics, governance dashboards, and DeFi data platforms built on SQD.

Follow the links from the [SQD Medium blog](https://medium.com/subsquid) for deep dives into the development process of larger projects built from scratch. They use the FireSquid version of the framework, so much of the involved code is outdated. Still, the general approaches they illustrate endure.

* [DeFi Dashboard](https://medium.com/subsquid/build-your-first-defi-dashboard-ad3ce1e9fc73). A React app for a real-time dashboard showing key statistics of the Moonwell lending protocol
* [Rave name service indexing on Fantom](https://medium.com/subsquid/building-a-fast-and-scalable-web3-api-on-fantom-blockchain-94c79933b55).
* [Indexing Uniswap data into parquets](https://medium.com/subsquid/how-to-scale-blockchain-data-science-for-large-datasets-b49d078c15eb). A step-by-step guide on how to extract, decode and analyze Uniswap trading data with SQD, Pandas and a Python notebook.
* [Analyzing Lens Protocol](https://medium.com/subsquid/how-to-analyze-lens-protocol-activity-data-with-subsquid-e1ee3b7b43fa). An end-to-end tutorial on how to extract and analyze the social data of the Lens protocol. Includes an extra step on how to build a UI for the dashboards.


# Use with Ganache or Hardhat
Source: https://docs.sqd.dev/en/sdk/squid-sdk/tutorials/evm-local

Use Squid SDK with Ganache or Hardhat — index a local Ethereum dev node from your own RPC endpoint without involving the SQD Network or Portal.

# Run squids with Ganache or Hardhat

## Objective

Here we show how it is possible to index an Ethereum development node running locally. This is useful for those who want to start developing a squid ETL or API in a local environment without waiting for the contract to be deployed to a testnet.

## Pre-requisites

* Either one of [Ganache](https://trufflesuite.com/ganache/) or [Hardhat](https://hardhat.org/)
* [Squid CLI](/en/sdk/squid-sdk/squid-cli/installation)
* Docker

## Setup

In this tutorial we will be using a layout in which the squid and the Hardhat/Ganache setup share the same folder. Create a new squid by running:

```bash theme={"system"}
sqd init ethereum-local-indexing -t evm
```

Here, `ethereum-local-indexing` is the project name and `evm` refers to the [`evm` template](https://github.com/subsquid-labs/squid-evm-template).

Install the dependencies and update `evm-processor` and `typeorm-store` to their ArrowSquid versions:

```bash theme={"system"}
cd ethereum-local-indexing
npm i
npm i @subsquid/evm-processor@next @subsquid/typeorm-store@next
```

Next, we set up a local EVM development environment. We consider two common options: [Hardhat](#hardhat) and [Ganache](#ganache).

## If you chose Hardhat

[Install the package](https://hardhat.org/hardhat-runner/docs/getting-started#installation).

#### 1. Create project

In a terminal, navigate to the squid project root folder. Prepare it by running

```bash theme={"system"}
rm README.md
mv tsconfig.json tsconfig.json.save
```

This will prevent collisions between SQD and Hardhat files.

Next, run

```bash theme={"system"}
npx hardhat
```

to initialize the Hardhat project. When prompted, choose **TypeScript project** and keep all other options at defaults. Finally, overwrite `tsconfig.json` provided by Hardhat:

```bash theme={"system"}
mv tsconfig.json.save tsconfig.json
```

#### 2. Configure Hardhat automining

Then, open the `hardhat.config.ts` and replace the `HardhatUserConfig` object with this one:

```typescript theme={"system"}
const config: HardhatUserConfig = {
  defaultNetwork: "hardhat",
  networks: {
    hardhat: {
      chainId: 1337,
      mining: {
        auto: false,
        interval: [4800, 5200]
      }
    },
  },
  solidity: "0.8.17",
};
```

The `mining` configuration will continuously mine blocks, even if no events or transactions are executed. This can be useful for debugging. You can read more about it on [hardhat official documentation](https://hardhat.org/hardhat-network/docs/reference#mining-modes).

#### 3. Sample contract

There should be a `contracts` subfolder in the project folder now, with a sample contract named `Lock.sol`. To compile this contract and verify its correctness, run

```bash theme={"system"}
npx hardhat compile
```

You should find the contract's ABI at `artifacts/contracts/Lock.sol/Lock.json`. **We will use this file in squid development.**

#### 4. Launch hardhat node

From the project root folder, run

```bash theme={"system"}
npx hardhat node
```

#### 5. Deploy the contract

The node will keep the console window busy. In a different terminal, run this command:

```bash theme={"system"}
npx hardhat run scripts/deploy.ts --network localhost
```

If the contract deployed successfully, the output should look like

```
Lock with 1 ETH and unlock timestamp 1704810454 deployed to 0x5FbDB2315678afecb367f032d93F642f64180aa3
```

Take note of the contract address, **you'll need it later**.

## If you chose Ganache

Install [Truffle](https://trufflesuite.com/docs/truffle/how-to/install/) and [Ganache](https://trufflesuite.com/docs/ganache/quickstart/#1-install-ganache) packages.

#### 1. Truffle project, sample contract

Let's create a new truffle project with a sample contract. In the project main folder run

```bash theme={"system"}
truffle unbox metacoin
```

Compile the contracts with

```bash theme={"system"}
truffle compile
```

You should find the contract's ABI at `build/contracts/MetaCoin.json`. **It will be useful for indexing**.

#### 2. Create a workspace

Launch the Ganache tool and select the *New Workspace (Ethereum)* option.

Next, provide a name for the workspace and link the Truffle project we just created to it. To do that, click **Add project** and select the `truffle-config.js` file in the project root folder. Finally, select the `Server` tab at the top.

In this window, change the server configuration to the exact values reported in this image.

The [`AUTOMINE` option](https://trufflesuite.com/docs/ganache/reference/cli-options/#miner) is disabled to ease the debugging.

Finally, click "Start" to launch the blockchain emulator.

#### 3. Deploy a smart contract

Configure Truffle by uncommenting the `development` section in `truffle-config.js` and setting its properties as follows:

```javascript theme={"system"}
    development: {
      host: "127.0.0.1",     // Localhost (default: none)
      port: 8545,            // Standard Ethereum port (default: none)
      network_id: "1337",    // Any network (default: none)
    },
```

Deploy the sample smart contract by running

```bash theme={"system"}
truffle migrate
```

Log information similar to this should be displayed:

```bash theme={"system"}
   Deploying 'MetaCoin'
   --------------------
   > transaction hash:    0xcdb820827306ebad7c6905d750d07536c3db93f4ef76fd777180bdac16eaa2ca
   > Blocks: 1            Seconds: 4
   > contract address:    0xd095211a90268241D75919f12397b389b1062f6F
   > block number:        329
   > block timestamp:     1673277461
   > account:             0x131D37F433BAf649111278c1d6E59843fFB26D28
   > balance:             99.98855876
   > gas used:            414494 (0x6531e)
   > gas price:           20 gwei
   > value sent:          0 ETH
   > total cost:          0.00828988 ETH

   > Saving artifacts
   -------------------------------------
   > Total cost:          0.01144124 ETH
```

Take note of the contract address, **you'll need it later**.

## Squid development

The indexing setup is ready to use. To test it, replace the contents of `src/processor.ts` with the following.

```ts theme={"system"}
import {TypeormDatabase} from '@subsquid/typeorm-store'
import {EvmBatchProcessor} from '@subsquid/evm-processor'

const processor = new EvmBatchProcessor()
  .setRpcEndpoint('http://localhost:8545')
  .setFinalityConfirmation(5)
  .addTransaction({})
  .setFields({
    transaction: {
      contractAddress: true
    }
  })

processor.run(new TypeormDatabase(), async (ctx) => {
  for (let blk of ctx.blocks) {
    for (let txn of blk.transactions) {
      console.log(txn)
    }
  }
})
```

This defines a squid that retrieves all chain transactions from the local node RPC endpoint without filtering, making sure to retrieve the addresses of the deployed contracts for deployment transactions. Run the squid with

```bash theme={"system"}
docker compose up -d
npm run build
npx squid-typeorm-migration apply
node -r dotenv/config lib/main.js
```

You should see the data of one (for Hardhat) or two (for Truffle+Ganache) contract deployment transactions printed to your terminal.

Now you can develop a SQD-based indexer alongside your contracts. Head over to the [dedicated tutorial](/en/sdk/squid-sdk/tutorials/bayc) for guidance on squid development. Use the **contract's ABI** ([here](#3-sample-contract) or [here](#1-truffle-project-sample-contract)) and contract **address** ([here](#5-deploy-the-contract) and [here](#3-deploy-a-smart-contract)) from previous steps and be mindful that the data source of the processor class needs to be set to the local node RPC endpoint, as in the example above:

```typescript theme={"system"}
// ...
  .setRpcEndpoint('http://localhost:8545')
// ...
```

You can also set the data source through an environment variable like it is done in [this complete end-to-end project example](https://medium.com/subsquid/how-to-build-a-performant-and-scalable-full-stack-nft-marketplace-63c12466b959) (outdated, but still useful). This will make sure the project code stays the same and only the environment variables change depending on where the project is deployed.


# Index to local CSV files
Source: https://docs.sqd.dev/en/sdk/squid-sdk/tutorials/file-csv

Tutorial: index blockchain data into local CSV files for analytics with Squid SDK — schema definitions, batch flushing, and post-processing examples.

# Save indexed data to local CSV files

## Objective

This tutorial describes how to use the SQD indexing framework for saving processed blockchain data to local CSV files. The intent is to show how Squid SDK can be used for data analytics prototyping.

File-based data formats like CSV are convenient for data analysis, especially in the early prototyping stages. This convenience motivated SQD Team to develop some extensions to allow saving processed data to file-based storage.

We chose MATIC transactions on Ethereum mainnet for this example. This selection provides enough data to highlight the performance of the SQD framework and is interesting from a data analysis standpoint.

An article about this demo project [has been published on Medium](https://link.medium.com/1NPC1S2czxb). The project source code can be found in the [`local-csv-indexing` repository](https://github.com/subsquid-labs/local-csv-indexing).

<Warning>
  As of 2023-12-17, the `local-csv-indexing` repo is mostly still sound, but already somewhat outdated. You can take a look at the less sophisticated, yet regularly updated example [here](https://github.com/subsquid-labs/file-store-csv-example).
</Warning>

## Pre-requisites

* [Squid CLI](/en/sdk/squid-sdk/squid-cli/installation)
* (optional) Python

## Setup

Let's start by creating a new blockchain indexer, or "squid" in SQD terminology. In a terminal, launch this command:

```bash theme={"system"}
sqd init local-csv-indexing -t evm
```

Here, `local-csv-indexing` is the name of the project, and can be changed to anything else. The `-t evm` option specifies that the [`evm` template](https://github.com/subsquid-labs/squid-evm-template) should be used as a starting point.

<Note>
  The template actually has more than what we need for this project. Unnecessary packages have been removed in the tutorial repository. You can grab [`package.json`](https://github.com/subsquid-labs/local-csv-indexing/blob/main/package.json) from there to do the same.

  Files-wise, `docker-compose.yml`, `schema.graphql` and `squid.yaml` were removed. [`commands.json`](/en/sdk/squid-sdk/squid-cli/commands-json), the list of local `sqd` scripts, has been significantly shortened ([here is the updated version](https://github.com/subsquid-labs/local-csv-indexing/blob/main/commands.json)).
</Note>

### ERC-20 token ABI

To be able to index transfers of a token, it's necessary to know the address and the ABI ([Application Binary Interface](https://docs.ethers.org/v5/api/utils/abi/)) of the token contract. The ABI defines the contract's functions and events, including their typed inputs and outputs.

Luckily, both of these can be found on block explorers like [Etherscan](https://medium.com/r/?url=https%3A%2F%2Fetherscan.io%2Ftoken%2F0x7d1afa7b718fb893db30a3abc0cfc608aacfebb0).

The indexer needs the ABI for locating the contract events or functions in the EVM execution trace and decoding their inputs. Squid SDK has a handy command to generate some boilerplate TypeScript code to achieve this:

```bash theme={"system"}
npx squid-evm-typegen src/abi 0x7D1AfA7B718fb893dB30A3aBc0Cfc608AaCfeBB0#matic
```

<Info>
  The typegen tool uses Etherscan API to fetch the contract ABI. Other compatible APIs are supported via the `--etherscan-api` flag. For example, if the contract was deployed to Polygon and its ABI was available from Polygonscan, it could still be fetched with the same command extended with `--etherscan-api https://api.polygonscan.io`.

  Alternatilvely, the same command can be used with a path (local or URL) to a JSON ABI in place of the contract address.
</Info>

This will generate some files under the `src/abi` folder, the most interesting of which is `matic.ts`.

### CSV, Tables and Databases

For writing local CSVs we will need the `file-store-csv` package:

```bash theme={"system"}
npm i @subsquid/file-store-csv
```

<Info>
  Packages are also available for [writing to parquet files](/en/sdk/squid-sdk/reference/store/file/parquet) and for [uploading to S3-compatible cloud services](/en/sdk/squid-sdk/reference/store/file/s3-dest).
</Info>

Next, let's create a new file at `src/tables.ts`. This is where it's possible to provide filenames for the CSV files, as well as configure their data structure, in much the same way as if they were a database table (the class name is no coincidence):

```typescript theme={"system"}
import {Table, Column, Types} from '@subsquid/file-store-csv'

export const Transfers = new Table(
  'transfers.csv',
  {
    blockNumber: Column(Types.Numeric()),
    timestamp: Column(Types.DateTime()),
    contractAddress: Column(Types.String()),
    from: Column(Types.String()),
    to: Column(Types.String()),
    amount: Column(Types.Numeric()),
  },
  {
    header: false,
  }
)
```

Let's create another file next, this time named `src/db.ts`, to configure the data abstraction layer. Here we export an instance of the [`Database` class](/en/sdk/squid-sdk/resources/persisting-data/overview) implementation from the `file-store` package (a dependency of `file-store-csv`). We will use this instance in much the same way as we would use a [`TypeormDatabase`](/en/sdk/squid-sdk/resources/persisting-data/typeorm) instance in a PostgreSQL-based squid.

```typescript theme={"system"}
import {Database, LocalDest, Store} from '@subsquid/file-store'
import { Transfers } from './tables'

export const db = new Database({
  tables: {
    Transfers,
  },
  dest: new LocalDest('./data'),
  chunkSizeMb: 100
})
```

<Info>
  Note the `chunkSizeMb` option. A new chunk (that is, a new folder with a new CSV file in it) will be written when either the amount of data stored in the processor buffer exceeds `chunkSizeMb`, or at the end of the batch during which [`ctx.store.setForceFlush()`](/en/sdk/squid-sdk/resources/persisting-data/file#setforceflush) was called.
</Info>

### Data indexing

All the indexing logic is defined in `src/processor.ts`, so let's open it and edit the `EvmBatchProcessor` class configuration. We should request data for the right smart contract and the right EVM event log:

```typescript theme={"system"}
export const contractAddress =
  '0x7D1AfA7B718fb893dB30A3aBc0Cfc608AaCfeBB0'.toLowerCase();

const processor = new EvmBatchProcessor()
  .setGateway('https://v2.archive.subsquid.io/network/ethereum-mainnet')
  .setRpcEndpoint({
    url: process.env.RPC_ENDPOINT,
    rateLimit: 10
  })
  .setFinalityConfirmation(75)
  .addLog({
    address: [contractAddress],
    topic0: [events.Transfer.topic]
  });
```

<Warning>
  Note: the RPC\_ENDPOINT environment variable is used, so make sure to edit the `.env` file and use a valid URL of an Ethereum RPC node, e.g.:

  ```bash theme={"system"}
  DB_NAME=squid
  DB_PORT=23798
  GQL_PORT=4350
  # JSON-RPC node endpoint, both wss and https endpoints are accepted
  RPC_ENDPOINT="<eth_rpc_endpoint_url>"
  ```
</Warning>

Let's define the logic to process a batch of EVM log data, and save it to CSV files.

A double loop is necessary to explore all the blocks in each batch and the logs in each block. The processor provides no guarantees that it won't provide any data not matching the filters, so it's necessary to check that the logs have been generated by the right address and that they have the right topic signature.

It's then possible to decode the events and prepare objects with the right data structure, which are then written to the `Transfers` CSV file.
Here is a short summary of the logic:

```typescript theme={"system"}
processor.run(db, async (ctx) => {
  let decimals = 18
  for (let block of ctx.blocks) {
    for (let log of block.logs) {
      if (log.address !== contractAddress) continue;
      if (log.topic0 !== events.Transfer.topic) continue;

      const { from, to, value } = events.Transfer.decode(log);

      ctx.store.Transfers.write(({
        blockNumber: block.header.height,
        timestamp: new Date(block.header.timestamp),
        contractAddress: log.address,
        from: from.toLowerCase(),
        to: to.toLowerCase(),
        amount: BigDecimal(value.toBigInt(), decimals).toNumber(),
      });
    }
  }
});
```

<Info>
  The file in the GitHub repository is slightly different, as there's some added logic to obtain the number of decimals for the token. For that, the processor [interacts with the smart contract deployed on chain](/en/sdk/squid-sdk/resources/tools/typegen/state-queries).
</Info>

### Launch the project

Build, then launch the project with

```bash theme={"system"}
npm run build
```

```bash theme={"system"}
node -r dotenv/config lib/main.js
```

And in a few minutes, a few sub-folders (whose names are the block ranges where the data is coming from) should be created under the `data` directory, each containing a `transfer.csv` file.

### Data analysis with Python

If you want to learn how to analyze this data using Python and Pandas, refer to [the Medium article dedicated to this demo project](https://link.medium.com/1NPC1S2czxb), or consult the [project's repository on GitHub](https://github.com/subsquid-labs/local-csv-indexing/local-csv-indexing).

## Conclusions

The purpose of this project was to demonstrate how to use SQD's indexing framework for data analytics prototyping: the indexer was able to ingest all this data, process it, and dump it to local CSV files in roughly 20 minutes (figure out of date). The simple Python script in the project's repository shows how to read multiple CSV files, and perform some data analysis with Pandas.

SQD Team seeks feedback on this new tool. If you want to share any thoughts or have any suggestions, feel free to reach out to us at [the SquidDevs Telegram channel](https://t.me/HydraDevs).


# Index to Parquet files
Source: https://docs.sqd.dev/en/sdk/squid-sdk/tutorials/file-parquet

Tutorial: index blockchain data into Apache Parquet files for analytics with Squid SDK — columnar schema, compression, and batch flushing examples.

# Save indexed data in Parquet files

## Objective

This tutorial describes how to use the SQD indexing framework to save the processed blockchain data to [Parquet files](https://parquet.apache.org/) instead of a database. The intent is to show how Squid SDK can be used for data analytics, this time with focus on tools suitable for larger datasets.

File-based data formats like [CSV](/en/sdk/squid-sdk/tutorials/file-csv) are convenient for data analysis, especially in the early prototyping stages. However, when working with large datasets the ability to read data files partially is a common requirement. This is rarely possible with CSV.

In contrast, the Parquet format is designed for efficient read/write operations without processing the entire file. To better serve data analysts' needs, the SQD Team developed a library for storing indexer data in this format.

The subject of this tutorial is the Uniswap DApp, namely the data from its pool contracts and positions held by investors. Uniswap was chosen because it generates a very large amount of information, and ultimately this helps to better show how to leverage a more performance-oriented format.

An article about this demo project [has been published on Medium](https://link.medium.com/7gU0BrLbDxb). The project source code can be found in the [`squid-parquet-storage` repo](https://github.com/subsquid-labs/squid-parquet-storage).

<Warning>
  As of 2023-12-17, the `squid-parquet-storage` repo is mostly still sound, but already somewhat outdated. You can take a look at the less sophisticated, yet regularly updated example [here](https://github.com/subsquid-labs/file-store-parquet-example/).
</Warning>

## Pre-requisites

* [Squid CLI](/en/sdk/squid-sdk/squid-cli/installation)
* (optional) Python

## Setup

Let's start by creating a new blockchain indexer, or a "squid" in SQD terminology. In a terminal, launch this command:

```bash theme={"system"}
sqd init local-parquet-indexing -t evm
```

Here, `local-parquet-indexing` is the name of the project, and can be changed to anything else. The `-t evm` option specifies that the [`evm` template](https://github.com/subsquid-labs/squid-evm-template) should be used as a starting point.

<Note>
  The template actually has more than what we need for this project. Unnecessary packages have been removed in the tutorial repository. You can grab [`package.json`](https://github.com/subsquid-labs/squid-parquet-storage/blob/main/package.json) from there to do the same.

  Files-wise, `docker-compose.yml`, `schema.graphql` and `squid.yaml` were removed. [`commands.json`](/en/sdk/squid-sdk/squid-cli/commands-json), the list of local `sqd` scripts, has been significantly shortened ([here is the updated version](https://github.com/subsquid-labs/squid-parquet-storage/blob/main/commands.json)).
</Note>

Finally, make sure to install the dependencies:

```bash theme={"system"}
npm i
```

### ERC-20 token ABI

Due to the size of this project most of the code will be explained rather than directly listed. The code of this section is no exception since this project indexes data from three kinds of Uniswap contracts, one of which is a factory that deploys new contracts. Additionally, it uses a Multicall contract.

For this project, you will need:

* The [Uniswap V3 Factory](https://etherscan.io/address/0x1f98431c8ad98523631ae4a59f267346ea31f984#code) smart contract address and ABI
* The ABI of [any one of the Uniswap V3 Pool](https://etherscan.io/address/0x390a4d096ba2cc450e73b3113f562be949127ceb#code) smart contracts deployed by the Factory
* The [Uniswap V3 Positions](https://etherscan.io/address/0xc36442b4a4522e871399cd717abdd847ab11fe88) smart contract address, and ABI
* The address of any deployed Maker DAO's Multicall smart contract. Luckily, [Uniswap have their own](https://etherscan.io/address/0x5ba1e12693dc8f9c48aad8770482f4739beed696)
* The ABI of an ERC-20 token (can be compiled from [OpenZeppelin repository](https://github.com/OpenZeppelin/openzeppelin-contracts/blob/master/contracts/token/ERC20/ERC20.sol), or downloaded, for example from [here](https://gist.github.com/veox/8800debbf56e24718f9f483e1e40c35c), or from the [repository of this article's project](https://github.com/subsquid-labs/squid-parquet-storage/tree/main/abi)). Save it as `abi/ERC20.json`.

<Note>
  The project also uses [`ERC20NameBytes`](https://github.com/subsquid-labs/squid-parquet-storage/blob/main/abi/ERC20NameBytes.json) and [`ERC20SymbolBytes`](https://github.com/subsquid-labs/squid-parquet-storage/blob/main/abi/ERC20SymbolBytes.json) ABIs, which OpenZeppelin defines in [`IERC20Metadata.sol`](https://github.com/OpenZeppelin/openzeppelin-contracts/blob/master/contracts/token/ERC20/extensions/IERC20Metadata.sol) and includes in [`ERC20.sol`](https://github.com/OpenZeppelin/openzeppelin-contracts/blob/master/contracts/token/ERC20/ERC20.sol). Save these files to `./abi`.
</Note>

<Note>
  You can find a pool and its ABI by inspecting the internal calls of one of the [Create Pool transactions of the Factory contract](https://etherscan.io/tx/0xd9e0fc1d737479e940d9807a535c75a31a2c8458792e2c66e2cc33f19f88e546#internal).
</Note>

Generate TypeScript code for them:

```bash theme={"system"}
npx squid-evm-typegen ./src/abi ./abi/*.json
npx squid-evm-typegen ./src/abi 0x1f98431c8ad98523631ae4a59f267346ea31f984#factory
npx squid-evm-typegen ./src/abi 0xc36442b4a4522e871399cd717abdd847ab11fe88#NonfungiblePositionManager
npx squid-evm-typegen ./src/abi 0x390a4d096ba2cc450e73b3113f562be949127ceb#pool --multicall
```

This should create a few files in the `src/abi` folder for you. The last command also automatically generates a Typescript ABI for the Multicall contract due to the `--multicall` flag.

### Tables and Databases

The `@subsquid/file-store` library defines the `Table` and `Database` classes.

The `Database` class gets its name from the interface that was originally developed to [access an actual database](/en/sdk/squid-sdk/resources/persisting-data/typeorm). Here, the interface is [used without modification](/en/sdk/squid-sdk/resources/persisting-data/overview) in a class designed to access a filesystem. `Table`s play a similar role to that of tables of an actual database: they represent collections of rows, all of which share same set of fields/columns. Each such data structure requires one or more data files to store it in both CSV and Parquet, hence the mapping of `Table`s to files.

To summarize, `Table` instances are used to define data files along with their schemas and hold file-specific settings. `Database` facilitates the interactions with the processor, coordinates writing to the files and maintains any state that facilitates that process (configuration, cloud connections and so on).

There are two main differences from the [CSV tutorial](/en/sdk/squid-sdk/tutorials/file-csv) and the first one is that for this project we will be using a `Table` implementation from `@subsquid/file-store-parquet`. Let's install it:

```bash theme={"system"}
npm i @subsquid/file-store-parquet
```

The other one is that this project is more involved and is, in fact, using ten different tables instead of one.

It's advisable to define these tables in a separate file. The original project has them under `src/tables.ts`. The syntax is pretty much the same as for the CSV tables, except now the `Table`, `Column`, and `Types` classes are imported from the `@subsquid/file-store-parquet` library and the set of available types is different. We will also configure the parquet file compression using the new `Compression` class.

Here's a snippet:

```typescript theme={"system"}
import {Table, Column, Compression, Types} from '@subsquid/file-store-parquet'

export const Tokens = new Table(
    'tokens.parquet',
    {
        blockNumber: Column(Types.Uint32()),
        timestamp: Column(Types.Timestamp()),
        contractAddress: Column(Types.String()),
        symbol: Column(Types.String()),
        name: Column(Types.String()),
        totalSupply: Column(Types.Uint64()),
        decimals: Column(Types.Uint16()),
    },
    {
        compression: Compression.ZSTD,
    }
)
```

The rest of the tables definitions can be found [here](https://github.com/subsquid-labs/squid-parquet-storage/blob/main/src/tables.ts).

Similarly, a `src/db.ts` file should be created to configure the `Database` class. Here we specify the tables used, as well as the destination and the size of the chunks in which the data is going to be split.

Here are the contents of this file in full:

```typescript theme={"system"}
import assert from 'assert'
import {Database, LocalDest, Store} from '@subsquid/file-store'
import {
    FactoryPairCreated,
    PoolBurn,
    PoolInitialize,
    PoolMint,
    PoolSwap,
    PositionCollect,
    PositionDecreaseLiquidity,
    PositionIncreaseLiquidity,
    PositionTransfer,
    Tokens,
} from './tables'
import {PoolsRegistry} from './utils'
import {S3Dest} from '@subsquid/file-store-s3'

type Metadata = {
    height: number
    pools: string[]
}

export const db = new Database({
    tables: {
        Tokens,
        FactoryPairCreated,
        PoolInitialize,
        PoolMint,
        PoolBurn,
        PoolSwap,
        PositionCollect,
        PositionDecreaseLiquidity,
        PositionIncreaseLiquidity,
        PositionTransfer,
    },
    dest: process.env.DEST === 'S3' ? new S3Dest('./uniswap', 'csv-store') : new LocalDest('./data'),
    hooks: {
        async onConnect(dest) {
            if (await dest.exists('status.json')) {
                let {height, pools}: Metadata = await dest.readFile('status.json').then(JSON.parse)
                assert(Number.isSafeInteger(height))

                let registry = PoolsRegistry.getRegistry()
                for (let pool of pools) {
                    registry.add(pool)
                }

                return height
            } else {
                return -1
            }
        },
        async onFlush(dest, range) {
            let metadata: Metadata = {
                height: range.to,
                pools: PoolsRegistry.getRegistry().values(),
            }
            await dest.writeFile('status.json', JSON.stringify(metadata))
        },
    },
    chunkSizeMb: 50,
})

export type Store_ = typeof db extends Database<infer R, any> ? Store<R> : never
```

<Note>
  The `chunkSizeMb` option defines the size (in MB) of a parquet file before it's saved on disk, and a new one is created.
</Note>

### Data indexing

The orchestration of the indexing logic is defined in the file named `src/processor.ts`:

```typescript theme={"system"}
import {EvmBatchProcessor} from '@subsquid/evm-processor'
import * as positionsAbi from './abi/NonfungiblePositionManager'
import * as factoryAbi from './abi/factory'
import * as poolAbi from './abi/pool'
import {db} from './db'
import {processFactory} from './mappings/factory'
import {processPools} from './mappings/pools'
import {FACTORY_ADDRESS, POSITIONS_ADDRESS} from './utils/constants'
import {processPositions} from './mappings/positions'

let processor = new EvmBatchProcessor()
    .setBlockRange({from: 12369621})
    .setGateway('https://v2.archive.subsquid.io/network/ethereum-mainnet')
    .setRpcEndpoint({
        url: process.env.ETH_CHAIN_NODE,
        rateLimit: 10
    })
    .setFinalityConfirmation(75)
    .addLog({
        address: [FACTORY_ADDRESS],
        topic0: [factoryAbi.events.PoolCreated.topic]
    })
    .addLog({
        topic0: [
            poolAbi.events.Burn.topic,
            poolAbi.events.Mint.topic,
            poolAbi.events.Initialize.topic,
            poolAbi.events.Swap.topic,
        ]
    })
    .addLog({
        address: [POSITIONS_ADDRESS],
        topic0: [
            positionsAbi.events.IncreaseLiquidity.topic,
            positionsAbi.events.DecreaseLiquidity.topic,
            positionsAbi.events.Collect.topic,
            positionsAbi.events.Transfer.topic,
        ]
    })

processor.run(db, async (ctx) => {
    await processFactory(ctx)
    await processPools(ctx)
    await processPositions(ctx)
})
```

Here's a brief explanation of the code above:

* The `EvmBatchProcessor` class is instantiated and set to connect to the Ethereum gateway of SQD Network, as well as a blockchain node, requesting data after a certain block (**make sure to add a node URL to `ETH_CHAIN_NODE` variable in the `.env` file**). Real-time consensus data will be considered final after 75 block confirmations / 15 minutes.

* It is also configured to request data for EVM logs generated by the Factory and Positions smart contracts, filtering for certain events (`PoolCreated` and `IncreaseLiquidity`, `DecreaseLiquidity`, `Collect`, `Transfer`, respectively).

* Processor is configured to also request EVM logs from **any address** with topic0 matching one of the signatures of the following Pool smart contract events: `Burn`, `Mint`, `Initialize`, `Swap`. This will guarantee that events generated by all Pool contracts are captured regardless of when the contracts were deployed.

* Finally, the processor is launched and data is processed in batches, by functions defined in `src/mappings`.

For a brief explanation of what [`processFactory`](https://github.com/subsquid-labs/squid-parquet-storage/blob/main/src/mappings/factory.ts), [`processPools`](https://github.com/subsquid-labs/squid-parquet-storage/blob/main/src/mappings/pools.ts) and [`processPositions`](https://github.com/subsquid-labs/squid-parquet-storage/blob/main/src/mappings/positions.ts) do, let's take the `processPositions` functions as an example:

* it needs to "unbundle" the batch of logs received

* for each EVM log found it checks that it belongs to one of the pool addresses generated by the factory

* compares the log's topic0 against the topics of the Events of the position NFT contract

* uses the corresponding Event TypeScript class to decode the log

* writes the decoded information to the corresponding table (parquet file)

To better understand how data is transformed, and how the other functions are defined as well, it's advised to [browse the repository](https://github.com/subsquid-labs/squid-parquet-storage/tree/main/src/mappings) and inspect the code. Be sure to check the [`utils`](https://github.com/subsquid-labs/squid-parquet-storage/tree/main/src/utils) folder as well, as there are some auxiliary files and functions used in the mapping logic.

### Launch the project

When the logic is fully implemented, to launch the project and start indexing, open a terminal and run these three commands:

```bash theme={"system"}
npm run build
```

```bash theme={"system"}
node -r dotenv/config lib/main.js
```

The indexer should be able to catch up with the Ethereum blockchain, and **reach the chain's head in a very short time**.

<Info>
  Bear in mind that this may vary a lot, depending on the Ethereum node used and on your hardware, as well as the connection, or physical distance from the Ethereum node. It took \~45 minutes while testing for this article. A test on a connection with a much higher latency and the same configuration finished indexing in 5 hours (figures out of date).
</Info>

The process will generate a series of sub-folders in the `data` folder, labelled after the block ranges where the data is coming from, and in each one of these folders there should be one `*.parquet` file for each of the *tables* we defined.

### Data analysis with Python

If you want to learn how to analyze this data using Python and Pandas, refer to [the Medium article dedicated to this demo project](https://link.medium.com/7gU0BrLbDxb).

## Conclusions

The purpose of this tutorial was to demonstrate how to use the SQD indexing framework for data analytics on larger datasets. The parquet data format is one of the most successful tools used in this setting, supported by the most common data analysis libraries such as Pandas and Pyarrow. We tested the performance of our parquet format tools on Uniswap, indexing the data from its pools and positions held by investors.

The project described here was able to index the entirety of Uniswap Pool events, across all the pools created by the Factory contract, as well as the Positions held by investors, in less than an hour (\~45 minutes) (figure out of date).

<Note>
  Indexing time may vary, depending on factors, such as the Ethereum node used, on the hardware, and quality of the connection.
</Note>

The simple Python script in the project's repository shows how to read multiple Parquet files, and perform some data analysis with Pandas.

SQD Team seeks feedback on this new tool. If you want to share any thoughts or have any suggestions, feel free to reach out to us at [the SquidDevs Telegram channel](https://t.me/HydraDevs).


# Frontier EVM-indexing squid
Source: https://docs.sqd.dev/en/sdk/squid-sdk/tutorials/frontier-evm

Build a Frontier-EVM indexing squid on Astar — Squid SDK tutorial for tracking NFTs on a Substrate chain with the Frontier EVM compatibility layer.

# Frontier EVM-indexing Squid

## Objective

The goal of this tutorial is to guide you through creating a simple blockchain indexer ("squid") using Squid SDK. The squid will be indexing the data from two contracts ([AstarDegens](https://blockscout.com/astar/address/0xd59fC6Bfd9732AB19b03664a45dC29B8421BDA9a) and [AstarCats](https://blockscout.com/astar/address/0x8b5d62f396Ca3C6cF19803234685e693733f9779)) deployed on the [Astar network](https://astar.network/). The objective will be to track ownership and transfers of all NFTs issued by these contracts.

A somewhat outdated version of the final result can be browsed [here](https://github.com/subsquid/squid-astar-example/tree/astar-degens).

## Pre-requisites

* Familiarity with Git
* A properly set up [development environment](/en/sdk/squid-sdk/how-to-start/development-environment-set-up) consisting of Node.js, Git and Docker
* [Squid CLI](/en/sdk/squid-sdk/squid-cli/installation)

## Scaffold using `sqd init`

We will start with the [`frontier-evm` squid template](https://github.com/subsquid-labs/squid-frontier-evm-template/) available through [`sqd init`](/en/sdk/squid-sdk/squid-cli/init). It is built to index EVM smart contracts deployed on Astar/Shiden, but it is also capable of indexing Substrate events. To retrieve the template and install the dependencies, run

```bash theme={"system"}
sqd init astar-evm-tutorial --template frontier-evm
cd astar-tutorial
npm i
```

## Define Entity Schema

Next, we ensure that the data [schema](/en/sdk/squid-sdk/reference/schema-file) of the squid defines [entities](/en/sdk/squid-sdk/reference/schema-file/entities) that we would like to track. We are interested in:

* Token transfers
* Ownership of tokens
* Contracts and their minted tokens

Here is a schema that defines the exact entities we need:

```graphql title="schema.graphql" theme={"system"}
type Token @entity {
  id: ID!
  owner: Owner
  uri: String
  transfers: [Transfer!]! @derivedFrom(field: "token")
  contract: Contract
}

type Owner @entity {
  id: ID!
  ownedTokens: [Token!]! @derivedFrom(field: "owner")
  balance: BigInt
}

type Contract @entity {
  id: ID!
  name: String
  symbol: String
  totalSupply: BigInt
  mintedTokens: [Token!]! @derivedFrom(field: "contract")
}

type Transfer @entity {
  id: ID!
  token: Token!
  from: Owner
  to: Owner
  timestamp: Int!
  block: Int!
}
```

It's worth noting a couple of things in this [schema definition](/en/sdk/squid-sdk/reference/schema-file):

* **`@entity`**: Signals that this type will be translated into an ORM model that is going to be persisted in the database.
* **`@derivedFrom`**: Signals that the field will not be persisted in the database. Instead, it will be [derived from](/en/sdk/squid-sdk/reference/schema-file/entity-relations) the entity relations.
* **type references** (e.g. `from: Owner`): When used on entity types, they establish a relation between two entities.

TypeScript entity classes have to be regenerated whenever the schema is changed, and to do that we use the `squid-typeorm-codegen` tool:

```bash theme={"system"}
npx squid-typeorm-codegen
```

The (re)generated entity classes can then be browsed at `src/model/generated`.

## ABI Definition and Wrapper

SQD maintains [tools](/en/sdk/squid-sdk/resources/tools/typegen/state-queries) for automated generation of TypeScript classes for handling EVM logs and transactions based on a [JSON ABI](https://docs.ethers.io/v5/api/utils/abi/) of the contract.

For our squid we will need such a module for the [ERC-721](https://eips.ethereum.org/EIPS/eip-721)-compliant part of the contracts' interfaces. Once again, the template repository already includes it, but it is still important to explain what needs to be done in case one wants to index a different type of contract.

Place any ABIs you requre for interfacing your contracts at `./abi` and run

```bash theme={"system"}
npx squid-evm-typegen ./src/abi ./abi/*.json --multicall
```

The results will be stored at `src/abi`. One module will be generated for each ABI file, and it will include constants useful for filtering and functions for decoding EVM events and functions defined in the ABI.

## Processor object and the batch handler

Squid SDK provides users with the [`SubstrateBatchProcessor` class](/en/sdk/squid-sdk). Its instances connect to [SQD Network](/en/network/overview) gateways at chain-specific URLs, to get chain data and apply custom transformations. The indexing begins at the starting block and keeps up with new blocks after reaching the tip.

`SubstrateBatchProcessor`s [expose methods](/en/sdk/squid-sdk/reference/processors/substrate-batch) that "subscribe" them to specific data such as Substrate events and calls. There are also [specialized methods](/en/sdk/squid-sdk/resources/substrate/frontier-evm) for subscribing to EVM logs and transactions by address. The actual data processing is then started by calling the `.run()` function. This will start generating requests to the SQD Network gateway for [*batches*](/en/sdk/squid-sdk/resources/batch-processing) of data specified in the configuration, and will trigger the callback function, or *batch handler* (passed to `.run()` as second argument) every time a batch is returned by the gateway.

It is in this callback function that all the mapping logic is expressed. This is where chain data decoding should be implemented, and where the code to save processed data on the database should be defined.

### Managing the EVM contract

Before we begin defining the mapping logic of the squid, we are going to write a `src/contracts.ts` utility module for managing the involved EVM contracts. It will export:

* Addresses of [astarDegens](https://blockscout.com/astar/address/0xd59fC6Bfd9732AB19b03664a45dC29B8421BDA9a) and [astarCats](https://blockscout.com/astar/address/0x8b5d62f396Ca3C6cF19803234685e693733f9779) contracts.
* A `Map` from the contract addresses to hardcoded `Contract` [entity](/en/sdk/squid-sdk/reference/schema-file/entities) instances.

Here are the full file contents:

```ts title="src/contracts.ts" theme={"system"}
import { Store } from '@subsquid/typeorm-store'
import { Contract } from './model'

export const astarDegensAddress = '0xd59fC6Bfd9732AB19b03664a45dC29B8421BDA9a'.toLowerCase();
export const astarCatsAddress = '0x8b5d62f396Ca3C6cF19803234685e693733f9779'.toLowerCase();

export const contractMapping: Map<string, Contract> = new Map()

contractMapping.set(astarDegensAddress, new Contract({
  id: astarDegensAddress,
  name: 'AstarDegens',
  symbol: 'DEGEN',
  totalSupply: 10000n,
  mintedTokens: []
}))

contractMapping.set(astarCatsAddress, new Contract({
  id: astarCatsAddress,
  name: 'AstarCats',
  symbol: 'CAT',
  totalSupply: 7777n,
  mintedTokens: []
})
```

## Create the processor object

The `src/processor.ts` file is where squids instantiate and configure their processor objects. We will use an instance of [`SubstrateBatchProcessor`](/en/sdk/squid-sdk).

We adapt the template code to handle two contracts instead of one and point the processor data source setting to the `astar` [SQD Network gateway URL](/en/data/evm). Here is the end result:

```ts title="src/processor.ts" theme={"system"}
import {assertNotNull} from '@subsquid/util-internal'
import {
    BlockHeader,
    DataHandlerContext,
    SubstrateBatchProcessor,
    SubstrateBatchProcessorFields,
    Event as _Event,
    Call as _Call,
    Extrinsic as _Extrinsic
} from '@subsquid/substrate-processor'
import * as erc721 from './abi/erc721'

import {astarDegensAddress, astarCatsAddress} from './contracts'

const processor = new SubstrateBatchProcessor()
  .setBlockRange({ from: 442693 })
  .setGateway('https://v2.archive.subsquid.io/network/astar-substrate')
  .setRpcEndpoint({
    url: assertNotNull(process.env.RPC_ENDPOINT),
    rateLimit: 10,
  })
  .addEvmLog({
    address: [astarDegensAddress],
    range: { from: 442693 },
    topic0: [erc721.events.Transfer.topic]
  })
  .addEvmLog({
    address: [astarCatsAddress],
    range: { from: 800854 },
    topic0: [erc721.events.Transfer.topic]
  })

export type Fields = SubstrateBatchProcessorFields<typeof processor>
export type Block = BlockHeader<Fields>
export type Event = _Event<Fields>
export type Call = _Call<Fields>
export type Extrinsic = _Extrinsic<Fields>
export type ProcessorContext<Store> = DataHandlerContext<Store, Fields>
```

<Warning>
  This code expects to find an URL of a working Astar RPC endpoint in the `RPC_ENDPOINT` environment variable. Set it in the `.env` file and in [SQD Cloud secrets](/en/cloud/resources/env-variables) if and when you deploy your squid there. We tested the code using a public endpoint available at `wss://astar.public.blastapi.io`; for production, we recommend using private endpoints or our [RPC addon](/en/cloud/resources/rpc-proxy).
</Warning>

## Define the batch handler

We change the batch handler logic taking care to avoid token ID clashing:

```ts title="src/main.ts" theme={"system"}
import { Store, TypeormDatabase } from '@subsquid/typeorm-store'
import { In } from 'typeorm'
import {
  astarDegensAddress,
  astarCatsAddress,
  contractMapping
} from './contracts'
import { Owner, Token, Transfer } from './model'
import * as erc721 from './abi/erc721'
import {
  processor,
  ProcessorContext,
  Event,
  Block
} from './processor'

var contractsSaved = false

processor.run(new TypeormDatabase(), async (ctx) => {
  const transfersData: TransferData[] = [];

  for (const block of ctx.blocks) {
    for (const event of block.events) {
      if (event.name === 'EVM.Log') {
        const transfer = handleTransfer(block.header, event)
        transfersData.push(transfer)
      }
    }
  }

  if (!contractsSaved) {
    await ctx.store.upsert([...contractMapping.values()])
    contractsSaved = true
  }
  await saveTransfers(ctx, transfersData)
})

type TransferData = {
  id: string
  from: string
  to: string
  token: bigint
  timestamp: number
  block: number
  contractAddress: string
}

function handleTransfer(block: Block, event: Event): TransferData {
  const { from, to, tokenId } = erc721.events.Transfer.decode(event)
  return {
    id: event.id,
    from,
    to,
    token: tokenId,
    timestamp: block.timestamp,
    block: block.height,
    contractAddress: event.args.address
  }
}

async function saveTransfers(
  ctx: ProcessorContext<Store>,
  transfersData: TransferData[]
) {
  const getTokenId = transferData => `${contractMapping.get(transferData.contractAddress)?.symbol ?? ""}-${transferData.token.toString()}`

  const tokensIds: Set<string> = new Set()
  const ownersIds: Set<string> = new Set()

  for (const transferData of transfersData) {
    tokensIds.add(getTokenId(transferData))
    ownersIds.add(transferData.from)
    ownersIds.add(transferData.to)
  }

  const tokens: Map<string, Token> = new Map(
    (await ctx.store.findBy(Token, { id: In([...tokensIds]) }))
      .map(token => [token.id, token])
  )

  const owners: Map<string, Owner> = new Map(
    (await ctx.store.findBy(Owner, { id: In([...ownersIds]) }))
      .map(owner => [owner.id, owner])
  )

  const transfers: Set<Transfer> = new Set()

  for (const transferData of transfersData) {
    const contract = new erc721.Contract(
      // temporary workaround for SDK issue 212
      // passing just the ctx as first arg may already work
      {_chain: {client: ctx._chain.rpc}},
      { height: transferData.block },
      transferData.contractAddress
    )

    let from = owners.get(transferData.from)
    if (from == null) {
      from = new Owner({ id: transferData.from, balance: 0n })
      owners.set(from.id, from)
    }

    let to = owners.get(transferData.to)
    if (to == null) {
      to = new Owner({ id: transferData.to, balance: 0n })
      owners.set(to.id, to)
    }

    const tokenId = getTokenId(transferData)
    let token = tokens.get(tokenId)
    if (token == null) {
      token = new Token({
        id: tokenId,
        uri: await contract.tokenURI(transferData.token),
        contract: contractMapping.get(transferData.contractAddress)
      })
      tokens.set(token.id, token)
    }

    token.owner = to

    const { id, block, timestamp } = transferData

    const transfer = new Transfer({
      id,
      block,
      timestamp,
      from,
      to,
      token
    })

    transfers.add(transfer)
  }

  await ctx.store.upsert([...owners.values()])
  await ctx.store.upsert([...tokens.values()])
  await ctx.store.insert([...transfers])
}
```

<Info>
  The `contract.tokenURI` call is accessing the **state** of the contract via a chain RPC endpoint. This is slowing down the indexing a little bit, but this data is only available this way. You'll find more information on accessing state in the [dedicated section of our docs](/en/sdk/squid-sdk/resources/substrate/frontier-evm#access-contract-state).
</Info>

## Database and the migration

Before giving your squid processor a local test, launch a PostgreSQL container with

```bash theme={"system"}
docker compose up -d
```

Squid projects automatically manage the database connection and schema via an [ORM abstraction](https://en.wikipedia.org/wiki/Object%E2%80%93relational_mapping). In this approach the schema is managed through migration files. Since we've made changes to the schema, we need to remove the existing migration(s) and create a new one. This involves the following steps:

1. Build the code:

   ```bash theme={"system"}
   npm run build
   ```

2. Make sure you start with a clean Postgres database. The following commands drop-create the Postgres instance in Docker:

   ```bash theme={"system"}
   docker compose down
   docker compose up -d
   ```

   Skip this step if you haven't used your database since the last `docker compose up -d`.

3. Regenerate the DB migration:

   ```bash theme={"system"}
   rm -r db/migrations
   ```

   ```bash theme={"system"}
   npx squid-typeorm-migration generate
   ```

4. Apply the migration:

   ```bash theme={"system"}
   npx squid-typeorm-migration apply
   ```

## Launch the Project

To launch the processor run the following command (this will block the current terminal):

```bash theme={"system"}
node -r dotenv/config lib/main.js
```

Finally, in a separate terminal window, launch the GraphQL server:

```bash theme={"system"}
npx squid-graphql-server
```

Visit [`localhost:4350/graphql`](http://localhost:4350/graphql) to access the [GraphiQL](https://github.com/graphql/graphiql) console. From this window, you can perform queries such as this one, to find out the account owners with the biggest balances:

```graphql theme={"system"}
query MyQuery {
  owners(limit: 10, where: {}, orderBy: balance_DESC) {
    balance
    id
  }
}
```

Or this other one, looking up the tokens owned by a given owner:

```graphql theme={"system"}
query MyQuery {
  tokens(where: {owner: {id_eq: "0x1210f3ea18ef463c162fff9084cee5b6e5ccab37"}}) {
    uri
    contract {
      id
      name
      symbol
      totalSupply
    }
  }
}
```

Have fun playing around with queries, after all, it's a *playground*!


# ink! contract indexing
Source: https://docs.sqd.dev/en/sdk/squid-sdk/tutorials/ink

Build a Squid SDK indexer for an ink! WASM smart contract — set up typegen, decode contract events, and persist them to Postgres for GraphQL queries.

## Objective

This tutorial starts with the [`substrate` squid template](https://github.com/subsquid-labs/squid-substrate-template) and goes through all the necessary changes to index the events of a WASM contract developed with [ink!](https://www.parity.io/blog/ink-3-0-paritys-rust-based-language-gets-a-major-update). This approach is taken to illustrate the development process. If you want to start indexing ASAP, consider starting with the [`ink` template](https://github.com/subsquid/squid-wasm-template) that contains the final code of this tutorial:

```bash theme={"system"}
sqd init <your squid name here> --template ink
```

Here we will use a simple test ERC20-type token contract deployed to [Shibuya](https://shibuya.subscan.io/) at `XnrLUQucQvzp5kaaWLG9Q3LbZw5DPwpGn69B5YcywSWVr5w`. Our squid will track all the token holders and account balances, together with the historical token transfers.

<Info>
  Squid SDK only supports WASM contracts executed by the [Contracts pallet](https://crates.parity.io/pallet_contracts/index.html) natively. The pallet is enabled by the following network runtimes:

  * `Astar` (a `Polkadot` parachain)
  * `Shibuya` (`Astar` testnet)
  * `Shiden` (`Kusama`-cousin of `Astar`)
  * `AlephZero` (a standalone Substrate-based chain)
</Info>

## Pre-requisites

* Familiarity with Git
* A properly set up [development environment](/en/sdk/squid-sdk/how-to-start/development-environment-set-up) consisting of Node.js, Git and Docker
* [Squid CLI](/en/sdk/squid-sdk/squid-cli/installation)

## Run the template

Retrieve the `substrate` template with `sqd init`:

```bash theme={"system"}
sqd init ink-tutorial --template substrate
cd ink-tutorial
```

and run it:

```bash theme={"system"}
npm i
npm run build
docker compose up -d
npx squid-typeorm-migration apply

node -r dotenv/config lib/main.js # should begin to ingest blocks

# open a separate terminal for this next command
npx squid-graphql-server # should begin listening on port 4350
```

After this test, shut down both processes with Ctrl-C and proceed.

## Define the data schema

The next step is to define the target data entities and their relations at `schema.graphql`, then use that file to autogenerate TypeORM entity classes.

We track:

* Wallet balances
* Token transfers

Our [schema definition](/en/sdk/squid-sdk/reference/schema-file) for modelling this data is straightforward:

```graphql theme={"system"}
# schema.graphql

type Owner @entity {
  id: ID!
  balance: BigInt! @index
}
 
type Transfer @entity {
  id: ID!
  from: Owner
  to: Owner
  amount: BigInt! @index
  timestamp: DateTime! @index
  block: Int!
}
 
```

Note:

* a one-to-many [relation](/en/sdk/squid-sdk/reference/schema-file/entity-relations) between `Owner` and `Transfer`;
* `@index` decorators for properties that we want to be able to filter the data by.

Next, we generate `TypeORM` entity classes from the schema with the `squid-typeorm-codegen` tool:

```bash theme={"system"}
npx squid-typeorm-codegen
```

The generated entity classes can be found under `src/model/generated`.

Finally, we create [database migrations](/en/sdk/squid-sdk/resources/persisting-data/typeorm) to match the changed schema. We restore the database to a clean state, then replace any existing migrations with the new one:

```bash theme={"system"}
docker compose down
docker compose up -d
rm -r db/migrations
npx squid-typeorm-migration generate
```

Apply the migration with

```bash theme={"system"}
npx squid-typeorm-migration apply
```

## WASM ABI Tools

The `Contracts` pallet stores the contract execution logs (calls and events) in a binary format. The decoding of this data is contract-specific and is done with the help of an ABI file typically published by the contract developer. ABI for ERC20 contracts like the one we're indexing can be found [here](https://raw.githubusercontent.com/subsquid-labs/squid-wasm-template/master/abi/erc20.json).

Download that file to the `abi` folder and install the following two tools from Squid SDK:

* `@subsquid/ink-abi` -- A performant library for decoding binary ink! contract data.
* `@subsquid/ink-typegen` -- A tool for making TypeScript modules for handling contract event and call data based on ABIs of contracts.

```bash theme={"system"}
npm i @subsquid/ink-abi && npm i @subsquid/ink-typegen --save-dev
```

Since `@subsquid/ink-typegen` is only used to generate source files, we install it as a dev dependency.

Generate the contract data handling module by running

```bash theme={"system"}
npx squid-ink-typegen --abi abi/erc20.json --output src/abi/erc20.ts
```

The generated `src/abi/erc20.ts` module defines interfaces to represent WASM data defined in the ABI, as well as functions necessary to decode this data (e.g. the `decodeEvent` function).

## Define the processor object

Squid SDK provides users with the [`SubstrateBatchProcessor` class](/en/sdk/squid-sdk). Its instances connect to [SQD Network](/en/network/overview) gateways at chain-specific URLs to get chain data and apply custom transformations. The indexing begins at the starting block and keeps up with new blocks after reaching the tip.

`SubstrateBatchProcessor`s [exposes methods](/en/sdk/squid-sdk/reference/processors/substrate-batch) to "subscribe" them to specific data such as Substrate events, extrinsics, storage items etc. The `Contracts` pallet emits `ContractEmitted` events wrapping the logs emitted by the WASM contracts. Processor [allows one](/en/sdk/squid-sdk/resources/substrate/ink) to subscribe to such events emitted by a specific contract.

The processor is instantiated and configured at the `src/processor.ts`. Here are the changes we need to make there:

* Change the gateway used to `https://v2.archive.subsquid.io/network/shibuya-substrate`.
* Remove the `addEvent` function call, and add `addContractsContractEmitted` instead, specifying the address of the contract we are interested in. The address should be represented as a hex string, so we need to decode our ss58 address of interest, `XnrLUQucQvzp5kaaWLG9Q3LbZw5DPwpGn69B5YcywSWVr5w`.

Here is the end result:

```ts title="src/processor.ts" theme={"system"}
import {assertNotNull} from '@subsquid/util-internal'
import {toHex} from '@subsquid/util-internal-hex'
import * as ss58 from '@subsquid/ss58'
import {
    BlockHeader,
    DataHandlerContext,
    SubstrateBatchProcessor,
    SubstrateBatchProcessorFields,
    Event as _Event,
    Call as _Call,
    Extrinsic as _Extrinsic
} from '@subsquid/substrate-processor'

export const SS58_NETWORK = 'astar' // used for the ss58 prefix, astar shares it with shibuya

const CONTRACT_ADDRESS_SS58 = 'XnrLUQucQvzp5kaaWLG9Q3LbZw5DPwpGn69B5YcywSWVr5w'
export const CONTRACT_ADDRESS = ss58.codec(SS58_NETWORK).decode(CONTRACT_ADDRESS_SS58)

export const processor = new SubstrateBatchProcessor()
    .setGateway('https://v2.archive.subsquid.io/network/shibuya-substrate')
    .setRpcEndpoint({
        url: assertNotNull(process.env.RPC_ENDPOINT),
        rateLimit: 10
    })
    .addContractsContractEmitted({
        contractAddress: [CONTRACT_ADDRESS],
        extrinsic: true
    })
    .setFields({
        block: {
            timestamp: true
        },
        extrinsic: {
            hash: true
        }
    })
    .setBlockRange({
        // genesis block happens to not have a timestamp, so it's easier
        // to start from 1 in cases when the deployment height is unknown
        from: 1
    })

export type Fields = SubstrateBatchProcessorFields<typeof processor>
export type Block = BlockHeader<Fields>
export type Event = _Event<Fields>
export type Call = _Call<Fields>
export type Extrinsic = _Extrinsic<Fields>
export type ProcessorContext<Store> = DataHandlerContext<Store, Fields>
```

## Define the batch handler

Once requested, the events can be processed by calling the `.run()` function that starts generating requests to SQD Network for [*batches*](/en/sdk/squid-sdk/resources/batch-processing) of data.

Every time a batch is returned by the Network, it will trigger the callback function, or *batch handler* (passed to `.run()` as second argument). It is in this callback function that all the mapping logic is expressed. This is where chain data decoding should be implemented, and where the code to save processed data on the database should be defined.

Batch handler is typically defined at the squid processor entry point, `src/main.ts`. Here is one that works for our task:

```ts title="src/main.ts" theme={"system"}
import {In} from 'typeorm'
import assert from 'assert'

import * as ss58 from '@subsquid/ss58'
import {Store, TypeormDatabase} from '@subsquid/typeorm-store'

import * as erc20 from './abi/erc20'
import {Owner, Transfer} from "./model"
import {
    processor,
    SS58_NETWORK,
    CONTRACT_ADDRESS,
    ProcessorContext
} from './processor'

processor.run(new TypeormDatabase({supportHotBlocks: true}), async ctx => {
    const txs: TransferRecord[] = getTransferRecords(ctx)

    const owners: Map<string, Owner> = await createOwners(ctx, txs)
    const transfers: Transfer[] = createTransfers(txs, owners)

    await ctx.store.upsert([...owners.values()])
    await ctx.store.insert(transfers)
})

interface TransferRecord {
    id: string
    from?: string
    to?: string
    amount: bigint
    block: number
    timestamp: Date
    extrinsicHash: string
}

function getTransferRecords(ctx: ProcessorContext<Store>): TransferRecord[] {
    const records: TransferRecord[] = []
    for (const block of ctx.blocks) {
        assert(block.header.timestamp, `Block ${block.header.height} had no timestamp`)
        for (const event of block.events) {
            if (event.name === 'Contracts.ContractEmitted' && event.args.contract === CONTRACT_ADDRESS) {
                assert(event.extrinsic, `Event ${event} arrived without a parent extrinsic`)
                const decodedEvent = erc20.decodeEvent(event.args.data)
                if (decodedEvent.__kind === 'Transfer') {
                    records.push({
                        id: event.id,
                        from: decodedEvent.from && ss58.codec(SS58_NETWORK).encode(decodedEvent.from),
                        to: decodedEvent.to && ss58.codec(SS58_NETWORK).encode(decodedEvent.to),
                        amount: decodedEvent.value,
                        block: block.header.height,
                        timestamp: new Date(block.header.timestamp),
                        extrinsicHash: event.extrinsic.hash
                    })
                }
            }
        }
    }
    return records
}

async function createOwners(ctx: ProcessorContext<Store>, txs: TransferRecord[]): Promise<Map<string, Owner>> {
    const ownerIds = new Set<string>()
    txs.forEach(tx => {
        if (tx.from) {
            ownerIds.add(tx.from)
        }
        if (tx.to) {
            ownerIds.add(tx.to)
        }
    })

    const ownersMap = await ctx.store.findBy(Owner, {
        id: In([...ownerIds])
    }).then(owners => {
        return new Map(owners.map(owner => [owner.id, owner]))
    })

    return ownersMap
}


function createTransfers(txs: TransferRecord[], owners: Map<string, Owner>): Transfer[] {
    return txs.map(tx => {
        const transfer = new Transfer({
            id: tx.id,
            amount: tx.amount,
            block: tx.block,
            timestamp: tx.timestamp,
            extrinsicHash: tx.extrinsicHash
        })

        if (tx.from) {
            transfer.from = owners.get(tx.from)
            if (transfer.from == null) {
                transfer.from = new Owner({id: tx.from, balance: 0n})
                owners.set(tx.from, transfer.from)
            }
            transfer.from.balance -= tx.amount
        }

        if (tx.to) {
            transfer.to = owners.get(tx.to)
            if (transfer.to == null) {
                transfer.to = new Owner({id: tx.to, balance: 0n})
                owners.set(tx.to, transfer.to)
            }
            transfer.to.balance += tx.amount
        }

        return transfer
    })
}
```

The `getTransferRecords` function generates a list of `TransferRecord` objects that contain the data we need to fill the models we have defined with our schema. This data is extracted from the events found in the batch context, `ctx`. We use the in the main body of the batch handler, the arrow function used as the second argument of the `.run()` function call, to fetch or create `Owner` instance and create a `Transfer` instance for every event found in the context.

Finally, these [TypeORM entity](/en/sdk/squid-sdk/reference/schema-file/entities) instances are saved to the database, all in one go. This is done to reduce the number of database queries.

<Info>
  In the `getTransferRecords` function we loop over the blocks and over the events contained in them, then filter the events with an `if`. The filtering is redundant when there's only one event type to process but will be needed when the processor is subscribed to multiple ones.
</Info>

## Launch the Project

Build, then launch the processor with

```bash theme={"system"}
npm run build
```

```bash theme={"system"}
node -r dotenv/config lib/main.js
```

This will block the current terminal. In a separate terminal window, launch the GraphQL server:

```bash theme={"system"}
npx squid-graphql-server
```

Visit [`localhost:4350/graphql`](http://localhost:4350/graphql) to access the [GraphiQl](https://github.com/graphql/graphiql) console. There you can perform queries such as this one, to find out the account owners with the biggest balances:

```graphql theme={"system"}
query MyQuery {
  owners(limit: 10, where: {}, orderBy: balance_DESC) {
    balance
    id
  }
}
```

Or this other one, looking up the largest transfers:

```graphql theme={"system"}
query MyQuery {
  transfers(limit: 10, orderBy: amount_DESC) {
    amount
    block
    id
    timestamp
    to {
      balance
      id
    }
    from {
      balance
      id
    }
  }
}

```

Have fun playing around with queries, after all, it's a *playground*!


# Simple Substrate squid
Source: https://docs.sqd.dev/en/sdk/squid-sdk/tutorials/substrate

Build a starter Substrate Squid SDK indexer — set up the processor, subscribe to balance transfer events, and persist to Postgres in 30 minutes.

## Objective

The goal of this tutorial is to guide you through creating a simple blockchain indexer ("squid") using Squid SDK. In this example we will query the [Crust storage network](https://crust.network). Our objective will be to observe which files have been added and deleted from the network. Additionally, our squid will be able to tell us the groups joined and the storage orders placed by a given account.

We will start with the `substrate` squid template, then go on to run the project, define a schema, and generate TypeScript interfaces. From there, we will be able to interact directly with SQD Network, using the and the metadata service to get a Crust types bundle.

We expect that experienced software developers should be able to complete this tutorial in around **10-15 minutes**.

## Pre-requisites

* Familiarity with Git
* A properly set up [development environment](/en/sdk/squid-sdk/how-to-start/development-environment-set-up) consisting of Node.js, Git and Docker
* [Squid CLI](/en/sdk/squid-sdk/squid-cli/installation)

## Scaffold with `sqd init`

Use [`sqd init`](/en/sdk/squid-sdk/squid-cli/init) and come up with some unique name for your squid. This tutorial will index data on Crust, a Substrate-based network, so use the `substrate` template:

```sh theme={"system"}
sqd init substrate-crust-tutorial --template substrate
cd substrate-crust-tutorial
```

### Run the project

Now you can follow the [quickstart](/en/sdk/squid-sdk/how-to-start/squid-development) guide to get the project up and running. Here is a summary:

```bash theme={"system"}
npm i
npm run build
docker compose up -d
npx squid-typeorm-migration apply
```

```bash theme={"system"}
node -r dotenv/config lib/main.js # will begin ingesting blocks
```

```bash theme={"system"}
# open a separate terminal for this next command
npx squid-graphql-server # should begin listening on port 4350
```

After this test, shut down both processes with Ctrl-C and proceed.

## Define the schema and generate entity classes

Next, we make changes to the data [schema](/en/sdk/squid-sdk/reference/schema-file) of the squid and define [entities](/en/sdk/squid-sdk/reference/schema-file/entities) that we would like to track. We are interested in:

* Files added to and deleted from the chain;
* Active accounts;
* Groups joined by accounts;
* Storage orders placed by accounts.

For this, we use the following `schema.graphql`:

```graphql title="schema.graphql" theme={"system"}
type Account @entity {
  id: ID! #Account address
  workReports: [WorkReport] @derivedFrom(field: "account")
  joinGroups: [JoinGroup] @derivedFrom(field: "member")
  storageOrders: [StorageOrder] @derivedFrom (field: "account")
}

type WorkReport @entity {
  id: ID! #event id
  account: Account!
  addedFiles: [[String]]
  deletedFiles: [[String]]
  extrinsicHash: String
  createdAt: DateTime
  blockNum: Int!
}

type JoinGroup @entity {
  id: ID!
  member: Account!
  owner: String!
  extrinsicHash: String
  createdAt: DateTime
  blockNum: Int!
}

type StorageOrder @entity {
  id: ID!
  account: Account!
  fileCid: String!
  extrinsicHash: String
  createdAt: DateTime
  blockNum: Int!
}
```

Notice that the `Account` entity is almost completely [derived](/en/sdk/squid-sdk/reference/schema-file/entity-relations). It is there to tie the other three entities together.

To finalize this step, run the `codegen` tool:

```bash theme={"system"}
npx squid-typeorm-codegen
```

This will automatically generate TypeScript entity classes for our schema. They can be found in the `src/model/generated` folder of the project.

## Generate TypeScript wrappers for events

We generate these using the [squid-substrate-typegen](/en/sdk/squid-sdk/tutorials/batch-processor-in-action) tool. Its configuration file is `typegen.json`; there, we need to

1. Set the `"specVersions"` field to a valid source of Crust chain runtime metadata. We'll use an URL of SQD-maintained metadata service:
   ```json theme={"system"}
   "specVersions": "https://v2.archive.subsquid.io/metadata/crust",
   ```
2. List all Substrate pallets we will need the data from. For each pallet we list all events, calls, storage items and constants needed.

<Info>
  Refer to [this note](/en/sdk/squid-sdk/resources/substrate/data-sourcing-miniguide) if you are unsure what Substrate data to use in your project.
</Info>

Our final `typegen.json` looks like this:

```json title="typegen.json" theme={"system"}
{
  "outDir": "src/types",
  "specVersions": "https://v2.archive.subsquid.io/metadata/crust",
  "pallets": {
    "Swork": {
      "events": [
        "WorksReportSuccess",
        "JoinGroupSuccess"
      ]
    },
    "Market": {
      "events": [
        "FileSuccess"
      ]
    }
  }
}
```

Once done with the configuration, we run the tool with

```bash theme={"system"}
npx squid-substrate-typegen ./typegen.json
```

The generated Typescript wrappers are at `src/types`.

## Set up the processor object

The next step is to create a [`SubstrateBatchProcessor`](/en/sdk/squid-sdk/reference/processors/substrate-batch) object which [subscribes](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests) to all the events we need. We do it at `src/processor.ts`:

```ts title="src/processor.ts" theme={"system"}
import {
  SubstrateBatchProcessor,
  SubstrateBatchProcessorFields,
  DataHandlerContext
} from '@subsquid/substrate-processor'
import {events} from './types' // the wrappers generated in previous section

const processor = new SubstrateBatchProcessor()
  .setGateway('https://v2.archive.subsquid.io/network/crust')
  .setRpcEndpoint({
    url: 'https://crust.api.onfinality.io/public',
    rateLimit: 10
  })
  .setBlockRange({ from: 583000 })
  .addEvent({
    name: [
      events.market.fileSuccess.name,
      events.swork.joinGroupSuccess.name,
      events.swork.worksReportSuccess.name
    ],
    call: true,
    extrinsic: true
  })
  .setFields({
    extrinsic: {
      hash: true
    },
    block: {
      timestamp: true
    }
  })

type Fields = SubstrateBatchProcessorFields<typeof processor>
export type ProcessorContext<Store> = DataHandlerContext<Store, Fields>
```

This creates a processor that

* Uses SQD Network as its main data source and a chain RPC for [real-time updates](/en/sdk/squid-sdk/resources/unfinalized-blocks). URLs of the SQD Network gateways are available on [this page](/en/data/substrate) and via [`sqd gateways`](/en/sdk/squid-sdk/squid-cli/gateways). See [this page](/en/sdk/squid-sdk/reference/processors/substrate-batch/general) for the reference on data sources configuration;
* [Subscribes](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests) to `Market.FileSuccess`, `Swork.JoinGroupSuccess` and `Swork.WorksReportSuccess` events emitted at heights starting at 583000;
* Additionally subscribes to calls that emitted the events and the corresponding extrinsics;
* [Requests](/en/sdk/squid-sdk/reference/processors/substrate-batch/field-selection) the `hash` data field for all retrieved extrinsics and the `timestamp` field for all block headers.

We also export the `ProcessorContext` type to be able to pass the sole argument of the batch handler function around safely.

## Define the batch handler

Squids [batch process](/en/sdk/squid-sdk/resources/batch-processing) chain data from multiple blocks. Compared to the [handlers](/en/sdk/squid-sdk/resources/batch-processing#migrate-from-handlers) approach this results in a much lower database load. Batch processing is fully defined by processor's [batch handler](/en/sdk/squid-sdk/reference/processors/architecture#processorrun), the callback supplied to the `processor.run()` call at the entry point of each processor (`src/main.ts` by convention).

We begin defining our batch handler by importing the entity model classes and Crust event types that we generated in previous sections. We also import the processor and its types:

```ts title="src/main.ts" theme={"system"}
import {Account, WorkReport, JoinGroup, StorageOrder} from './model'
import {processor, ProcessorContext} from './processor'
```

Let's skip for now the `process.run()` call - we are going to come back to it in a second - and scroll down to the `getTransferEvents` function. In the template repository this function loops through the items contained in the context, extracts the events data and stores it in a list of objects.

For this project we are still going to extract events data from the context, but this time we have more than one event type so we have to sort them. We also need to handle the account information. Let's start with deleting the `TransferEvent` interface and defining this instead:

```typescript theme={"system"}
type Tuple<T,K> = [T,K]
interface EventsInfo {
  joinGroups: Tuple<JoinGroup, string>[]
  marketFiles: Tuple<StorageOrder, string>[]
  workReports: Tuple<WorkReport, string>[]
  accountIds: Set<string>
}
```

Now, let's replace the `getTransferEvents` function with the below snippet that

* extracts event information in a manner specific to its name (known from `e.name`);
* stores event information in an object (we are going to use entity classes for that) and extracts `accountId`s from it;
* store all `accountId`s in a set.

```typescript theme={"system"}
import {toHex} from '@subsquid/substrate-processor'
import * as ss58 from '@subsquid/ss58'

function getEventsInfo(ctx: ProcessorContext<Store>): EventInfo {
  let eventsInfo: EventsInfo = {
    joinGroups: [],
    marketFiles: [],
    workReports: [],
    accountIds: new Set<string>()
  }
  for (let block of ctx.blocks) {
    const blockTimestamp = block.header.timestamp ? new Date(block.header.timestamp) : undefined
    for (let e of block.events) {
      if (e.name === events.swork.joinGroupSuccess.name) {
        const decoded = events.swork.joinGroupSuccess.v1.decode(e)
        const memberId = ss58.codec('crust').encode(decoded[0])
        eventsInfo.joinGroups.push([
          new JoinGroup({
            id: e.id,
            owner: ss58.codec('crust').encode(decoded[1]),
            blockNum: block.header.height,
            createdAt: blockTimestamp,
            extrinsicHash: e.extrinsic?.hash
          }),
          memberId
        ])
        // add encountered account ID to the set of unique accountIDs
        eventsInfo.accountIds.add(memberId)
      }
      if (e.name === events.market.fileSuccess.name) {
        const decoded = events.market.fileSuccess.v1.decode(e)
        const accountId = ss58.codec('crust').encode(decoded[0])
        eventsInfo.marketFiles.push([
          new StorageOrder({
            id: e.id,
            fileCid: toHex(decoded[1]),
            blockNum: block.header.height,
            createdAt: blockTimestamp,
            extrinsicHash: e.extrinsic?.hash
          }),
          accountId
        ])
        eventsInfo.accountIds.add(accountId)
      }
      if (e.name === events.swork.worksReportSuccess.name) {
        const decoded = events.swork.worksReportSucces.v1.decode(e)
        const accountId = ss58.codec('crust').encode(decoded[0])

        const addedExtr = e.call?.args.addedFiles
        const deletedExtr = e.call?.args.deletedFiles

        const addedFiles = addedExtr.map(v => v.map(ve => String(ve)))
        const deletedFiles = deletedExtr.map(v => v.map(ve => String(ve)))

        if (addedFiles.length > 0 || deletedFiles.length > 0) {
          eventsInfo.workReports.push([
            new WorkReport({
              id: e.id,
              addedFiles: addedFiles,
              deletedFiles: deletedFiles,
              blockNum: block.header.height,
              createdAt: blockTimestamp,
              extrinsicHash: e.extrinsic?.hash,
            }),
            accountId
          ])
          eventsInfo.accountIds.add(accountId)
        }
      }
    }
  }
  return eventsInfo
}
```

Next, we want to create an entity (`Account`) object for every `accountId` in the set, then add the `Account` information to every event entity object. Finally, we save all the created and modified entity models into the database.

Take the code inside `processor.run()` and change it so that it looks like this:

```typescript theme={"system"}
processor.run(new TypeormDatabase(), async (ctx) => {
  const eventsInfo = getEventsInfo(ctx)

  let accounts = await ctx.store
    .findBy(Account, { id: In([...eventsInfo.accountIds]) })
    .then(accounts => new Map(accounts.map(a => [a.id, a]))
  for (let aid of eventsInfo.accountIds) {
    if (!accounts.has(aid)) {
      accounts.set(aid, new Account({ id: aid }))
    }
  }

  for (const jg of eventsInfo.joinGroups) {
    // necessary to add this field to the previously created model
    // because now we have the Account created.
    jg[0].member = accounts.get(jg[1])
  }
  for (const mf of eventsInfo.marketFiles) {
    mf[0].account = accounts.get(mf[1])
  }
  for (const wr of eventsInfo.workReports) {
    wr[0].account = accounts.get(wr[1])
  }

  await ctx.store.save([...accounts.values()]);
  await ctx.store.insert(eventsInfo.joinGroups.map(el => el[0]));
  await ctx.store.insert(eventsInfo.marketFiles.map(el => el[0]));
  await ctx.store.insert(eventsInfo.workReports.map(el => el[0]));
})
```

## Apply changes to the database

Squid projects automatically manage the database connection and schema via an [ORM abstraction](https://en.wikipedia.org/wiki/Object%E2%80%93relational_mapping) provided by [TypeORM](https://typeorm.io). Previously we changed the data schema at `schema.graphql` and reflected these changes in our Typescript code using `npx squid-typeorm-codegen`. Here, we [apply the corresponding changes to the database itself](/en/sdk/squid-sdk/resources/persisting-data/typeorm).

First, we'll need to compile our updated project code. Do this with:

```bash theme={"system"}
npm run build
```

Next we ensure that the database is at blank state:

```bash theme={"system"}
docker compose down
docker compose up -d
```

Then we replace any old migrations with the new one with

```bash theme={"system"}
rm -r db/migrations
```

```bash theme={"system"}
npx squid-typeorm-migration generate
```

The new migration will be generated from the TypeORM entity classes we previously made out of `schema.graphql`. Apply it with:

```bash theme={"system"}
npx squid-typeorm-migration apply
```

## Launch the project

It's finally time to run the project! Run

```bash theme={"system"}
node -r dotenv/config lib/main.js
```

in one terminal, then open another one and run

```bash theme={"system"}
npx squid-graphql-server
```

Now you can see the results of our hard work by visiting [`localhost:4350/graphql`](http://localhost:4350/graphql) in a browser and accessing the [GraphiQL](https://github.com/graphql/graphiql) console.

From this window we can perform queries. This one displays info on ten latest work reports, including all involved files and the account id:

```graphql theme={"system"}
query MyQuery {
  workReports(limit: 10, orderBy: blockNum_DESC) {
    account {
      id
    }
    addedFiles
    deletedFiles
  }
}
```

## Credits

This sample project is adapted from a real integration, developed by our very own [Mikhail Shulgin](https://github.com/ma-shulgin). Credit for building it and helping with the guide goes to him.


# DipDup
Source: https://docs.sqd.dev/en/sdk/alternative-clients/dipdup

Python indexing framework that can use SQD Network as a data source

[DipDup](https://dipdup.io) is a Python indexing framework that can use [SQD Network](/en/network/overview) as a data source. It offers:

* SQLite, PostgreSQL and TimescaleDB data sinks
* GraphQL APIs based on Hasura

Development workflow uses the `dipdup` tool to generate a stub project. Once done with that, all you have to do is to define the data schema and the handlers. Take a look at their [quickstart](https://dipdup.io/docs/quickstart-evm) for more details.

With its handler-based architecture and the choice of Python as the transform logic language, DipDup is easier to develop for than [Squid SDK](/en/sdk/squid-sdk/overview), but has higher requirements on database IO bandwith and CPU. The IO bandwidth issue is partially solved by DipDup's caching layer used for database access.

## Getting Started

Visit the [DipDup documentation](https://dipdup.io/docs/) to get started with building Python-based indexers using SQD Network.


# Migrate SDK squids to Portal
Source: https://docs.sqd.dev/en/sdk/migration/evm-gateway-to-portal

Replace gateway data sources with Portal for improved performance and stability

<Card title="← Back to Portal Setup" icon="arrow-left" href="/en/portal/migration">
  Return to the Portal setup overview to explore other deployment options.
</Card>

This guide walks you through migrating SDK-based squids from gateways of the open private version of SQD Network to portals of the permissionless SQD Networkas your primary data source.

## Prerequisites

* An existing Squid SDK-based squid using `@subsquid/evm-processor` with `.setGateway()`
* Access to an SQD portal URL (see Step 2 for options)

## Migration steps

<Steps>
  <Step title="Install portal-api version">
    Navigate to your squid's folder and install the `portal-api` version of `@subsquid/evm-processor`:

    ```bash theme={"system"}
    npm i @subsquid/evm-processor@portal-api
    ```
  </Step>

  <Step title="Obtain a portal URL">
    Obtain a portal URL for your dataset. Choose the option that matches your setup:

    <Tabs>
      <Tab title="SQD Cloud user">
        If you're an existing user of [the SQD Cloud](https://app.subsquid.io):

        1. [Enable the preview mode](https://app.subsquid.io/preview)
        2. Navigate to [the portals page](https://app.subsquid.io/portal)
        3. Click on the tile of your network and follow the instructions

        <Check>
          Once configured, you can run your portal-powered squid both locally and in the Cloud. You can skip Step 3, it's already configured automatically.
        </Check>
      </Tab>

      <Tab title="Self-hosted portal">
        If you want to use a private portal:

        1. [Set up your portal](/en/portal/self-hosting)
        2. Get your portal's [dataset-specific API URL](/en/portal/self-hosting#using-your-portal) for the network you're interested in
      </Tab>

      <Tab title="Public portal (testing)">
        If you don't have an SQD Cloud account and want to test a portal, use the following URL template:

        ```bash theme={"system"}
        https://portal.sqd.dev/datasets/<dataset-slug>
        ```

        Replace `<dataset-slug>` with the last path segment of the gateway URL for your network found on the [EVM Networks listing](/en/data/evm).

        **Example:** The URL for the Ethereum dataset is:

        ```bash theme={"system"}
        https://portal.sqd.dev/datasets/ethereum-mainnet
        ```
      </Tab>
    </Tabs>
  </Step>

  <Step title="Update processor configuration">
    Configure your squid to ingest data from the portal by replacing the `.setGateway()` call in your processor configuration (typically in `src/processor.ts`) with `.setPortal()`.

    <Tabs>
      <Tab title="Local or self-hosted portal">
        Replace `.setGateway()` with `.setPortal()` using your portal URL directly:

        ```typescript src/processor.ts theme={"system"}
        export const processor = new EvmBatchProcessor()
          .setPortal('<portal URL for your dataset>')
        ```
      </Tab>

      <Tab title="Cloud deployment">
        If you're deploying to [the Cloud](/en/cloud/), source the portal URL from an environment variable configured in your manifest:

        ```typescript src/processor.ts theme={"system"}
        import { assertNotNull } from '@subsquid/util-internal'

        export const processor = new EvmBatchProcessor()
          .setPortal(
            assertNotNull(
              process.env.PORTAL_URL,
              'Required env variable PORTAL_URL is missing'
            )
          )
        ```
      </Tab>
    </Tabs>

    <Warning>
      If your squid uses an RPC endpoint, keep that configuration for now. Portals do not currently support real-time data ingestion, but this capability is coming soon.
    </Warning>
  </Step>
</Steps>

<Check>
  Your squid is now configured to source data from Portal. Test it locally to verify everything works correctly.
</Check>


# Migrate Solana SDK squids to Portal
Source: https://docs.sqd.dev/en/sdk/migration/solana-gateway-to-portal

Replace gateway and RPC data sources with the public Portal for real-time Solana data

<Card title="← Back to Portal Setup" icon="arrow-left" href="/en/portal/migration">
  Return to the Portal setup overview to explore other deployment options.
</Card>

The newest version of the SQD Network portal serves real-time data. It can replace both gateways of the open private version of SQD Network and RPC endpoints.

## Prerequisites

* An existing Squid SDK-based Solana squid using `@subsquid/solana-stream` with `.setGateway()` and/or `.setRpc()`

## Migration steps

<Steps>
  <Step title="Install @portal-api packages">
    **A.** Enter your squid's folder.

    **B.** Remove both your lock file and the `node_modules` folder:

    <Tabs>
      <Tab title="NPM">
        ```bash theme={"system"}
        rm -r node_modules package-lock.json
        ```
      </Tab>

      <Tab title="Yarn">
        ```bash theme={"system"}
        rm -r node_modules yarn.lock
        ```
      </Tab>

      <Tab title="PNPM">
        ```bash theme={"system"}
        rm -r node_modules pnpm-lock.yaml
        ```
      </Tab>
    </Tabs>

    **C.** Upgrade all SQD packages that have a `@portal-api` version to it:

    ```bash theme={"system"}
    npx --yes npm-check-updates --filter "@subsquid/*" --target "@portal-api" --upgrade
    ```

    **D.** Freeze the versions of `@portal-api` packages by removing any version range specifiers (`^`, `~`, `<`, `>`, `>=`, `<=`) preceding the package versions. Here's a script:

    ```bash theme={"system"}
    sed -i -e 's/[\^~=<>]*\([0-9\.]*-portal-api\.[0-9a-f]\{6\}\)/\1/g' package.json
    ```

    <Accordion title="You can also do it manually">
      Here's an example edit:

      ```diff theme={"system"}
         "dependencies": {
      -    "@subsquid/batch-processor": "^1.0.0-portal-api.18ef40",
      +    "@subsquid/batch-processor": "1.0.0-portal-api.18ef40",
           "@subsquid/borsh": "^0.2.0",
      -    "@subsquid/solana-objects": ">=0.0.3-portal-api.18ef40",
      -    "@subsquid/solana-stream": "<1.0.0-portal-api.18ef40",
      +    "@subsquid/solana-objects": "0.0.3-portal-api.18ef40",
      +    "@subsquid/solana-stream": "1.0.0-portal-api.18ef40",
           "@subsquid/typeorm-migration": "^1.3.0",
      -    "@subsquid/typeorm-store": "~1.6.0-portal-api.18ef40",
      +    "@subsquid/typeorm-store": "1.6.0-portal-api.18ef40",
           "dotenv": "^16.4.7",
           "pg": "^8.13.1",
           "typeorm": "^0.3.20"
         },
         "devDependencies": {
      -    "@subsquid/solana-typegen": "^0.4.1-portal-api.18ef40",
      +    "@subsquid/solana-typegen": "0.4.1-portal-api.18ef40",
           "@types/node": "^22.13.1",
           "typescript": "~5.7.3"
         }
      ```
    </Accordion>

    **E.** Install the dependencies:

    <Tabs>
      <Tab title="NPM">
        ```bash theme={"system"}
        npm install
        ```
      </Tab>

      <Tab title="Yarn">
        ```bash theme={"system"}
        yarn install
        ```
      </Tab>

      <Tab title="PNPM">
        ```bash theme={"system"}
        pnpm install
        ```
      </Tab>
    </Tabs>
  </Step>

  <Step title="Update your code">
    **A.** Replace all existing data sources with the portal:

    ```diff theme={"system"}
    +  .setPortal({
    +    url: 'https://portal.sqd.dev/datasets/solana-mainnet',
    +      http: {
    +        retryAttempts: Infinity
    +      }
    +   })
    -  .setGateway('https://v2.archive.subsquid.io/network/solana-mainnet')
    -  .setRpc({
    -    client: new SolanaRpcClient({
    -      url: process.env.SOLANA_NODE
    -    })
    -  })
    ```

    Also, please remove any mentions of `SolanaRpcClient`, for example:

    ```diff theme={"system"}
    -import {DataSourceBuilder, SolanaRpcClient} from '@subsquid/solana-stream'
    +import {DataSourceBuilder} from '@subsquid/solana-stream'
    ```

    **B.** Replace any block height literals with **slot number** literals.

    ```diff theme={"system"}
    +  .setBlockRange({from: 325000000})
    -  .setBlockRange({from: 303262650})
    ```

    Use this converter to translate block heights into slot numbers. It bisects the chain through the public Portal:

    **C.** If you used the `slot` field of block headers anywhere in your code, replace it with `.number`:

    ```diff theme={"system"}
    -  slot: block.header.slot,
    +  slot: block.header.number,
    ```

    **D.** If you need the block height (for example to stay compatible with your old code) request it in the `.setFields` call:

    ```diff theme={"system"}
       .setFields({
         block: { // block header fields
           timestamp: true,
    +      height: true
         },
    ```
  </Step>

  <Step title="Test by re-syncing">
    We highly recommend that all squids migrated to Portal are tested by re-syncing them. This will allow you to make sure that everything works as expected for the whole length of the chain and catch any bugs early.

    To resync your squid, follow the zero-downtime update procedure:

    1. Deploy your squid into a new slot.
    2. Wait for it to sync, observing the improved data fetching.
    3. Assign your production tag to the new deployment to redirect the GraphQL requests there.

    See [the slots and tags guide](/en/cloud/resources/slots-and-tags#zero-downtime-updates) for details.

    <Accordion title="A workaround that allows continuous operation without a resync (not recommended)">
      Assuming your squid is version-controlled and has one processor:

      1. Commit your updated squid code.

      2. Stop your Cloud squid deployment.

         1. Reset your repo to the state before the updates.
         2. Add the `to` field to the argument of the [`.setBlockRange`](/en/sdk/squid-sdk/solana-indexing/sdk/solana-batch/general#set-block-range) `DataSourceBuilder` call (likely in `./src/main.ts`). Set it to the current slot number.
         3. [Update your deployment](/en/sdk/squid-sdk/squid-cli/deploy).
         4. Take a look at your squid's logs. It should be repeatedly terminating (due to having nothing to do) and restarting.

      3. Connect to the squid's database and update the `height` field of the [status schema](/en/sdk/squid-sdk/faq#how-do-squids-keep-track-of-their-sync-progress) to contain **slot** instead of the block height of the block mentioned there.

      4. Verify that the `height` field updated successfully by re-reading the status schema.

      5. Reset your codebase to its updated version and redeploy your squid again.
    </Accordion>
  </Step>
</Steps>

<Check>
  Your squid is now configured to source data from Portal.
</Check>

## What's next?

Take a look at the [Soldexer project](https://soldexer.dev) for a peek at the new client architecture and the most detailed and up-to-date documentation on the updated Portal API.


# Data freshness monitoring
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/solana/guides/advanced-topics/latency-monitoring

Compare Portal data freshness against external RPC providers

The `solanaRpcLatencyWatcher` subscribes to RPC endpoints via WebSocket and measures when blocks arrive at the Portal versus when they appear at the RPC endpoints.

<Warning>
  The measured values include client-side network data freshness. For RPC endpoints, only the arrival time of blocks is measured—this does not capture the node’s internal processing or response latency if queried directly. Results represent end-to-end delays as experienced by the client, not pure Portal or RPC processing performance.
</Warning>

```ts theme={"system"}
import { formatBlock } from '@subsquid/pipes'
import { solanaPortalSource, solanaRpcLatencyWatcher } from '@subsquid/pipes/solana'

/**
 * This example demonstrates how to track and compare block indexing latency
 * between the Subsquid Portal and external RPC providers.
 * It listens for new block heads and measures the time until blocks are
 * observed on the client side through both RPC endpoints and the Portal.
 *
 ******************************************************************************
 * ⚠️ Important:
 * - The measured values INCLUDE client-side network latency.
 * - For RPC, only the *arrival time* of the block is measured — this does NOT
 *   capture the node's internal processing or response latency if queried directly.
 *****************************************************************************
 *
 * In other words, the results represent end-to-end delays as experienced by the client,
 * not the pure Portal latency or RPC processing performance.
 */

async function main() {
  // Create a stream of new blocks from Solana mainnet portal
  const stream = solanaPortalSource({
    portal: 'https://portal.sqd.dev/datasets/solana-mainnet',
    outputs: new SolanaQueryBuilder()
      .addFields({ block: { number: true, hash: true, timestamp: true } })
      .includeAllBlocks({ from: 'latest' })
      .build(),
  }).pipe(
    solanaRpcLatencyWatcher({
      rpcUrl: ['https://api.mainnet-beta.solana.com'], // RPC endpoints to monitor
    }).pipe({
      profiler: { id: 'expose metrics' },
      transform: (data, { metrics }) => {
        if (!data) return // Skip if no latency data

        // For each RPC endpoint, update the latency gauge metric
        for (const rpc of data.rpc) {
          metrics
            .gauge({
              name: 'rpc_latency_ms',
              help: 'RPC Latency in ms',
              labelNames: ['url'],
            })
            .set({ url: rpc.url }, data.rpc[0].portalDelayMs)
        }

        return data
      },
    }),
  )

  // Iterate over the stream, logging block and RPC latency data
  for await (const { data } of stream) {
    if (!data) continue // Skip if no block data

    // Log block number and timestamp
    console.log(`-------------------------------------`)
    console.log(`BLOCK DATA: ${formatBlock(data.number)} / ${data.timestamp.toString()}`)
    // Log RPC latency table for the block
    console.table(data.rpc)
  }
}

void main()
```

## Output format

Data freshness data includes:

* `url`: RPC endpoint URL
* `receivedAt`: Timestamp when the RPC endpoint received the block
* `portalDelayMs`: Milliseconds between RPC arrival and Portal availability

```
Block: 36,046,611 / Fri Sep 26 2025 14:29:29 GMT+0400
┌───┬─────────────────────────────────┬──────────────────────────┬───────────────┐
│   │ url                             │ receivedAt               │ portalDelayMs │
├───┼─────────────────────────────────┼──────────────────────────┼───────────────┤
│ 0 │ https://base.drpc.org           │ 2025-09-26T10:29:29.134Z │ 646           │
│ 1 │ https://base-rpc.publicnode.com │ 2025-09-26T10:29:29.130Z │ 642           │
└───┴─────────────────────────────────┴──────────────────────────┴───────────────┘
```

## Custom metrics integration

Export data freshness metrics to Prometheus or other monitoring systems.

```ts expandable theme={"system"}
import { solanaPortalSource, solanaRpcLatencyWatcher } from '@subsquid/pipes/solana'
import { Registry, Counter } from 'prom-client'

const registry = new Registry()
const latencyCounter = new Counter({
  name: 'portal_latency_ms_total',
  help: 'Total Portal latency in milliseconds',
  registers: [registry],
})

const stream = solanaPortalSource({
  portal: 'https://portal.sqd.dev/datasets/solana-mainnet',
  outputs: new SolanaQueryBuilder()
    .addFields({ block: { number: true, hash: true, timestamp: true } })
    .includeAllBlocks({ from: 'latest' })
    .build(),
}).pipe(
  solanaRpcLatencyWatcher({
    rpcUrl: ['https://api.mainnet-beta.solana.com'],
  }).pipe({
    transform: (data) => {
      if (!data) return

      // Record latency metrics
      for (const rpc of data.rpc) {
        latencyCounter.inc(rpc.portalDelayMs)
      }

      return data
    },
  }),
)

// Expose metrics endpoint
import http from 'http'
const server = http.createServer(async (req, res) => {
  if (req.url === '/metrics') {
    res.setHeader('Content-Type', registry.contentType)
    res.end(await registry.metrics())
  } else {
    res.end('OK')
  }
})
server.listen(9090)

for await (const { data } of stream) {
  // Process blocks
}
```


# Logging
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/solana/guides/advanced-topics/logging

Customize logging with Pino-compatible transports

[solanaPortalSource()](../../reference/basic-components/source) accepts a Pino-compatible `logger`, allowing you to integrate custom log transports and send logs to external services like GCP Cloud Logging, Sentry, or any other Pino-compatible destination.

## Basic custom logger

Pass a custom logger to the source to configure logging for your entire pipeline.

```ts theme={"system"}
import { createTarget } from "@subsquid/pipes";
import { solanaPortalSource } from "@subsquid/pipes/solana";
import pino from "pino";

async function main() {
  const transport = pino.transport({
    target: "pino-pretty",
    options: {
      colorize: true,
      translateTime: "HH:MM:ss",
    },
  });

  const source = solanaPortalSource({
    portal: "https://portal.sqd.dev/datasets/solana-mainnet",
    logger: pino(transport),
  });

  const target = createTarget({
    write: async ({ logger, read }) => {
      for await (const { data } of read()) {
        logger.info({ count: data.length }, "Processed batch");
      }
    },
  });

  await source.pipeTo(target);
}

void main()
```

## Integration with cloud services

You can use any Pino transport to send logs to cloud services. Pass the configured logger to the source.

<Tabs>
  <Tab title="GCP Cloud Logging">
    ```ts theme={"system"}
    import { createTarget } from "@subsquid/pipes";
    import { solanaPortalSource } from "@subsquid/pipes/solana";
    import pino from "pino";

    async function main() {
      const transport = pino.transport({
        target: "@google-cloud/logging-pino",
        options: {
          projectId: "your-project-id",
          logName: "pipes-indexer",
        },
      });

      const source = solanaPortalSource({
        portal: "https://portal.sqd.dev/datasets/solana-mainnet",
        logger: pino(transport),
      });

      const target = createTarget({
        write: async ({ logger, read }) => {
          for await (const { data } of read()) {
            logger.info(
              {
                blocksProcessed: data.blocks?.length,
                eventsCount: data.transfer?.length,
              },
              "Batch processed"
            );
          }
        },
      });

      await source.pipeTo(target);
    }

    void main()
    ```
  </Tab>

  <Tab title="Sentry">
    ```ts theme={"system"}
    import { createTarget } from "@subsquid/pipes";
    import { solanaPortalSource } from "@subsquid/pipes/solana";
    import pino from "pino";

    async function main() {
      const transport = pino.transport({
        target: "pino-sentry-transport",
        options: {
          sentry: {
            dsn: process.env.SENTRY_DSN,
            environment: "production",
          },
          level: "error", // Only send errors to Sentry
        },
      });

      const source = solanaPortalSource({
        portal: "https://portal.sqd.dev/datasets/solana-mainnet",
        logger: pino(transport),
      });

      const target = createTarget({
        write: async ({ logger, read }) => {
          for await (const { data } of read()) {
            try {
              await processData(data);
              logger.info({ count: data.length }, "Batch processed");
            } catch (error) {
              logger.error({ error, data }, "Failed to process batch");
            }
          }
        },
      });

      await source.pipeTo(target);
    }

    void main()
    ```
  </Tab>

  <Tab title="Multiple Transports">
    ```ts theme={"system"}
    import { createTarget } from "@subsquid/pipes";
    import { solanaPortalSource } from "@subsquid/pipes/solana";
    import pino from "pino";

    async function main() {
      const transport = pino.transport({
        targets: [
          {
            target: "pino-pretty",
            options: { colorize: true },
            level: "info",
          },
          {
            target: "@google-cloud/logging-pino",
            options: { projectId: "your-project-id" },
            level: "info",
          },
          {
            target: "pino-sentry-transport",
            options: { sentry: { dsn: process.env.SENTRY_DSN } },
            level: "error",
          },
        ],
      });

      const source = solanaPortalSource({
        portal: "https://portal.sqd.dev/datasets/solana-mainnet",
        logger: pino(transport),
      });

      const target = createTarget({
        write: async ({ logger, read }) => {
          for await (const { data } of read()) {
            logger.info({ count: data.length }, "Processed batch");
          }
        },
      });

      await source.pipeTo(target);
    }

    void main()
    ```
  </Tab>
</Tabs>

<Tip>
  The `ctx.logger` in transformers and targets is the same logger instance passed to the source. Configure logging at the source level, then use `ctx.logger` throughout your pipeline for consistent logging.
</Tip>


# Metrics
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/solana/guides/advanced-topics/metrics

Track custom Prometheus metrics in Solana pipes

Pipes SDK can expose a Prometheus-compatible metrics server. You can customize it to add counters, gauges, histograms, and summaries.

```ts theme={"system"}
import { solanaInstructionDecoder, solanaPortalStream } from "@subsquid/pipes/solana";
import { metricsServer } from "@subsquid/pipes/metrics/node";

import * as orcaWhirlpool from './abi/orca_whirlpool/index.js'

async function main() {
  const stream = solanaPortalStream({
    id: 'solana-decoder',
    portal: 'https://portal.sqd.dev/datasets/solana-mainnet',
    outputs: solanaInstructionDecoder({
      range: {
        from: 'latest'
      },
      programId: orcaWhirlpool.programId,
      instructions: {
        swap: orcaWhirlpool.instructions.swap,
        swapV2: orcaWhirlpool.instructions.swapV2,
      },
    }),
    metrics: metricsServer({
      port: 9090
    }), // equivalent to metricsServer(), as 9090 is the default port
  })

  for await (const { data, ctx } of stream) {
    // Add custom counter metric
    ctx.metrics
      .counter({
        name: "my_swaps_counter",
        help: "Number of processed swaps",
      })
      .inc(data.swap.length + data.swapV2.length);
  }
}

void main()
```

<Check>
  Access metrics at `http://localhost:9090/metrics` to verify they're being exposed correctly.

  ```
  # HELP my_swaps_counter Number of processed swaps
  # TYPE my_swaps_counter counter
  my_swaps_counter 218598
  ```
</Check>

<Tip>
  Use Grafana dashboards to visualize block processing rate, error rates, and latency trends from your Prometheus metrics.
</Tip>

## Available metric types

You can create different types of Prometheus metrics:

```ts theme={"system"}
for await (const { data, ctx } of stream) {
  // Counter - monotonically increasing value
  ctx.metrics.counter({ name: "events_total", help: "Total events" }).inc();

  // Gauge - value that can go up or down
  ctx.metrics
    .gauge({ name: "queue_size", help: "Current queue size" })
    .set(queueSize);

  // Histogram - observations with configurable buckets
  ctx.metrics
    .histogram({ name: "batch_size", help: "Batch size distribution" })
    .observe(data.transfers.length);
}
```

<Info>
  Expose metrics with `metricsServer()` on your source, then visualize them with [Pipes UI](../basic-development/pipes-ui).
</Info>

See the [Profiling](./profiling) guide for the built-in per-batch profiler exposed on the same metrics endpoint.


# Profiling
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/solana/guides/advanced-topics/profiling

Measure where time is spent in a Solana pipe

Pipes SDK ships a built-in per-batch profiler. It records how long each part of the pipeline takes. When a [`metricsServer()`](../../reference/utility-components/metrics-server) is attached to the source, the profiler output is served as JSON at `http://localhost:<metrics_port>/profiler` and rendered live in [Pipes UI](../basic-development/pipes-ui).

## Enabling the profiler

The profiler is **on by default** when `process.env.NODE_ENV !== 'production'` and **off** otherwise. Override explicitly with `profiler: true` or `profiler: false` on the source.

```ts theme={"system"}
solanaPortalStream({
  portal: 'https://portal.sqd.dev/datasets/solana-mainnet',
  outputs: /* ... */,
  metrics: metricsServer({ port: 9090 }),
  profiler: true,   // force on — useful in production
})
```

## Interpreting the output

The pipeline is represented as a tree. Each node reports how long was spent in that stage of a batch. A typical tree looks like:

```
batch
├── fetch data
├── apply transformers
│   ├── track progress
│   └── Solana decoder
├── clickhouse
│   ├── data handler
│   ├── insert cursor
│   └── cleanup cursors
└── metrics processing
```

## Custom spans

Wrap any code in your target or transformer to get it to appear as a tree node. `ctx.profiler.start()` is a no-op when the profiler is disabled, so the instrumentation is safe to leave in place.

```ts theme={"system"}
onData: async ({ data, ctx }) => {
  const span = ctx.profiler.start('my measure')
  await myDataProcessing(data)
  span.end()
},
```

The named span appears under its parent node in the tree:

```
batch
...
├── clickhouse
│   ├── data handler
│   │   └── my measure
...
```


# Railway deployment
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/solana/guides/advanced-topics/railway-deployment

Deploying a Pipes SDK project to Railway

This guide walks you through deploying a Pipes SDK indexer to [Railway](https://railway.app). You'll end up with four services running together: your indexer, a database (PostgreSQL or ClickHouse), and the Pipe UI dashboard.

## Prerequisites

* A [Railway account](https://railway.app)
* Your project pushed to a **public or private GitHub repository**
* Your project built with `pipes init` (which generates a `Dockerfile` and `docker-compose.yaml`)

***

## Option A — Drag & Drop via the Railway Dashboard

The quickest way to get started is to drop your `docker-compose.yaml` directly onto the Railway project canvas.

### Step 1 — Create a new project on Railway

1. Go to [railway.app/new](https://railway.app/new) and click **Empty Project**.

### Step 2 — Drag and drop your `docker-compose.yaml`

1. Open your project canvas.
2. Drag the `docker-compose.yaml` file from your project root and drop it anywhere on the canvas.

Railway parses the file and creates a service for each entry. For a typical Pipes SDK project this produces:

| Service                    | Image / Source                     |
| -------------------------- | ---------------------------------- |
| Your indexer               | Built from your local `Dockerfile` |
| `postgres` or `clickhouse` | Official Docker images             |

### Step 3 — Link the indexer service to your GitHub repository

1. Click the indexer service card on the canvas.
2. Go to **Settings → Source** and choose **GitHub Repo**.
3. Select your repository and branch.

Railway will now redeploy automatically on every push.

### Step 4 — Add the Pipe UI service

1. Click **+ New Service** on the canvas.
2. Choose **Docker Image** and enter `iankguimaraes/pipe-ui:latest`.
3. Go to the service's **Variables** tab and add:
   ```
   METRICS_SERVER_URL=${{Pipes.RAILWAY_PRIVATE_DOMAIN}}:9090
   ```
   Replace `Pipes` with the actual name Railway assigned to your indexer service.

### Step 5 — Generate public domains

For each service that needs a public URL:

1. Click the service card.
2. Go to **Settings → Networking → Public Networking**.
3. Click **Generate Domain** and set the correct port (`3000` for Pipe UI, `5432` for PostgreSQL, `8123` for ClickHouse).

***

## Option B — Railway CLI

If you prefer the terminal, the Railway CLI gives you full control over every service and environment variable.

### Step 1 — Install the Railway CLI

```bash theme={"system"}
# macOS / Linux
curl -fsSL https://railway.app/install.sh | sh

# or via npm
npm install -g @railway/cli
```

### Step 2 — Log in to Railway

```bash theme={"system"}
railway login
```

This opens a browser window for OAuth authentication. After approving, the CLI is authenticated for the current session.

### Step 3 — Initialize the Railway project

Run this from the root of your indexer project:

```bash theme={"system"}
railway init --name "your-project-name"
```

Use the same name as the `name` field in your `package.json`. This creates a new project on Railway and links the current directory to it.

### Step 4 — Add the database service

**If your project uses PostgreSQL** (projects with `drizzle.config.ts`):

```bash theme={"system"}
railway add -d postgres
```

Railway provisions a managed PostgreSQL instance and injects a `DATABASE_URL` variable automatically.

**If your project uses ClickHouse:**

```bash theme={"system"}
railway add \
  --service Clickhouse \
  --image clickhouse/clickhouse-server:latest \
  --variables CLICKHOUSE_DB=pipes \
  --variables CLICKHOUSE_USER=default \
  --variables CLICKHOUSE_PASSWORD=password
```

### Step 5 — Add the indexer service

Replace `your-org/your-repo` with your actual GitHub repository slug.

**PostgreSQL project:**

```bash theme={"system"}
railway add \
  --service Pipes \
  --repo your-org/your-repo \
  --variables "DB_CONNECTION_STR=\${{Postgres.DATABASE_URL}}"
```

**ClickHouse project:**

```bash theme={"system"}
railway add \
  --service Pipes \
  --repo your-org/your-repo \
  --variables "CLICKHOUSE_URL=http://\${{Clickhouse.RAILWAY_PRIVATE_DOMAIN}}:8123" \
  --variables CLICKHOUSE_DB=pipes \
  --variables CLICKHOUSE_USER=default \
  --variables CLICKHOUSE_PASSWORD=password
```

> **Note on variable syntax:** `${{ServiceName.VARIABLE}}` is Railway's cross-service reference syntax. The shell requires escaping the `$` as `\$` when passing it through the CLI; Railway resolves it at runtime.

The `--repo` flag links this service to your GitHub repository and enables automatic deployments on every push to the default branch.

### Step 6 — Add the Pipe UI dashboard

```bash theme={"system"}
railway add \
  --service PipeUI \
  --image iankguimaraes/pipe-ui:latest \
  --variables "METRICS_SERVER_URL=\${{Pipes.RAILWAY_PRIVATE_DOMAIN}}:9090"
```

The Pipe UI connects to your indexer's metrics endpoint (port 9090, exposed by the indexer at runtime).

### Step 7 — Generate public domains

Give each service a publicly accessible URL:

**PostgreSQL project:**

```bash theme={"system"}
# Expose the database
railway domain --service Postgres --port 5432

# Expose the UI
railway domain --service PipeUI --port 3000
```

**ClickHouse project:**

```bash theme={"system"}
# Expose the database
railway domain --service Clickhouse --port 8123

# Expose the UI
railway domain --service PipeUI --port 3000
```

### Step 8 — Open the Railway dashboard

```bash theme={"system"}
railway open
```

This opens your project in the Railway web UI where you can monitor deployments, view logs, and manage environment variables.

***

## Service Architecture

```
┌──────────────────────────────────────────────────────┐
│                   Railway Project                    │
│                                                      │
│  ┌─────────────┐     private network                 │
│  │   Database  │◄────────────────────┐               │
│  │  (Postgres  │                     │               │
│  │ /Clickhouse)│                     │               │
│  └──────┬──────┘                     │               │
│         │ public domain (optional)   │               │
│                               ┌──────┴──────┐        │
│                               │   Indexer   │        │
│                               │   (Pipes)   │        │
│                               └──────┬──────┘        │
│                                      │ :9090 metrics  │
│                               ┌──────▼──────┐        │
│                               │   Pipe UI   │        │
│                               │  (PipeUI)   │        │
│                               └──────┬──────┘        │
│                                      │ public domain  │
└──────────────────────────────────────┼───────────────┘
                                       ▼
                                  Browser / API
```

Services communicate over Railway's private network using `${{ServiceName.RAILWAY_PRIVATE_DOMAIN}}` references. Only the UI (and optionally the database) need public domains.

***

## Environment Variables Reference

### Indexer (PostgreSQL)

| Variable            | Value                        |
| ------------------- | ---------------------------- |
| `DB_CONNECTION_STR` | `${{Postgres.DATABASE_URL}}` |

### Indexer (ClickHouse)

| Variable              | Value                                                |
| --------------------- | ---------------------------------------------------- |
| `CLICKHOUSE_URL`      | `http://${{Clickhouse.RAILWAY_PRIVATE_DOMAIN}}:8123` |
| `CLICKHOUSE_DB`       | `pipes`                                              |
| `CLICKHOUSE_USER`     | `default`                                            |
| `CLICKHOUSE_PASSWORD` | `password`                                           |

### ClickHouse service

| Variable              | Value      |
| --------------------- | ---------- |
| `CLICKHOUSE_DB`       | `pipes`    |
| `CLICKHOUSE_USER`     | `default`  |
| `CLICKHOUSE_PASSWORD` | `password` |

### Pipe UI

| Variable             | Value                                    |
| -------------------- | ---------------------------------------- |
| `METRICS_SERVER_URL` | `${{Pipes.RAILWAY_PRIVATE_DOMAIN}}:9090` |

***

## Dockerfile Overview

Your project's `Dockerfile` (generated by `pipes init`) uses a two-stage build:

1. **Builder stage** — installs dependencies with `pnpm`, compiles TypeScript to `dist/`.
2. **Runner stage** — copies only the production build, runs migrations (PostgreSQL only), then starts the indexer.

The indexer exposes **port 9090** for metrics, which Pipe UI connects to.

```dockerfile theme={"system"}
EXPOSE 9090
CMD ["sh", "-lc", "pnpm db:generate && pnpm db:migrate && node dist/index.js"]
# (PostgreSQL only; ClickHouse projects skip the migration step)
```

***

## Troubleshooting

**Indexer fails to start — cannot connect to database**
The database service may not be healthy yet. Railway starts services in parallel; the indexer's health-check retry logic should handle this, but you can also set a startup delay under **Settings → Deploy → Start Command**.

**`${{...}}` variables show as literal strings**
Cross-service references are resolved at deploy time. Make sure both services are in the same Railway project and the referenced service name matches exactly (case-sensitive).

**ClickHouse connection refused**
Confirm `CLICKHOUSE_URL` uses the private domain (`RAILWAY_PRIVATE_DOMAIN`), not a public URL, and that port 8123 is correct.

**Pipe UI shows no data**
Check that `METRICS_SERVER_URL` points to the private domain of the indexer service and that port 9090 is included.


# Stateful transforms
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/solana/guides/advanced-topics/stateful-transforms

Six approaches to maintaining state across pipe batches, with trade-offs and examples

A stateful transform is any step in your pipeline that produces output based on more than the current batch — for example, running balances, sliding-window aggregates, or enrichment lookups. This page surveys the available approaches and when to choose each one.

## Before you add state

The cleanest solution is often to emit raw events and let the downstream database derive the state at query time. ClickHouse materialized views and Postgres views both work for this. If your logic can be expressed as SQL and you can tolerate slightly higher query latency, prefer this over transformer state — it eliminates crash recovery and fork handling entirely on the transformer side.

If your logic is hard to express in SQL, or if the derived state must be pre-computed before reaching the target, read on.

## At a glance

| Approach                                                | State lives in | Persistence across restarts    | Fork handling           | Extra infra   | Best for                                     |
| ------------------------------------------------------- | -------------- | ------------------------------ | ----------------------- | ------------- | -------------------------------------------- |
| [A. Pure in-RAM](#a-pure-in-ram)                        | JS heap        | rebuilt from portal on startup | `fork()` callback       | none          | sliding windows, candles, rolling aggregates |
| [B. ClickHouse MVs](#b-clickhouse-materialized-views)   | ClickHouse     | ✓                              | `sign = -1` rows        | ClickHouse    | SQL-expressible analytics                    |
| [C. SQLite transformer](#c-sqlite-transformer)          | Local file     | ✓ (delta table)                | `fork()` callback       | none          | moderate state                               |
| [D. Postgres/Drizzle target](#d-postgresdrizzle-target) | Postgres       | ✓                              | ✓ automatic             | Postgres      | atomic state + output, Postgres target       |
| [E. Apache Flink](#e-apache-flink)                      | Flink cluster  | ✓                              | via compensating events | Kafka + Flink | TB-scale distributed state                   |
| [F. External KV store](#f-external-kv-store)            | Redis / Valkey | ✓ (with AOF/RDB)               | `fork()` callback       | Redis         | µs-latency lookups, multi-process state      |

***

## A. Pure in-RAM

Keep state in a JavaScript `Map` or array inside the transformer closure. No external storage is involved.

**When to use:**

* State can be derived from a bounded window of recent blocks (e.g., last N blocks or last M seconds).
* You can afford to replay that window on restart (warm-up time ∝ window size).
* State loss is contained: at most one window's worth of history needs to be replayed.

**When not to use:**

* State grows without bound (e.g., all-time ERC-20 balances). Use [Postgres approach D](#d-postgresdrizzle-target) instead — specifically the in-memory + Postgres mirror sub-approach.
* The warm-up window is too large to replay quickly on every restart.

### The warm-up pattern

When the process restarts, the target's cursor tells you where the pipeline left off. The in-RAM state is gone. To rebuild it, call `portal.getStream()` in the `start()` callback — the raw portal client API, independent of the main stream:

```typescript theme={"system"}
start: async ({ portal, state, logger }) => {
  if (!state.current) return  // first ever run: start empty

  const warmupFrom = Math.max(state.initial, state.current.number - LOOKBACK_BLOCKS)
  if (warmupFrom >= state.current.number) return

  for await (const { blocks } of portal.getStream({
    type: 'evm',
    fromBlock: warmupFrom,
    toBlock: state.current.number,
    fields: { block: { number: true }, log: { data: true } },
    logs: [{ address: [CONTRACT], topic0: [EVENT_TOPIC] }],
  })) {
    for (const block of blocks) {
      for (const log of block.logs) {
        // rebuild in-RAM state from block and log fields
      }
    }
  }
}
```

`portal` is a live `PortalClient` already connected to the dataset. The warm-up query's `toBlock` is the saved cursor, so it terminates immediately after the pipeline resumes from `cursor + 1`. Multiple in-RAM transformers each run their own `start()` warm-up in parallel (the SDK calls child `start()` callbacks concurrently).

### Fork handling

`target.fork()` fires first (ClickHouse `onRollback` or drizzle snapshot rollback), then the transformer's `fork()` callback. At that point the database already reflects pre-fork state. In `fork()`, drop in-RAM entries for blocks beyond the rollback cursor:

```typescript theme={"system"}
fork: async (cursor, { logger }) => {
  recentEntries = recentEntries.filter(e => e.blockNumber <= cursor.number)
}
```

### Composability

Use the same `initQueue`/`WriteQueue` pattern as the Postgres examples. For ClickHouse targets the queue holds closures over `ClickhouseStore` instead of a Postgres `Transaction`:

```typescript theme={"system"}
type CHS = { insert(params: { table: string; values: unknown[]; format: string }): Promise<unknown> }

class WriteQueue {
  private ops: Array<(store: CHS) => Promise<void>> = []
  push(op: (store: CHS) => Promise<void>): void { this.ops.push(op) }
  async flush(store: CHS): Promise<void> { for (const op of this.ops) await op(store) }
}
```

<Expandable title="Complete example — rolling 1-hour SQD transfer volume">
  See [`13.stateful-transform-in-ram.example.ts`](https://github.com/subsquid-labs/pipes-sdk-docs/blob/master/src/advanced/evm/13.stateful-transform-in-ram.example.ts) for the full implementation: a rolling \~1-hour transfer volume tracker for the SQD token on Arbitrum, with portal warm-up, fork handling, and the WriteQueue composability pattern targeting ClickHouse.
</Expandable>

***

## B. ClickHouse materialized views

Write raw events to a base table; let ClickHouse compute derived state via materialized views (MVs). The transformer is stateless — it only emits events, not pre-computed state.

**When to use:**

* Your aggregation logic is expressible in SQL.
* ClickHouse is already your target.
* You want derived state updated automatically without any transformer code.

**When not to use:**

* Logic requires imperative iteration (e.g., order-dependent simulation).
* Each MV chain adds latency — avoid long dependency chains for latency-sensitive consumers.
* Very frequent writes on lightweight data: prefer plain (non-materialized) views if you have spare CPU on the database machine.

### The core limitation: MVs see only new rows

A materialized view fires when new rows are inserted into its source table. Its `SELECT` clause only operates on the **newly inserted batch**, not the full table. Running totals like cumulative balance cannot be written directly in the MV `SELECT`.

### Workaround: auxiliary aggregating tables

Maintain a separate "current state" table using `AggregatingMergeTree` with `argMaxState`. The MV reads this table alongside the new rows to resolve the latest value before the current batch:

```sql theme={"system"}
-- Stores latest balance per pool (AggregatingMergeTree = efficient upsert)
CREATE TABLE current_balances (
    pool_address        String,
    token_a_balance_raw AggregateFunction(argMax, Int256, Tuple(DateTime, UInt16, UInt16)),
    token_b_balance_raw AggregateFunction(argMax, Int256, Tuple(DateTime, UInt16, UInt16))
) ENGINE = AggregatingMergeTree()
ORDER BY pool_address;

-- MV that keeps current_balances up to date
CREATE MATERIALIZED VIEW current_balances_mv TO current_balances AS
SELECT
    pool_address,
    argMaxState(token_a_balance_raw, (timestamp, transaction_index, log_index)) AS token_a_balance_raw,
    argMaxState(token_b_balance_raw, (timestamp, transaction_index, log_index)) AS token_b_balance_raw
FROM balances_history
GROUP BY pool_address;
```

A downstream MV that needs the running balance queries `current_balances` with `argMaxMerge()`:

```sql theme={"system"}
latest_pool_balances AS (
    SELECT
        pool_address,
        argMaxMerge(token_a_balance_raw) AS balance_token_a_raw,
        argMaxMerge(token_b_balance_raw) AS balance_token_b_raw
    FROM current_balances
    WHERE pool_address IN (SELECT pool_address FROM unique_pools_to_insert)
    GROUP BY pool_address
)
```

### Temporal joins with ASOF JOIN

When you need "the most recent price before each event", use `ASOF JOIN`:

```sql theme={"system"}
SELECT ...
FROM liquidity_events_raw ml
ASOF JOIN latest_prices wp
    ON wp.pool_address = ml.pool_address
    AND wp.ts_num + wp.transaction_index * 100_000 + wp.log_index
        <= ml.ts_num + ml.transaction_index * 100_000 + ml.log_index
WHERE ml.protocol = 'uniswap_v4'
```

The `ASOF JOIN` selects the latest row in `latest_prices` whose ordering key is ≤ the current event's key — effectively "last price before this log".

### Fork rollback

ClickHouse is non-transactional. Use `CollapsingMergeTree` with a `sign` column: insert `sign = 1` rows on the way forward and `sign = -1` rows to cancel them on rollback. Your `onRollback` handler computes which blocks to cancel and inserts the negating rows.

<Expandable title="Complete example — DEX liquidity tracking">
  See [`pipes-sqdgn-dex-example/pipes/evm/liquidity/liquidity.sql`](https://github.com/subsquid-labs/pipes-sqdgn-dex-example/blob/master/pipes/evm/liquidity/liquidity.sql) for a full production SQL schema: `liquidity_events_raw` as the base table, `CollapsingMergeTree` for rollback, `AggregatingMergeTree` + `argMaxState` for current pool balances, an `ASOF JOIN` MV for V4 liquidity, and separate V2/V3/V4 MV chains targeting `balances_history`.
</Expandable>

***

## C. SQLite transformer

Keep transformer state in a local SQLite database. The transformer reads and writes SQLite; the downstream target (typically ClickHouse) receives the pre-computed rows.

**When to use:**

* State is too large for RAM.
* You need random-access lookups (e.g., "current balance of address X") that would be slow as a linear scan of in-RAM arrays.
* You're not using Postgres as your target (otherwise see [approach D](#d-postgresdrizzle-target) for better atomicity).
* The indexer runs on persistent infrastructure (SQLite file must survive restarts).

**When not to use:**

* The indexer runs on ephemeral infrastructure (containers, spot VMs). SQLite is lost on restart.
* State requires complex analytical SQL (window functions, multi-table joins) — consider DuckDB as a drop-in alternative with full analytical query support.

### The delta table pattern

SQLite and the downstream target commit separately — a crash between them leaves the two out of sync. A `balance_deltas` table records the net change per address per block, allowing `rollbackTo(blockNumber)` to invert any set of blocks atomically:

```typescript theme={"system"}
// Schema
db.exec(`
  CREATE TABLE IF NOT EXISTS balance_deltas (
    address      TEXT    NOT NULL,
    block_number INTEGER NOT NULL,
    delta        TEXT    NOT NULL,
    PRIMARY KEY (address, block_number)
  )
`)

function rollbackTo(blockNumber: number) {
  db.transaction(() => {
    const deltas = db.prepare('SELECT address, delta FROM balance_deltas WHERE block_number > ?').all(blockNumber)
    const net = new Map<string, bigint>()
    for (const { address, delta } of deltas) net.set(address, (net.get(address) ?? 0n) + BigInt(delta))
    for (const [address, delta] of net) {
      const { balance } = db.prepare('SELECT balance FROM balances WHERE address = ?').get(address) as any
      db.prepare('INSERT INTO balances (address, balance) VALUES (?, ?) ON CONFLICT(address) DO UPDATE SET balance = excluded.balance')
        .run(address, (BigInt(balance) - delta).toString())
    }
    db.prepare('DELETE FROM balance_deltas WHERE block_number > ?').run(blockNumber)
    db.prepare('DELETE FROM processed_blocks WHERE block_number > ?').run(blockNumber)
  })()
}
```

### Crash recovery in start()

Compare the SQLite high-water mark with the pipeline cursor. If SQLite is ahead, roll back to match:

```typescript theme={"system"}
start: async ({ state }) => {
  const sqliteLastBlock  = db.prepare('SELECT MAX(block_number) as m FROM processed_blocks').get().m ?? null
  const pipelineLastBlock = state.current?.number ?? null

  if (sqliteLastBlock !== null && (pipelineLastBlock === null || sqliteLastBlock > pipelineLastBlock)) {
    rollbackTo(pipelineLastBlock ?? -1)  // SQLite crashed ahead of cursor — roll back
  } else if (sqliteLastBlock !== pipelineLastBlock) {
    throw new Error(`State mismatch: SQLite=${sqliteLastBlock}, cursor=${pipelineLastBlock}. Delete the SQLite file to rebuild.`)
  }
}
```

### Historical-only variant

If you're indexing only finalized data and will never see forks, drop the delta table and accept that a crash requires rebuilding from scratch:

```typescript theme={"system"}
start: async ({ state }) => {
  if (sqliteLastBlock !== pipelineLastBlock) {
    throw new Error(`Delete ${SQLITE_DB_PATH} to rebuild.`)
  }
}
```

<Expandable title="Complete examples">
  * [`09.stateful-transform-on-sqlite.example.ts`](https://github.com/subsquid-labs/pipes-sdk-docs/blob/master/src/advanced/evm/09.stateful-transform-on-sqlite.example.ts) — full delta-table implementation with fork and crash recovery
  * [`10.stateful-transform-on-sqlite-no-forks.example.ts`](https://github.com/subsquid-labs/pipes-sdk-docs/blob/master/src/advanced/evm/10.stateful-transform-on-sqlite-no-forks.example.ts) — simpler historical-only variant
</Expandable>

***

## D. Postgres/Drizzle target

When your target is already Postgres, the `drizzleTarget` can commit transformer state and output rows inside the **same serializable transaction** as the cursor save. This gives the strongest atomicity guarantees of any approach: a crash between `transform()` and the cursor save is impossible because both commit together.

**When to use:**

* Postgres is your output target.
* You want zero crash recovery code (atomicity handles it automatically).
* Fork rollback should be automatic (drizzleTarget installs snapshot triggers).

**When not to use:**

* Your target is ClickHouse or another non-Postgres database.
* State is too large for Postgres (rare).

### The WriteQueue / initQueue pattern

Multiple stateful transformers all need to write inside the same transaction. The `WriteQueue` collects their write closures; `initQueue` wraps each batch in a `Piped<T>` with a fresh queue; `onData` flushes everything:

```typescript theme={"system"}
class WriteQueue {
  private ops: Array<(tx: Transaction) => Promise<void>> = []
  push(op: (tx: Transaction) => Promise<void>): void { this.ops.push(op) }
  async flush(tx: Transaction): Promise<void> { for (const op of this.ops) await op(tx) }
}

function initQueue<T>() {
  return createTransformer<T, Piped<T>>({
    transform: (data) => ({ payload: data, writes: new WriteQueue() }),
  })
}

// Pipeline:
stream
  .pipe(initQueue<DecodedBatch>())
  .pipe(transformerA(db))
  .pipe(transformerB(db))
  .pipeTo(drizzleTarget({
    db,
    tables: [tableA, tableB],
    onData: async ({ tx, data }) => { await data.writes.flush(tx) },
  }))
```

`onData` stays a one-liner regardless of how many transformers are chained.

### Sub-approach 1 — Stateless transform (per-batch DB reads)

`transform()` reads current state from Postgres (the last committed snapshot), computes the delta, and pushes write closures to the queue. No in-RAM Map survives between batches.

* ✓ No RAM limit on state size
* ✓ Zero fork handling code (snapshot triggers on `tables` cover rollback)
* ✗ One `SELECT … WHERE address IN (…)` per batch

<Expandable title="Complete example — stateless (example 11)">
  See [`11.stateful-transforms-postgres-stateless.example.ts`](https://github.com/subsquid-labs/pipes-sdk-docs/blob/master/src/advanced/evm/11.stateful-transforms-postgres-stateless.example.ts): two transformers (`BalanceTransformer` + `TransferCountTransformer`) reading from Postgres each batch and writing atomically via WriteQueue.
</Expandable>

### Sub-approach 2 — In-memory + Postgres mirror

`start()` loads the full state into in-RAM Maps. `transform()` reads/writes the Maps with no DB round trips per batch. `fork()` reloads the Maps from Postgres after drizzleTarget commits the snapshot rollback.

* ✓ No per-batch DB reads — all reads from memory after startup
* ✓ Fast for large batches with many distinct keys
* ✗ Full state must fit in RAM
* ✗ Startup time is O(state size)
* ✗ `fork()` callbacks required to resync Maps after rollback

<Expandable title="Complete example — in-memory (example 12)">
  See [`12.stateful-transforms-postgres-in-memory.example.ts`](https://github.com/subsquid-labs/pipes-sdk-docs/blob/master/src/advanced/evm/12.stateful-transforms-postgres-in-memory.example.ts): same two transformers with in-RAM Maps loaded from Postgres at startup and reloaded on fork.
</Expandable>

<Note>
  For both sub-approaches, all state tables must be listed in `drizzleTarget`'s `tables` array. This installs PostgreSQL snapshot triggers that roll them back automatically on a blockchain reorg. The `onStart` callback can run `CREATE TABLE IF NOT EXISTS` for quick setup; in production, use [drizzle-kit migrations](https://orm.drizzle.team/docs/migrations) instead.
</Note>

***

## E. Apache Flink

[Apache Flink](https://flink.apache.org) is a distributed stateful stream-processing framework. The Pipes SDK acts as a data source feeding Flink via Kafka or a direct connector.

**When to use:**

* State is too large for a single machine (terabytes).
* Your problem requires stateful joins across multiple independent streams (e.g., correlate DEX trades with lending liquidations across different chains).
* You need exactly-once semantics across multiple heterogeneous sinks.

**When not to use:**

* Single-node deployments — the operational overhead (JVM runtime, cluster management, ZooKeeper or KRaft, checkpoint storage) is only justified when the problem genuinely requires distributed state.

**Architecture:** The Pipes SDK emits raw events to Kafka (one topic per event type). On a blockchain fork, it emits compensating rows (e.g., `sign = -1`) that Flink sees as normal data and can handle with a subtract-and-recompute pattern. Flink manages its own checkpoints; crash recovery is handled entirely by Flink.

***

## F. External KV store

Use Redis, Valkey, or a similar key-value store as a fast external state backend.

**When to use:**

* Multiple parallel pipeline instances must share state (horizontal scaling of the indexer).
* Per-key lookups must complete in under 1 ms (e.g., enriching 50 k events per second with metadata from a 100 M-entry map that doesn't fit in RAM).

**When not to use:**

* A single-process indexer is sufficient — adding Redis increases operational complexity for no benefit.
* You need transactional state + output commits (use approach D instead).

**Fork handling:** The transformer's `fork()` callback must delete or undo the Redis keys written for rolled-back blocks. Keep a per-block write log (similar to the SQLite delta table) to know which keys to revert.

**Crash safety:** Redis is not durable by default. Enable AOF or RDB persistence, or treat Redis purely as a warm cache and accept that a Redis restart requires a replay from the pipeline cursor.

***

## Fork callbacks and crash recovery

The fork handling responsibilities differ by approach:

| Approach            | `fork()` needed in transformer        | How DB state is rolled back        |
| ------------------- | ------------------------------------- | ---------------------------------- |
| A. In-RAM           | ✓ — prune entries > cursor            | n/a (state is RAM-only)            |
| B. ClickHouse MVs   | ✗ — handled in `onRollback`           | `sign = -1` rows via `onRollback`  |
| C. SQLite           | ✓ — calls `rollbackTo(cursor.number)` | `rollbackTo()` reverts delta table |
| D. Postgres/drizzle | ✗ — automatic                         | snapshot triggers via `tables`     |
| E. Flink            | ✗ — compensating events               | Flink checkpoint rollback          |
| F. External KV      | ✓ — revert write log                  | manual key deletion                |

**Ordering guarantee:** `target.fork()` always fires before transformer `fork()` callbacks. By the time your transformer's `fork()` runs, the target (ClickHouse `onRollback`, drizzleTarget snapshot rollback) has already committed the database rollback. It is safe to read the database in `fork()`.

**Crash recovery** (approaches A, B, C only): a crash between the transformer's store write and the target's cursor save leaves state ahead of the pipeline cursor. Handle this in `start()` by comparing your local high-water mark to `state.current`:

```typescript theme={"system"}
start: async ({ state }) => {
  const localLastBlock    = /* read your checkpoint */
  const pipelineLastBlock = state.current?.number ?? null

  if (localLastBlock !== null && (pipelineLastBlock === null || localLastBlock > pipelineLastBlock)) {
    rollbackTo(pipelineLastBlock ?? -1)  // crash recovery: undo ahead-of-cursor writes
  }
}
```

Approaches D (Postgres/drizzle) and E (Flink) are immune to this problem: state and cursor commit atomically.

***

## Composing multiple stateful transformers

When multiple stateful transformers write to the same target, they must not each independently call the target's write API. Use the `WriteQueue` / `initQueue` pattern to collect all writes and flush them in a single `onData` call:

1. **`initQueue<T>()`** wraps the raw batch in `Piped<T>` with a fresh `WriteQueue`. Place it as the first `.pipe()`.
2. **Each transformer** receives `Piped<T>`, pushes closures to `writes`, and returns `Piped<T>` unchanged.
3. **`onData`** calls `data.writes.flush(store_or_tx)` — a one-liner that scales to any number of transformers.

```typescript theme={"system"}
type Piped<T> = { payload: T; writes: WriteQueue }

function initQueue<T>() {
  return createTransformer<T, Piped<T>>({
    transform: (data) => ({ payload: data, writes: new WriteQueue() }),
  })
}
```

Because every domain transformer takes `Piped<T>` as input and produces `Piped<T>` as output, none of them assume a fixed position in the chain — they are all order-independent and can be added or removed without touching the others.

For Postgres targets, `WriteQueue` closures take a `Transaction`; for ClickHouse, they take the structural `CHS` type shown in approach A. The pattern is identical in both cases.


# Cursor management
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/solana/guides/architecture-deep-dives/cursor-management

How pipelines track progress and resume after restarts

A cursor records the last successfully processed slot. Built-in targets (ClickHouse, Drizzle) handle persistence automatically. When using `createTarget` directly you own the full lifecycle.

## The cursor object

```typescript theme={"system"}
type BlockCursor = {
  number: number      // stream resumes from number + 1
  hash?: string       // slot hash — used as parentBlockHash for fork detection
  timestamp?: number  // slot timestamp in seconds
}
```

`hash` is the fork detection tripwire: the SDK sends `parentBlockHash = cursor.hash` in each portal request. An absent hash silently skips fork detection for that request. See [cursor semantics](./fork-handling#5-cursor-semantics) for the full picture.

## Startup: range.from and stored cursors

`range.from` in the decoder sets where the stream begins on a first run — before any cursor exists:

```typescript theme={"system"}
solanaInstructionDecoder({
  range: { from: 'latest' },           // chain head
  // range: { from: 200_000_000 },     // slot number
  // range: { from: '2024-01-01' },    // ISO date string
  // range: { from: new Date() },      // Date object
})
```

Once a cursor is stored, `range.from` is ignored — the stream resumes from `cursor.number + 1`.

## The stream id

The `id` on `solanaPortalStream` is the primary key for all stored state:

```typescript theme={"system"}
solanaPortalStream({ id: 'my-pipeline', ... })
```

Both built-in targets use it to isolate state records, so multiple streams can share one physical table. **Never rename an active stream's id** — the stored cursor is keyed on it, and renaming causes the pipeline to restart from `range.from`.

## ClickHouse target

`clickhouseTarget` saves the cursor after every successful `onData` call and resolves fork and crash-recovery callbacks automatically.

```typescript theme={"system"}
clickhouseTarget({
  client,
  settings: {
    id: 'my-stream',      // overrides solanaPortalStream id (default: 'stream')
    table: 'sync',        // state table name (default: 'sync')
    database: 'default',  // ClickHouse database
    maxRows: 10_000,      // cursor rows to keep per stream id (default: 10,000)
  },
  onData: ...,
  onRollback: ...,
})
```

**`onRollback` is called in two situations:**

* `type: 'offset_check'` — on every startup when a cursor exists. ClickHouse is non-transactional: a crash between `onData` and the cursor save leaves rows newer than the saved cursor. Delete them here. See [non-transactional databases](./fork-handling#4-state-rollback-atomicity).
* `type: 'blockchain_fork'` — when the portal signals a reorg. The rollback cursor is resolved automatically from stored history; your callback only needs to delete rows after `safeCursor.number`.

The same implementation typically serves both:

```typescript theme={"system"}
onRollback: async ({ store, safeCursor }) => {
  await store.removeAllRows({
    tables: ['my_table'],
    where: `slot_number > {n:UInt32}`,
    params: { n: safeCursor.number },
  })
},
```

**State table.** Each row stores the cursor, the last finalized slot, and the unfinalized slot history used for fork recovery. Rows beyond `maxRows` are pruned every 25 saves. Set `maxRows` to cover your network's worst-case reorg depth — see [rollback depth](./fork-handling#3-rollback-depth-and-history-limits).

## Drizzle target

`drizzleTarget` saves the cursor inside the same PostgreSQL transaction as the data write — fully atomic, no crash-recovery pass needed.

```typescript theme={"system"}
drizzleTarget({
  db: drizzle(DB_URL),
  tables: [swapsTable],       // every table onData writes to — required
  settings: {
    state: {
      id: 'my-stream',
      schema: 'public',
      table: 'sync',
      unfinalizedBlocksRetention: 1000,  // cursor rows to keep (default: 1,000)
    },
    transaction: { isolationLevel: 'serializable' },  // default
  },
  onData: async ({ tx, data }) => {
    await tx.insert(swapsTable).values(...)
  },
})
```

**`tables` is required** for every table written in `onData`. At startup the target installs a PostgreSQL trigger on each listed table; the trigger copies the pre-change row into a `<name>__snapshots` table (keyed by slot number and primary key). On a fork the target replays these snapshots in reverse, restoring pre-fork state automatically. Writing to a table not in `tables` raises a runtime error.

Snapshotting only fires for slots at or above the current finalized head — historical slots can never be reorged.

**Advisory lock.** Every batch acquires `pg_try_advisory_xact_lock(hashtext(id))` inside the transaction, preventing concurrent writers on the same stream. Two `drizzleTarget` instances sharing the same `id` will serialize correctly; two with different `id`s run independently.

**Retention.** Snapshot rows below `min(current, finalizedHead) - unfinalizedBlocksRetention` are deleted every 25 batches. Set this to cover your network's worst-case reorg depth.

**Rollback hooks.** `onBeforeRollback` and `onAfterRollback` receive `{ tx, cursor }` and run inside the fork transaction. Use them to perform additional cleanup that the snapshot mechanism cannot cover (e.g., rows in tables not tracked by `tables`).

## Async iterator

When consuming a pipeline with `for await...of` instead of `pipeTo`, the native `[Symbol.asyncIterator]()` always calls `read()` with no cursor — it has no way to accept one. The stream therefore starts from `range.from` on every run.

**Finalized streams.** If the stream only consumes already-finalized slots (no forks possible), rebuilding the stream with `range.from` set to the stored cursor is sufficient:

```typescript theme={"system"}
let cursor = loadCursor()  // BlockCursor | undefined

const stream = solanaPortalStream({
  id: 'my-pipeline',
  portal: '...',
  outputs: solanaInstructionDecoder({
    // cursor.number is the last processed slot; resume from the next one
    range: { from: cursor ? cursor.number + 1 : 0 },
  }),
})

for await (const { data, ctx } of stream) {
  await processData(data)
  saveCursor(ctx.stream.state.current)  // { number, hash, timestamp }
}
```

Save `ctx.stream.state.current` — the full `BlockCursor` of the batch's last slot — not just the number. The `hash` is needed if you later switch to real-time or need the cursor as a fork anchor.

**Real-time streams.** Setting `range.from` to a stored number loses the slot hash. On restart the first request carries no `parentBlockHash`, so fork detection is silently disabled for that request. For real-time streams, use the `pipeToIterator` helper from the [async iteration tab of the fork handling guide](./fork-handling), which accepts an `initialCursor` and passes it directly to `read()` inside `pipeTo`:

```typescript theme={"system"}
const stream = pipeToIterator(
  solanaPortalStream({ id: 'my-pipeline', portal: '...', outputs: solanaInstructionDecoder({ range: { from: 'latest' } }) }),
  loadCursor(),   // full BlockCursor with hash — passed to read(), not range.from
  onFork,
)

for await (const { data, ctx } of stream) {
  await processData(data)
  saveCursor(ctx.stream.state.current)
}
```

`pipeToIterator` preserves `parentBlockHash` across fork rounds because it uses `pipeTo` internally. On a fresh first run, pass `undefined` as `initialCursor` and the stream begins from `range.from` as normal.

## Custom cursor management

When using `createTarget` directly, you own the full cursor lifecycle.

At the start of `write`, fetch the stored cursor and pass it to `read`:

```typescript theme={"system"}
write: async ({ read }) => {
  const cursor = await db.getLatestCursor()
  for await (const { data, ctx } of read(cursor)) {
    // ...
  }
}
```

After processing each batch, persist the cursor together with the fork-recovery state:

```typescript theme={"system"}
await db.transaction(async (tx) => {
  await writeData(tx, data)
  await tx.saveCursor({
    cursor:        ctx.stream.state.current,
    rollbackChain: ctx.stream.state.rollbackChain,
    finalized:     ctx.stream.head.finalized,
  })
})
```

For **transactional stores** (Postgres): save all three fields in the same transaction as the data write. For **non-transactional stores** (ClickHouse): write data first, cursor last, and implement a startup check that detects and corrects any data written after the last cursor save. See [state rollback atomicity](./fork-handling#4-state-rollback-atomicity).

The `fork` callback and the algorithm for resolving rollback cursors from stored history are covered in detail in the [fork handling guide](./fork-handling).

<Card title="Example: cursor management with createTarget" icon="github" href="https://github.com/subsquid-labs/pipes-sdk-docs/blob/master/src/basics/06-cursor-from-target.ts">
  A minimal example showing manual cursor passing in createTarget
</Card>

<Card title="Example: ClickHouse target" icon="github" href="https://github.com/subsquid-labs/pipes-sdk-docs/blob/master/src/basics/08-clickhouse-target.ts">
  Full pipeline with onRollback and onData
</Card>

<Card title="Example: Drizzle / PostgreSQL target" icon="github" href="https://github.com/subsquid-labs/pipes-sdk/blob/main/docs/examples/evm/08.drizzle.example.ts">
  Full pipeline including GraphQL API
</Card>


# Fork handling
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/solana/guides/architecture-deep-dives/fork-handling

Handle Solana forks and rollbacks in real-time streams

When consuming a real-time stream near the chain head, the portal can detect that the client's view of the chain has diverged from the canonical chain — a situation known as a fork or reorg. The portal signals this with an HTTP 409 response containing a sample of slots from the new canonical chain. Your code must find the highest slot that both chains agree on, roll back any state written after that point, and replay from there.

<Note>
  Fork handling is only needed for real-time streams (`range.from: 'latest'`). Historical streams consume already-finalized data and never produce forks. See [Fork detection scope](#7-fork-detection-scope-real-time-streams-only) below.
</Note>

The SDK provides two patterns for consuming a stream. Both use the same state-tracking logic; they differ in how the fork signal is delivered.

<Note>
  If your pipeline includes a [stateful transformer](../advanced-topics/stateful-transforms#fork-callbacks-in-stateful-transformers), it must also implement a `fork` callback to roll back its own state in lockstep with the target.
</Note>

<Tabs>
  <Tab title="Via pipeTo / targets">
    The `pipeTo(createTarget({write, fork}))` pattern keeps fork handling completely separate from batch processing. The SDK catches the 409 internally and calls `fork()` with the portal's consensus slot sample; `write()` never sees the interruption and continues iterating batches without restarting.

    <Steps>
      <Step title="Declare state">
        Two variables span the lifetime of the stream:

        ```typescript theme={"system"}
        let recentUnfinalizedBlocks: BlockCursor[] = []
        let finalizedHighWatermark: BlockCursor | undefined
        ```

        `recentUnfinalizedBlocks` is the local history of unfinalized slots used to find the common ancestor during a fork. `finalizedHighWatermark` tracks the highest finalized slot ever seen — stored as a full `BlockCursor` (number **and** hash) so it can double as a rollback cursor when needed. Both must be declared outside `pipeTo` so `fork()` can access them.
      </Step>

      <Step title="Collect rollback history">
        Inside `write()`, append each batch's unfinalized slots to the local history:

        ```typescript theme={"system"}
        ctx.stream.state.rollbackChain.forEach((bc) => {
          recentUnfinalizedBlocks.push(bc)
        })
        ```

        `ctx.stream.state.rollbackChain` contains only the slots from **this batch** that are above the current finalized head — it is a per-batch delta, not a full snapshot. Always append to the end; never replace or reorder.
      </Step>

      <Step title="Prune finalized blocks">
        After collecting history, prune slots that are now finalized and cap the queue:

        ```typescript theme={"system"}
        if (ctx.stream.head.finalized) {
          if (!finalizedHighWatermark || ctx.stream.head.finalized.number > finalizedHighWatermark.number) {
            finalizedHighWatermark = ctx.stream.head.finalized
          }
          recentUnfinalizedBlocks = recentUnfinalizedBlocks.filter(b => b.number >= finalizedHighWatermark!.number)
        }
        recentUnfinalizedBlocks = recentUnfinalizedBlocks.slice(recentUnfinalizedBlocks.length - 1000)
        ```

        Portal instances behind a load balancer can report different finalized heads. Using the **maximum** seen so far (the high-water mark) prevents the pruning threshold from moving backwards when the stream reconnects to a lagging instance. See [consideration 6](#6-load-balanced-portals-and-a-non-monotonic-finalized-head) for details.
      </Step>

      <Step title="Implement fork()">
        `fork()` receives `previousBlocks` — the portal's current-chain sample — and must return the last good slot cursor, or `null` if recovery is impossible:

        ```typescript theme={"system"}
        fork: async (newConsensusBlocks) => {
          const rollbackIndex = findRollbackIndex(recentUnfinalizedBlocks, newConsensusBlocks)
          if (rollbackIndex >= 0) {
            recentUnfinalizedBlocks.length = rollbackIndex + 1
            return recentUnfinalizedBlocks[rollbackIndex]
          }
          if (finalizedHighWatermark &&
              newConsensusBlocks.every(b => b.number < finalizedHighWatermark!.number)) {
            recentUnfinalizedBlocks = recentUnfinalizedBlocks.filter(b => b.number <= finalizedHighWatermark!.number)
            return finalizedHighWatermark
          }
          return null
        }
        ```

        Three cases: (1) a common ancestor is found in local history — truncate and return it; (2) all `previousBlocks` fall below the finalized high-water mark, meaning the portal's sample doesn't reach local history — return the high-water mark cursor; (3) no recovery possible — return `null`, which surfaces a `ForkCursorMissingError`.
      </Step>
    </Steps>

    <Expandable title="Complete example">
      ```typescript theme={"system"}


      import { BlockCursor, createTarget } from '@subsquid/pipes'
      import { solanaPortalStream, solanaInstructionDecoder } from '@subsquid/pipes/solana'
      import * as orcaWhirlpool from './abi/orca_whirlpool/index.js'

      async function main() {
        let recentUnfinalizedSlots: BlockCursor[] = []
        let finalizedHighWatermark: BlockCursor | undefined

        await solanaPortalStream({
          id: 'forks',
          portal: 'https://portal.sqd.dev/datasets/solana-mainnet',
          outputs: solanaInstructionDecoder({
            programId: orcaWhirlpool.programId,
            instructions: { swap: orcaWhirlpool.instructions.swap },
            range: { from: 'latest' }
          }),
        })
          .pipeTo(createTarget({
            write: async ({read}) => {
              for await (const {data, ctx} of read(recentUnfinalizedSlots[recentUnfinalizedSlots.length-1])) {
                console.log(`Got ${data.swap.length} swaps`)
                ctx.stream.state.rollbackChain.forEach((bc) => { recentUnfinalizedSlots.push(bc) })
                if (ctx.stream.head.finalized) {
                  if (!finalizedHighWatermark || ctx.stream.head.finalized.number > finalizedHighWatermark.number) {
                    finalizedHighWatermark = ctx.stream.head.finalized
                  }
                  recentUnfinalizedSlots = recentUnfinalizedSlots.filter(b => b.number >= finalizedHighWatermark!.number)
                }
                recentUnfinalizedSlots = recentUnfinalizedSlots.slice(recentUnfinalizedSlots.length - 1000)
              }
            },
            fork: async (newConsensusSlots) => {
              const rollbackIndex = findRollbackIndex(recentUnfinalizedSlots, newConsensusSlots)
              if (rollbackIndex >= 0) {
                recentUnfinalizedSlots.length = rollbackIndex + 1
                return recentUnfinalizedSlots[rollbackIndex]
              }
              if (finalizedHighWatermark &&
                  newConsensusSlots.every(b => b.number < finalizedHighWatermark!.number)) {
                recentUnfinalizedSlots = recentUnfinalizedSlots.filter(b => b.number <= finalizedHighWatermark!.number)
                return finalizedHighWatermark
              }
              return null
            }
          }))
      }

      main().then(() => { console.log('\ndone') })


      function findRollbackIndex(chainA: BlockCursor[], chainB: BlockCursor[]): number {
        let aIndex = 0, bIndex = 0, lastCommonIndex = -1
        while (aIndex < chainA.length && bIndex < chainB.length) {
          const a = chainA[aIndex], b = chainB[bIndex]
          if (a.number < b.number) { aIndex++; continue }
          if (a.number > b.number) { bIndex++; continue }
          if (a.hash !== b.hash) return lastCommonIndex
          lastCommonIndex = aIndex; aIndex++; bIndex++
        }
        return lastCommonIndex
      }
      ```
    </Expandable>
  </Tab>

  <Tab title="Via async iteration (workaround)">
    The native `[Symbol.asyncIterator]()` on a `PortalSource` cannot handle forks that require multiple 409 rounds. After a fork, the only option with native iteration is to re-create the stream — but the re-created stream's first request carries no `parentBlockHash`, so the portal cannot detect whether the client is still on the wrong chain and will not send the second 409.

    The root cause: `pipeTo`'s internal `read()` generator maintains a `cursor` variable across fork rounds. After `target.fork()` returns a rollback cursor it sets `cursor = forkedCursor` before re-entering `self.read(cursor)`, keeping `parentBlockHash` populated on every subsequent request. The native async iterator calls `this.read()` with no cursor and has no equivalent mechanism.

    **Workaround:** wrap `pipeTo` in a helper called `pipeToIterator` that bridges its push-based `write()` into a pull-based iterator via a single-item queue with producer acknowledgement. This preserves the `for await...of` interface while using `pipeTo`'s cursor-tracking machinery internally.

    <Steps>
      <Step title="Declare state">
        Same two variables as the `pipeTo` approach — no extra `resumeCursor` needed, since `pipeTo` handles cursor updates internally:

        ```typescript theme={"system"}
        let recentUnfinalizedBlocks: BlockCursor[] = []
        let finalizedHighWatermark: BlockCursor | undefined
        ```
      </Step>

      <Step title="Write the fork callback">
        The fork callback passed to `pipeToIterator` is identical to `fork()` in the `pipeTo` example — the same three-case logic, the same state mutations:

        ```typescript theme={"system"}
        async (newConsensusBlocks) => {
          const rollbackIndex = findRollbackIndex(recentUnfinalizedBlocks, newConsensusBlocks)
          if (rollbackIndex >= 0) {
            recentUnfinalizedBlocks.length = rollbackIndex + 1
            return recentUnfinalizedBlocks[rollbackIndex]
          }
          if (finalizedHighWatermark &&
              newConsensusBlocks.every(b => b.number < finalizedHighWatermark!.number)) {
            recentUnfinalizedBlocks = recentUnfinalizedBlocks.filter(b => b.number <= finalizedHighWatermark!.number)
            return finalizedHighWatermark
          }
          return null
        }
        ```

        The SDK awaits this callback before resuming the stream, so `recentUnfinalizedBlocks` is safe to mutate here without additional locking.
      </Step>

      <Step title="Wrap the source and iterate">
        Pass the source, the initial cursor, and the fork callback to `pipeToIterator`, then iterate normally:

        ```typescript theme={"system"}
        const stream = pipeToIterator(source, recentUnfinalizedBlocks.at(-1), onFork)

        for await (const {data, ctx} of stream) {
          // batch processing — identical to the pipeTo example
        }
        ```
      </Step>
    </Steps>

    <Expandable title="pipeToIterator implementation">
      ```typescript theme={"system"}
      // WORKAROUND — see explanation above the tab
      function pipeToIterator<T>(
        source: { pipeTo(t: ReturnType<typeof createTarget<T>>): Promise<void> },
        initialCursor: BlockCursor | undefined,
        onFork: (previousBlocks: BlockCursor[]) => Promise<BlockCursor | null>
      ): AsyncIterableIterator<{ data: T; ctx: any }> {

        type Slot =
          | { k: 'batch'; v: { data: T; ctx: any } }
          | { k: 'end' }
          | { k: 'error'; err: unknown }

        const queue: Slot[] = []
        let consumerWake: (() => void) | null = null
        let producerAck:  (() => void) | null = null
        const wake = () => { consumerWake?.(); consumerWake = null }

        ;(source.pipeTo as any)(createTarget({
          write: async ({ read }: any) => {
            for await (const batch of read(initialCursor)) {
              queue.push({ k: 'batch', v: batch })
              wake()
              await new Promise<void>(r => { producerAck = r })
            }
            queue.push({ k: 'end' })
            wake()
          },
          fork: onFork,
        })).catch((err: unknown) => { queue.push({ k: 'error', err }); wake() })

        return {
          async next(): Promise<IteratorResult<{ data: T; ctx: any }>> {
            if (!queue.length) await new Promise<void>(r => { consumerWake = r })
            const slot = queue.shift()!
            if (slot.k === 'end')   return { done: true,  value: undefined as any }
            if (slot.k === 'error') throw slot.err
            producerAck?.(); producerAck = null
            return { done: false, value: slot.v }
          },
          [Symbol.asyncIterator]() { return this },
        }
      }
      ```
    </Expandable>

    <Expandable title="Complete example">
      ```typescript theme={"system"}


      import { BlockCursor, createTarget } from '@subsquid/pipes'
      import { solanaPortalStream, solanaInstructionDecoder } from '@subsquid/pipes/solana'
      import * as orcaWhirlpool from './abi/orca_whirlpool/index.js'

      // WORKAROUND — pipeToIterator defined above (see implementation expandable)

      async function main() {
        let recentUnfinalizedSlots: BlockCursor[] = []
        let finalizedHighWatermark: BlockCursor | undefined

        const stream = pipeToIterator(
          solanaPortalStream({
            id: 'forks-async',
            portal: 'https://portal.sqd.dev/datasets/solana-mainnet',
            outputs: solanaInstructionDecoder({
              programId: orcaWhirlpool.programId,
              instructions: { swap: orcaWhirlpool.instructions.swap },
              range: { from: 'latest' }
            }),
          }),
          recentUnfinalizedSlots.at(-1),
          async (newConsensusSlots) => {
            const rollbackIndex = findRollbackIndex(recentUnfinalizedSlots, newConsensusSlots)
            if (rollbackIndex >= 0) {
              recentUnfinalizedSlots.length = rollbackIndex + 1
              return recentUnfinalizedSlots[rollbackIndex]
            }
            if (finalizedHighWatermark &&
                newConsensusSlots.every(b => b.number < finalizedHighWatermark!.number)) {
              recentUnfinalizedSlots = recentUnfinalizedSlots.filter(b => b.number <= finalizedHighWatermark!.number)
              return finalizedHighWatermark
            }
            recentUnfinalizedSlots.length = 0
            return null
          }
        )

        for await (const {data, ctx} of stream) {
          console.log(`Got ${data.swap.length} swaps`)
          ctx.stream.state.rollbackChain.forEach((bc: BlockCursor) => { recentUnfinalizedSlots.push(bc) })
          if (ctx.stream.head.finalized) {
            if (!finalizedHighWatermark || ctx.stream.head.finalized.number > finalizedHighWatermark.number) {
              finalizedHighWatermark = ctx.stream.head.finalized
            }
            recentUnfinalizedSlots = recentUnfinalizedSlots.filter(b => b.number >= finalizedHighWatermark!.number)
          }
          recentUnfinalizedSlots = recentUnfinalizedSlots.slice(recentUnfinalizedSlots.length - 1000)
        }
      }

      main().then(() => { console.log('\ndone') })


      function findRollbackIndex(chainA: BlockCursor[], chainB: BlockCursor[]): number {
        let aIndex = 0, bIndex = 0, lastCommonIndex = -1
        while (aIndex < chainA.length && bIndex < chainB.length) {
          const a = chainA[aIndex], b = chainB[bIndex]
          if (a.number < b.number) { aIndex++; continue }
          if (a.number > b.number) { bIndex++; continue }
          if (a.hash !== b.hash) return lastCommonIndex
          lastCommonIndex = aIndex; aIndex++; bIndex++
        }
        return lastCommonIndex
      }
      ```
    </Expandable>
  </Tab>
</Tabs>

## The common-ancestor search

Both approaches use the same merge-sort scan. Given two ascending-sorted arrays of `BlockCursor` — local history and the portal's `previousBlocks` — `findRollbackIndex` returns the index in local history of the last entry that both chains agree on (same slot number **and** hash):

```typescript theme={"system"}
function findRollbackIndex(chainA: BlockCursor[], chainB: BlockCursor[]): number {
  let aIndex = 0, bIndex = 0, lastCommonIndex = -1
  while (aIndex < chainA.length && bIndex < chainB.length) {
    const a = chainA[aIndex], b = chainB[bIndex]
    if (a.number < b.number) { aIndex++; continue }
    if (a.number > b.number) { bIndex++; continue }
    if (a.hash !== b.hash) return lastCommonIndex   // chains diverged here
    lastCommonIndex = aIndex; aIndex++; bIndex++
  }
  return lastCommonIndex
}
```

The scan advances the pointer for the lower-numbered entry until both point to the same slot number. A hash mismatch means the chains diverged at this number; `lastCommonIndex` holds the last agreement point. Returning `-1` means no common ancestor was found in the sample.

## Edge cases and considerations

<AccordionGroup>
  <Accordion title="1. Rollback history bootstrap">
    **Empty history at stream start.** The rollback chain is built batch-by-batch from `ctx.stream.state.rollbackChain`. Until the first batch arrives the history is empty. A fork arriving before any batch has been processed means `fork()` will find no common ancestor and must return `null`, which the SDK turns into a fatal error. For a long-running process this window is typically acceptable, but it matters for freshly started consumers.

    **History gaps from fast-moving finalization.** `rollbackChain` in each batch contains only the slots from *that batch* that are strictly above the current finalized head. A slot that was already at or below the finalized head when its batch was fetched will never appear in any rollback chain and will therefore be absent from history. This can leave gaps in the number sequence. Algorithms that assume a contiguous history will fail; always match by both number *and* hash.

    **No finalized-head info in a batch.** When `batch.head.finalized` is absent, no history is accumulated. On networks or portal deployments that do not yet surface finality data, the rollback chain stays empty indefinitely. On such networks fork recovery is impossible unless unfinalized slots are tracked through another mechanism.
  </Accordion>

  <Accordion title="2. The previousBlocks payload from the 409">
    **Ascending order, match by hash *and* number.** The API spec requires matching on both. Matching only by number is wrong — different chains can have the same slot number. The array is ordered ascending (lowest number first); the last entry is the most recent slot the portal knows about.

    **`previousBlocks` may have no overlap with local history.** The portal sends a bounded sample. If `findRollbackIndex` finds no agreement point at all (returns -1) and no HWM fallback applies, fork recovery is impossible — return `null`. The SDK will surface a `ForkCursorMissingError`. Do not silently roll back to slot 0 or crash.

    **Multiple consecutive 409s converge to the common ancestor.** These two cases are distinct from each other: when `findRollbackIndex` *does* find an overlap point, the stream rolls back there and resumes. If the true common ancestor is deeper still — because the `previousBlocks` sample only reached partway — the portal detects another mismatch and sends a fresh 409 with an older window, this time closer to the true ancestor. The stream converges over several rounds. `fork()` must be idempotent across these calls; truncating the history array in place handles this correctly, since each call receives a shorter local history. Database-backed approaches must also handle re-entrant rollback calls.
  </Accordion>

  <Accordion title="3. Rollback depth and history limits">
    **Fork deeper than your history.** If you cap rollback history (e.g. to 1000 slots), a reorg deeper than the cap is unrecoverable. Choose the cap based on the worst-case reorg depth for your target network. Ethereum mainnet finalizes within \~64 slots (\~2 epochs), but PoW or pre-finality networks can reorg much deeper. Fail loudly rather than silently replaying from slot 0.

    **The finalized slot as the last-resort anchor.** Keep the current finalized slot *in* your rollback history even though it is technically not unfinalized. It is the guaranteed safe floor: the portal will never ask you to roll back past it. Having it available means `fork()` can always return a valid cursor for the deepest possible reorg. Pruning with `number > finalized` instead of `number >= finalized` removes this anchor and makes very deep reorgs unrecoverable.

    **History that never gets pruned.** If the portal never sends a finalized head, rollback history will grow without bound. Apply a slot-count cap as a secondary safeguard.
  </Accordion>

  <Accordion title="4. State rollback atomicity">
    **Business state and rollback-chain history must be rolled back atomically.** For databases with transactions (Postgres), both must be updated in the same transaction — a crash between the two leaves `fork()` computing the wrong rollback point.

    **For non-transactional databases (ClickHouse), atomicity is not achievable; use a crash-recovery callback instead.** Write application data first, write the rollback-chain checkpoint second. A crash after data but before the checkpoint save leaves the checkpoint pointing to the previous batch. On every restart, before the stream resumes, the checkpoint cursor should be read and used to purge any rows written after it — this closes the gap. This is how the Pipes SDK ClickHouse target works: `onRollback` is invoked with `type: 'offset_check'` on every startup so user code can delete the partial batch. Because ClickHouse `DELETE`s are asynchronous and unsafe under concurrent writes, the SDK inserts tombstone rows (`sign = -1`) via `CollapsingMergeTree` instead of issuing true deletes; queries that need to see only live rows must use the `FINAL` modifier.

    **Rolling back spans multiple batches.** A single reorg can invalidate data written across many batches. Your rollback mechanism must undo *all* rows/documents written after the rollback point, not just the last batch.

    **Idempotency of re-processing.** After a rollback the stream replays slots from the rollback cursor forward. Write logic that is not idempotent (e.g. unconditional INSERT instead of UPSERT, incrementing a counter instead of setting it) will corrupt state on replay. Design writes so they are safe to run more than once for the same slot.

    **Side effects that cannot be rolled back.** Database writes can be undone; emails, webhook calls, and Kafka publishes cannot. Either defer all external side effects until the slot is finalized, or build a separate reconciliation layer. Treating unfinalized state as permanent is the most common source of production incidents in real-time blockchain consumers.
  </Accordion>

  <Accordion title="5. Cursor semantics">
    **The cursor returned from `fork()` is inclusive.** Return the last slot you consider good; the SDK resumes from `cursor.number + 1`. Off-by-one errors cause either duplicate re-processing or skipped slots.

    **The cursor hash must be set.** The SDK sends `parentBlockHash = cursor.hash` in the next request so the portal can detect the next fork. A cursor with a missing hash silently disables fork detection for that request.

    **The cursor in `write()`'s `read()` call is only the initial startup cursor.** `pipeTo()` handles post-fork cursor updates inside the `read()` generator; `write()` runs continuously through forks and is never restarted by the SDK. The cursor you pass to `read()` is only relevant if `write()` is re-invoked by an external retry mechanism. For in-memory implementations the cursor is effectively always `undefined`.

    **Process restart loses in-memory rollback history.** An in-memory rollback chain survives forks but not process restarts. After a restart you have no history. For services that must survive restarts, persist the rollback chain alongside application state and restore it on startup. See [Cursor management](./cursor-management) for patterns.
  </Accordion>

  <Accordion title="6. Load-balanced portals and a non-monotonic finalized head">
    **The `X-Sqd-Finalized-Head-Number` header can go backwards.** Portal instances behind a load balancer can be at different heights. When a reconnected stream lands on a lagging instance, the `finalized` value in `batch.head.finalized` may be lower than what was previously reported. Do not use the current batch's finalized number as a pruning threshold directly.

    **Treat the finalized head as a high-water mark.** Maintain the highest finalized number seen across all batches and key all pruning on that value. For database-backed implementations this is critical: a DELETE keyed on the current (possibly lower) finalized number will over-retain rows on some batches, and under-retain them if the logic is structured the other way.

    **A 409 from a lagging instance may have `previousBlocks` entirely below the high-water mark.** Two cases:

    * *All* of `previousBlocks` are strictly below the high-water mark. The lagging instance's sample doesn't reach local history. Because the high-water mark is truly final, every correct instance agrees on it: the fork is somewhere *above* it. Return the high-water mark cursor. This requires storing the finalized head as a full `BlockCursor` (number **and** hash), not just a number — the hash is needed for the next request's `parentBlockHash`.

    * Some of `previousBlocks` are at or above the high-water mark but no hash match is found. This is a genuine inconsistency at a height the client already considers final. Return `null` and surface the error.
  </Accordion>

  <Accordion title="7. Fork detection scope (real-time streams only)">
    **Forks only occur in the real-time (unfinalized) portion of the stream.** The `/finalized-stream` endpoint never returns a 409. Fork handling is only needed when consuming the `/stream` endpoint with `fromBlock` near or at the chain head. If your range is bounded and entirely in the past, you will never see a fork.

    **`parentBlockHash` is the tripwire.** Every request to the portal includes the hash of the last slot the client has seen. A mismatch triggers a 409. Anything that disrupts this — starting from a cursor with a wrong or missing hash, replaying from a checkpoint that has drifted from the chain — will produce spurious fork events.
  </Accordion>

  <Accordion title="8. The rollbackChain field contract">
    **`rollbackChain` is per-batch, not cumulative.** It contains only the slots in *this batch* that are above the current finalized head. Treat it as a delta to append to running history, not as a full snapshot of the current unfinalized chain.

    **Slots near the finality boundary move between finalized and unfinalized.** A slot that appears in one batch's `rollbackChain` may be at or below the finalized head in the next batch. The pruning filter must remove these once they are finalized, or rollback history will slowly fill with slots that can never be the subject of a reorg.

    **Empty `rollbackChain` is valid.** It means either (a) the batch contained no slots above the finalized head, or (b) the finalized head was unknown. Do not treat an empty rollback chain as an error.
  </Accordion>

  <Accordion title="9. Algorithm correctness for common-ancestor search">
    **Both arrays must be in ascending order.** The merge-sort scan breaks silently if either array is unsorted. Local history is ascending if you always append to the end; `previousBlocks` from the portal is ascending by protocol convention. After a rollback, the truncated history remains ascending.

    **Gaps in slot numbers do not break correctness, only efficiency.** A gap (e.g. slots 100, 101, 103 — 102 missing because it was already finalized) means a fork at 102 resolves by rolling back to 101. The extra re-processing of 102 is harmless because finalized slots are immutable.

    **Duplicate entries break the scan.** If the same slot number appears more than once with different hashes in your history, the scan may report the wrong common ancestor. UPSERT rather than INSERT when persisting rollback chain entries to a store.

    **Hash comparison requires both sides to be non-null.** `BlockCursor.hash` is optional in the type system. If either side is `undefined`, `undefined !== "0x..."` evaluates to `true`, which looks like a fork on a slot that may be fine. Always verify hashes are present before comparing.
  </Accordion>

  <Accordion title="10. Concurrency and ordering invariants">
    **`fork()` is called synchronously relative to the batch stream.** The SDK awaits `fork()` before resuming the stream. No new batches arrive while `fork()` is running. It is safe to mutate shared state inside `fork()` without additional locking.

    **`write()` and `fork()` share mutable state without synchronization.** This is safe only because the SDK never calls them concurrently. If you introduce background workers or async tasks that also read or write rollback state, you must add explicit synchronization.

    **The order in which you update rollback history and application state matters.** If you update application state first and crash before updating rollback history, the next restart will not know how far to roll back. Prefer database transactions that update both atomically, or update rollback history first so a crash leaves you conservative — you can always re-process a slot you have already seen.
  </Accordion>
</AccordionGroup>


# Pipe anatomy
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/solana/guides/basic-development/anatomy

How a pipe is put together

<Frame>
  <img alt="Pipe anatomy" />
</Frame>

A Solana pipe made with SQD's Pipes SDK consists of:

* A **source** - typically made with `solanaPortalSource()`. Can have one or more outputs.

* **Queries** - tell the source which data has to be retrieved to compute each output. A query is defined by a chain call terminated by `.build()`. Here's an example:

  ```ts theme={"system"}
  solanaQuery()
    .addFields({
      block: { timestamp: true },
      instruction: { programId: true, transactionIndex: true },
    })
    .addInstruction({
      programId: [ ORCA_WHIRLPOOL_PROGRAM_ID ]
    })
    .build()
  ```

* **Per-query transforms** (optional) - you can pass data from each query through a chain of simple transforms:
  ```ts theme={"system"}
  query
    .pipe(data => data.map(item => ({
      funkyNumber: item.header.timestamp + item.header.number,
      ...item
    })))
    .pipe(someOtherSimpleTransformCallback)
  ```
  Source object streams the data you get out of each chain of transforms as the value of the corresponding output field.

* Making utils that return reusable **query-transform combos** is a very useful pattern. In particular, on Solana it is often convenient to keep retrieval and decoding of program instructions in a single module. You can easily make such combos with the `solanaInstructionDecoder()` function - see the [Handling instructions](./handling-instructions) guide.

* **Whole pipe transformers** (optional) - use this if you need to compute something based on data originating from multiple queries, or if you need access to per-batch context (cursor, logger, profiler, fork callbacks). Use `createTransformer()` so the SDK can thread cursor and rollback information ([1](../architecture-deep-dives/cursor-management), [2](../architecture-deep-dives/fork-handling)) through your transform:
  ```ts theme={"system"}
  import { createTransformer } from '@subsquid/pipes'

  const enrichSwaps = createTransformer<
    { swaps: Swap[]; swapsV2: SwapV2[] },
    { events: EnrichedSwap[] }
  >({
    transform: ({ swaps, swapsV2 }, ctx) => {
      ctx.logger.info({ batch: ctx.stream.state.current?.number }, 'enriching')
      return {
        events: [
          ...swaps.map((s) => ({ version: 'v1' as const, ...s })),
          ...swapsV2.map((s) => ({ version: 'v2' as const, ...s })),
        ],
      }
    },
  })

  solanaPortalStream({ /* ... */ }).pipe(enrichSwaps)
  ```

* Pipe termination: a plain async iterator or a **target**.
  * If you use the pipe as an async iterator it will throw exceptions if the underlying chain is experiencing reorgs, see [Fork handling](../architecture-deep-dives/fork-handling).
  * We offer two targets out of the box:

    * Postgres via Drizzle
    * ClickHouse

    You can make your own using [`createTarget()`](../../reference/basic-components/target/create-target).


# Developing pipes
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/solana/guides/basic-development/flow

Typical development workflow for pipes

Begin by making sure you know

* which on-chain data items (instructions, token balance updates etc) you need;
* how you are going to transform these items into usable data, based on your business logic;
* what is your preferred mode of consuming the transformed data.

Use [Pipes CLI](../../quickstart) to quickly generate a starter project. The process of getting from here to a useful data pipeline follows directly from [Pipe anatomy](./anatomy).

## Adding queries and developing transforms

Here are some things to keep in mind as you're developing the heart of your pipeline.

### Writing maintainable pipes

Recall that there are two kinds of transforms in Pipes SDK:

* **Per-query transforms** work on subsets of raw data. They can be bundled with their queries, making them logically self-contained and easily reusable.
* **Whole pipe transforms** process outputs of all per-query transforms at the same time. This unlocks arbitrary data combinations, but it also means that that each such transform might need to be changed whenever any of the upstream transforms changes.

For maximum mainatainability you'll have to balance the following two objectives:

1. Aim to push as much of your business logic into per-query transforms. Make the code of any whole pipe transforms as simple as possible.

2. Use ready-made, validated query-transform combos. For example, [solanaInstructionDecoder()](./handling-instructions) fetches and decodes Solana program instructions. It'll often be preferable to build your pipeline out of such modules, even if that happens to make whole-pipe transforms slightly more complicated.

### Stateful transforms

It's often the case that you need access to some part of the previously processed data to do the transform. For example, to make one minute candles out of raw DEx swap prices you need to know the previous OHLC values. There are multiple ways to accomplish this in Pipes SDK, each with its advantages and disadvantages. Consult the [Stateful transforms](../advanced-topics/stateful-transforms) guide.

## Writing data

### Postgres and ClickHouse

If you need your transformed data in Postgres or ClickHouse, you should already have a basic configuration generated by [Pipes CLI](../../quickstart).

If you're working with real-time data, **it is very important to**

* **on Postgres** when adding or removing any relevant tables: update the list of tables in the target configuration;
* **on ClickHouse** when any data dependencies or structure of the stored data changes: update the `onRollback()` callback.

Consult the [Postgres via Drizzle](./targets/postgres-drizzle) and [ClickHouse](./targets/clickhouse) guides.

### Plain iterator

A complete pipeline without a `.pipeTo` is a valid async iterator.

* The pipeline will produce some logs by default. Disable them by setting `logger: false` when creating the data source. If you're looking to convert an existing standalone pipe into a module in a larger program and wish to get rid of any side effects, consult the [Running bare bones](./running-bare-bones) guide.

* If you're working with unfinalized data (default setting of the source), the iterator will throw `ForkException`s on blockchain reorgs. You should catch these and process them correctly. Consult the [fork handling guide](../architecture-deep-dives/fork-handling) for details.

  Alternatively, configure the data source to use final data only:

  ```ts theme={"system"}
  const source = evmPortalSource({
    portal: {
      url: '<portal_dataset_url>',
      finalized: true,
    },
    ...
  })
  ```

* By default, the pipeline is stateless: when re-created it'll restart from the earliest block relevant to any of the queries. If you want the pipeline to persist its sync state between restarts, you'll have to manage the state by yourself. See [Cursor management](../architecture-deep-dives/cursor-management).

### Developing your own target

Use the [createTarget() function](../../reference/basic-components/target/create-target).

* If you're working with unfinalized data (default setting of the source), you must define a fork handler callback. Consult the [fork handling guide](../architecture-deep-dives/fork-handling) for details.

  Alternatively, configure the data source to use final data only:

  ```ts theme={"system"}
  const source = evmPortalSource({
    portal: {
      url: '<portal_dataset_url>',
      finalized: true,
    },
    ...
  })
  ```

* If you want your pipeline to preserve its sync state between restarts, you'll have to manage this state in your `write` callback. See [Cursor management](../architecture-deep-dives/cursor-management).


# Handling program instructions
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/solana/guides/basic-development/handling-instructions

Fetching and decoding Solana program instructions with solanaInstructionDecoder

`solanaInstructionDecoder()` bundles a Solana instruction query with a decoding transform into a single reusable module. Pass the result as an output to `solanaPortalStream`:

```ts theme={"system"}
import * as orcaWhirlpool from './abi/orca_whirlpool'
import { solanaInstructionDecoder, solanaPortalStream } from '@subsquid/pipes/solana'

const stream = solanaPortalStream({
  portal: 'https://portal.sqd.dev/datasets/solana-mainnet',
  outputs: {
    orcaWhirlpool: solanaInstructionDecoder({
      range: { from: 340_000_000 },
      programId: orcaWhirlpool.programId,
      instructions: {
        swap: orcaWhirlpool.instructions.swap,
        swapV2: orcaWhirlpool.instructions.swapV2,
      },
    }),
  },
})

for await (const { data } of stream) {
  console.log(`parsed ${data.orcaWhirlpool.swap.length} swaps`)
}
```

`solanaInstructionDecoder()` can:

* Fetch from **one or multiple programs** — pass a single address or an array to `programId`.
* Filter by **instruction type** — only instructions whose discriminator matches one of the entries in `instructions` are fetched and decoded.
* Handle decode errors with a custom **`onError` callback** instead of letting them propagate.

See the [solanaInstructionDecoder() reference](../../reference/utility-components/instruction-decoder) for all parameters.

## Generating a typed ABI module

Typed ABI modules for Solana programs are generated from [Anchor IDL](https://www.anchor-lang.com/docs/basics/idl) files by `@subsquid/solana-typegen`. Each generated module exports typed decode functions, so decoded instruction field types are statically known at compile time — you get precise type checking and IDE autocompletion across the entire pipeline.

**Install:**

```bash theme={"system"}
npm install -D @subsquid/solana-typegen
```

### From a local IDL file

```bash theme={"system"}
npx squid-solana-typegen src/abi whirlpool.json
```

The output module name is derived from the filename (`whirlpool`). Override it with a `#name` fragment:

```bash theme={"system"}
npx squid-solana-typegen src/abi whirlpool.json#orca_whirlpool
```

### From a program address

For Anchor programs that publish their IDL on-chain, pass the program address directly:

```bash theme={"system"}
npx squid-solana-typegen src/abi whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc#orca_whirlpool
```

The tool tries two sources in parallel: the on-chain IDL account (defaulting to Solana mainnet) and the [SolanaFM](https://solana.fm) IDL database. It fails only if both sources fail. Use `--solana-rpc-endpoint` to point at a different RPC endpoint:

```bash theme={"system"}
npx squid-solana-typegen \
  --solana-rpc-endpoint https://my-rpc.example.com \
  src/abi \
  whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc#orca_whirlpool
```

Without the `#name` fragment the program address itself becomes the module name.

### From a URL

```bash theme={"system"}
npx squid-solana-typegen src/abi https://example.com/whirlpool.json#orca_whirlpool
```

## Generated module structure

For an input named `orca_whirlpool`, the tool creates:

```
src/abi/
  abi.support.ts              # shared runtime, generated once per output dir
  orca_whirlpool/
    index.ts                  # re-exports + programId
    instructions.ts           # one typed constant per instruction
    events.ts                 # one typed constant per event (if any)
    types.ts                  # shared struct types
```

Each entry in `instructions.ts` is a TypeScript interface (the decoded shape) alongside a constant carrying the discriminator and decode logic:

```ts theme={"system"}
export interface CloseBundledPosition {
    bundleIndex: number
}

export const closeBundledPosition = instruction(
    { d8: '0x2924d8f51b556743' },
    {
        bundledPosition: 0,
        positionBundle: 1,
        positionBundleTokenAccount: 2,
        positionBundleAuthority: 3,
        receiver: 4,
        whirlpoolProgram: 5,
    },
    struct({ bundleIndex: u16 })
)
```

The `d8` discriminator and account index map are embedded in the constant — `solanaInstructionDecoder()` reads them automatically to build the source query filter.

## Accessing decoded data

The keys of the `instructions` map become field names in the decoded output batch. Each field is an array of decoded instruction objects:

```ts theme={"system"}
for await (const { data } of stream) {
  for (const ins of data.orcaWhirlpool.swap) {
    const { accounts, data: fields } = ins.instruction
    console.log(accounts.tokenAuthority, fields.amount)
  }
}
```

See the [decoded instruction structure](../../reference/utility-components/instruction-decoder#decoded-instruction-structure) reference for the full field list.


# Pipes UI
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/solana/guides/basic-development/pipes-ui

Live dashboard for monitoring a running pipe

Pipes UI is a local web dashboard that connects to a running pipe and visualises its progress, speed, portal query, and profiler breakdown. It reads the metrics server that the SDK exposes on the pipe process — nothing needs to be deployed or hosted.

<Frame>
  <video />
</Frame>

## Expose metrics on the pipe

Attach [`metricsServer()`](../advanced-topics/metrics) to the source. Listens on `localhost:9090` by default.

```ts theme={"system"}
import { evmPortalStream, evmDecoder, commonAbis } from '@subsquid/pipes/evm'
import { metricsServer } from '@subsquid/pipes/metrics/node'

evmPortalStream({
  id: 'my-pipe',                       // shows up in the dashboard as the pipe name
  portal: 'https://portal.sqd.dev/datasets/ethereum-mainnet',
  outputs: evmDecoder({
    profiler: { name: 'transfers' },   // labels this span in the profiler tree
    range: { from: 'latest' },
    events: { transfers: commonAbis.erc20.events.Transfer },
  }),
  metrics: metricsServer(),            // exposes /metrics, /stats, /profiler, /health on :9090
})
```

Start the pipe as usual (`ts-node`, `bun`, compiled JS, etc.).

## Run the dashboard

In a second terminal:

```bash theme={"system"}
npx @subsquid/pipes-ui@alpha
```

The UI serves on `http://localhost:3000` and polls the metrics server at `http://localhost:9090`. Open the URL in a browser — the page auto-refreshes once the pipe starts producing batches.

## What it shows

Per pipe (keyed by the `id` passed to the source):

* chain / dataset, with the inferred chain kind (EVM, Solana, …)
* progress: current block, target block, percent complete, ETA
* throughput: blocks/s and bytes/s over the last 30 samples
* the serialised portal query (helpful for reviewing what your decoder actually asked for)
* memory usage of the pipe process and SDK version

When [profiling is on](../advanced-topics/profiling) (the default in non-production environments), the UI also renders the per-batch span tree — useful for seeing which stage (`fetch data`, `apply transformers`, a named decoder, your own `ctx.profiler.start('…')` spans) is dominating batch time. Decorate spans you want to track with `profiler: { name: '…' }` on transformers and decoders.

Any [custom metrics](../advanced-topics/metrics) you register via `ctx.metrics.counter()`, `.gauge()`, `.histogram()`, or `.summary()` show up on the pipe's `/metrics` endpoint (as Prometheus text). The dashboard does not render arbitrary custom series — if you need charts for your own metrics, scrape `/metrics` with Prometheus and graph with Grafana.

The full list of HTTP endpoints served by the metrics process (useful for ad-hoc `curl` inspection) is in the [metricsServer reference](../../reference/utility-components/metrics-server#endpoints).

## Troubleshooting

* **"Failed to reach metrics server"** on the UI — the pipe is not running, or `metricsServer()` is not attached to the source, or it listens on a non-default port. Start the pipe first, then reload the dashboard.
* **UI shows no pipes** — the source config is missing an `id`. Add `id: 'my-pipe'` to the source options.
* **Profiler tab is empty** — the pipe has `profiler: false` set on the source, or `NODE_ENV=production` (the default is to enable profiling only outside production). Set `profiler: true` on the source to force it on. See [Profiling](../advanced-topics/profiling).


# Running bare bones
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/solana/guides/basic-development/running-bare-bones

Using a pipe as a plain async iterator without extra services

By default, `solanaPortalSource` activates a console logger. Passing `metrics` and `progress` enables those services. To embed a pipe into external code with no side effects, disable them all:

```ts theme={"system"}
import { solanaQuery, solanaPortalStream } from '@subsquid/pipes/solana'

const stream = solanaPortalStream({
  portal: 'https://portal.sqd.dev/datasets/solana-mainnet',
  outputs: solanaQuery()
    .addFields({ block: { number: true, timestamp: true } })
    .includeAllBlocks()
    .build(),
  logger: false,   // disable all log output
  profiler: false, // profiler disabled under all circumstances
  // omit `metrics` — no metrics server
  // omit `progress` — no progress reporting
})

for await (const { data } of stream) {
  // data is Block[]
  console.log(data.length)
}
```

The source becomes a plain async iterable that yields `{ data, ctx }` per batch. `ctx.logger` is a no-op when `logger: false`.


# ClickHouse
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/solana/guides/basic-development/targets/clickhouse

Store pipe output in ClickHouse

Install the ClickHouse Node.js client:

```bash theme={"system"}
npm install @clickhouse/client
```

At a glance, the pipeline looks like this:

```ts theme={"system"}
import { createClient } from '@clickhouse/client'
import { clickhouseTarget } from '@subsquid/pipes/targets/clickhouse'

await solanaPortalSource({ ... }).pipeTo(
  clickhouseTarget({
    client: createClient({ url: 'http://localhost:8123' }),
    onData: async ({ store, data }) => {
      store.insert({ table: 'swaps', values: data.swap.map(...), format: 'JSONEachRow' })
    },
    onRollback: async ({ store, safeCursor }) => {
      await store.removeAllRows({ tables: ['swaps'], where: `slot > ${safeCursor.number}` })
    },
  }),
)
```

## Table design

Use `CollapsingMergeTree` with a `sign Int8 DEFAULT 1` column. This engine enables efficient fork rollbacks: to cancel rows, the target re-inserts them with `sign = -1` and ClickHouse merges the pair during background processing.

```sql theme={"system"}
CREATE TABLE IF NOT EXISTS swaps (
  slot              UInt64 CODEC(DoubleDelta, ZSTD),
  instruction_index UInt32,
  program_id        String,
  sign              Int8 DEFAULT 1
) ENGINE = CollapsingMergeTree(sign)
  ORDER BY (slot, instruction_index);
```

Design notes:

* Apply `DoubleDelta + ZSTD` codecs to monotonically increasing columns such as block numbers and timestamps.
* Use `LowCardinality` for columns with low cardinality like addresses to reduce storage and speed up filtering.
* Store 256-bit integers as `UInt256`; serialize JavaScript `BigInt` values to strings before insertion.

Create the table in `onStart` using `store.command()`:

```ts theme={"system"}
onStart: async ({ store }) => {
  await store.command({ query: `CREATE TABLE IF NOT EXISTS transfers ( ... )` })
}
```

## `onData`

Call `store.insert()` to queue an insert. The call is non-blocking — inserts fire concurrently and are fully flushed when the target closes:

```ts theme={"system"}
onData: async ({ store, data }) => {
  store.insert({
    table: 'swaps',
    values: data.swap.map((d) => ({
      slot:              d.block.slot,
      instruction_index: d.instruction.instructionIndex,
      program_id:        d.programId,
    })),
    format: 'JSONEachRow',
  })
}
```

## `onRollback`

Implement `onRollback` to handle blockchain forks. It is invoked in two situations:

* `type: 'offset_check'` — on startup, when a saved cursor is found, to discard writes from a previous crashed or partial run
* `type: 'blockchain_fork'` — when the stream detects a chain reorganisation

Use `store.removeAllRows()` to cancel rows past the safe point. For `CollapsingMergeTree` tables this re-inserts matching rows with `sign = -1`:

```ts theme={"system"}
onRollback: async ({ store, safeCursor }) => {
  await store.removeAllRows({
    tables: ['swaps'],
    where: `slot > ${safeCursor.number}`,
  })
}
```

## Complete example

```ts expandable theme={"system"}
import { solanaInstructionDecoder, solanaPortalSource } from '@subsquid/pipes/solana'
import { clickhouseTarget } from '@subsquid/pipes/targets/clickhouse'
import { createClient } from '@clickhouse/client'

const client = createClient({ url: 'http://localhost:8123' })

await solanaPortalSource({
  portal: 'https://portal.sqd.dev/datasets/solana-mainnet',
  outputs: solanaInstructionDecoder({
    range: { from: '340000000' },
    programId: '...',
    instructions: { swap: swapInstruction },
  }),
}).pipeTo(
  clickhouseTarget({
    client,
    onStart: async ({ store }) => {
      await store.command({
        query: `
          CREATE TABLE IF NOT EXISTS swaps (
            slot              UInt64 CODEC(DoubleDelta, ZSTD),
            instruction_index UInt32,
            program_id        String,
            sign              Int8 DEFAULT 1
          ) ENGINE = CollapsingMergeTree(sign)
            ORDER BY (slot, instruction_index)
        `,
      })
    },
    onData: async ({ store, data }) => {
      store.insert({
        table: 'swaps',
        values: data.swap.map((d) => ({
          slot:              d.block.slot,
          instruction_index: d.instruction.instructionIndex,
          program_id:        d.programId,
        })),
        format: 'JSONEachRow',
      })
    },
    onRollback: async ({ store, safeCursor }) => {
      await store.removeAllRows({
        tables: ['swaps'],
        where: `slot > ${safeCursor.number}`,
      })
    },
  }),
)
```

## Docker setup

```yaml docker-compose.yml theme={"system"}
services:
  clickhouse:
    image: clickhouse/clickhouse-server:latest
    ports:
      - "8123:8123"
      - "9000:9000"
    environment:
      CLICKHOUSE_DB: default
      CLICKHOUSE_USER: default
      CLICKHOUSE_PASSWORD: default
    volumes:
      - clickhouse-data:/var/lib/clickhouse

volumes:
  clickhouse-data:
```

```bash theme={"system"}
docker compose up -d
```

See the [clickhouseTarget reference](../../../reference/basic-components/target/clickhouse) for the full API.


# Postgres via Drizzle
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/solana/guides/basic-development/targets/postgres-drizzle

Store pipe output in PostgreSQL using Drizzle ORM

Install Drizzle ORM and the PostgreSQL driver:

```bash theme={"system"}
npm install drizzle-orm pg
npm install -D drizzle-kit @types/pg
```

At a glance, the pipeline looks like this:

```ts theme={"system"}
await solanaPortalSource({ ... }).pipeTo(
  drizzleTarget({
    db: drizzle('postgresql://...'),
    tables: [swapsTable],
    onData: async ({ tx, data }) => {
      for (const batch of batchForInsert(data.swap)) {
        await tx.insert(swapsTable).values(batch.map(...))
      }
    },
  }),
)
```

## Schema

Define your tables with Drizzle ORM. Every table needs a primary key, and every table written to in [`onData`](#ondata) must appear in [`tables`](#tables).

```ts theme={"system"}
import { bigint, integer, pgTable, primaryKey, varchar } from 'drizzle-orm/pg-core'

const swapsTable = pgTable('swaps', {
  slot:             bigint({ mode: 'bigint' }).notNull(),
  instructionIndex: integer().notNull(),
  programId:        varchar().notNull(),
}, (t) => [primaryKey({ columns: [t.slot, t.instructionIndex] })])
```

## `onData` and `batchForInsert`

`onData` runs inside a serializable transaction. Use `batchForInsert` to split data arrays into chunks that fit within PostgreSQL's 32,767-parameter limit — chunk size is calculated automatically from the number of columns:

```ts theme={"system"}
import { batchForInsert, drizzleTarget } from '@subsquid/pipes/targets/drizzle/node-postgres'

onData: async ({ tx, data }) => {
  for (const batch of batchForInsert(data.swap)) {
    await tx.insert(swapsTable).values(
      batch.map((d) => ({
        slot:             BigInt(d.block.slot),
        instructionIndex: d.instruction.instructionIndex,
        programId:        d.programId,
      })),
    )
  }
}
```

Pass an explicit second argument to `batchForInsert` to cap chunk size:

```ts theme={"system"}
for (const batch of batchForInsert(data.transfers, 100)) { ... }
```

## `tables`

Every table written to in `onData` must be listed in `tables`. At startup, the target installs PostgreSQL trigger functions on these tables to track row-level changes for automatic fork handling. Inserting into an unlisted table throws at runtime.

## Schema migrations

Use [Drizzle Kit](https://orm.drizzle.team/docs/kit-overview) to generate and apply migrations:

```bash theme={"system"}
npx drizzle-kit generate
npx drizzle-kit migrate
```

Alternatively, run migrations automatically on startup via `onStart`:

```ts theme={"system"}
import { migrate } from 'drizzle-orm/node-postgres/migrator'

drizzleTarget({
  db,
  tables: [...],
  onStart: async ({ db }) => {
    await migrate(db, { migrationsFolder: './drizzle' })
  },
  onData: async ({ tx, data }) => { ... },
})
```

## Rollback handling

Fork handling is fully automatic. Each batch runs inside a transaction that snapshots row-level changes. When the stream detects a fork, the target replays those snapshots in reverse to restore the pre-fork state.

Use `onBeforeRollback` and `onAfterRollback` to run custom logic around a rollback. Both callbacks receive the Drizzle transaction and the `cursor` (`BlockCursor`) to which state was rolled back:

```ts theme={"system"}
drizzleTarget({
  db,
  tables: [...],
  onBeforeRollback: async ({ tx, cursor }) => { /* e.g. log or acquire an external lock */ },
  onAfterRollback:  async ({ tx, cursor }) => { /* e.g. invalidate a cache */ },
  onData: async ({ tx, data }) => { ... },
})
```

## Complete example

```ts expandable theme={"system"}
import { solanaInstructionDecoder, solanaPortalSource } from '@subsquid/pipes/solana'
import { batchForInsert, drizzleTarget } from '@subsquid/pipes/targets/drizzle/node-postgres'
import { drizzle } from 'drizzle-orm/node-postgres'
import { bigint, integer, pgTable, primaryKey, varchar } from 'drizzle-orm/pg-core'

const swapsTable = pgTable('swaps', {
  slot:             bigint({ mode: 'bigint' }).notNull(),
  instructionIndex: integer().notNull(),
  programId:        varchar().notNull(),
}, (t) => [primaryKey({ columns: [t.slot, t.instructionIndex] })])

await solanaPortalSource({
  portal: 'https://portal.sqd.dev/datasets/solana-mainnet',
  outputs: solanaInstructionDecoder({
    range: { from: '340000000' },
    programId: '...',
    instructions: { swap: swapInstruction },
  }),
}).pipeTo(
  drizzleTarget({
    db: drizzle('postgresql://postgres:postgres@localhost:5432/postgres'),
    tables: [swapsTable],
    onData: async ({ tx, data }) => {
      for (const batch of batchForInsert(data.swap)) {
        await tx.insert(swapsTable).values(
          batch.map((d) => ({
            slot:             BigInt(d.block.slot),
            instructionIndex: d.instruction.instructionIndex,
            programId:        d.programId,
          })),
        )
      }
    },
  }),
)
```

See the [drizzleTarget reference](../../../reference/basic-components/target/postgres-drizzle) for the full API.


# Quickstart
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/solana/quickstart

Bootstrap a Pipes SDK project

# Using with AI

The fastest way to get an AI coding agent productive on a Pipes SDK project is to install the official [Pipes SDK Agent Skill](/en/ai/agent-skills#pipes-sdk-skill):

```bash theme={"system"}
npx skills add subsquid-labs/skills/pipes-sdk
```

The skill activates automatically on tasks like *"create an indexer for Uniswap V3 swaps"* or *"my indexer is syncing slowly, help me optimize it"*. It covers scaffolding, runtime error diagnosis, sync tuning, and data-quality checks.

Pair the skill with one or both MCP servers so the agent can read live data and look things up:

* [Portal MCP server](/en/ai/mcp-server) — 29 tools for querying blocks, transactions, logs, instructions, and analytics across 225+ datasets. No API key.
* [Documentation MCP server](/en/ai/mcp-server-docs) — search and retrieve these docs from inside the agent.

If you'd rather feed docs into a model directly, the static [`llms.txt`](/llms.txt) (index) and [`llms-full.txt`](/llms-full.txt) (full content) files are kept in sync with the site. See the [AI Development overview](/en/ai/ai-development) for the full menu.

# Scaffolding with Pipes CLI

<Note>
  `pipes-cli` is a work in progress.
</Note>

In a few minutes, you'll have a running pipe that indexes Orca Whirlpool swap instructions on Solana mainnet into a local PostgreSQL database.

## Prerequisites

* Node.js 22.15+
* `pnpm`
* Docker (for the bundled PostgreSQL container)

## Initialize the project

Run the CLI in the directory where you want the project folder to land:

```bash theme={"system"}
pnpx @subsquid/pipes-cli@1.0.0-alpha.4 init
```

The CLI prompts for the project folder name, package manager (please stick to `pnpm` for now), sink (please use `ClickHouse` or `Postgres`), network type, network, and template; then installs dependencies and writes a runnable project.

You can supply a JSON config instead of filling the prompts manually. Here's the configuration for Orca Whirlpool swap instructions mentioned above:

```bash theme={"system"}
pnpx @subsquid/pipes-cli@1.0.0-alpha.4 init --config '{
  "projectFolder": "orca-example",
  "packageManager": "pnpm",
  "sink": "postgresql",
  "networkType": "svm",
  "network": "solana-mainnet",
  "templates": [
    {
      "templateId": "custom",
      "params": {
        "contracts": [
          {
            "contractAddress": "whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc",
            "contractName": "OrcaWhirlpool",
            "contractEvents": [
              {
                "name": "swap",
                "type": "instruction",
                "inputs": [
                  { "name": "amount", "type": "u64" },
                  { "name": "sqrtPriceLimit", "type": "u128" }
                ]
              }
            ],
            "range": { "from": "latest" }
          }
        ]
      }
    }
  ]
}'
```

`--config` also accepts a path to a JSON file.

To inspect the full config schema run

```
pnpx @subsquid/pipes-cli@1.0.0-alpha.4 init --schema
```

## Run the pipeline

The generated project ships with a `docker-compose.yml` that brings up the sink database and the pipeline together:

```bash theme={"system"}
cd orca-example
docker compose --profile with-pipeline up
```

For an iterative dev loop, run the database in Docker and the pipeline locally:

```bash theme={"system"}
docker compose up -d         # Postgres on :5432
pnpm run db:migrate          # apply the generated migration
pnpm run dev                 # tsx src/index.ts
```

Either way, rows start landing in the `orca_whirlpool_swap` table within a minute.

## What was generated

The project layout:

```
orca-example/
├── src/
│   ├── index.ts        # the pipe — source, decoder, target
│   ├── schemas.ts      # Drizzle table definitions
│   ├── contracts/      # generated ABI bindings (per program ID)
│   └── utils/
├── migrations/         # SQL migrations generated by drizzle-kit
├── docker-compose.yml  # Postgres + optional pipeline service
├── Dockerfile
├── drizzle.config.ts
├── package.json
├── .env                # DB_CONNECTION_STR — points at local Postgres
└── README.md
```

The pipe lives in `src/index.ts`. The decoder block defines what to extract + a light transform:

```ts theme={"system"}
const custom = solanaInstructionDecoder({
  range: { from: 'latest' },
  programId: ['whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc'],
  instructions: {
    swap: orcaWhirlpoolInstructions.swap,
  },
}).pipe(enrichEvents)
```

The decoder asks the Portal for `swap` instructions on the Orca Whirlpool program. `enrichEvents` (from `src/utils/`) reshapes each decoded instruction into a row matching the Drizzle table. See the [Pipe anatomy](./guides/basic-development/anatomy) and [Handling contract events](./guides/basic-development/handling-events) guides for more info on `evmDecoder()`.

The `main()` function wires the decoder to a [drizzleTarget](./reference/basic-components/target/postgres-drizzle):

```ts theme={"system"}
export async function main() {
  await solanaPortalSource({
    id: 'solana-orca-pipe',
    portal: 'https://portal.sqd.dev/datasets/solana-mainnet',
    outputs: { custom },
  }).pipeTo(
    drizzleTarget({
      db: drizzle(env.DB_CONNECTION_STR),
      tables: [orcaWhirlpoolSwapTable],
      onData: async ({ tx, data }) => {
        for (const values of chunk(data.custom.swap)) {
          await tx.insert(orcaWhirlpoolSwapTable).values(values)
        }
      },
    }),
  )
}
```

The `id` is a per-pipeline identifier — keep it stable so the [target's cursor](./guides/architecture-deep-dives/cursor-management) survives restarts. See [solanaPortalSource](./reference/basic-components/source) for the full source API and [Pipe anatomy](./guides/basic-development/anatomy) for how the pieces fit together.

## Other examples

<AccordionGroup>
  <Accordion title="Token balances">
    The `tokenBalances` template indexes pre/post token balances directly from blocks — no program ID needed. The generated pipe uses [solanaQuery()](./reference/basic-components/query-builder) instead of an instruction decoder.

    ```json theme={"system"}
    pnpx @subsquid/pipes-cli@1.0.0-alpha.4 init --config '{
      "projectFolder": "solana-tokens",
      "packageManager": "pnpm",
      "sink": "postgresql",
      "networkType": "svm",
      "network": "solana-mainnet",
      "templates": [
        {
          "templateId": "tokenBalances"
        }
      ]
    }'
    ```
  </Accordion>
</AccordionGroup>

The CLI ships one built-in SVM template — `tokenBalances` — plus the open-ended `custom` template shown at the top of the page.


# Query builder
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/solana/reference/basic-components/query-builder

API reference for SolanaQueryBuilder

`SolanaQueryBuilder` assembles a typed portal query from a field selection and one or more data-request clauses. Pass `.build()` to `outputs` (or to `.pipe()`) on a Solana source. The resulting object is consumed by [`solanaPortalStream()`](./source) and by the [solanaInstructionDecoder](../utility-components/instruction-decoder) which composes on top of it.

## `solanaQuery()`

Returns a fresh `SolanaQueryBuilder`.

```ts theme={"system"}
import { solanaQuery } from '@subsquid/pipes/solana'

const query = solanaQuery()
  .addFields({
    block: { number: true, timestamp: true },
    instruction: { programId: true, data: true, accounts: true },
  })
  .addInstruction({
    range: { from: 200_000_000 },
    request: {
      programId: ['whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc'],
    },
  })
  .build()
```

## `SolanaQueryBuilder<F>`

```ts theme={"system"}
class SolanaQueryBuilder<F extends FieldSelection = {}> {
  addFields<T>(fields: Subset<T, FieldSelection>): SolanaQueryBuilder<F & T>
  addInstruction(options: RequestOptions<InstructionRequest>): this
  addTransaction(options: RequestOptions<TransactionRequest>): this
  addLog(options: RequestOptions<LogRequest>): this
  addBalance(options: RequestOptions<BalanceRequest>): this
  addTokenBalance(options: RequestOptions<TokenBalanceRequest>): this
  addReward(options: RequestOptions<RewardRequest>): this
  includeAllBlocks(range?: Range): this
  addRange(range: PortalRange): this
  merge(other?: SolanaQueryBuilder<F>): this
  build(opts?: { setupQuery?: SetupQueryFn<SolanaQueryBuilder<F>> }): QueryAwareTransformer
}
```

The generic parameter `F` narrows the block type produced by the stream — only fields explicitly selected with `.addFields()` appear on the decoded records, at both compile and runtime.

### `RequestOptions<R>` and `Range`

```ts theme={"system"}
type RequestOptions<R> = { range: PortalRange; request: R }
type PortalRange = { from?: number | string | 'latest' | Date; to?: number | string | Date }
type Range        = { from: number; to?: number }
```

`PortalRange.from` defaults to `0`; a `Date` or numeric timestamp is resolved to a slot via the portal at query start. `'latest'` is resolved to the current head. `includeAllBlocks()` accepts the stricter `Range`, so dates and `'latest'` are not allowed there.

In Solana, `number` identifies a **slot**. Slot and block height differ: skipped slots do not increment `height`.

***

## `.addFields(fields)`

Add to the field selection. Repeated calls are merged recursively.

### `block`

| Field          | Type                                       |
| -------------- | ------------------------------------------ |
| `number`       | `number` (slot; always returned)           |
| `hash`         | `string` (always returned)                 |
| `height`       | `number` (block height, skips not counted) |
| `parentNumber` | `number`                                   |
| `parentHash`   | `string`                                   |
| `timestamp`    | `number` (Unix seconds)                    |

### `transaction`

| Field                         | Type                                                 |
| ----------------------------- | ---------------------------------------------------- |
| `transactionIndex`            | `number`                                             |
| `version`                     | `'legacy' \| number`                                 |
| `accountKeys`                 | `string[]`                                           |
| `addressTableLookups`         | `{ accountKey, readonlyIndexes, writableIndexes }[]` |
| `numReadonlySignedAccounts`   | `number`                                             |
| `numReadonlyUnsignedAccounts` | `number`                                             |
| `numRequiredSignatures`       | `number`                                             |
| `recentBlockhash`             | `string`                                             |
| `signatures`                  | `string[]`                                           |
| `err`                         | `null \| object` (error details, `null` on success)  |
| `computeUnitsConsumed`        | `bigint`                                             |
| `fee`                         | `bigint` (lamports)                                  |
| `loadedAddresses`             | `{ readonly: string[]; writable: string[] }?`        |
| `hasDroppedLogMessages`       | `boolean`                                            |

### `instruction`

| Field                   | Type                                                      |
| ----------------------- | --------------------------------------------------------- |
| `transactionIndex`      | `number`                                                  |
| `instructionAddress`    | `number[]` (path in the call tree)                        |
| `programId`             | `string`                                                  |
| `accounts`              | `string[]`                                                |
| `data`                  | `string`                                                  |
| `computeUnitsConsumed`  | `bigint?`                                                 |
| `error`                 | `unknown?`                                                |
| `isCommitted`           | `boolean` (`false` when the enclosing transaction failed) |
| `hasDroppedLogMessages` | `boolean`                                                 |

### `log`

| Field                | Type                         |
| -------------------- | ---------------------------- |
| `transactionIndex`   | `number`                     |
| `logIndex`           | `number`                     |
| `instructionAddress` | `number[]`                   |
| `programId`          | `string`                     |
| `kind`               | `'log' \| 'data' \| 'other'` |
| `message`            | `string`                     |

### `balance`

SOL balance change for one account within one transaction.

| Field              | Type                       |
| ------------------ | -------------------------- |
| `transactionIndex` | `number`                   |
| `account`          | `string`                   |
| `pre`              | `bigint` (lamports before) |
| `post`             | `bigint` (lamports after)  |

### `tokenBalance`

SPL-token balance change. Records are a tagged union: some updates carry only `pre*` fields (token account removed), some only `post*` (token account created), and some both.

| Field                                                                  | Type                     |
| ---------------------------------------------------------------------- | ------------------------ |
| `transactionIndex`                                                     | `number`                 |
| `account`                                                              | `string` (token account) |
| `preProgramId`, `preMint`, `preDecimals`, `preOwner`, `preAmount`      | before state             |
| `postProgramId`, `postMint`, `postDecimals`, `postOwner`, `postAmount` | after state              |

### `reward`

| Field         | Type                                           |
| ------------- | ---------------------------------------------- |
| `pubkey`      | `string`                                       |
| `lamports`    | `bigint`                                       |
| `postBalance` | `bigint`                                       |
| `rewardType`  | `string?` (`fee`, `rent`, `voting`, `staking`) |
| `commission`  | `number?`                                      |

***

## `.addInstruction(options)`

Filter instructions. List filters AND across fields; values within a field OR. Matches instructions anywhere in the call tree, not just top-level.

| Field                      | Type       | Meaning                                                                             |
| -------------------------- | ---------- | ----------------------------------------------------------------------------------- |
| `programId`                | `string[]` | Program that executes the instruction.                                              |
| `d1`, `d2`, `d4`, `d8`     | `string[]` | First 1/2/4/8 bytes of `data` in `0x`-prefixed hex — used as Anchor discriminators. |
| `a0`–`a9`                  | `string[]` | Account at position 0..9 of the instruction's account list.                         |
| `isCommitted`              | `boolean`  | Restrict to committed (successful) instructions.                                    |
| `transaction`              | `boolean`  | Also fetch the parent transaction.                                                  |
| `transactionBalances`      | `boolean`  | Also fetch SOL balance updates for parent transactions.                             |
| `transactionTokenBalances` | `boolean`  | Also fetch token balance updates for parent transactions.                           |
| `transactionInstructions`  | `boolean`  | Also fetch sibling instructions executed by the parent transaction.                 |
| `innerInstructions`        | `boolean`  | Also fetch all inner instructions (entire subtrees) of matching instructions.       |
| `logs`                     | `boolean`  | Also fetch logs produced by matching instructions.                                  |

An empty `request: {}` matches every instruction in the range.

```ts theme={"system"}
solanaQuery().addInstruction({
  range: { from: 200_000_000 },
  request: {
    programId: ['whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc'],
    d8: ['0xf8c69e91e17587c8'], // 8-byte Anchor discriminator for Whirlpool `swap`
    isCommitted: true,
    innerInstructions: true,
  },
})
```

<Note>
  The portal accepts account-position filters up to `a15` and a `mentionsAccount` filter that matches an account anywhere in the list. These are not exposed by the builder; use a raw query via [`portal.getStream()`](./source#symboliterator) if you need them.
</Note>

## `.addTransaction(options)`

Filter transactions.

| Field          | Type       | Meaning                                                        |
| -------------- | ---------- | -------------------------------------------------------------- |
| `feePayer`     | `string[]` | Fee-payer account.                                             |
| `instructions` | `boolean`  | Also fetch all instructions executed in matching transactions. |
| `logs`         | `boolean`  | Also fetch all logs produced by matching transactions.         |

## `.addLog(options)`

Filter program log messages.

| Field         | Type                             | Meaning                                              |
| ------------- | -------------------------------- | ---------------------------------------------------- |
| `programId`   | `string[]`                       | Program that produced the log.                       |
| `kind`        | `('log' \| 'data' \| 'other')[]` | Log type.                                            |
| `transaction` | `boolean`                        | Also fetch the parent transaction.                   |
| `instruction` | `boolean`                        | Also fetch the instruction the log was emitted from. |

## `.addBalance(options)`

Filter SOL balance updates.

| Field                     | Type       | Meaning                                             |
| ------------------------- | ---------- | --------------------------------------------------- |
| `account`                 | `string[]` | Account whose balance changed.                      |
| `transaction`             | `boolean`  | Also fetch the parent transaction.                  |
| `transactionInstructions` | `boolean`  | Also fetch all instructions of parent transactions. |

## `.addTokenBalance(options)`

Filter SPL-token balance updates.

| Field                           | Type       | Meaning                                             |
| ------------------------------- | ---------- | --------------------------------------------------- |
| `account`                       | `string[]` | Token account.                                      |
| `preProgramId`, `postProgramId` | `string[]` | Token program ID before / after.                    |
| `preMint`, `postMint`           | `string[]` | Mint before / after.                                |
| `preOwner`, `postOwner`         | `string[]` | Owner before / after.                               |
| `transaction`                   | `boolean`  | Also fetch the parent transaction.                  |
| `transactionInstructions`       | `boolean`  | Also fetch all instructions of parent transactions. |

## `.addReward(options)`

Filter validator rewards.

| Field    | Type       | Meaning               |
| -------- | ---------- | --------------------- |
| `pubkey` | `string[]` | Validator public key. |

## `.includeAllBlocks(range?)`

Request every slot in `range` regardless of whether any other filter matches. Useful when you want uninterrupted timestamps or slot progression even across slots with no targeted activity. Without `range`, applies from slot 0 onward.

```ts theme={"system"}
solanaQuery().includeAllBlocks({ from: 200_000_000, to: 200_001_000 })
```

## `.addRange(range)`

Push a range-only request with no filters — typically used to bound a stream built from other sources.

## `.merge(other)`

Merge another builder's requests and fields in-place. Overlapping ranges are reconciled at build time.

## `.build(opts?)`

Return a `QueryAwareTransformer` suitable for use as a source output.

```ts theme={"system"}
solanaPortalStream({
  portal: 'https://portal.sqd.dev/datasets/solana-mainnet',
  outputs: { swaps: solanaQuery().addInstruction({/*…*/}).build() },
})
```

`opts.setupQuery` is an advanced hook called when the query is finalized: it receives `{ query, logger }` and can mutate `query` (e.g. merge additional requests from runtime data). Default behaviour is to merge `this` into the stream's root query.

***

## See also

* [Source](./source) — how queries attach to the stream.
* [solanaInstructionDecoder](../utility-components/instruction-decoder) — typed wrapper that emits a pre-built query plus instruction decoding.
* [Handling program instructions](../../guides/basic-development/handling-instructions) — higher-level guide.
* [Portal API (Solana) OpenAPI](/en/portal/solana/api) — raw wire protocol this builder serialises to.


# Source
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/solana/reference/basic-components/source

API reference for Solana Portal source

The source component connects to SQD Portal and streams blockchain data to your pipeline. It's the starting point for all Pipes SDK data flows.

## solanaPortalSource

Create a Portal source for Solana chains.

```ts theme={"system"}
solanaPortalSource(config: SolanaPortalSourceConfig): Source
```

**Parameters:**

* `id`: (required) Pipeline ID. Must be unique within any infra shared with other pipelines (DB, logging sinks etc).

* `portal`: (required) Portal API URL or config object.
  * String: `"https://portal.sqd.dev/datasets/ethereum-mainnet"`
  * Object: `{ url: string, finalized?: boolean }`. When `finalized: true` is set the stream will consist of finalized blocks only and none of the [fork handling machinery](../../guides/architecture-deep-dives/fork-handling) will be required.

* `outputs`: (required) A single query-transformers chain combo or record of named outputs.

* `cache`: (optional) Portal cache instance. If supplied, saves portal responses locally and reuses them when the pipeline re-runs.

* `logger`: (optional) A pino-compatible `Logger` instance or a log level string. Accepted level values: `'fatal'`, `'error'`, `'warn'`, `'info'`, `'debug'`, `'trace'`, `'silent'`, `false`, `null`. Passing `false` or `null` silences all log output. When omitted, a default console logger is used.

* `metrics`: (optional) `metricsServer()` instance for exposing Prometheus metrics.

* `progress`: (optional) Options for progress tracking.

* `profiler`: (optional) Enable the built-in per-batch profiler. See [Profiling](../../guides/advanced-topics/profiling).

**Example:**

```ts theme={"system"}
import { solanaPortalSource } from "@subsquid/pipes/solana";
import { portalSqliteCache } from "@subsquid/pipes/portal-cache/node";

const source = solanaPortalSource({
  id: "orca-swaps",
  portal: "https://portal.sqd.dev/datasets/solana-mainnet",
  outputs: solanaInstructionDecoder({
    range: { from: 200000000 },
    programId: orcaWhirlpool.programId,
    instructions: { swap: orcaWhirlpool.instructions.swap },
  }),
  cache: portalSqliteCache({ path: "./cache.sqlite" }),
});
```

### Finalized Blocks

You can configure the source to only receive finalized blocks:

```ts theme={"system"}
const source = solanaPortalSource({
  portal: {
    finalized: true,
    url: 'https://portal.sqd.dev/datasets/solana-mainnet'
  }
});
```

Using finalized blocks eliminates the need for rollback handlers in your targets, simplifying the logic of your pipeline.

## Pipe methods

### pipe()

Chain a single [whole-pipe transformer](./transformer) to the source.

```ts theme={"system"}
source.pipe(transformer)
```

The returned value behaves exactly as the source.

See also: [Stateful transformers](../../guides/advanced-topics/stateful-transforms).

### pipeTo()

Connect the pipeline to a [target](./target).

```ts theme={"system"}
source.pipeTo(target)
```

This is a terminal operation: you cannot continue piping after calling this method.

If you want your stream to resume on restarts and properly handle unfinalized data, make sure that the target [manages cursors](../../guides/architecture-deep-dives/cursor-management) and [handles forks](../../guides/architecture-deep-dives/fork-handling) correctly.

### \*[Symbol.asyncIterator]()

Use the pipeline as an async iterator:

```ts theme={"system"}
for await (const { data } of stream) {
  // ... do something with data ...
}
```

On blockchain forks this will throw `ForkException`s - see [Fork handling](../../guides/architecture-deep-dives/fork-handling).


# clickhouseTarget
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/solana/reference/basic-components/target/clickhouse

ClickHouse target for Pipes SDK

See the [ClickHouse guide](../../../guides/basic-development/targets/clickhouse) for usage examples, table design, and setup instructions.

```ts theme={"system"}
import { clickhouseTarget } from '@subsquid/pipes/targets/clickhouse'
```

## `clickhouseTarget`

```ts theme={"system"}
clickhouseTarget<T>({
  client: ClickHouseClient,
  onStart?: (ctx: { store: ClickhouseStore; logger: Logger }) => unknown | Promise<unknown>,
  onData: (ctx: { store: ClickhouseStore; data: T; ctx: Ctx }) => unknown | Promise<unknown>,
  onRollback?: (ctx: {
    type: 'offset_check' | 'blockchain_fork'
    store: ClickhouseStore
    safeCursor: BlockCursor
    /** @deprecated Use `safeCursor` instead. */
    cursor: BlockCursor
  }) => unknown | Promise<unknown>,
  settings?: Settings,
})
```

| Parameter    | Required | Description                                                                  |
| ------------ | -------- | ---------------------------------------------------------------------------- |
| `client`     | Yes      | Client from `@clickhouse/client`.                                            |
| `onStart`    | No       | Runs once before processing starts. Use for table creation or other setup.   |
| `onData`     | Yes      | Called for each batch.                                                       |
| `onRollback` | No       | Called on startup (`'offset_check'`) and on each fork (`'blockchain_fork'`). |
| `settings`   | No       | Configuration for the internal cursor state table. See `Settings` below.     |

**`Settings`:**

| Field      | Default                      | Description                                             |
| ---------- | ---------------------------- | ------------------------------------------------------- |
| `database` | Client's configured database | ClickHouse database for the state table.                |
| `table`    | `'sync'`                     | Name of the state table.                                |
| `id`       | `'stream'`                   | Stream identifier within the state table.               |
| `maxRows`  | `10000`                      | Maximum rows retained per stream id in the state table. |

## `ClickhouseStore` methods

| Method                                                  | Description                                                                                                                               |
| ------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| `store.insert(params)`                                  | Queues an insert. Non-blocking — returns a `Promise` but need not be awaited inside `onData`; inserts are flushed when the target closes. |
| `store.query(params)`                                   | Passthrough to `client.query()`.                                                                                                          |
| `store.command(params)`                                 | Passthrough to `client.command()`.                                                                                                        |
| `store.removeAllRows({ tables, where, params? })`       | Cancels rows matching `where` by re-inserting them with `sign = -1`. Requires `CollapsingMergeTree`.                                      |
| `store.removeAllRowsByQuery({ table, query, params? })` | Like `removeAllRows`, but uses a custom `SELECT` to identify the rows to cancel.                                                          |
| `store.executeFiles(dir)`                               | Executes all `.sql` files found in `dir`.                                                                                                 |


# createTarget
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/solana/reference/basic-components/target/create-target

API reference for createTarget

Build a custom data sink. A target drains batches from the pipe and is responsible for persisting them.

```ts theme={"system"}
createTarget<In>(config: Target<In>): Target<In>
```

**Config fields:**

* `write`: (required) Async function `({ read, logger }) => Promise<void>`. Iterate the stream by calling `read()` and consuming `{ data, ctx }` batches. The function returns when the stream ends.
* `fork`: (optional) `(previousBlocks: BlockCursor[]) => Promise<BlockCursor | null>`. Called when the source detects a chain reorg. Return the last safe cursor to roll back to, or `null` if no common ancestor can be determined (the stream will throw). See [Fork handling](../../../guides/architecture-deep-dives/fork-handling). You don't need this callback when the source is configured to [read only finalized blocks](../source#finalized-blocks).

## The `write` context

```ts theme={"system"}
type WriteCtx<In> = {
  read: (cursor?: BlockCursor) => AsyncIterableIterator<PortalBatch<In>>
  logger: Logger
}
```

| Field    | Description                                                                                                           |
| -------- | --------------------------------------------------------------------------------------------------------------------- |
| `read`   | Opens an async iterator over pipeline batches. Pass `cursor` to resume from a specific slot the target has persisted. |
| `logger` | Pino-compatible logger scoped to this target.                                                                         |

## Per-batch context (`ctx`)

Each `{ data, ctx }` yielded by `read()` carries the same `BatchContext` that transformers receive. Fields:

| Field                        | Type                        | Description                                                                                                 |
| ---------------------------- | --------------------------- | ----------------------------------------------------------------------------------------------------------- |
| `id`                         | `string`                    | Pipeline ID — the `id` passed to `solanaPortalStream()`.                                                    |
| `logger`                     | `Logger`                    | Batch-scoped logger.                                                                                        |
| `metrics`                    | `Metrics`                   | Prometheus metrics registry. See [Metrics](../../../guides/advanced-topics/metrics).                        |
| `profiler`                   | `Profiler`                  | Open a span with `ctx.profiler.start('label')`. See [Profiling](../../../guides/advanced-topics/profiling). |
| `stream.dataset`             | `ApiDataset`                | Dataset metadata.                                                                                           |
| `stream.head.finalized`      | `BlockCursor \| undefined`  | Current finalized head.                                                                                     |
| `stream.head.latest`         | `BlockCursor \| undefined`  | Current unfinalized head.                                                                                   |
| `stream.state.initial`       | `number`                    | First slot the stream was configured to read.                                                               |
| `stream.state.last`          | `number`                    | Last slot the stream intends to read.                                                                       |
| `stream.state.current`       | `BlockCursor`               | Latest slot in this batch.                                                                                  |
| `stream.state.rollbackChain` | `BlockCursor[]`             | Tail of unfinalized cursors subject to rollback.                                                            |
| `stream.progress`            | `ProgressEvent['progress']` | Progress metrics when `progress` is enabled.                                                                |
| `stream.query`               | `{ url, hash, raw }`        | Portal query details for the batch.                                                                         |
| `batch.blocksCount`          | `number`                    | Number of slots in this batch.                                                                              |
| `batch.bytesSize`            | `number`                    | Compressed payload size received from the portal.                                                           |
| `batch.requests`             | `Record<number, number>`    | Map of HTTP status code → count of responses that produced this batch.                                      |
| `batch.lastBlockReceivedAt`  | `Date`                      | Wall-clock time the last block was received.                                                                |

## Example

```ts theme={"system"}
const target = createTarget({
  write: async ({ read, logger }) => {
    for await (const { data, ctx } of read()) {
      const span = ctx.profiler.start('save')
      await database.save(data)
      span.end()
      logger.info(
        { block: ctx.stream.state.current.number, rows: ctx.batch.blocksCount },
        'saved batch',
      )
    }
  },
  fork: async (previousBlocks) => {
    // Return a cursor from your persisted state; null to fail hard.
    return previousBlocks[previousBlocks.length - 1] ?? null
  },
})
```

## Resuming from a persisted cursor

Stateful targets typically persist a cursor and resume from it on restart:

```ts theme={"system"}
createTarget({
  write: async ({ read, logger }) => {
    const lastSaved = await database.getCursor() // BlockCursor | undefined
    for await (const { data, ctx } of read(lastSaved)) {
      await database.save(data)
      await database.saveCursor(ctx.stream.state.current)
    }
  },
})
```


# drizzleTarget
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/solana/reference/basic-components/target/postgres-drizzle

PostgreSQL target for Pipes SDK via Drizzle ORM

See the [Postgres via Drizzle guide](../../../guides/basic-development/targets/postgres-drizzle) for usage examples and setup instructions.

```ts theme={"system"}
import { drizzleTarget } from '@subsquid/pipes/targets/drizzle/node-postgres'
```

## `drizzleTarget`

```ts theme={"system"}
drizzleTarget<T>({
  db: NodePgDatabase,
  tables: Table[] | Record<string, Table>,
  onStart?: (ctx: { db: NodePgDatabase }) => Promise<unknown>,
  onData: (ctx: { tx: Transaction; data: T; ctx: Ctx }) => Promise<unknown>,
  onBeforeRollback?: (ctx: { tx: Transaction; cursor: BlockCursor }) => Promise<unknown> | unknown,
  onAfterRollback?: (ctx: { tx: Transaction; cursor: BlockCursor }) => Promise<unknown> | unknown,
  settings?: {
    state?: StateOptions
    transaction?: {
      isolationLevel?: 'read uncommitted' | 'read committed' | 'repeatable read' | 'serializable'
    }
  },
})
```

| Parameter                             | Required | Description                                                                                     |
| ------------------------------------- | -------- | ----------------------------------------------------------------------------------------------- |
| `db`                                  | Yes      | Drizzle `NodePgDatabase` instance. Must expose `$client` (a `pg` Pool or Client).               |
| `tables`                              | Yes      | Tables tracked for automatic fork rollback. All tables written to in `onData` must appear here. |
| `onStart`                             | No       | Runs once before processing starts. Receives `{ db }`.                                          |
| `onData`                              | Yes      | Called for each batch inside a serializable transaction.                                        |
| `onBeforeRollback`                    | No       | Called inside the rollback transaction before snapshots are replayed.                           |
| `onAfterRollback`                     | No       | Called inside the rollback transaction after snapshots are replayed.                            |
| `settings.state`                      | No       | Configuration for the internal cursor state table. See `StateOptions` below.                    |
| `settings.transaction.isolationLevel` | No       | Transaction isolation level. Defaults to `'serializable'`.                                      |

**`StateOptions`:**

| Field                        | Default    | Description                                                           |
| ---------------------------- | ---------- | --------------------------------------------------------------------- |
| `schema`                     | `'public'` | PostgreSQL schema for the state table.                                |
| `table`                      | `'sync'`   | Name of the state table.                                              |
| `id`                         | `'stream'` | Stream identifier within the state table.                             |
| `unfinalizedBlocksRetention` | `1000`     | Number of unfinalized blocks retained in state for rollback purposes. |

## `batchForInsert`

```ts theme={"system"}
import { batchForInsert } from '@subsquid/pipes/targets/drizzle/node-postgres'
```

```ts theme={"system"}
function batchForInsert<T>(data: readonly T[], size?: number): Generator<T[]>
```

Splits an array into chunks that fit within PostgreSQL's 32,767-parameter limit. Chunk size is `Math.floor(32767 / columnsPerRecord)` by default. Pass `size` to set a smaller cap; values exceeding the computed maximum are silently clamped.

`chunk` is a deprecated alias for `batchForInsert`.


# Transformer
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/solana/reference/basic-components/transformer

API reference for createTransformer and pipe transforms

## createTransformer

Construct a whole-pipe transformer.

```ts theme={"system"}
createTransformer<I, O>(config: TransformerOptions<I, O>): Transformer<I, O>
```

**Config fields:**

* `transform`: (required) `(data: I, ctx: BatchContext) => O | Promise<O>`. Called once per batch.
* `start`: (optional) `(ctx: StartCtx) => void | Promise<void>`. Called once when the pipe starts. Use this to load state, warm up caches, or query the portal for historical data before the main stream begins.
* `stop`: (optional) `(ctx: StopCtx) => void | Promise<void>`. Called once when the pipe stops.
* `fork`: (optional) `(cursor: BlockCursor, ctx: Ctx) => void | Promise<void>`. Called before the next batch whenever the source detects a chain reorg. `cursor` identifies the last safe slot. See [Fork handling](../../guides/architecture-deep-dives/fork-handling).
* `profiler`: (optional) `{ name: string; hidden?: boolean }`. Overrides the transformer's node name in the [profiler](../../guides/advanced-topics/profiling) tree.

**Example:**

```ts theme={"system"}
const transformer = createTransformer({
  transform: async (data, ctx) => {
    ctx.logger.info({ slot: ctx.stream.state.current.number }, 'batch')
    return data.map((b) => b.instructions)
  },
})
```

## Context variables

Each callback receives a context object. The fields differ by callback.

### `transform(data, ctx: BatchContext)`

`ctx` is the full per-batch context. Fields:

| Field      | Type                 | Description                                                                                                                                                                                    |
| ---------- | -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `id`       | `string`             | Pipeline ID — the `id` passed to `solanaPortalStream()`.                                                                                                                                       |
| `logger`   | `Logger`             | Pino-compatible logger scoped to this batch. Defaults to the source-level logger.                                                                                                              |
| `metrics`  | `Metrics`            | Prometheus metrics registry. Use `ctx.metrics.counter()`, `.gauge()`, `.histogram()`, `.summary()` to register and update custom metrics. See [Metrics](../../guides/advanced-topics/metrics). |
| `profiler` | `Profiler`           | Open a span with `ctx.profiler.start('label')`. See [Profiling](../../guides/advanced-topics/profiling).                                                                                       |
| `stream`   | `BatchStreamContext` | Per-stream state (see below).                                                                                                                                                                  |
| `batch`    | `BatchMetadata`      | Per-batch volume info (see below).                                                                                                                                                             |

#### `ctx.stream: BatchStreamContext`

| Field                 | Type                        | Description                                                                               |
| --------------------- | --------------------------- | ----------------------------------------------------------------------------------------- |
| `dataset`             | `ApiDataset`                | Dataset metadata returned by the portal (chain name, genesis, tier).                      |
| `head.finalized`      | `BlockCursor \| undefined`  | Current finalized head known to the portal, if advertised.                                |
| `head.latest`         | `BlockCursor \| undefined`  | Current unfinalized head.                                                                 |
| `state.initial`       | `number`                    | First slot the stream was configured to read.                                             |
| `state.last`          | `number`                    | Last slot the stream intends to read (often `Infinity`).                                  |
| `state.current`       | `BlockCursor`               | Latest slot in this batch. Cursor has `{ number, hash?, timestamp? }`.                    |
| `state.rollbackChain` | `BlockCursor[]`             | Unfinalized-chain tail — cursors the stream will need to roll back if a fork is detected. |
| `progress`            | `ProgressEvent['progress']` | Progress metrics when `progress` is configured on the source; otherwise undefined.        |
| `query`               | `{ url, hash, raw }`        | Debug info for the portal query feeding this batch.                                       |

#### `ctx.batch: BatchMetadata`

| Field                 | Type                     | Description                                                             |
| --------------------- | ------------------------ | ----------------------------------------------------------------------- |
| `blocksCount`         | `number`                 | Number of slots in this batch.                                          |
| `bytesSize`           | `number`                 | Compressed payload size received from the portal.                       |
| `requests`            | `Record<number, number>` | Map of HTTP status code → number of responses that produced this batch. |
| `lastBlockReceivedAt` | `Date`                   | Wall-clock time the last block was received.                            |

### `start(ctx: StartCtx)`

Fired once, before any batch. Use to warm up caches or run one-off queries.

| Field           | Type                       | Description                                                                 |
| --------------- | -------------------------- | --------------------------------------------------------------------------- |
| `id`            | `string`                   | Pipeline ID.                                                                |
| `logger`        | `Logger`                   | Same as in `BatchContext`.                                                  |
| `metrics`       | `Metrics`                  | Same as in `BatchContext`.                                                  |
| `portal`        | `PortalClient`             | Live portal client. Use `portal.getStream(query)` for warm-up reads.        |
| `state.initial` | `number`                   | First slot the stream was configured to read.                               |
| `state.current` | `BlockCursor \| undefined` | Cursor persisted by the previous run, if any. `undefined` on a fresh start. |

### `fork(cursor, ctx: Ctx)`

Fired before the next batch whenever a reorg is detected. `cursor` is the last slot to keep; drop state produced for anything after it.

| Field      | Type       | Description                |
| ---------- | ---------- | -------------------------- |
| `logger`   | `Logger`   | Same as in `BatchContext`. |
| `profiler` | `Profiler` | Same as in `BatchContext`. |

### `stop(ctx: StopCtx)`

Fired once when the pipe stops.

| Field    | Type     | Description                |
| -------- | -------- | -------------------------- |
| `logger` | `Logger` | Same as in `BatchContext`. |


# solanaInstructionDecoder
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/solana/reference/utility-components/instruction-decoder

API reference for Solana instruction decoder

See the [Handling program instructions](../../guides/basic-development/handling-instructions) guide for usage examples and ABI generation.

Returns a query-transformer combo that instructs the [source](../basic-components/source) to fetch and decode Solana program instructions.

```ts theme={"system"}
solanaInstructionDecoder<T>(config: SolanaInstructionDecoderConfig<T>): Transformer
```

**Parameters:**

* `range`: Slot range `{ from: number | string | 'latest', to?: number }` (required)
* `programId`: Program address(es) `string | string[]` (required)
* `instructions`: Map of instruction names to ABI instruction objects (required)
* `profiler`: Profiler config shard that'll be used for labeling the transformer in profiling data `{ name: string }` (optional)
* `onError`: Error handler `(ctx: BatchCtx, error: any) => unknown | Promise<unknown>` (optional)

**Example:**

```ts theme={"system"}
const transformer = solanaInstructionDecoder({
  range: { from: 200000000, to: 201000000 },
  programId: orcaWhirlpool.programId,
  instructions: {
    swap: orcaWhirlpool.instructions.swap,
    swapV2: orcaWhirlpool.instructions.swapV2,
  },
});
```

## Decoded instruction structure

Each entry in the output arrays is a `DecodedInstruction` object:

| Field               | Type                               | Description                                                        |
| ------------------- | ---------------------------------- | ------------------------------------------------------------------ |
| `instruction`       | `{ accounts, data }`               | Decoded accounts (by name) and instruction data fields             |
| `block`             | `{ number: number, hash: string }` | Slot number and block hash                                         |
| `timestamp`         | `Date`                             | Block timestamp                                                    |
| `transaction`       | `Transaction`                      | Parent transaction (index, signatures)                             |
| `innerInstructions` | `Instruction[]`                    | CPI calls made by this instruction                                 |
| `rawInstruction`    | `Instruction`                      | Raw instruction as received from the portal, including `programId` |
| `tokenBalances`     | `TokenBalance[]`                   | Token balance changes in the parent transaction                    |
| `blockNumber`       | `number`                           | Deprecated alias for `block.number`                                |

## Instruction discriminators

Instructions are filtered by discriminators (prefix bytes of instruction data). The discriminator names indicate the number of hex characters returned:

| Name | Hex Characters | Bytes   | Common Use                    |
| ---- | -------------- | ------- | ----------------------------- |
| `d1` | 4 chars        | 2 bytes | Simple programs               |
| `d2` | 8 chars        | 4 bytes | Some native programs          |
| `d4` | 14 chars       | 7 bytes | Custom programs               |
| `d8` | 18 chars       | 9 bytes | Anchor programs (most common) |

The decoder automatically extracts the appropriate discriminator from the ABI instruction definition.

## Multiple programs

Decode instructions from multiple programs:

```ts theme={"system"}
const decoder = solanaInstructionDecoder({
  range: { from: 200000000 },
  programId: [
    orcaWhirlpool.programId,
    raydiumAmm.programId,
  ],
  instructions: {
    orcaSwap: orcaWhirlpool.instructions.swap,
    raydiumSwap: raydiumAmm.instructions.swapBaseIn,
  },
});
```

## Error handling

Handle decoding errors:

```ts theme={"system"}
const decoder = solanaInstructionDecoder({
  range: { from: 200000000 },
  programId: programId,
  instructions: { swap: instructions.swap },
  onError: (ctx, error) => {
    ctx.logger.warn(`Failed to decode instruction: ${error.message}`);
    // Return null to skip this instruction, or throw to stop processing
    return null;
  },
});
```


# metricsServer
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/solana/reference/utility-components/metrics-server

Expose Prometheus metrics and the live-stats API from your pipe

Start a metrics server on the pipe process. Required by [Pipes UI](../../guides/basic-development/pipes-ui) and by anything that scrapes Prometheus (Grafana, Alertmanager, etc.).

```ts theme={"system"}
import { metricsServer } from "@subsquid/pipes/metrics/node";
import { solanaPortalStream } from "@subsquid/pipes/solana";

solanaPortalStream({
  // ...
  metrics: metricsServer({ port: 9090 }),
  // ...
});
```

**Parameters:**

* `port`: HTTP port for the server (default: `9090`).

## Endpoints

`metricsServer()` serves four HTTP endpoints on the configured port. They are all also useful for ad-hoc inspection with `curl`.

| Path        | Content                                                                                                                             |
| ----------- | ----------------------------------------------------------------------------------------------------------------------------------- |
| `/stats`    | JSON — per-pipe progress, speed, portal query, SDK version. This is what [Pipes UI](../../guides/basic-development/pipes-ui) polls. |
| `/metrics`  | Prometheus text — built-in `sqd_*` series plus any custom metrics you registered. Scrape this from Prometheus.                      |
| `/profiler` | JSON — recent per-batch span trees. See [Profiling](../../guides/advanced-topics/profiling). Empty when profiling is disabled.      |
| `/health`   | Responds with `ok`.                                                                                                                 |

## Custom metrics

Register counters, gauges, histograms, and summaries via `ctx.metrics` in [whole-pipe transformers](../basic-components/transformer), [targets](../basic-components/target), or when consuming the pipe as an async iterator. See the [Metrics guide](../../guides/advanced-topics/metrics).

## See also

* [Pipes UI](../../guides/basic-development/pipes-ui) — visual dashboard that consumes `/stats` and `/profiler`.
* [Metrics](../../guides/advanced-topics/metrics) — walkthrough for exposing Prometheus metrics and adding custom series.
* [Profiling](../../guides/advanced-topics/profiling) — interpreting the `/profiler` span tree.


# solanaRpcLatencyWatcher
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/solana/reference/utility-components/rpc-latency-watcher

Monitor RPC latency and compare with Portal performance

The `solanaRpcLatencyWatcher` function monitors the latency between Portal and external RPC providers, helping you track indexing performance.

## Import

```ts theme={"system"}
import { solanaRpcLatencyWatcher } from '@subsquid/pipes/solana'
```

## Signature

```ts theme={"system"}
function solanaRpcLatencyWatcher(options: {
  rpcUrl: string[]
}): RpcLatencyWatcher
```

## Parameters

| Parameter | Type       | Description                           |
| --------- | ---------- | ------------------------------------- |
| `rpcUrl`  | `string[]` | Array of RPC endpoint URLs to monitor |

## Return Value

Returns a transformer that can be piped to add latency data to the stream.

## Output Data Structure

Each output contains:

```ts theme={"system"}
{
  number: number,           // Slot number
  timestamp: Date,          // Block timestamp
  rpc: Array<{
    url: string,            // RPC endpoint URL
    receivedAt: Date,       // When block was received from RPC
    portalDelayMs: number,  // Delay between RPC and Portal (ms)
  }>
}
```

## Basic Usage

```ts theme={"system"}
import { solanaPortalSource, solanaRpcLatencyWatcher } from '@subsquid/pipes/solana'

const stream = solanaPortalSource({
  portal: 'https://portal.sqd.dev/datasets/solana-mainnet',
  outputs: new SolanaQueryBuilder().addFields({ block: { number: true, hash: true, timestamp: true } }).includeAllBlocks({ from: 'latest' }).build(),
}).pipe(
  solanaRpcLatencyWatcher({
    rpcUrl: ['https://api.mainnet-beta.solana.com'],
  })
)

for await (const { data } of stream) {
  if (!data) continue

  console.log(`Slot: ${data.number}`)
  console.table(data.rpc)
}
```

## Example Output

```
-------------------------------------
BLOCK DATA: 369,377,455 / Fri Sep 26 2025 15:31:36 GMT+0400
┌───┬─────────────────────────────────────┬──────────────────────────┬───────────────┐
│   │ url                                 │ receivedAt               │ portalDelayMs │
├───┼─────────────────────────────────────┼──────────────────────────┼───────────────┤
│ 0 │ https://api.mainnet-beta.solana.com │ 2025-09-26T11:31:37.075Z │ 358           │
└───┴─────────────────────────────────────┴──────────────────────────┴───────────────┘
```

## Multiple RPC Endpoints

Monitor multiple RPC providers simultaneously:

```ts theme={"system"}
const stream = solanaPortalSource({
  portal: 'https://portal.sqd.dev/datasets/solana-mainnet',
  outputs: new SolanaQueryBuilder().addFields({ block: { number: true, hash: true, timestamp: true } }).includeAllBlocks({ from: 'latest' }).build(),
}).pipe(
  solanaRpcLatencyWatcher({
    rpcUrl: [
      'https://api.mainnet-beta.solana.com',
      'https://solana-mainnet.rpc.extrnode.com',
    ],
  })
)
```

## Metrics Integration

Export latency data to Prometheus or other monitoring systems:

```ts theme={"system"}
const stream = solanaPortalSource({
  portal: 'https://portal.sqd.dev/datasets/solana-mainnet',
  outputs: new SolanaQueryBuilder().addFields({ block: { number: true, hash: true, timestamp: true } }).includeAllBlocks({ from: 'latest' }).build(),
}).pipe(
  solanaRpcLatencyWatcher({
    rpcUrl: ['https://api.mainnet-beta.solana.com'],
  }).pipe({
    profiler: { id: 'expose metrics' },
    transform: (data, { metrics }) => {
      if (!data) return

      for (const rpc of data.rpc) {
        metrics
          .gauge({
            name: 'rpc_latency_ms',
            help: 'RPC Latency in ms',
            labelNames: ['url'],
          })
          .set({ url: rpc.url }, rpc.portalDelayMs)
      }

      return data
    },
  })
)
```

## Prometheus Export

Full example with HTTP metrics endpoint:

```ts theme={"system"}
import { solanaPortalSource, solanaRpcLatencyWatcher } from '@subsquid/pipes/solana'
import { Registry, Counter } from 'prom-client'
import http from 'http'

const registry = new Registry()
const latencyCounter = new Counter({
  name: 'portal_latency_ms_total',
  help: 'Total Portal latency in milliseconds',
  registers: [registry],
})

const stream = solanaPortalSource({
  portal: 'https://portal.sqd.dev/datasets/solana-mainnet',
  outputs: new SolanaQueryBuilder().addFields({ block: { number: true, hash: true, timestamp: true } }).includeAllBlocks({ from: 'latest' }).build(),
}).pipe(
  solanaRpcLatencyWatcher({
    rpcUrl: ['https://api.mainnet-beta.solana.com'],
  }).pipe({
    transform: (data) => {
      if (!data) return

      for (const rpc of data.rpc) {
        latencyCounter.inc(rpc.portalDelayMs)
      }

      return data
    },
  })
)

// Expose metrics endpoint
const server = http.createServer(async (req, res) => {
  if (req.url === '/metrics') {
    res.setHeader('Content-Type', registry.contentType)
    res.end(await registry.metrics())
  } else {
    res.end('OK')
  }
})
server.listen(9090)

for await (const { data } of stream) {
  // Process blocks
}
```

## How It Works

The RPC latency watcher:

1. Subscribes to slot updates via WebSocket (`slotsUpdatesSubscribe`)
2. Listens for `optimisticConfirmation` events from each RPC
3. Compares the arrival time of blocks between RPC and Portal
4. Reports the delay as `portalDelayMs`

<Warning>
  The measured values include client-side network latency. For RPC, only the arrival time of the block is measured—this does not capture the node's internal processing latency.
</Warning>

## Use Cases

* **Performance monitoring** - Track indexing latency in production
* **RPC comparison** - Compare performance across different RPC providers
* **Alerting** - Trigger alerts when latency exceeds thresholds
* **Optimization** - Identify bottlenecks in your indexing pipeline


# portalSqliteCache
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/solana/reference/utility-components/sqlite-cache

SQLite cache for Portal responses

Create SQLite cache for Portal responses. Use with `solanaPortalSource` to cache Portal API responses locally.

```ts theme={"system"}
portalSqliteCache(config: { path: string }): PortalCache
```

**Example:**

```ts theme={"system"}
import { portalSqliteCache } from "@subsquid/pipes/portal-cache/node";
import { solanaPortalSource } from "@subsquid/pipes/solana";

const source = solanaPortalSource({
  portal: "https://portal.sqd.dev/datasets/solana-mainnet",
  cache: portalSqliteCache({
    path: "./portal-cache.sqlite",
  }),
});
```

<Note>
  Import from `@subsquid/pipes/portal-cache/node` instead of `@subsquid/pipes/portal-cache`.
</Note>

### When to Use

* Development iteration
* Testing pipelines
* Repeated processing of same slot ranges.


# Why Pipes SDK?
Source: https://docs.sqd.dev/en/sdk/pipes-sdk/solana/why-pipes-sdk

And when you might want to use it

<CardGroup>
  <Card title="EVM version" icon="book" href="/en/sdk/pipes-sdk/evm/quickstart">
    Pipes SDK is also available for EVM
  </Card>
</CardGroup>

Pipes SDK is a TypeScript library for retrieving blockchain data from SQD Portals and transforming it. It features:

* **All features of the SQD Portal API:**
  * Data is downloaded in big chunks and at a high speed.
  * It is filtered on the server side - you only download what you need.
  * Real-time data is supported.
  * Information on blockchain reorganizations and finality is available; in standard modules these are handled automatically.
* **Being a library**: although Pipes SDK can be used to build full-featured blockchain indexers, it is easy to embed it into larger applications, microservices, or data processing workflows.
* **Reusable modules:** data filters can be bundled with transformation logic, and the resulting modules can be mixed and matched. For example you can create modules to get you data on Raydium swaps and SPL token balance updates, then just plug them into your pipe when you need either.
* **Simplicity of extension:** we've made adding new modules as simple as possible.
  * This includes data sinks: adding support for your database, data lake or message queue is no longer a hassle.

## When to Use Pipes SDK

1. You want common protocols (SPL, Jupiter AMM, Raydium etc) handled for you. If all of your data is like that, you can start uploading it into your database in minutes.

2. You need deep customization of any part of the data pipeline.

## When to Use Alternatives

1. If you want to use Portal data in an non-JS app, consider using
   * [Raw Portal API](/en/portal/solana/api) for all languages.

2. Consider using [Squid SDK](/en/sdk/squid-sdk/evm) if:
   * You're making a self-contained Web3 data service such as a GraphQL API.
   * You're looking for an indexing framework similar to TheGraph, Ponder or Envio.


# ApeWorx plugin
Source: https://docs.sqd.dev/en/sdk/squid-sdk/apeworx

Use SQD Network as a data source for ApeWorx — Python plugin for retrieving blocks, events, and transactions in bulk via the subsquid plugin.

# ApeWorx SQD plugin

<Info>
  The `subsquid` ApeWorx plugin is currently in beta. Please report any bugs or suggestions to [Squid Devs](https://t.me/HydraDevs) or open an issue at the [GitHub repo](https://github.com/subsquid/ape-subsquid/) of the plugin.
</Info>

[ApeWorx](https://apeworx.io) is a modular Web3 development framework for Python programmers. Among other things, it is capable of [retrieving blockchain data in bulk](https://docs.apeworx.io/ape/stable/userguides/data.html). The data can come from various sources, including SQD Network.

The network provides free access to blocks and event logs data. On long block ranges (>1k blocks) data retrieval is orders of magnitude (>10x) faster compared to RPC-based data sources.

## Basic usage

In an existing [ApeWorx installation](https://docs.apeworx.io/ape/stable/userguides/quickstart.html#installation), run the following to install the `subsquid` plugin:

```bash theme={"system"}
ape plugins install "ape-subsquid@git+https://github.com/subsquid/ape-subsquid.git@main"
```

To source data from SQD Network, start `ape console` as usual, e.g. with

```bash theme={"system"}
ape console --network ethereum:mainnet:geth
```

then pass

```python theme={"system"}
engine_to_use='subsquid'
```

to your data query methods. You can speed up the following calls:

1. Bulk block data retrieval with `chain.blocks.query`:
   ```python theme={"system"}
   df = chain.blocks.query(
       '*',
       start_block=18_000_000,
       stop_block=19_000_000,
       engine_to_use='subsquid'
   )
   ```
   This query retrieves data on 1M blocks in about 11 minutes.

2. Contract events retrieval:
   ```python theme={"system"}
   contract = Contract('0xdac17f958d2ee523a2206206994597c13d831ec7', abi='usdt.json')
   df = contract.Transfer.query(
       '*',
       start_block=18_000_000,
       stop_block=18_100_000,
       engine_to_use='subsquid'
   )
   ```
   This query retrieves 1.6M events emitted over 100k block in about 17 minutes.

<Warning>
  At the moment, all SQD Network datasets are updated only once every several thousands of blocks. The current dataset height can be retrieved with `get_network_height()`:

  ```python theme={"system"}
  from ape_subsquid import get_network_height

  get_network_height() # currently 19212115 while the chain is at 19213330
  ```

  Queries that request blocks above the current dataset height **will fail** with an `ape_subsquid.exceptions.DataRangeIsNotAvailable` exception. That includes queries without an explicitly set `stop_block`. If you don't need the recent data, you can explicitly request the data up to the block height of the SQD dataset, e.g.

  ```python theme={"system"}
  subsquid_height = get_network_height()
  df = chain.blocks.query(
      '*',
      start_block=19_000_000,
      stop_block=subsquid_height,
      engine_to_use='subsquid'
  )
  ```

  To get the latest data, retrieve the tail with the default engine and append it to the dataframe:

  ```python theme={"system"}
  taildf = chain.blocks.query('*', start_block=subsquid_height+1)
  df = pd.concat([df, taildf], ignore_index=True, copy=False)
  ```
</Warning>

<Info>
  When working with block ranges much longer than 1M blocks, the plugin may occasionally fail due to HTTP 503 errors rarely returned by the network. If you encounter this issue, split your block range into sub-1M blocks intervals and retrieve the data for each interval separately, retrying when the queries throw exceptions.
</Info>

## Networks support

The following networks available via plugins from `ape plugins list --all` are supported:

* `arbitrum`
* `avalanche`
* `base`
* `bsc`
* `ethereum`
* `fantom`
* `optimism`
* `polygon`
* `polygon-zkevm`

## Extra functionality

### Low-level queries

The plugin supports the following [low-level queries](https://docs.apeworx.io/ape/stable/methoddocs/api.html#module-ape.api.query):

* [`BlockQuery`](https://docs.apeworx.io/ape/stable/methoddocs/api.html#ape.api.query.BlockQuery)
* [`ContractEventQuery`](https://docs.apeworx.io/ape/stable/methoddocs/api.html#ape.api.query.ContractEventQuery)
* [`ContractCreationQuery`](https://docs.apeworx.io/ape/stable/methoddocs/api.html#ape.api.query.ContractCreationQuery) (only for [networks with traces support](/en/data/evm))
* [`AccountTransactionQuery`](https://docs.apeworx.io/ape/stable/methoddocs/api.html#ape.api.query.AccountTransactionQuery)

The following queries are **NOT** supported:

* [`BlockTransactionQuery`](https://docs.apeworx.io/ape/stable/methoddocs/api.html#ape.api.query.BlockTransactionQuery): uses hashes instead of numbers to identify blocks. This is not supported by SQD Network.
* [`ContractMethodQuery`](https://docs.apeworx.io/ape/stable/methoddocs/api.html#ape.api.query.ContractMethodQuery): not used by any high-level methods at the moment. If you need it, let us know at the [Squid Devs channel](https://t.me/HydraDevs).

### Account history retrieval

The plugin can be used to retrieve historical data for individual accounts, although this is much slower than working with specialized APIs such as Etherscan.

```python theme={"system"}
accountHistory = chain.history._get_account_history('vitalik.eth')
accountHistory.query('value', engine_to_use='subsquid')
```


# Squid SDK Examples
Source: https://docs.sqd.dev/en/sdk/squid-sdk/examples

A library of example Squid SDK indexers — DEX swaps, NFTs, stablecoins, multichain, ink! contracts — to learn from and bootstrap your own project.

<Columns>
  <Card title="Subsquid Labs on GitHub" icon="github" href="https://github.com/subsquid-labs">
    Example squid repositories.
  </Card>

  <Card title="Examples on docs.sqd.ai" icon="list" href="https://docs.sqd.ai/sdk/examples">
    Tag-filtered list of example squids.
  </Card>
</Columns>


# Squid SDK FAQ
Source: https://docs.sqd.dev/en/sdk/squid-sdk/faq

Frequently asked questions about Squid SDK — common errors, processor configuration, GraphQL setup, and migration guidance for indexer developers.

<AccordionGroup>
  <Accordion title="What are some real-world applications for which Squid SDK was a good fit?">
    Here is an incomplete list:

    * DeFi dashboards, tracking addresses and internal transactions
    * NFT marketplaces, with a dynamic sets of NFT contracts to watch
    * Historical price feeds, tracking Uniswap trades and Chainlink oracle contracts
    * Mining smart contract deployments and the bytecode
    * Real-time bots (\<1sec delay) triggered by on-chain activity
  </Accordion>

  <Accordion title="How does Squid SDK handle unfinalized blocks?">
    The SQD Network only serves finalized blocks and is typically \~1000 blocks behind the tip. The most recent blocks, as well as the unfinalized blocks are seamlessly handled by the SDK from a complementary RPC data source, set by the `chain` config. Potential chain reorgs are automatically handled under the hood. See [Indexing unfinalized blocks](/en/sdk/squid-sdk/resources/unfinalized-blocks) for details.
  </Accordion>

  <Accordion title="What is the latency for the data served by the squid?">
    Since the ArrowSquid release, the Squid SDK has the option to ingest unfinalized blocks directly from an RPC endpoint, making the indexing real-time.
  </Accordion>

  <Accordion title="How do I enable GraphQL subscriptions for local runs?">
    Add `--subscription` flag to the `serve` command defined in `commands.json`. See [Subscriptions](/en/sdk/squid-sdk/reference/openreader-server/configuration/subscriptions) for details.
  </Accordion>

  <Accordion title="How do squids keep track of their sync progress?">
    Depends on the data sink used. Squid processors that use [`TypeormDatabase`](/en/sdk/squid-sdk/resources/persisting-data/typeorm) keep their state in a [schema](https://www.postgresql.org/docs/current/sql-createschema.html), not in a table. By default the schema is called `squid_processor` (name must be overridden in [multiprocessor squids](/en/sdk/squid-sdk/resources/multichain)). You can view it with

    ```sql theme={"system"}
    select * from squid_processor.status;
    ```

    and manually drop it with

    ```sql theme={"system"}
    drop schema squid_processor cascade;
    ```

    to reset the processor status.

    Squids that store their data in [file-based datasets](/en/sdk/squid-sdk/resources/persisting-data/file) store their status in `status.txt` by default. This can be overridden by defining custom [database hooks](/en/sdk/squid-sdk/resources/persisting-data/file#hooks).
  </Accordion>

  <Accordion title="Is there a healthcheck endpoint for the indexer?">
    Yes, the processor exposes the key prometheus metrics at the `${process.env.PROMETHEUS_PORT}/metric` endpoint. The squids deployed to the SQD Cloud also publicly explose the metrics, see [Monitoring in the Cloud](/en/cloud/resources/monitoring)
  </Accordion>

  <Accordion title="Do squids have a debug mode?">
    Yes. To see all debug messages, set the `SQD_DEBUG` anv variable to `*`:

    ```bash title=".env" theme={"system"}
    SQD_DEBUG=*
    ```

    You can also enable debug messages just for a specific namespace. For example,

    ```bash title=".env" theme={"system"}
    SQD_DEBUG=sqd:processor:archive
    ```

    will show only the messages related to your squid's queries to the SQD Network.
  </Accordion>
</AccordionGroup>


# Squid SDK Glossary
Source: https://docs.sqd.dev/en/sdk/squid-sdk/glossary

Definitions of key Squid SDK terms — archives, gateways, FireSquid, processors, batch handlers, store, schema, and the SQD Network terminology.

### Archives

Deprecated term used for [SQD Network](/en/network) and for the data sourcing service of the deprecated FireSquid SDK version. Occasionally refers to a chain-specific endpoint available from either source (e.g. "an Ethereum archive"). The new terminology is:

* "Archives" as an abstract collection of services for some networks is replaced by "[SQD Network](/en/network)" (when referring to data location) or "SQD Network gateway" (when referring to the service)
* "public Archives" are replaced by the [open private version](/en/network/overview#open-private-network) of SQD Network
* "an archive" for a particular network is replaced by "a SQD Network gateway"

List of gateways for the open private SQD Network is available in [these docs](/en/data/evm) and via [`sqd gateways`](/en/sdk/squid-sdk/squid-cli/gateways).

**Not to be confused with [archive blockchain nodes](https://ethereum.org/developers/docs/nodes-and-clients/archive-nodes)**.

### `archive-registry`

The deprecated NPM package `@subsquid/archive-registry` that was used to look up squid data sources by network aliases (with `lookupArchive()` and a small CLI). We now recommend using raw gateway URLs instead of `lookupArchive()` calls in processor configuration. The exploratory CLI is replaced by [`sqd gateways`](/en/sdk/squid-sdk/squid-cli/gateways); list of available network-specific gateways is also [available in these docs](/en/data/evm).

### Block

An atomic state transition of a blockchain. Typically an ordered collection of transactions.

### Call

On [Substrate](#substrate), a call is a sub-routine changing the runtime state. An extrinsic consists of a root call which in turn may have sub-calls, thus calls executed by an extrinsic have parent-child relationship. For example, `util.batch` extrinsic has a single root call and multiple child calls. SQD processor is call-based rather than extrinsic based, as normally one is interested in specific calls changing the substrate state, no matter if it was part of a batch extrinsic, or it was wrapped in a sudo or proxy call.

### Cloud (former Aquarium)

A [cloud service](/en/cloud) for deploying [squids](#squid) in a serverless fashion maintained by Subsquid Labs GmbH.

### Contracts pallet

[`Contracts`](https://substrate-developer-hub.github.io/substrate-how-to-guides/docs/pallet-design/add-contracts-pallet/) is a [pallet](#pallet) developed by Parity to execute WASM-based smart contracts.

### ETL

Stands for Extract-Transform-Load. A pipeline to extract data from the source, enrich, transform, normalize and load into the target data store.

### Event

An operation performed during a blockchain state transition that results in emission of a structured log message. Subsequently, the message can be retrieved from blockchain nodes.

### EVM

Stands for the Ethereum Virtual Machine. An instruction set and the runtime specification originally developed for the Ethereum network which was later adopted by many other chains (e.g. BSC, Polygon).

### Extrinsic

On [Substrate](#substrate), a generalized transaction externally submitted to a Substrate runtime for execution. There are [technical nuances](https://substrate.stackexchange.com/questions/2248/is-a-transaction-an-extrinsic) differentiating transactions and extrinsics.

### FRAME pallets

A collection of standard [Substrate](#substrate) pallets maintained by Parity. Currently kept in the [Substrate Repo](https://github.com/paritytech/substrate/tree/master/frame).

### Frontier pallet

[Frontier](https://github.com/polkadot-evm/frontier) is a [Substrate](#substrate) [pallet](#pallet) implementing the [Ethereum Virtual Machine](#evm). In particular, Substrate chains with Frontier pallet support EVM-based smart contracts.

### GraphQL

A declarative query language and an API specification language developed by Facebook as an alternative to REST. See the official [GraphQL docs](https://graphql.org/) for more details.

### Hot blocks

An alternative term for [unfinalized blocks](/en/sdk/squid-sdk/resources/unfinalized-blocks) that emphasizes the fact that the data extracted from these blocks is volatile. Sometimes used in Squid SDK code and comments.

### ink!

An SDK (software development kit) and a smart-contract language for developing WASM-based smart contracts, maintained by Parity. The contracts developed with ink! are compiled into a WASM blob compatible with the API exposed by the [`Contracts` pallet](#contracts-pallet). More details [here](https://use.ink).

### OpenReader

1. SQD's own open source [GraphQL server](/en/sdk/squid-sdk/reference/openreader-server/overview), built in-house. No longer recommended for new projects running PostgreSQL due to its [limitations](/en/sdk/squid-sdk/reference/openreader-server/overview#limitations). See [Serving GraphQL](/en/sdk/squid-sdk/resources/serving-graphql) to learn more.

2. The GraphQL [schema generation library](https://github.com/subsquid/squid-sdk/tree/master/graphql/openreader) at the heart of 1. Implements [OpenCRUD](https://www.opencrud.org/).

### Pallet

A portable module that can be added to a [Substrate](#substrate) runtime. Typically, contains a self-contained implementation of a business logic that can be re-used across multiple chains.

### Schema file

A file describing the target data schema for a squid, normally called `schema.graphql`. The file uses a GraphQL dialect to define entities, properties and relations. See [details here](/en/sdk/squid-sdk/reference/schema-file).

### State

A key-value map defining the internal worldview of an EVM contract of a Substrate runtime at a specific point in time. The consensus algorithm ensures that the honest majority of the nodes agree on the runtime state.

### Storage

On [Substrate](#substrate), a persistent key-value database kept by the chain nodes. It is used to access the current and historical [state](#state). See details on the [Substrate docs page](https://docs.substrate.io/fundamentals/state-transitions-and-storage/)

### Squid

A project consisting of an [ETL](#etl) for extracting and transforming on-chain data (squid processor), and optionally an API to present the data (squid API).

### Squid processor

The [ETL](#etl) part of the squid. Extracts on-chain data from an [SQD Network](/en/network) gateway and/or directly from chain RPC, then transforms and optionally enriches it with external data. Saves the result into a target [data sink](/en/sdk/squid-sdk/reference/store).

### Squid API

The data presentation part of the squid. Typically, it's an [OpenReader](#openreader) GraphQL API auto-generated from a schema file.

### Substrate

[Substrate](https://substrate.io) is a framework for developing blockchain runtimes. Used to develop the Polkadot, Kusama chains and all the parachains.

### Substrate Runtime

The code that defines the state transition logic of a blockchain, and by extention its business logic. See details on the [Substrate docs page](https://docs.substrate.io/fundamentals/architecture/).

### Squid SDK

[Squid SDK](/en/sdk/squid-sdk/overview) is a collection of open-source libraries for building [squids](#squid).

### Typegen

A tool generating strongly typed data access classes from a metadata in some format. Squid SDK includes typegen tools:

* for accessing EVM smart contract data based on the contract ABI
* for accessing event, extrinsics and storage data based on [Substrate](#substrate) metadata
* for accessing [ink!](#ink) smart contract data based on the contract metadata

See [this section](/en/sdk/squid-sdk/resources/tools/typegen/generation) for documentation on these tools.

### WASM

A portable binary code format and an execution environment specification. WASM programs enjoy deterministic outputs and near-native execution speeds, which makes WASM an attractive alternative to EVM.


# Development flow
Source: https://docs.sqd.dev/en/sdk/squid-sdk/how-to-start/squid-development

Recommended Squid SDK development workflow — model the schema, write the processor, run migrations, iterate locally, and deploy to SQD Cloud.

This page is a definitive end-to-end guide into practical squid development. It uses templates to simplify the process. Check out [Squid from scratch](/en/sdk/squid-sdk/how-to-start/squid-from-scratch) for a more educational barebones approach.

<Info>
  Feel free to also use the template-specific `sqd` scripts defined in [`commands.json`](/en/sdk/squid-sdk/squid-cli/commands-json) to simplify your workflow. See [sqd CLI cheatsheet](/en/sdk/squid-sdk/how-to-start/cli-cheatsheet) for a short intro.
</Info>

## Prepare the environment

* Node v16.x or newer
* Git
* [Squid CLI](/en/sdk/squid-sdk/squid-cli/installation)
* Docker (if your squid will store its data to PostgreSQL)

See also the [Environment set up](/en/sdk/squid-sdk/how-to-start/development-environment-set-up) page.

## Understand your technical requirements

Consider your business requirements and find out

1. How the data should be delivered. Options:
   * [PostgreSQL](/en/sdk/squid-sdk/resources/persisting-data/typeorm) with an optional [GraphQL API](/en/sdk/squid-sdk/resources/serving-graphql) - can be real-time
   * [file-based dataset](/en/sdk/squid-sdk/resources/persisting-data/file) - local or on S3
   * [Google BigQuery](/en/sdk/squid-sdk/resources/persisting-data/bigquery)
2. What data should be delivered
3. What are the technologies powering the blockchain(s) in question. Supported options:

   * Ethereum Virtual Machine (EVM) chains like [Ethereum](https://ethereum.org) - [supported networks](/en/data/evm)
   * [Substrate](https://substrate.io)-powered chains like [Polkadot](https://polkadot.network) and [Kusama](https://kusama.network) - [supported networks](/en/data/substrate)

   Note that you can use SQD via [RPC ingestion](/en/sdk/squid-sdk/resources/unfinalized-blocks) even if your network is not listed.
4. What exact data should be retrieved from blockchain(s)
5. Whether you need to mix in any [off-chain data](/en/sdk/squid-sdk/resources/external-api)

#### Example requirements

<AccordionGroup>
  <Accordion title="DEX analytics on Polygon">
    Suppose you want to train a prototype ML model on all trades done on Uniswap Polygon since the v3 upgrade.

    1. A delay of a few hours typically won't matter for training, so you may want to deliver the data as files for easier handling.
    2. The output could be a simple list of swaps, listing pair, direction and token amounts for each.
    3. Polygon is an EVM chain.
    4. All the required data is contained within `Swap` events emitted by the pair pool contracts. Uniswap deploys these [dynamically](/en/sdk/squid-sdk/resources/evm/factory-contracts), so you will also have to capture `PoolCreated` events from the factory contract to know which `Swap` events are coming from Uniswap and map them to pairs.
    5. No off-chain data will be necessary for this task.
  </Accordion>

  <Accordion title="NFT ownership on Ethereum">
    Suppose you want to make a website that shows the image and ownership history for ERC721 NFTs from a certain Ethereum contract.

    1. For this application it makes sense to deliver a GraphQL API.
    2. Output data might have `Token`, `Owner` and `Transfer` database tables / [entities](/en/sdk/squid-sdk/reference/schema-file/entities), with e.g. `Token` supplying all the fields necessary to show ownership history and the image.
    3. Ethereum is an EVM chain.
    4. Data on token mints and ownership history can be derived from `Transfer(address,address,uint256)` EVM event logs emitted by the contract. To render images, you will also need token metadata URLs that are only available by [querying the contract state](/en/sdk/squid-sdk/resources/tools/typegen/state-queries) with the `tokenURI(uint256)` function.
    5. You'll need to retrieve the off-chain token metadata (usually from IPFS).
  </Accordion>

  <Accordion title="Kusama transfers BigQuery dataset">
    Suppose you want to create a BigQuery dataset with Kusama native tokens transfers.

    1. The delivery format is BigQuery.
    2. A single table with `from`, `to` and `amount` columns may suffice.
    3. Kusama is a Substrate chain.
    4. The required data is available from `Transfer` events emitted by the `Balances` pallet. Take a look at our [Substrate data sourcing miniguide](/en/sdk/squid-sdk/resources/substrate/data-sourcing-miniguide) for more info on how to figure out which pallets, events and calls are necessary for your task.
    5. No off-chain data will be necessary for this task.
  </Accordion>
</AccordionGroup>

## Start from a template

Although it is possible to [compose a squid from individual packages](/en/sdk/squid-sdk/how-to-start/squid-from-scratch), in practice it is usually easier to start from a template.

<Tabs>
  <Tab title="EVM">
    <AccordionGroup>
      <Accordion title="Templates for the PostgreSQL+GraphQL data destination">
        * A minimal template intended for developing EVM squids. Indexes ETH burns.
          ```bash theme={"system"}
          sqd init my-squid-name -t evm
          ```
        * A starter squid for indexing ERC20 transfers.
          ```bash theme={"system"}
          sqd init my-squid-name -t https://github.com/subsquid-labs/squid-erc20-template
          ```
        * Classic [example Subgraph](https://github.com/graphprotocol/example-subgraph) after a [migration](/en/sdk/squid-sdk/resources/migrate/migrate-subgraph) to SQD.
          ```bash theme={"system"}
          sqd init my-squid-name -t gravatar
          ```
        * A template showing how to [combine data from multiple chains](/en/sdk/squid-sdk/resources/multichain). Indexes USDC transfers on Ethereum and Binance.
          ```bash theme={"system"}
          sqd init my-squid-name -t multichain
          ```
      </Accordion>

      <Accordion title="Templates for storing data in files">
        * USDC transfers -> local CSV
          ```bash theme={"system"}
          sqd init my-squid-name -t https://github.com/subsquid-labs/file-store-csv-example
          ```
        * USDC transfers -> local Parquet
          ```bash theme={"system"}
          sqd init my-squid-name -t https://github.com/subsquid-labs/file-store-parquet-example
          ```
        * USDC transfers -> CSV on S3
          ```bash theme={"system"}
          sqd init my-squid-name -t https://github.com/subsquid-labs/file-store-s3-example
          ```
      </Accordion>

      <Accordion title="Templates for the Google BigQuery data destination">
        * USDC transfers -> BigQuery dataset
          ```bash theme={"system"}
          sqd init my-squid-name -t https://github.com/subsquid-labs/squid-bigquery-example
          ```
      </Accordion>
    </AccordionGroup>
  </Tab>

  <Tab title="Substrate">
    <Accordion title="Templates for the PostgreSQL+GraphQL data destination">
      * Native events emitted by Substrate-based chains
        ```bash theme={"system"}
        sqd init my-squid-name -t substrate
        ```
      * ink! smart contracts
        ```bash theme={"system"}
        sqd init my-squid-name -t ink
        ```
      * Frontier EVM contracts on Astar and Moonbeam
        ```bash theme={"system"}
        sqd init my-squid-name -t frontier-evm
        ```
    </Accordion>
  </Tab>
</Tabs>

After retrieving the template of choice install its dependencies:

```bash theme={"system"}
cd my-squid-name
npm i
```

Test the template locally. The procedure varies depending on the data sink:

<Tabs>
  <Tab title="PostgreSQL+GraphQL">
    1. Launch a PostgreSQL container with
       ```bash theme={"system"}
       docker compose up -d
       ```

    2. Build the squid with
       ```bash theme={"system"}
       npm run build
       ```

    3. Apply the DB migrations with
       ```bash theme={"system"}
       npx squid-typeorm-migration apply
       ```

    4. Start the squid processor with
       ```bash theme={"system"}
       node -r dotenv/config lib/main.js
       ```
       You should see output that contains lines like these ones:
       ```bash theme={"system"}
       04:11:24 INFO  sqd:processor processing blocks from 6000000
       04:11:24 INFO  sqd:processor using archive data source
       04:11:24 INFO  sqd:processor prometheus metrics are served at port 45829
       04:11:27 INFO  sqd:processor 6051219 / 18079056, rate: 16781 blocks/sec, mapping: 770 blocks/sec, 544 items/sec, eta: 12m
       ```

    5. Start the GraphQL server by running
       ```bash theme={"system"}
       npx squid-graphql-server
       ```
       in a separate terminal, then visit the [GraphiQL console](http://localhost:4350/graphql) to verify that the GraphQL API is up.

    When done, shut down and erase your database with `docker compose down`.
  </Tab>

  <Tab title="filesystem dataset">
    1. (for the S3 template only) Set the credentials and prepare a bucket for your data as described in the [template README](https://github.com/subsquid-labs/file-store-s3-example/blob/main/README.md).

    2. Build the squid with
       ```bash theme={"system"}
       npm run build
       ```

    3. Start the squid processor with
       ```bash theme={"system"}
       node -r dotenv/config lib/main.js
       ```
       The output should contain lines like these ones:
       ```bash theme={"system"}
       04:11:24 INFO  sqd:processor processing blocks from 6000000
       04:11:24 INFO  sqd:processor using archive data source
       04:11:24 INFO  sqd:processor prometheus metrics are served at port 45829
       04:11:27 INFO  sqd:processor 6051219 / 18079056, rate: 16781 blocks/sec, mapping: 770 blocks/sec, 544 items/sec, eta: 12m
       ```
       You should see a `./data` folder populated with indexer data appear in a bit. A local folder looks like this:
       ```bash theme={"system"}
       $ tree ./data/
       ./data/
       ├── 0000000000-0007242369
       │   └── transfers.tsv
       ├── 0007242370-0007638609
       │   └── transfers.tsv
       ...
       └── status.txt
       ```
  </Tab>

  <Tab title="BigQuery">
    Create a dataset with your BigQuery account, then follow the [template README](https://github.com/subsquid-labs/squid-bigquery-example/blob/master/README.md).
  </Tab>
</Tabs>

## The bottom-up development cycle

The advantage of this approach is that the code remains buildable at all times, making it easier to catch issues early.

### I. Regenerate the task-specific utilities

<Tabs>
  <Tab title="EVM">
    Retrieve JSON ABIs for all contracts of interest (e.g. from Etherscan), taking care to get ABIs for implementation contracts and not [proxies](/en/sdk/squid-sdk/resources/evm/proxy-contracts) where appropriate. Assuming that you saved the ABI files to `./abi`, you can then regenerate the utilities with

    ```bash theme={"system"}
    npx squid-evm-typegen ./src/abi ./abi/*.json --multicall
    ```

    Or if you would like the tool to retrieve the ABI from Etherscan in your stead, you can run e.g.

    ```bash theme={"system"}
    npx squid-evm-typegen \
      src/abi \
      0xdAC17F958D2ee523a2206206994597C13D831ec7#usdt
    ```

    The utility classes will become available at `src/abi`.

    See also [EVM typegen code generation](/en/sdk/squid-sdk/resources/tools/typegen/generation).
  </Tab>

  <Tab title="Substrate">
    Follow the respective reference configuration pages of each typegen tool:

    * [Substrate typegen configuration](/en/sdk/squid-sdk/resources/tools/typegen/generation)
    * [ink! typegen configuration](/en/sdk/squid-sdk/resources/tools/typegen/generation)

    <Accordion title="A tip for `frontier-evm` squids">
      These squids use both Substrate typegen *and* EVM typegen. To generate all the required utilities, [configure the Substrate part](/en/sdk/squid-sdk/resources/tools/typegen/generation), then save all relevant JSON ABIs to `./abi`, then run

      ```bash theme={"system"}
      npx squid-evm-typegen ./src/abi ./abi/*.json --multicall
      ```

      followed by

      ```bash theme={"system"}
      npx squid-substrate-typegen ./typegen.json
      ```
    </Accordion>
  </Tab>
</Tabs>

### II. Configure the data requests

Data requests are [customarily](/en/sdk/squid-sdk/how-to-start/layout) defined at `src/processor.ts`. The details depend on the network type:

<Tabs>
  <Tab title="EVM">
    Edit the definition of `const processor` to

    1. Use a data source appropriate for your chain and task.
       * It is possible to [use RPC](/en/sdk/squid-sdk/reference/processors/evm-batch/general#set-rpc-endpoint) as the only data source, but [adding](/en/sdk/squid-sdk/reference/processors/evm-batch/general#set-gateway) a [SQD Network](/en/data/evm) data source will make your squid sync much faster.
       * RPC is a hard requirement if you're building a real-time API.
       * If you're using RPC as one of your data sources, make sure to [set the number of finality confirmations](/en/sdk/squid-sdk/reference/processors/evm-batch/general#set-finality-confirmation) so that [hot blocks ingestion](/en/sdk/squid-sdk/resources/unfinalized-blocks) works properly.

    2. Request all [event logs](/en/sdk/squid-sdk/reference/processors/evm-batch/logs), [transactions](/en/sdk/squid-sdk/reference/processors/evm-batch/transactions), [execution traces](/en/sdk/squid-sdk/reference/processors/evm-batch/traces) and [state diffs](/en/sdk/squid-sdk/reference/processors/evm-batch/state-diffs) that your task requires, with any necessary related data (e.g. parent transactions for event logs).

    3. [Select all data fields](/en/sdk/squid-sdk/reference/processors/evm-batch/field-selection) necessary for your task (e.g. `gasUsed` for transactions).

    See [reference documentation](/en/sdk/squid-sdk/reference/processors/evm-batch) for more info and [processor configuration showcase](/en/sdk/squid-sdk/examples) for a representative set of examples.
  </Tab>

  <Tab title="Substrate">
    Edit the definition of `const processor` to

    1. Use a data source appropriate for your chain and task
       * [Use](/en/sdk/squid-sdk/reference/processors/substrate-batch/general#set-gateway) a [SQD Network gateway](/en/data/substrate) whenever it is available. [RPC](/en/sdk/squid-sdk/reference/processors/evm-batch/general#set-rpc-endpoint) is still required in this case.
       * For networks without a gateway use just the RPC.

    2. Request all [events](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#events) and [calls](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#calls) that your task requires, with any necessary related data (e.g. parent extrinsics).

    3. If your squid indexes any of the following:

       * an [ink! contract](/en/sdk/squid-sdk/resources/substrate/ink)
       * an EVM contract running on the [Frontier EVM pallet](/en/sdk/squid-sdk/resources/substrate/frontier-evm)
       * [Gear messages](/en/sdk/squid-sdk/resources/substrate/gear)

       then you can use some of the [specialized data requesting methods](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#specialized-setters) to retrieve data more selectively.

    4. [Select all data fields](/en/sdk/squid-sdk/reference/processors/substrate-batch/field-selection) necessary for your task (e.g. `fee` for extrinsics).

    See [reference documentation](/en/sdk/squid-sdk/reference/processors/substrate-batch) for more info. Processor config examples can be found in the tutorials:

    * [general Substrate](/en/sdk/squid-sdk/tutorials/substrate)

    * [ink!](/en/sdk/squid-sdk/tutorials/ink)

    * [Frontier EVM](/en/sdk/squid-sdk/tutorials/frontier-evm)
  </Tab>
</Tabs>

### III. Decode and normalize the data

Next, change the batch handler to decode and normalize your data.

In templates, the batch handler is defined at the [`processor.run()`](/en/sdk/squid-sdk/reference/processors/architecture#processorrun) call in `src/main.ts` as an inline function. Its sole argument `ctx` contains:

* at `ctx.blocks`: all the requested data for a batch of blocks
* at `ctx.store`: the means to save the processed data
* at `ctx.log`: a [`Logger`](/en/sdk/squid-sdk/reference/logger)
* at `ctx.isHead`: a boolean indicating whether the batch is at the current chain head
* at `ctx._chain`: the means to access RPC for [state calls](#external-data)

This structure ([reference](/en/sdk/squid-sdk/reference/processors/architecture#batch-context)) is common for all processors; the structure of `ctx.blocks` items varies.

<Tabs>
  <Tab title="EVM">
    Each item in `ctx.blocks` contains the data for the requested logs, transactions, traces and state diffs for a particular block, plus some info on the block itself. See [EVM batch context reference](/en/sdk/squid-sdk/reference/processors/evm-batch/context-interfaces).

    Use the `.decode` methods from the [contract ABI utilities](#typegen) to decode events and transactions, e.g.

    ```ts theme={"system"}
    import * as erc20abi from './abi/erc20'

    processor.run(db, async ctx => {
      for (let block of ctx.blocks) {
        for (let log of block.logs) {
          if (log.topics[0]===erc20abi.events.Transfer.topic) {
            let {from, to, value} = erc20.events.Transfer.decode(log)
          }
        }
      }
    })
    ```

    See also the [EVM data decoding](/en/sdk/squid-sdk/resources/tools/typegen/decoding).
  </Tab>

  <Tab title="Substrate">
    Each item in `ctx.blocks` contains the data for the requested events, calls and, if requested, any related extrinsics; it also has some info on the block itself. See [Substrate batch context reference](/en/sdk/squid-sdk/reference/processors/substrate-batch/context-interfaces).

    Use the `.is()` and `.decode()` functions to decode the data for each runtime version, e.g. like this:

    ```ts theme={"system"}
    import {events} from './types'

    processor.run(db, async ctx => {
      for (let block of ctx.blocks) {
        for (let event of block.events) {
          if (event.name == events.balances.transfer.name) {
            let rec: {from: string; to: string; amount: bigint}
            if (events.balances.transfer.v1020.is(event)) {
              let [from, to, amount] = events.balances.transfer.v1020.decode(event)
              rec = {from, to, amount}
            }
            else if (events.balances.transfer.v1050.is(event)) {
              let [from, to, amount] = events.balances.transfer.v1050.decode(event)
              rec = {from, to, amount}
            }
            else if (events.balances.transfer.v9130.is(event)) {
              rec = events.balances.transfer.v9130.decode(event)
            }
            else {
              throw new Error('Unsupported spec')
            }
          }
        }
      }
    })

    ```

    See also the [Substrate data decoding](/en/sdk/squid-sdk/resources/tools/typegen/decoding).

    You can also decode the data of certain pallet-specific events and transactions with specialized tools:

    * use the utility classes made with `@substrate/squid-ink-typegen` to [decode events emitted by ink! contracts](/en/sdk/squid-sdk/resources/tools/typegen/decoding)
    * use the [`@subsquid/frontier` utils](/en/sdk/squid-sdk/reference/frontier) and the [EVM typegen](/en/sdk/squid-sdk/resources/tools/typegen/decoding) to decode event logs and transactions of EVM contracts
  </Tab>
</Tabs>

### (Optional) IV. Mix in external data and chain state calls output

If you need external (i.e. non-blockchain) data in your transformation, take a look at the [External APIs and IPFS](/en/sdk/squid-sdk/resources/external-api) page.

If any of the on-chain data you need is unavalable from the processor or incovenient to retrieve with it, you have an option to get it via [direct chain queries](/en/sdk/squid-sdk/resources/tools/typegen/state-queries).

### V. Prepare the store

At `src/main.ts`, change the [`Database`](/en/sdk/squid-sdk/resources/persisting-data/overview) object definition to accept your output data. The methods for saving data will be exposed by `ctx.store` within the [batch handler](/en/sdk/squid-sdk/reference/processors/architecture).

<Tabs>
  <Tab title="PostgreSQL+GraphQL">
    1. Define the schema of the database (and the [core schema of the OpenReader GraphQL API](/en/sdk/squid-sdk/reference/openreader-server/api) if it is used) at [`schema.graphql`](/en/sdk/squid-sdk/reference/schema-file).

    2. Regenerate the TypeORM model classes with
       ```bash theme={"system"}
       npx squid-typeorm-codegen
       ```
       The classes will become available at `src/model`.

    3. Compile the models code with
       ```bash theme={"system"}
       npm run build
       ```

    4. Ensure that the squid has access to a blank database. The easiest way to do so is to start PostgreSQL in a Docker container with

       ```bash theme={"system"}
       docker compose up -d
       ```

       If the container is running, stop it and erase the database with

       ```bash theme={"system"}
       docker compose down
       ```

       before issuing an `docker compose up -d`.

       The alternative is to connect to an external database. See [this section](/en/sdk/squid-sdk/reference/store/typeorm#database-connection-parameters) to learn how to specify the connection parameters.

    5. Regenerate a migration with
       ```bash theme={"system"}
       rm -r db/migrations
       ```
       ```bash theme={"system"}
       npx squid-typeorm-migration generate
       ```

    You can now use the async functions [`ctx.store.upsert()`](/en/sdk/squid-sdk/reference/store/typeorm#upsert) and [`ctx.store.insert()`](/en/sdk/squid-sdk/reference/store/typeorm#insert), as well as various [TypeORM lookup methods](/en/sdk/squid-sdk/reference/store/typeorm#typeorm-methods) to access the database.

    See the `typeorm-store` [guide](/en/sdk/squid-sdk/resources/persisting-data/typeorm) and [reference](/en/sdk/squid-sdk/reference/store/typeorm) for more info.
  </Tab>

  <Tab title="filesystem dataset">
    Filesystem dataset writing, as performed by the `@subsquid/file-store` package and its extensions, stores the data into one or more flat tables. The exact table definition format depends on the output file format.

    1. Decide on the file format you're going to use:

       * [Parquet](/en/sdk/squid-sdk/reference/store/file/parquet)
       * [CSV](/en/sdk/squid-sdk/reference/store/file/csv)
       * [JSON/JSONL](/en/sdk/squid-sdk/reference/store/file/json)

       If your template does not have any of the necessary packages, install them.

    2. Define any tables you need at the `tables` field of the `Database` constructor argument:
       ```ts theme={"system"}
       import { Database } from '@subsquid/file-store'

       const dbOptions = {
         tables: {
           FirstTable: new Table(/* ... */),
           SecondTable: new Table(/* ... */),
           // ...
         },
         // ...
       }

       processor.run(new Database(dbOptions), async ctx => { // ...
       ```

    3. Define the destination filesystem via the `dest` field of the `Database` constructor argument. Options:
       * local folder - use `LocalDest` from `@subsquid/file-store`
       * S3-compatible file storage service - install `@subsquid/file-store-s3` and use [`S3Dest`](/en/sdk/squid-sdk/reference/store/file/s3-dest)

    Once you're done you'll be able to enqueue data rows for saving using the `write()` and `writeMany()` methods of the context store-provided table objects:

    ```ts theme={"system"}
    ctx.store.FirstTable.writeMany(/* ... */)
    ctx.store.SecondTable.write(/* ... */)
    ```

    The store will write the files automatically as soon as the buffer reaches the size set by the `chunkSizeMb` field of the `Database` constructor argument, or at the end of the batch if a call to [`setForceFlush()`](/en/sdk/squid-sdk/resources/persisting-data/file#setforceflush) was made anywhere in the batch handler.

    See the `file-store` [guide](/en/sdk/squid-sdk/resources/persisting-data/file) and the [reference pages of its extensions](/en/sdk/squid-sdk/reference/store/file).
  </Tab>

  <Tab title="BigQuery">
    Follow the [guide](/en/sdk/squid-sdk/resources/persisting-data/bigquery).
  </Tab>
</Tabs>

### VI. Persist the transformed data to your data sink

Once your data is [decoded](#batch-handler-decoding), optionally [enriched with external data](#external-data) and transformed the way you need it to be, it is time to save it.

<Tabs>
  <Tab title="PostgreSQL+GraphQL">
    For each batch, create all the instances of all TypeORM model classes at once, then save them with the minimal number of calls to `upsert()` or `insert()`, e.g.:

    ```ts theme={"system"}
    import { EntityA, EntityB } from './model'

    processor.run(new TypeormDatabase(), async ctx => {
      const aEntities: Map<string, EntityA> = new Map() // id -> entity instance
      const bEntities: EntityB = []

      for (let block of ctx.blocks) {
        // fill the containets aEntities and bEntities
      }

      await ctx.store.upsert([...aEntities.values()])
      await ctx.store.insert(bEntities)
    })
    ```

    It will often make sense to keep the entity instances in maps rather than arrays to make it easier to reuse them when defining instances of other entities with [relations](/en/sdk/squid-sdk/reference/schema-file/entity-relations) to the previous ones. The process is described in more detail in the [step 2 of the BAYC tutorial](/en/sdk/squid-sdk/tutorials/bayc/step-two-deriving-owners-and-tokens).

    If you perform any [database lookups](/en/sdk/squid-sdk/reference/store/typeorm#typeorm-methods), try to do so in batches and make sure that the entity fields that you're searching over are [indexed](/en/sdk/squid-sdk/reference/schema-file/indexes-and-constraints).

    See also the [patterns](/en/sdk/squid-sdk/resources/batch-processing#patterns) and [anti-pattens](/en/sdk/squid-sdk/resources/batch-processing#anti-patterns) sections of the Batch processing guide.
  </Tab>

  <Tab title="filesystem dataset">
    You can enqueue the transformed data for writing whenever convenient without any sizeable impact on performance.

    At low output data rates (e.g. if your entire dataset is in tens of Mbytes or under) take care to call [`ctx.store.setForceFlush()`](/en/sdk/squid-sdk/resources/persisting-data/file#setforceflush) when appropriate to make sure your data actually gets written.
  </Tab>

  <Tab title="BigQuery">
    You can enqueue the transformed data for writing whenever convenient without any sizeable impact on performance. The actual data writing will happen automatically at the end of each batch.
  </Tab>
</Tabs>

## The top-down development cycle

The [bottom-up development cycle](#bottom-up-development) described above is convenient for inital squid development and for trying out new things, but it has the disadvantage of not having the means of saving the data ready at hand when initially writing the data decoding/transformation code. That makes it necessary to come back to that code later, which is somewhat inconvenient e.g. when adding new squid features incrementally.

The alternative is to do the same steps in a different order:

1. [Update the store](#store)
2. If necessary, [regenerate the utility classes](#typegen)
3. [Update the processor configuration](#processor-config)
4. [Decode and normalize the added data](#batch-handler-decoding)
5. [Retrieve any external data](#external-data) if necessary
6. [Add the persistence code for the transformed data](#batch-handler-persistence)

## GraphQL options

[Store your data to PostgreSQL](/en/sdk/squid-sdk/resources/persisting-data/typeorm), then consult [Serving GraphQL](/en/sdk/squid-sdk/resources/serving-graphql) for options.

## Scaling up

If you're developing a large squid, make sure to use [batch processing](/en/sdk/squid-sdk/resources/batch-processing) throughout your code.

A common mistake is to make handlers for individual event logs or transactions; for updates that require data retrieval that results in lots of small database lookups and ultimately in poor syncing performance. Collect all the relevant data and process it at once. A simple architecture of that type is discussed in the [BAYC tutorial](/en/sdk/squid-sdk/tutorials/bayc).

You should also check the [Cloud best practices page](/en/cloud/resources/best-practices) even if you're not planning to deploy to [SQD Cloud](/en/cloud) - it contains valuable performance-related tips.

Many issues commonly arising when developing larger squids are addressed by the third party [`@belopash/typeorm-store` package](https://github.com/belopash/squid-typeorm-store). Consider using it.

For complete examples of complex squids take a look at the [Giant Squid Explorer](https://github.com/subsquid-labs/giant-squid-explorer) and [Thena Squid](https://github.com/subsquid-labs/thena-squid) repos.

## Next steps

* Learn about [batch processing](/en/sdk/squid-sdk/resources/batch-processing).
* Learn how squid deal with [unfinalized blocks](/en/sdk/squid-sdk/resources/unfinalized-blocks).
* [Use external APIs and IPFS](/en/sdk/squid-sdk/resources/external-api) in your squid.
* See how squid should be set up for the [multichain setting](/en/sdk/squid-sdk/resources/multichain).
* Deploy your squid [on own infrastructure](/en/sdk/squid-sdk/resources/self-hosting) or to [SQD Cloud](/en/cloud).


# Indexer from scratch
Source: https://docs.sqd.dev/en/sdk/squid-sdk/how-to-start/squid-from-scratch

Build a Squid SDK indexer from scratch — learn how to compose evm-processor, typeorm-store, and graphql-server NPM packages without a starter template.

Here's an example of how SDK packages can be combined into a working indexer (called *squid*).

This page goes through all the technical details to make the squid architecture easier to understand. If you would like to get to a working indexer ASAP, [bootstrap from a template](/en/sdk/squid-sdk/how-to-start/squid-development#templates).

## USDT transfers API

<Info>
  **Pre-requisites**: NodeJS 20.x or newer, Docker.
</Info>

Suppose the task is to track transfers of USDT on Ethereum, then save the resulting data to PostgreSQL and serve it as a GraphQL API. From this description we can immediately put together a list of [packages](/en/sdk/squid-sdk/overview):

* `@subsquid/evm-processor` - for retrieving Ethereum data
* the triad of `@subsquid/typeorm-store`, `@subsquid/typeorm-codegen` and `@subsquid/typeorm-migration` - for saving data to PostgreSQL

We also assume the following choice of *optional* packages:

* `@subsquid/evm-typegen` - for decoding Ethereum data and useful constants such as event topic0 values
* `@subsquid/evm-abi` - as a peer dependency for the code generated by `@subsquid/evm-typegen`
* `@subsquid/graphql-server` / [OpenReader](/en/sdk/squid-sdk/reference/openreader-server)

To make the indexer, follow these steps:

<Steps>
  <Step title="Create a new folder and initialise a new project">
    * create `package.json`
      ```bash theme={"system"}
      npm init
      ```
    * add `.gitignore`
      ```bash title=".gitignore" theme={"system"}
      node_modules
      lib
      ```
  </Step>

  <Step title="Install the packages">
    ```bash theme={"system"}
    npm i dotenv typeorm @subsquid/evm-processor @subsquid/typeorm-store @subsquid/typeorm-migration @subsquid/graphql-server @subsquid/evm-abi
    ```

    ```bash theme={"system"}
    npm i typescript @subsquid/typeorm-codegen @subsquid/evm-typegen --save-dev
    ```
  </Step>

  <Step title="Add a minimal `tsconfig.json`">
    ```json title="tsconfig.json" theme={"system"}
    {
      "compilerOptions": {
        "rootDir": "src",
        "outDir": "lib",
        "module": "commonjs",
        "target": "es2020",
        "esModuleInterop": true,
        "skipLibCheck": true,
        "experimentalDecorators": true,
        "emitDecoratorMetadata": true
      }
    }
    ```
  </Step>

  <Step title="Define the schema for both the database and the core GraphQL API">
    Define the schema for both the database and the core GraphQL API in [`schema.graphql`](/en/sdk/squid-sdk/reference/schema-file):

    ```graphql title="schema.graphql" theme={"system"}
    type Transfer @entity {
      id: ID!
      from: String! @index
      to: String! @index
      value: BigInt!
    }
    ```
  </Step>

  <Step title="Generate TypeORM classes based on the schema">
    ```bash theme={"system"}
    npx squid-typeorm-codegen
    ```

    The TypeORM classes are now available at `src/model/index.ts`.
  </Step>

  <Step title="Prepare the database">
    * create `.env` and `docker-compose.yaml` files
      ```bash title=".env" theme={"system"}
      DB_NAME=squid
      DB_PORT=23798
      RPC_ETH_HTTP=https://rpc.ankr.com/eth
      ```
      ```yaml title="docker-compose.yaml" theme={"system"}
      services:
        db:
          image: postgres:15
          environment:
            POSTGRES_DB: "${DB_NAME}"
            POSTGRES_PASSWORD: postgres
          ports:
            - "${DB_PORT}:5432"
      ```
    * start the database container
      ```bash theme={"system"}
      docker compose up -d
      ```
    * compile the TypeORM classes
      ```bash theme={"system"}
      npx tsc
      ```
    * generate the migration file
      ```bash theme={"system"}
      npx squid-typeorm-migration generate
      ```
    * apply the migration with
      ```bash theme={"system"}
      npx squid-typeorm-migration apply
      ```
  </Step>

  <Step title="Generate utility classes for decoding USDT contract data">
    [Generate utility classes](/en/sdk/squid-sdk/resources/tools/typegen/generation) for decoding [USDT contract](https://etherscan.io/address/0xdac17f958d2ee523a2206206994597c13d831ec7) data based on its ABI.

    * Create an `./abi` folder:
      ```bash theme={"system"}
      mkdir abi
      ```

    * Find the ABI at the ["Contract" tab of the contract page on Etherscan](https://etherscan.io/address/0xdAC17F958D2ee523a2206206994597C13D831ec7#code). Scroll down a bit:

    * Copy the ABI, then paste to a new file at `./abi/usdt.json`.

    * Run the utility classes generator:
      ```bash theme={"system"}
      npx squid-evm-typegen src/abi ./abi/*
      ```

    The utility classes are now available at `src/abi/usdt.ts`
  </Step>

  <Step title="Tie all the generated code together with `src/main.ts`">
    Tie all the generated code together with a `src/main.ts` executable with the following code blocks:

    * Imports
      ```ts theme={"system"}
      import { EvmBatchProcessor } from '@subsquid/evm-processor'
      import { TypeormDatabase } from '@subsquid/typeorm-store'
      import * as usdtAbi from './abi/usdt'
      import { Transfer } from './model'
      ```
    * [`EvmBatchProcessor`](/en/sdk/squid-sdk/reference/processors/evm-batch) object definition
      ```ts theme={"system"}
      const processor = new EvmBatchProcessor()
        .setGateway('https://v2.archive.subsquid.io/network/ethereum-mainnet')
        .setRpcEndpoint({
          url: process.env.RPC_ETH_HTTP,
          rateLimit: 10
        })
        .setFinalityConfirmation(75) // 15 mins to finality
        .addLog({
          address: [ '0xdAC17F958D2ee523a2206206994597C13D831ec7' ],
          topic0: [ usdtAbi.events.Transfer.topic ]
        })
      ```
    * [`TypeormDatabase`](/en/sdk/squid-sdk/reference/store/typeorm) object definition
      ```ts theme={"system"}
      const db = new TypeormDatabase()
      ```
    * A call to [`processor.run()`](/en/sdk/squid-sdk/reference/processors/architecture#processorrun) with an inline definition of the [batch handler](/en/sdk/squid-sdk/reference/processors/architecture#batch-context)
      ```ts theme={"system"}
      processor.run(db, async ctx => {
        const transfers: Transfer[] = []
        for (let block of ctx.blocks) {
          for (let log of block.logs) {
            let {from, to, value} = usdtAbi.events.Transfer.decode(log)
            transfers.push(new Transfer({
              id: log.id,
              from, to, value
            }))
          }
        }
        await ctx.store.insert(transfers)
      })
      ```
      Note how supplying a `TypeormDatabase` to the function caused `ctx.store` to be a [PostgreSQL-compatible `Store` object](/en/sdk/squid-sdk/reference/store/typeorm#store-interface).
  </Step>

  <Step title="Compile the project and start the processor process">
    Compile the project and start the [processor process](/en/sdk/squid-sdk/overview#processor)

    ```bash theme={"system"}
    npx tsc
    ```

    ```bash theme={"system"}
    node -r dotenv/config lib/main.js
    ```
  </Step>

  <Step title="Start the GraphQL server">
    In a separate terminal, configure the GraphQL port and start the GraphQL server:

    ```diff title=".env" theme={"system"}
     DB_NAME=squid
     DB_PORT=23798
     RPC_ETH_HTTP=https://rpc.ankr.com/eth
    +GRAPHQL_SERVER_PORT=4350
    ```

    ```bash theme={"system"}
    npx squid-graphql-server
    ```
  </Step>
</Steps>

The finished GraphQL API with GraphiQL is available at [localhost:4350/graphql](http://localhost:4350/graphql).

Final code for this mini-tutorial is available in [this repo](https://github.com/subsquid-labs/squid-from-scratch).

<Tip>
  The commands listed here are often abbreviated as [custom `sqd` commands](/en/sdk/squid-sdk/squid-cli/commands-json) in squids. If you'd like to do that too you can use the [`commands.json` file of the EVM template](https://github.com/subsquid-labs/squid-evm-template/blob/main/commands.json) as a starter.
</Tip>


# Squid SDK Overview
Source: https://docs.sqd.dev/en/sdk/squid-sdk/overview

Squid SDK architecture overview — processor, batch handlers, schema, and Store roles in building TypeScript blockchain indexers with GraphQL APIs.

A *squid* is an indexing project built with [Squid SDK](https://github.com/subsquid/squid-sdk) to retrieve and process blockchain data from the [SQD Network](/en/network/overview)
(either permissioned or decentralized instance). The Squid SDK is a set of open source Typescript libraries that retrieve, decode, transform and persist the data. It can also make the transformed data available via an API. All stages of the indexing pipeline, from the data extraction to transformation to persistence are performed on [batches of blocks](/en/sdk/squid-sdk/resources/batch-processing) to maximize the indexing speed. Modular architecture of the SDK makes it possible to extend indexing projects (squids) with custom plugins and data targets.

## Required squid components

### Processor

*Processor* is the word used for

1. The main NodeJS process of the squid.
2. The main object (`processor`) of this process: its method call `processor.run()` is the entry point.

`processor` objects handle data retrieval and transformation; data persistence is handled by a separate object called [Store](#store). Squid SDK offers two processor classes:

* [`EvmBatchProcessor`](/en/sdk/squid-sdk/reference/processors/evm-batch) via the [`@subsquid/evm-processor`](https://www.npmjs.com/package/@subsquid/evm-processor) NPM package - for Ethereum-compatible [networks](/en/data/evm)
* [`SubstrateBatchProcessor`](/en/sdk/squid-sdk/reference/processors/substrate-batch) via [`@subsquid/substrate-processor`](https://www.npmjs.com/package/@subsquid/substrate-processor) - for [networks](/en/data/substrate) based on [Substrate](https://substrate.io) such as [Polkadot](https://polkadot.network)

### Store

A *store* is an object that processors use to persist their data. SQD offers three store classes:

* [`TypeormStore`](/en/sdk/squid-sdk/resources/persisting-data/typeorm) for saving data to PostgreSQL, via

  * [`@subsquid/typeorm-store`](https://www.npmjs.com/package/@subsquid/typeorm-store)
  * [`@subsquid/typeorm-codegen`](https://www.npmjs.com/package/@subsquid/typeorm-codegen) (a code generator, install with `--save-dev`)
  * [`@subsquid/typeorm-migration`](https://www.npmjs.com/package/@subsquid/typeorm-migration)

  Install all three packages to use this store.
* [`file-store`](/en/sdk/squid-sdk/resources/persisting-data/file) via [`@subsquid/file-store`](https://www.npmjs.com/package/@subsquid/file-store) - for saving data to filesystems. It is a modular system with a variety of [extensions](/en/sdk/squid-sdk/reference/store/file) for various formats and destinations.
* [`bigquery-store`](/en/sdk/squid-sdk/resources/persisting-data/bigquery) via [`@subsquid/bigquery-store`](https://www.npmjs.com/package/@subsquid/bigquery-store) - for saving data to [Google BigQuery](https://cloud.google.com/bigquery).

You can mix and match any store class with any processor class.

## Optional squid components

### Typegen

A *typegen* is a tool for generating utility code for technology-specific operations such as decoding. Here are the typegens available:

<Tabs>
  <Tab title="On EVM">
    * [`squid-evm-typegen`](/en/sdk/squid-sdk/resources/tools/typegen/generation) via [`@subsquid/evm-typegen`](https://www.npmjs.com/package/@subsquid/evm-typegen):
      * decodes smart contract data
      * handles direct calls to contract methods
      * exposes useful constants such as event topics and function signature hashes
        The generated code depends on [`@subsquid/evm-abi`](https://www.npmjs.com/package/@subsquid/evm-abi), SQD's own high performance, open source EVM codec.
  </Tab>

  <Tab title="On Substrate">
    * [`squid-substrate-typegen`](/en/sdk/squid-sdk/resources/tools/typegen/generation) via [`@subsquid/substrate-typegen`](https://www.npmjs.com/package/@subsquid/substrate-typegen):
      * general purpose pallet data decoding (aware of runtime versions)
      * handles direct storage queries
    * [`squid-ink-typegen`](/en/sdk/squid-sdk/resources/tools/typegen/generation) via [`@subsquid/ink-typegen`](https://www.npmjs.com/package/@subsquid/ink-typegen) for decoding the data of [ink!](https://use.ink) contracts
  </Tab>
</Tabs>

Install these with `--save-dev`.

### GraphQL server

Squids that store their data in PostgreSQL can subsequently make it available as a GraphQL API via a variety of supported servers. See [Serving GraphQL](/en/sdk/squid-sdk/resources/serving-graphql).

Among other alternatives, SQD provides its own server called [OpenReader](/en/sdk/squid-sdk/reference/openreader-server) via the [`@subsquid/graphql-server`](https://www.npmjs.com/package/@subsquid/graphql-server) package. The server runs as a separate process. [Core API](/en/sdk/squid-sdk/reference/openreader-server/api) is automatically derived from the schema file; it is possible to extend it with [custom queries](/en/sdk/squid-sdk/reference/openreader-server/configuration/custom-resolvers) and [basic access control](/en/sdk/squid-sdk/reference/openreader-server/configuration/authorization).

### Misc utilities

<Tabs>
  <Tab title="On EVM">
    * [Squid CLI](/en/sdk/squid-sdk/squid-cli/installation) is a utility for [retrieving squid templates](/en/sdk/squid-sdk/squid-cli/init), managing [chains of commands](https://github.com/subsquid/squid-sdk/tree/master/util/commands) commonly used in development and running [all squid processes at once](/en/sdk/squid-sdk/squid-cli/run). It can also be used for deploying to [SQD Cloud](/en/cloud).
  </Tab>

  <Tab title="On Substrate">
    * [Squid CLI](/en/sdk/squid-sdk/squid-cli/installation) is a utility for [retrieving squid templates](/en/sdk/squid-sdk/squid-cli/init), managing [chains of commands](https://github.com/subsquid/squid-sdk/tree/master/util/commands) commonly used in development and running [all squid processes at once](/en/sdk/squid-sdk/squid-cli/run). It can also be used for deploying to [SQD Cloud](/en/cloud).
    * [`@subsquid/ss58`](https://www.npmjs.com/package/@subsquid/ss58) handles encoding and decoding of SS58 addresses
    * [`@subsquid/frontier`](https://www.npmjs.com/package/@subsquid/frontier) decodes events and calls of the [Frontier EVM pallet](https://paritytech.github.io/frontier/frame/evm.html) to make them decodable with [`squid-evm-typegen`](/en/sdk/squid-sdk/resources/tools/typegen/generation)
  </Tab>
</Tabs>


# Squid SDK Quickstart
Source: https://docs.sqd.dev/en/sdk/squid-sdk/quickstart

5-minute Squid SDK quickstart — build a TypeScript indexer that streams USDC transfers from Ethereum into local Postgres for GraphQL queries.

This is a 5 min quickstart on how to build an indexer using Squid SDK.

The indexer (squid) will:

* Fetch all historical USDC transfers on Ethereum from the SQD Network
* Decode it
* Save it to a local Postgres database
* Start a GraphQL server with a rich API to query the historical USDC transfers.

## Prerequisites

* (On Windows) [WSL](https://learn.microsoft.com/en-us/windows/wsl/install)
* Node.js v18+
* Git
* Docker (for running Postgres)

<Steps>
  <Step title="Step 1">
    Install Squid CLI:

    ```bash theme={"system"}
    npm i -g @subsquid/cli
    ```

    Squid CLI is a multi-purpose utility tool for scaffolding and managing the indexers, both locally and in SQD Cloud.
  </Step>

  <Step title="Step 2">
    Scaffold the indexer project (squid) from an example repo using Squid CLI:

    ```bash theme={"system"}
    sqd init hello-squid -t https://github.com/subsquid-labs/showcase01-all-usdc-transfers
    cd hello-squid
    ```

    The example is just a public GitHub repo with a squid project pre-configured to index USDC.
  </Step>

  <Step title="Step 3">
    Inspect the `./src` folder:

    <Tree>
      <Tree.Folder name="src">
        <Tree.Folder name="abi">
          <Tree.File name="usdc.ts" />
        </Tree.Folder>

        <Tree.File name="main.ts" />

        <Tree.Folder name="model">
          <Tree.Folder name="generated">
            <Tree.File name="index.ts" />

            <Tree.File name="marshal.ts" />

            <Tree.File name="usdcTransfer.model.ts" />
          </Tree.Folder>

          <Tree.File name="index.ts" />
        </Tree.Folder>
      </Tree.Folder>
    </Tree>

    Here,

    * `src/abi/usdc` is a utility module generated from the JSON ABI of the USDC contract. It contains methods for event decoding, direct RPC queries and some useful constants.
    * `src/model` contains TypeORM model classes autogenerated from `schema.graphql`. Squids use them to populate Postgres.

    `main.ts` is the main executable. In this example, it also contains all the data retrieval configuration:

    ```ts theme={"system"}
    const processor = new EvmBatchProcessor()
      // SQD Network gateways are the primary source of blockchain data in
      // squids, providing pre-filtered data in chunks of roughly 1-10k blocks.
      // Set this for a fast sync.
      .setGateway('https://v2.archive.subsquid.io/network/ethereum-mainnet')
      // Another data source squid processors can use is chain RPC.
      // In this particular squid it is used to retrieve the very latest chain data
      // (including unfinalized blocks) in real time. It can also be used to
      //   - make direct RPC queries to get extra data during indexing
      //   - sync a squid without a gateway (slow)
      .setRpcEndpoint('https://rpc.ankr.com/eth')
      // The processor needs to know how many newest blocks it should mark as "hot".
      // If it detects a blockchain fork, it will roll back any changes to the
      // database made due to orphaned blocks, then re-run the processing for the
      // main chain blocks.
      .setFinalityConfirmation(75)
      // .addXXX() methods request data items. In this case we're asking for
      // Transfer(address,address,uint256) event logs emitted by the USDC contract.
      //
      // We could have omitted the "address" filter to get Transfer events from
      // all contracts, or the "topic0" filter to get all events from the USDC
      // contract, or both to get all event logs chainwide. We also could have
      // requested some related data, such as the parent transaction or its traces.
      //
      // Other .addXXX() methods (.addTransaction(), .addTrace(), .addStateDiff()
      // on EVM) are similarly feature-rich.
      .addLog({
        range: { from: 6_082_465 },
        address: [USDC_CONTRACT_ADDRESS],
        topic0: [usdcAbi.events.Transfer.topic],
      })
      // .setFields() is for choosing data fields for the selected data items.
      // Here we're requesting hashes of parent transaction for all event logs.
      .setFields({
        log: {
          transactionHash: true,
        },
      })
    ```

    The rest of the file is about data processing and storage:

    ```ts theme={"system"}
    // TypeormDatabase objects store the data to Postgres. They are capable of
    // handling the rollbacks that occur due to blockchain forks.
    //
    // There are also Database classes for storing data to files and BigQuery
    // datasets.
    const db = new TypeormDatabase({supportHotBlocks: true})

    // The processor.run() call executes the data processing. Its second argument is
    // the handler function that is executed once on each batch of data. Processor
    // object provides the data via "ctx.blocks". However, the handler can contain
    // arbitrary TypeScript code, so it's OK to bring in extra data from IPFS,
    // direct RPC calls, external APIs etc.
    processor.run(db, async (ctx) => {
      // Making the container to hold that which will become the rows of the
      // usdc_transfer database table while processing the batch. We'll insert them
      // all at once at the end, massively saving IO bandwidth.
      const transfers: UsdcTransfer[] = []

      // The data retrieved from the SQD Network gatewat and/or the RPC endpoint
      // is supplied via ctx.blocks
      for (let block of ctx.blocks) {
        // On EVM, each block has four iterables - logs, transactions, traces,
        // stateDiffs
        for (let log of block.logs) {
          if (log.address === USDC_CONTRACT_ADDRESS &&
              log.topics[0] === usdcAbi.events.Transfer.topic) {
            // SQD's very own EVM codec at work - about 20 times faster than ethers
            let {from, to, value} = usdcAbi.events.Transfer.decode(log)
            transfers.push(new UsdcTransfer({
              id: log.id,
              block: block.header.height,
              from,
              to,
              value,
              txnHash: log.transactionHash
            }))
          }
        }
      }

      // Just one insert per batch!
      await ctx.store.insert(transfers)
    })
    ```
  </Step>

  <Step title="Step 4">
    Install the dependencies and build

    ```bash theme={"system"}
    npm i
    npm run build
    ```
  </Step>

  <Step title="Step 5">
    The processor is a background process that continously fetches the data, decodes it and stores it in a local Postgres. All the logic is defined in `main.ts` and is fully customizable.

    To run the processor, we first start a local Postgres where the decoded data is persisted (the template comes with a Docker compose file):

    ```bash theme={"system"}
    docker compose up -d
    ```

    Processor will connect to Postgres using the connection parameters from `.env`.

    Apply database migrations with

    ```bash theme={"system"}
    npx squid-typeorm-migration apply
    ```

    then start the processor with

    ```bash theme={"system"}
    node -r dotenv/config lib/main.js
    ```

    The indexer is now running.
  </Step>

  <Step title="Step 6">
    Start the GraphQL API serving the transfers data from Postgres:

    ```bash theme={"system"}
    npx squid-graphql-server
    ```

    The server comes with a GraphQL playground available at [`localhost:4350/graphql`](http://localhost:4350/graphql).
  </Step>

  <Step title="Step 7">
    Query the data!
  </Step>
</Steps>


# The frontier package
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/frontier

Index EVMs running on Substrate (Frontier) with Squid SDK — use the @subsquid/frontier package to decode Frontier-EVM events and contract calls.

# `@subsquid/frontier`

The way the Frontier EVM pallet exposes EVM logs and transaction may change due to runtime upgrades. [`@subsquid/frontier`](https://github.com/subsquid/squid-sdk/tree/master/substrate/frontier) provides helper methods that are aware of the upgrades:

#### `getEvmLog(event: Event): EvmLog`

Extract the EVM log data from `EVM.Log` event.

#### `getTransaction(call: Call): LegacyTransaction | EIP2930Transaction | EIP1559Transaction`

Extract the transaction data from `Ethereum.transact` call with additional fields depending on the EVM transaction type.

#### `getTransactionResult(ethereumExecuted: Event): {from: string, to: string, transactionHash: string, status: 'Succeed' | 'Error' | 'Revert' | 'Fatal', statusReason: string}`

Extract transaction result from an `Ethereum.Executed` event.

See also the [Frontier EVM guide](/en/sdk/squid-sdk/resources/substrate/frontier-evm).

#### Example

```typescript theme={"system"}
const processor = new SubstrateBatchProcessor()
  .setGateway('https://v2.archive.subsquid.io/network/astar-substrate')
  .setRpcEndpoint('https://astar-rpc.dwellir.com')
  .addEthereumTransaction({})
  .addEvmLog({})

processor.run(new TypeormDatabase(), async ctx => {
  for (const block of ctx.blocks) {
    for (const event of block.events) {
      if (event.name === 'EVM.Log') {
        // no need to supply any extra data to determine
        // the runtime version: event has all necessary references
        const {address, data, topics} = getEvmLog(event)

        // process evm log data
      }
    }
    for (const call of block.calls) {
      if (call.name==='Ethereum.transact') {
        const txn = getTransaction(call)
        // process evm txn data
      }   
    }
  }
})
```


# Squid SDK Logger
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/logger

Native logger module of Squid SDK — pino-based structured logging with log levels, JSON output, and integration with batch processor lifecycle.

A [`Logger`](https://github.com/subsquid/squid/tree/master/util/logger) interface is injected into the [handler context](/en/sdk/squid-sdk/reference/processors/architecture#batch-context) with `ctx.log`. It is bound to the namespace `sqd:processor:mapping`. The context logger is a recommended way of logging for squid processors.

`Logger` exposes the following logging levels, in order of increasing severity:

* `TRACE`
* `DEBUG`
* `INFO`
* `WARN`
* `ERROR`
* `FATAL`

By default, the logging level is set to `INFO`.

And here is an example:

```typescript theme={"system"}
processor.run(new TypeormDatabase(), async (ctx) => {
  ctx.log.trace("Trace Log example");
  ctx.log.debug("Debug Log example");
  ctx.log.info("Info Log example");
  ctx.log.warn("Warn Log example");
  ctx.log.error("Error Log example");
  ctx.log.fatal("Fatal Log example")
});
```

## Overriding the log level

The log level can be overridden by setting a matching namespace selector to one of the `SQD_TRACE`, ..., `SQD_FATAL` env variables. In particular, to set the handler logs level to `DEBUG` set the environment variable `SQD_DEBUG` to `sqd:processor:mapping`:

```bash theme={"system"}
SQD_DEBUG=sqd:processor:mapping
```

The namespace selector supports wildcards, so one can also enable internal debug logs of `@subsquid/substrate-processor` with

```
SQD_DEBUG=sqd:processor*
```

since all processor context loggers inherit the processor-level namespace `sqd:processor`.

## Accessing logs of a deployed Squid

Processor logs can be inspected once the squid is deployed to Cloud:

```bash theme={"system"}
sqd logs -n <name> -s <slot> -f --level=<level>
```

or

```bash theme={"system"}
sqd logs -n <name> -t <tag> -f --level=<level>
```

<Accordion title="For older version-based deployments...">
  ...the slot string is `v${version}`, so use

  ```bash theme={"system"}
  sqd logs -n <name> -s v<version> -f --level=<level>
  ```

  Check out the [Slots and tags guide](/en/cloud/resources/slots-and-tags) to learn more.
</Accordion>

The available levels are:

* `info`
* `warning`
* `debug`
* `error` - will fetch messages emitted by
  * `ctx.log.error`
  * `ctx.log.fatal`
  * `ctx.log.trace`

See [CLI Reference](/en/sdk/squid-sdk/squid-cli/logs) or `sqd logs --help` for a full list of log options supported by SQD Cloud.


# DoS protection
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/openreader-server/configuration/dos-protection

Protect your Squid SDK GraphQL API from denial-of-service queries — depth, complexity, and rate-limit configuration in OpenReader.

The squid [GraphQL API server](https://github.com/subsquid/squid-sdk/tree/master/graphql/graphql-server) accepts the following optional start arguments to fend off heavy queries.

To enable the protection, add the corresponding flags to `serve` and `serve:prod` command definitions at [`commands.json`](/en/sdk/squid-sdk/squid-cli/commands-json):

```json theme={"system"}
...
  "serve": {
    "description": "Start the GraphQL API server",
    "cmd": ["squid-graphql-server", "--max-root-fields", "10", "--max-response-size", "1000"]
  },
  "serve:prod": {
    "description": "Start the GraphQL API server in prod",
    "cmd": ["squid-graphql-server", "--max-root-fields", "10", "--max-response-size", "1000", "--dumb-cache", "in-memory"]
  },
...
```

When deploying to [SQD Cloud](/en/cloud), make sure to use `serve:prod` to start the GraphQL `api` service in the [deployment manifest](/en/cloud/reference/manifest#deploy):

```yaml title="squid.yaml" theme={"system"}
# ...
deploy:
  # other services ...
  api:
    cmd: [ "sqd", "serve:prod" ]
```

**`--max-request-size <kb>`**

The argument limits the size of a request in kilobytes. It is set to `256kb` by default.

**`--max-root-fields <count>`**

The maximal allowed number of root-level queries in a single GraphQL request.

**`--max-response-size <nodes>`**

This option limits the *estimated* query response size and makes server return an error if it exceeds the provided value. Note that the estimated size depends only on the decorators in `schema.graphql` and the requested fields.

The estimate is the product of the cardinality of the entity list and the response item weight.

The cardinality is estimated as the minimum of

* the `limit` argument of the query (`Infinity` if not provided)
* `@cardinality` value defined in `schema.graphql` (if the requested entity type is decorated in the schema file, `Infinity` otherwise)
* the size of the argument list of the `_eq` and `id_in` filters in the `where` clause (if applicable)

In particular, if there are no `@cardinality` decorators in `schema.graphql`, the client queries must explicitly provide limits or where filters to pass through.

The response item weight is calculated recursively:

* `byteWeight` for each scalar field or `1` if it's not decorated
* for non-scalar fields, the estimated weight times the estimated cardinality (if it's a list)
* each non-leaf node in the query AST tree adds a weight of `1`

In a nutshell, assuming that the schema file is properly decorated with `@cardinality` and `@byteWeight`, the estimated response size should roughly be at the same scale as the byte size of the query result.

**`--subscription-max-response-size <nodes>`**

Same as `--max-response-size` but for live query [subscriptions](/en/sdk/squid-sdk/reference/openreader-server/configuration/subscriptions).

#### Example

Assume the schema is defined as follows, and the server is launched with `--max-response-size 1000`.

```ts title="schema.graphql" theme={"system"}
type Foo @entity {
    id: ID!
    // a large string 
    bigField: String! @byteWeight(value: 1000.0)
    bar: Bar!
}

type Bar @entity {
    id: ID!
    // bar.foos typically contain about 100 items
    foos: [Foo!]! @derivedFrom(field: "bar") @cardinality(value: 100)
    bazs: [Baz!]! @derivedFrom(field: "bar")
}

// there are around 100 entities of type Baz
type Baz @entity @cardinality(value: 100) {
    id: ID!
    bar: Bar!
}
```

The following queries will be bounced:

```graphql theme={"system"}
query A {
    bars {
        id
    }
}

query B {
    bars(limit: 1001) {
        id
    }
}

query C {
    bars(limit: 100) {
        id 
        foos(limit: 10) {
            id
        }
    }
}

query D {
    bars(limit: 10) {
        id 
        foos {
            id
        }
    }
}

query E {
    foos(id_eq: "1") {
        id
        bigField
    }
}
```

* The estimated cardinality of query A is `Infinity`
* The estimated cardinality of query B `1001` and so the expected size exceeds the limit
* The estimated cardinality of query C is `100` while the item size is `13`, so the size is estimated to `1300`.
* The estimated cardinality of query D is `10` while the item size is `103`, so the size is estimated to `1030`.
* The estimated cardinality of query E is `1` while the item size is `1001` (due to `bigField` having weight `1000`).

At the same time, the following queries will go through:

```graphql theme={"system"}
query A {
    bars(limit: 100) {
        id
    }
}

query B {
    bars(limit: 3) {
        id
        foos {
            id
        }
        bazs {
            id
        }
    }
}

query C {
    bars {
        id
        bar {
            foos(where: { id_in ["1", "2" ]}) {
                id
            }
            id
        }
    }
}
```


# Subscriptions
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/openreader-server/configuration/subscriptions

Push real-time entity updates over a websocket in OpenReader subscriptions — Squid SDK GraphQL subscriptions for live API consumers.

<Warning>
  RAM usage of subscriptions scales poorly under high load, making the feature unsuitable for most production uses. There are currently no plans to fix this issue.
</Warning>

OpenReader supports [GraphQL subscriptions](https://www.apollographql.com/docs/react/data/subscriptions/) via live queries. To use these, a client opens a websocket connection to the server and sends a `subscription` query there. The query body is then repeatedly executed (every 5 seconds by default) and the results are sent to the client whenever they change.

To enable subscriptions, add the additional `--subscriptions` flag to the `squid-graphql-server` startup command. The poll interval is configured with the `--subscription-poll-interval` flag. For details and a full list of available options, run

```bash theme={"system"}
npx squid-graphql-server --help
```

For each entity types, the following queries are supported for subscriptions:

* `${EntityName}ById` -- query a single entity
* `${EntityName}s` -- query multiple entities with a `where` filter
  Note that despite being [deprecated](/en/sdk/squid-sdk/reference/openreader-server/overview#supported-queries) from the regular query set, `${EntityName}s` queries will continue to be available for subscriptions going forward.

## Local runs

To enable subscriptions, add the `--subscriptions` flag to `serve` and `serve:prod` commands at [`commands.json`](/en/sdk/squid-sdk/squid-cli/commands-json):

```json title="commands.json" theme={"system"}
...
  "serve": {
    "description": "Start the GraphQL API server",
    "cmd": ["squid-graphql-server", "--subscriptions"]
  },
  "serve:prod": {
    "description": "Start the GraphQL API server in prod",
    "cmd": ["squid-graphql-server", "--subscriptions", "--dumb-cache", "in-memory"]
  },
...
```

A `ws` endpoint will be available the usual `localhost:<GQL_PORT>/graphql` URL.

## SQD Cloud deployments

For SQD Cloud deployments, make sure to use the updated `serve:prod` command in the [deployment manifest](/en/cloud/reference/manifest#deploy):

```yaml title="squid.yaml" theme={"system"}
# ...
deploy:
  # other services ...
  api:
    cmd: [ "sqd", "serve:prod" ]
```

The subscription `wss` endpoint will be available at the canonical API endpoint `wss://{org}.subsquid.io/{name}/v/v{version}/graphql`.

## Example

Let's take the following simple schema with account and transfer entities:

```graphql file=schema.graphql theme={"system"}
type Account @entity {
  "Account address"
  id: ID!
  transfersTo: [Transfer!] @derivedFrom(field: "to")
  transfersFrom: [Transfer!] @derivedFrom(field: "from")
}

type Transfer @entity {
  id: ID!
  timestamp: DateTime! @index
  from: Account!
  to: Account!
  amount: BigInt! @index
}
```

After modifying `commands.json` start the GraphQL server with subscriptions with

```bash theme={"system"}
npx squid-graphql-server
```

The following sample [script](https://github.com/subsquid/squid-substrate-template/blob/main/scripts/sub-client.js) will subscribe to the most recent transfers (by `timestamp`).

```typescript theme={"system"}
const WebSocket = require('ws')
const { createClient } = require('graphql-ws');

const port = process.env.GQL_PORT || 4350
const host = process.env.GQL_HOST || 'localhost'
const proto = process.env.GQL_PROTO || 'ws'


const client = createClient({
  webSocketImpl: WebSocket,
  url: `${proto}://${host}:${port}/graphql`,
});

client.subscribe(
  {
    query: `
    subscription {
        transfers(limit: 5, orderBy: timestamp_DESC) {
            amount
            blockNumber
            from {
              id
            }
            to {
              id
            }
        }
    }  
    `,
  },
  {
    next: (data) => {
      console.log(`New transfers: ${JSON.stringify(data)}`);
    },
    error: (error) => {
      console.error('error', error);
    },
    complete: () => {
      console.log('done!');
    },
  }
);
```


# Block data for Substrate
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/processors/substrate-batch/context-interfaces

Block data interfaces for Substrate indexing with Squid SDK — type definitions for blocks, extrinsics, events, and calls in the processor context.

`SubstrateBatchProcessor` follows the common [squid processor architecture](/en/sdk/squid-sdk/overview), in which data processing happens within the [batch handler](/en/sdk/squid-sdk/reference/processors/architecture#processorrun), a function repeatedly called on batches of on-chain data. The function takes a single argument called "batch context". Its structure follows the [common batch context layout](/en/sdk/squid-sdk/reference/processors/architecture#batch-context), with `ctx.blocks` being an array of `Block` objects containing the data to be processed, aligned at the block level.

For `SubstrateBatchProcessor` the `Block` interface is defined as follows:

```ts theme={"system"}
export type Block<F extends FieldSelection = {}> = {
  header: BlockHeader<F>
  extrinsics: Extrinsic<F>[]
  calls: Call<F>[]
  events: Event<F>[]
}
```

`F` here is the type of the argument of the [`setFields()`](/en/sdk/squid-sdk/reference/processors/substrate-batch/field-selection) processor method.

`Block.header` contains the block header data. The rest of the fields are iterables containing four kinds of blockchain data. The canonical ordering within each iterable is the same as it is within the blocks.

The exact fields available in each data item type are inferred from the `setFields()` call argument. They are documented on the [Field selection](/en/sdk/squid-sdk/reference/processors/substrate-batch/field-selection) page:

* [extrinsics section](/en/sdk/squid-sdk/reference/processors/substrate-batch/field-selection#extrinsics);
* [calls section](/en/sdk/squid-sdk/reference/processors/substrate-batch/field-selection#calls);
* [events section](/en/sdk/squid-sdk/reference/processors/substrate-batch/field-selection#events);
* [block header section](/en/sdk/squid-sdk/reference/processors/substrate-batch/field-selection#block-headers).

## Example

The handler below simply outputs all the `Balances.transfer_all` calls on Kusama in [real time](/en/sdk/squid-sdk/resources/unfinalized-blocks):

```ts theme={"system"}
import {SubstrateBatchProcessor} from '@subsquid/substrate-processor'
import {TypeormDatabase} from '@subsquid/typeorm-store'

const processor = new SubstrateBatchProcessor()
  .setGateway('https://v2.archive.subsquid.io/network/kusama')
  .setRpcEndpoint('https://kusama-rpc.polkadot.io')
  .setBlockRange({from: 19_600_000})
  .addCall({
    name: ['Balances.transfer_all'],
  })
  .setFields({
    call: {
      origin: true,
      success: true 
    }
  }) 

processor.run(new TypeormDatabase(), async ctx => {
  for (let block of ctx.blocks) {
    for (let call of block.calls) {
      ctx.log.info(call, `Call:`)
    }
  }
})
```

One can experiment with the [`setFields()`](/en/sdk/squid-sdk/reference/processors/substrate-batch/field-selection) argument and see how the output changes.


# Data requests
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests

Subscribe to Substrate events and calls with addEvent and addCall in Squid SDK — filter by pallet, name, and arguments to drive your processor.

<Warning>
  Setting a data request field to `[]` *selects no data*. Pass `undefined` for a wildcard selection:

  ```typescript theme={"system"}
  .addEvent({name: []}) // selects no events
  .addEvent({}) // selects all events
  ```
</Warning>

<Tip>
  All array-valued data request fields (e.g. `name?: string[]` of `addEvent()` are *arrays* of acceptable values. That means that you can request e.g. multiple events with a single `addEvent()` call. Requesting data that way is more efficient than requesting data with many individual calls.
</Tip>

## Events

**`addEvent(options)`**: Subscribe to Substrate runtime events. `options` has the following structure:

```ts theme={"system"}
{
  // data requests
  name?: string[]
  range?: {from: number, to?: number}

  // related data retrieval
  call?: boolean
  stack?: boolean
  extrinsic?: boolean
}
```

`name`s must follow the convention `${Pallet}.${NameWithinPallet}` with both parts usually capitalized, e.g. `Balances.Transfer`.

You may also request the data related to the events:

* with `call = true` the processor will retrieve the parent call and add it to the `calls` iterable within the [block data](/en/sdk/squid-sdk/reference/processors/substrate-batch/context-interfaces);
* with `stack = true` it will do that with all calls in the entire call stack;
* with `extrinsic = true` it will add the parent extrinsic to the `extrinsics` block data iterable.

Note than events can also be requested by the `addCall()` method as related data.

Selection of the exact data to be retrieved for each log and its optional parent transaction is done with the `setFields()` method [documented](/en/sdk/squid-sdk/reference/processors/substrate-batch/field-selection#events) on the Fields selection page.

## Calls

**`addCall(options)`**: Subscribe to runtime calls (even if wrapped into a `system.sudo` or `util.batch` extrinsic). `options` has the following structure:

```ts theme={"system"}
{   
  // data requests
  name?: string[]
  range?: {from: number, to?: number}

  // related data retrieval
  events?: boolean
  stack?: boolean
  extrinsic?: boolean
}
```

The name must follow the convention `${Pallet}.${call_name}`. The pallet name is normally capitalized, and the call name is in the snake\_case format, as in `Balances.transfer_keep_alive`.

By default, both successful and failed calls are fetched. Select the `call.success` field and later check it within the batch handler if you need to disambiguate.

You may also request the data related to the calls:

* with `events = true` the processor will retrieve all the events that the call emitted and add them to the `events` iterable within the [block data](/en/sdk/squid-sdk/reference/processors/substrate-batch/context-interfaces);
* with `stack = true` it will add all calls in the stack of each matching call, including itself, to the `calls` iterable;
* with `extrinsic = true` it will add the parent extrinsic to the `extrinsics` block data iterable.

Note than calls can also be requested by the `addEvent()` method as related data.

Selection of the exact data to be retrieved for each log and its optional parent transaction is done with the `setFields()` method [documented](/en/sdk/squid-sdk/reference/processors/substrate-batch/field-selection#calls) on the Fields selection page.

## Specialized setters

#### **`addEvmLog(options)`**

A `SubstrateBatchProcessor` configuration setter that subscribes it to `EVM.Log` events by contract address(es) and/or EVM log topics. `options` have the following structure:

```ts theme={"system"}
{
  // data requests
  address?: string[]
  topic0?: string[]
  topic1?: string[]
  topic2?: string[]
  topic3?: string[]
  range?: {from: number, to?: number}

  // related data retrieval
  call?: boolean
  stack?: boolean
  extrinsic?: true
} 
```

Related data retrieval and field selection are identical to [`addEvent()`](#events).

#### **`addEthereumTransaction(options)`**

A `SubstrateBatchProcessor` configuration setter that subscribes it to `Ethereum.transact` calls by contract address(es) and/or function [sighashes](https://www.4byte.directory). `options` have the following structure:

```ts theme={"system"}
{
  // data requests
  to?: string[] // contract addresses
  sighash?: string[]
  range?: {from: number, to?: number}

  // related data retrieval
  events: boolean
  stack: boolean
  extrinsic: boolean
}
```

Related data retrieval and field selection are identical to that [`addCall()`](#calls).

The processor with fetch both successful and failed transactions. Further, there's a difference between the success of a Substrate call and the internal EVM transaction. The transaction may fail even if the enclosing Substrate call has succeeded. Use `events = true` to retrieve `Ethereum.Executed` events that can be used to figure out the EVM transaction status (see [`getTransactionResult()`](/en/sdk/squid-sdk/reference/frontier#get-transaction-result)).

#### **`addContractsContractEmitted(options)`**

Subscribe to the ink! events (`Contracts.ContractEmitted`) of the WASM runtime. `options` has the following structure:

```ts theme={"system"}
{
  // data requests
  contractAddress?: string[]
  range?: {from: number, to?: number}

  // related data retrieval
  call?: boolean
  stack?: boolean
  extrinsic?: boolean
}
```

Contract addresses must be specified as hex strings, so make sure to decode them if yours are encoded with ss58.

Related data retrieval and field selection are identical to [`addEvent()`](#events).

#### **`addGearMessageQueued(options)`**

#### **`addGearUserMessageSent(options)`**

Structure of `options` is identical for both methods:

```ts theme={"system"}
{
  // data requests
  programId?: string[]
  range?: {from: number, to?: number}

  // related data retrieval
  call?: boolean
  stack?: boolean
  extrinsic?: boolean
}
```

The methods above subscribe to the events [`Gear.MessageQueued`](https://wiki.gear-tech.io/docs/api/events/#messagequeued) and [`Gear.UserMessageSent`](https://wiki.gear-tech.io/docs/api/events/#usermessagesent) emitted by the specified Gear program.

Related data retrieval and field selection are identical to [`addEvent()`](#events).


# Substrate Field Selection
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/processors/substrate-batch/field-selection

Fine-tune Substrate data requests with setFields() in Squid SDK — choose only the block, extrinsic, event, and call fields you need to index efficiently.

#### `setFields(options)`

Set the fields to be retrieved for data items of each supported type. The `options` object has the following structure:

```ts theme={"system"}
{
  event?:     // field selector for logs
  call?:      // field selector for transactions
  extrinsic?: // field selector for state diffs
  block?:     // field selector for block headers
}
```

Every field selector is a collection of boolean fields mapping one-to-one to the fields of data items within the batch context [iterables](/en/sdk/squid-sdk/reference/processors/substrate-batch/context-interfaces). Setting a field of a field selector of a given type to true, like this

```ts theme={"system"}
let processor = new SubstrateBatchProcessor()
  .setFields({
    call: {
      origin: true
    }
  })
```

will cause the processor to populate the corresponding field (in this case `origin`) of all data items of that type (in this case `Call`). The field will be available in all data items of the type, including nested items. Suppose we used the processor defined above to subscribe to some calls as well as some events, and for each event we requested a parent call:

```ts theme={"system"}
processor
  .addEvent({
    // some event data requests
    call: true
  })
  .addCall({
    // some call data requests
  })
```

As a result, the `origin` field would be available both within the call items retrieved due to `.addCall()` and within the call items that provide parent call information for the events:

```ts theme={"system"}
processor.run(db, async ctx => {
  for (let block of ctx.blocks) {
    for (let call of block.calls) {
      let callOrigin = call.origin // OK
    }
    for (let event of block.events) {
      let parentCallOrigin = event.call?.origin // also OK!
    }
  }
})
```

Some data fields are enabled by default but can be disabled by setting a field of a field selector to `false`. For example, this code will not compile:

```ts theme={"system"}
let processor = new SubstrateBatchProcessor()
  .setFields({
    call: {
      name: false
    }
  })
  .addCall({
    // some call data requests
  })

processor.run(db, async ctx => {
  for (let block of ctx.blocks) {
    for (let call of block.calls) {
      let callName = call.name // ERROR: no such field
    }
  }
})
```

Disabling unused fields will improve sync performance, as the disabled fields will not be fetched from SQD Network.

## Data item types and field selectors

<Tip>
  Most IDEs support smart suggestions to show the possible field selectors. For VS Code, press `Ctrl+Space`:
</Tip>

Here we describe the data item types as functions of the field selectors. In `SubstrateBatchProcessor` each field of a field selector maps to the eponymous field of its corresponding data item type. Item data fields are divided into three categories:

* Fields that are added independently of the `setFields()` call. These are either fixed or depend on the related data retrieval flags (e.g. `call` for events).
* Fields that can be disabled by `setFields()`. E.g. the `args` field will be fetched for calls by default, but can be disabled by setting `args: false` within the `call` field selector.
* Fields that can be requested by `setFields()`. E.g. the `tip` field will only be available in extrinsics if the `extrinsic` field selector sets `tip: true`.

In addition to data fields, data items have methods that simplify access to related data, e.g. `Call.getExtrinsic()`.

### Events

`Event` data items may have the following fields:

```ts theme={"system"}
Event {
  // can be requested with field selectors
  phase: 'ApplyExtrinsic' | 'Initialization' | 'Finalization'

  // can be disabled with field selectors
  name: string
  args: any

  // independent of field selectors
  id: string
  index: number
  block: BlockHeader
  callAddress?: number[]
  call?: Call
  extrinsicIndex?: number
  extrinsic?: Extrinsic

  // methods
  getCall(): Call
  getExtrinsic(): Extrinsic
}
```

Definition of the types mentioned are available in their respective sections:

* [`BlockHeader`](#block-headers),
* [`Call`](#calls),
* [`Extrinsic`](#extrinsics).

### Calls

`Call` data items may have the following fields:

```ts theme={"system"}
Call {
  // can be requested with field selectors
  error?: any
  origin?: any
  success: boolean
  
  // can be disabled with field selectors
  name: string
  args: any

  // independent of field selectors
  id: string
  address: number[]
  block: BlockHeader
  events: Event[]
  parentCall?: Call
  subcalls: Call[]
  extrinsicIndex: number
  extrinsic?: Extrinsic

  // methods
  getParentCall(): Call
  getExtrinsic(): Extrinsic
}
```

Definition of the types mentioned are available in their respective sections:

* [`BlockHeader`](#block-headers),
* [`Event`](#events),
* [`Extrinsic`](#extrinsics).

### Extrinsics

`Extrinsic` data items may have the following fields:

```ts theme={"system"}
Extrinsic {
  // can be requested with field selectors
  success: boolean
  error?: unknown
  fee?: bigint
  hash: string
  signature?: {
    // no per-field selection here:
    // "signature: true" gets all fields
    address: unknown
    signature: unknown
    signedExtensions: unknown
  }
  tip?: bigint
  version: number
  
  // independent of field selectors
  id: string
  index: number
  block: BlockHeader
  events: Event[]
  call?: Call
  subcalls: Call[]

  // methods
  getCall(): Call
}
```

Definition of mentioned types are available in their respective sections:

* [`BlockHeader`](#block-headers),
* [`Event`](#events),
* [`Call`](#calls),

### Block headers

`BlockHeader` data items may have the following fields:

```ts theme={"system"}
BlockHeader{
  // can be requested with field selectors
  digest: { // request with "digest: true"
    logs: string[]
  }
  extrinsicsRoot: string
  stateRoot: string
  timestamp?: number
  validator?: string

  // independent of field selectors
  id: string
  hash: string
  height: number
  implName: string
  implVersion: number
  parentHash: string
  specName: string
  specVersion: number
  _runtime: Runtime
  _runtimeOfPrevBlock: Runtime
  
  // methods
  getParent(): {
    _runtime: Runtime
    hash: string
    height: number
  }
}
```

`Runtime` is an internal type that tools like [squid-substrate-typegen](/en/sdk/squid-sdk/resources/tools/typegen) use for dealing with Substrate runtime versions. The curious may take a look at the definition [here](https://github.com/subsquid/squid-sdk/blob/master/substrate/substrate-runtime/src/runtime/runtime.ts).

## A complete example

Fetch `Balances.Transfer` events and `Balances.transfer_keep_alive` calls along with their parent extrinsics. Enrich the extrinsics with hashes, fees and success flags:

```ts theme={"system"}
const processor = new SubstrateBatchProcessor()
  .addEvent({
    name: ['Balances.Transfer'],
    extrinsic: true
  })
  .addCall({
    name: ['Balances.transfer_keep_alive'],
    extrinsic: true
  })
  .setFields({
    extrinsic: {
      hash: true,
      fee: true,
      success: true
    }
  })
```


# Substrate Batch General Settings
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/processors/substrate-batch/general

General settings for Squid SDK SubstrateBatchProcessor — configure data source, types bundle, batch size, finality, metrics, and runtime options.

<Tip>
  The method documentation is also available inline and can be accessed via suggestions in most IDEs.
</Tip>

The following setters configure the global settings of `SubstrateBatchProcessor`. They return the modified instance and can be chained.

Calling [`setRpcEndpoint()`](#set-rpc-endpoint) is a hard requirement on Substrate, as chain RPC is used to retrieve chain metadata. Adding a [SQD Network gateway](/en/data/substrate) with [`setGateway()`](#set-gateway) is optional but highly recommended, as it greatly reduces RPC usage.

To reduce it further, you can explicitly disable [RPC ingestion](/en/sdk/squid-sdk/resources/unfinalized-blocks) by calling [`setRpcDataIngestionSettings({ disabled: true })`](#set-rpc-data-ingestion-settings): in this scenario the RPC will only be used for metadata retrieval and to perform any [direct RPC queries](/en/sdk/squid-sdk/resources/tools/typegen/state-queries) you might be doing in your squid code. This will, however, introduce a delay of a few thousands of blocks between the chain head and the highest block available to your squid.

### `setGateway(url: string | GatewaySettings)`

Adds a [SQD Network](/en/network) data source. The argument is either a string URL of a SQD Network gateway or

```ts theme={"system"}
{
  url: string // gateway URL
  requestTimeout?: number // in milliseconds
}
```

See [Substrate gateways](/en/data/substrate).

### `setRpcEndpoint(rpc: ChainRpc)`

Adds a RPC data source. If added, it will be used for

* [RPC ingestion](/en/sdk/squid-sdk/resources/unfinalized-blocks) (unless explicitly disabled with [`setRpcDataIngestionSettings()`](#set-rpc-data-ingestion-settings))
* any [direct RPC queries](/en/sdk/squid-sdk/resources/tools/typegen/state-queries) you make in your squid code

A node RPC endpoint can be specified as a string URL or as an object:

```ts theme={"system"}
type ChainRpc = string | {
  url: string // http, https, ws and wss are supported
  capacity?: number // num of concurrent connections, default 10
  maxBatchCallSize?: number // default 100
  rateLimit?: number // requests per second, default is no limit
  requestTimeout?: number // in milliseconds, default 30_000
  headers: Record<string, string> // http headers
}
```

Setting `maxBatchCallSize` to `1` disables batching completely.

<Tip>
  We recommend using private endpoints for better performance and stability of your squids. For SQD Cloud deployments you can use the [RPC addon](/en/cloud/resources/rpc-proxy). If you use an external private RPC, keep the endpoint URL in a [Cloud secret](/en/cloud/resources/env-variables#secrets).
</Tip>

### `setDataSource(ds: {archive?: string, chain?: ChainRpc})` (deprecated)

Replaced by [`setGateway()`](#set-gateway) and [`setRpcEndpoint()`](#set-rpc-endpoint).

### `setRpcDataIngestionSetting(settings: RpcDataIngestionSettings)`

Specify the [RPC ingestion](/en/sdk/squid-sdk/resources/unfinalized-blocks) settings.

```ts theme={"system"}
type RpcDataIngestionSettings = {
  disabled?: boolean
  headPollInterval?: number
  newHeadTimeout?: number
}
```

Here,

* `disabled`: Explicitly disables data ingestion from an RPC endpoint. **RPC endpoint is still required on Substrate because `SubstrateBatchProcessor` relies on it for metadata.** The only effect of this setting is to have the processor stop once it reaches the max SQD Network dataset height.
* `headPollInterval`: Poll interval for new blocks in milliseconds. Poll mechanism is used to get new blocks via HTTP connections. Default: 5000.
* `newHeadTimeout`: When ingesting from a websocket, this setting specifies the timeout in milliseconds after which the connection will be reset and subscription re-initiated if no new blocks were received. Default: no timeout.

### `setBlockRange({from: number, to?: number})`

Limits the range of blocks to be processed. When the upper bound is specified, processor will terminate with exit code 0 once it reaches it.

Note that block ranges can also be specified separately for each data request. This method sets global bounds for all block ranges in the configuration.

### `includeAllBlocks(range?: {from: number, to?: number})`

By default, processor will fetch only blocks which contain requested items. This method modifies such behavior to fetch all chain blocks. Optionally a range of blocks can be specified for which the setting should be effective.

### `setTypesBundle(bundle: string | OldTypesBundle | OldSpecsBundle | PolkadotjsTypesBundle)`

Sets a [types bundle](https://substrate.stackexchange.com/a/1231/4655).

Types bundle is only required for historical blocks which have metadata version below 14 and only if we don't have built-in support for the chain in question. Most chains listed in the [polkadot.js app](https://polkadot.js.org/apps/#/explorer) are supported.

SQD project has its own types bundle format, however, most of polkadotjs types bundles will work as well.

Types bundles can be specified in 2 different ways:

1. as a name of a JSON file:
   ```ts theme={"system"}
   processor.setTypesBundle('typesBundle.json')
   ```
2. as an [`OldTypesBundle`/`OldSpecsBundle`](https://github.com/subsquid/squid-sdk/blob/master/substrate/substrate-runtime/src/metadata/old/types.ts) or [`PolkadotjsTypesBundle`](https://github.com/subsquid/squid-sdk/blob/master/substrate/substrate-runtime/src/metadata/old/types.ts) object:
   ```ts theme={"system"}
   // OldTypesBundle object
   processor.setTypesBundle({
     types: {
       Foo: 'u8'
     }
   })
   ```

There a [mini-guide](/en/sdk/squid-sdk/resources/substrate/types-bundle-miniguide) on how to obtain type bundles for Substrate chains without relying on SQD tools.

### `setPrometheusPort(port: string | number)`

Sets the port for a built-in prometheus health metrics server (serving at `http://localhost:${port}/metrics`). By default, the value of PROMETHEUS\_PORT environment variable is used. When it is not set, processor will pick an ephemeral port.


# Batch processing
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/batch-processing

Batch-based data transformation model in Squid SDK — why processors handle ranges of blocks at once for higher indexing throughput than per-block models.

Batch data processing model employed by Squid SDK relies on the following principles:

* Minimize the number of database hits by grouping multiple single-row transactions into multi-row batch transactions.
* Transform the data in memory using vectorized operators.
* Use the [MakerDAO `Multicall` contract](/en/sdk/squid-sdk/resources/tools/typegen/state-queries#batch-state-queries) to batch EVM state queries.
* Use [`XXX.getMany()`](/en/sdk/squid-sdk/resources/tools/typegen/state-queries) to batch Substrate state queries.

In practice, batching is a more flexible (compared to handler parallelization) way to speed up the inherently sequential indexing of on-chain transactions and logs.

To illustrate, assume the processor must infer the current state of an on-chain record. It is configured to listen to the two on-chain events, `Create` and `Update`, that are emitted once the record is created or updated. The [data batch](/en/sdk/squid-sdk/reference/processors/architecture#ctxblocks) received by the processor is then an array of event items, i.e.

```ts theme={"system"}
[
    Create({id: 1, name: 'Alice'}), 
    Update({id: 1, name: 'Bob'}),
    Create({id: 2, name: 'Mikee'}), 
    Update({id: 1, name: 'Carol'}), 
    Update({id: 2, name: 'Mike'})
]
```

Following the principles above, a processor would update the intermediary entity states in memory, persisting only the final state:

```ts theme={"system"}
[{id: 1, name: 'Carol'}, {id: 2, name: 'Mike'}]
```

in a single transaction.

Let's see the batch processing principles in action.

## Patterns

An idiomatic use of [`processor.run()`](/en/sdk/squid-sdk/reference/processors/architecture#processorrun) is as follows:

```ts theme={"system"}
processor.run(new TypeormDatabase(), async (ctx) => {
    // a decoded and normalized ctx.blocks data
    // (just the logs in this example)
    const logDataBatch = []

    for (const block of ctx.blocks) {
      for (const log of c.logs) {
        // transform and normalize the raw logs data
        // based on the onchain data
        logDataBatch.push(decodeAndTransformToMyData(log))
      }
    }

    // the set of my target entity IDs to be updated/created
    const myEntityIds = new Set()
    for (const d of logDataBatch) {
        // the business logic mapping 
        // the on-chain data with the target
        // entity ID
        myEntityIds.add(extractEntityId(d))
    }

    // load the enities by the list of IDs and 
    // put into an ID map
    const myEntities: Map<string, MyEntity> = new Map(
       // batch-load using IN operator
      (await ctx.store.findBy(MyEntity, { id: In([...myEntityIds]) }))
        // put the result into the ID map
        .map((entity) => [entity.id, entity])
    );

    // calculate the updated state of the entities
    for (const d of logDataBatch) {
        const myEntity = myEntities.get(extractEntityId(d))
        if (myEntity == null) {
            // create a new instance with d and
            // add to myEntities map
        } else {
            // update myEntity using d
        }
    }

    // batch-update all entities in the map
    await ctx.store.save([...myEntities.values()])
});
```

For a full implementation of the above pattern, see [EVM squid example](https://github.com/subsquid-labs/evm-logs-example/tree/master/src) or [Substrate squid example](https://github.com/subsquid/squid-substrate-examples/tree/master/1-events).

## Anti-patterns

Avoid loading or persisting single entities unless strictly necessary. For example, here is a possible antipattern for the [Gravatar example](https://github.com/subsquid/gravatar-squid):

```ts theme={"system"}
processor.run(new TypeormDatabase(), async (ctx) => {
  for (const c of ctx.blocks) {
    for (const log of c.logs) {
      // making sure that we process only the relevant logs
      if (log.address !== GRAVATAR_CONTRACT ||
          (log.topics[0] !== events.NewGravatar.topic &&
           log.topics[0] !== events.UpdatedGravatar.topic)) continue
      const { id, owner, displayName, imageUrl } = extractData(log)
      // ANTIPATTERN!!!
      // Doing an upsert per event drastically decreases the indexing speed
      await ctx.store.save(Gravatar, new Gravatar({
        id: id.toHexString(),
        owner: decodeHex(owner),
        displayName,
        imageUrl
      }))
    }
  }
})
```

Instead, use an in-memory cache, and batch upserts:

```ts theme={"system"}
processor.run(new TypeormDatabase(), async (ctx) => {
  const gravatars: Map<string, Gravatar> = new Map();
  for (const c of ctx.blocks) {
    for (const log of c.logs) {
      if (log.address !== GRAVATAR_CONTRACT ||
          (log.topics[0] !== events.NewGravatar.topic &&
           log.topics[0] !== events.UpdatedGravatar.topic)) continue
      const { id, owner, displayName, imageUrl } = extractData(log)
      gravatars.set(id.toHexString(), new Gravatar({
        id: id.toHexString(),
        owner: decodeHex(owner),
        displayName,
        imageUrl
      }))
    }
  }
  await ctx.store.save([...gravatars.values()])
})
```

## Migrate from handlers

Batch-based processing can be used as a drop-in replacement for the handler-based mappings employed by e.g. subgraphs. While the handler-based processing is significantly slower due to excessive database lookups and writes, it may be a good intermediary step while [migrating an existing subgraph to Squid SDK](/en/sdk/squid-sdk/resources/migrate/migrate-subgraph).

One can simply re-use the existing handlers while looping over the `ctx` items:

```ts theme={"system"}
processor.run(new TypeormDatabase(), async (ctx) => {
  for (const c of ctx.blocks) {
    for (const log of c.logs) {
      switch (log.topics[0]) {
        case abi.events.FooEvent.topic:
          await handleFooEvent(ctx, log)
          continue
        case abi.events.BarEvent.topic:
          await handleFooEvent(ctx, log)
          continue
        default:
          continue
      }
    }
    for (const txn of c.transactions) {
      // 0x + 4 bytes
      const sighash = txn.input.slice(0, 10)
      switch (sighash) {
        case '0xa9059cbb': // transfer(address,uint256) sighash
          await handleTransferTx(ctx, txn)
          continue
        case abi.functions.approve.sighash:
          await handleApproveTx(ctx, txn)
          continue
        default:
          continue
      }
    }
  }
})
```

## Block hooks

Similarly, one can implement pre- and post- block hooks:

```ts theme={"system"}
processor.run(new TypeormDatabase(), async (ctx) => {
  for (const c of ctx.blocks) {
    await preBlockHook(ctx, c)
    for (const log of c.logs) {
      // some logic
    }
    for (const txn of c.transactions) {
      // some more logic
    }
    await postBlockHook(ctx, c)
  }
})
```


# Factory contracts
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/evm/factory-contracts

Index a dynamic set of EVM contracts deployed by a factory — Squid SDK guide to tracking new instances and decoding events from each created contract.

In some cases the set of contracts to be indexed by the squid is not known in advance. For example, a DEX contract typically creates a new contract for each trading pair added, and each such trading contract is of interest.

While the set of handler subscriptions is static and defined at the processor creation, one can leverage wildcard subscriptions and filter for contracts of interest at runtime.

Let's consider how it works in a DEX example, with a contract emitting `PoolCreated` log when a new pool contract is created by the main contract. Full code is available in the [examples repo](https://github.com/subsquid-labs/factory-example).

```typescript title="src/processor.ts" theme={"system"}
export const processor = new EvmBatchProcessor()
  .setGateway('https://v2.archive.subsquid.io/network/ethereum-mainnet')
  .setRpcEndpoint('<my_eth_rpc_url>')
  .setBlockRange({
    from: 12_369_621,
  })
  .setFields({
    log: {
      topics: true,
      data: true,
    },
    transaction: {
      hash: true,
    },
  })
  .addLog({
    address: [FACTORY_ADDRESS],
    topic0: [factoryAbi.events.PoolCreated.topic],
  })
  .addLog({
    topic0: [poolAbi.events.Swap.topic],
    transaction: true,
  })
```

```typescript title="src/main.ts" theme={"system"}
let factoryPools: Set<string>

processor.run(new TypeormDatabase(), async (ctx) => {
  if (!factoryPools) {
    factoryPools = await ctx.store.findBy(Pool, {}).then((q) => new Set(q.map((i) => i.id)))
  }

  let pools: PoolData[] = []
  let swaps: SwapEvent[] = []

  for (let block of ctx.blocks) {
    for (let log of block.logs) {
      if (log.address === FACTORY_ADDRESS) {
        pools.push(getPoolData(ctx, log))
      } else if (factoryPools.has(log.address)) {
        swaps.push(getSwap(ctx, log))
      }
    }
  }

  await createPools(ctx, pools)
  await processSwaps(ctx, swaps)
})
```

## Two-pass indexing for factory contracts

Squids built with the pattern shown above get the job done, but retrieve a lot of data that ends up discarded in the process. Complete elimination of this overhead would require dynamically changing the processor configuration, which is not currently possible. However, the configuration can be changed at a fixed block and that can be used to eliminate most of the overhead, drastically reducing the sync time.

The technique has a couple of limitations:

* The number of newly deployed contracts should be moderate (roughly up to tens of thousands). If your factory contract deploys contracts by millions (e.g. Pancakeswap), then vanilla factory pattern will be faster.
* You will need to periodically perform an extra action to keep the syncing overhead of your squid to a minimum.

The idea is to retrieve the list of the contracts that the factory deploys up to a certain block *before the main sync starts*. Then all data of interest up to that block can be requested only for these contracts. Once that data is retrieved, the contract can switch back to retrieving the data chain-wide and filtering it in processor. The example above can be changed to:

```typescript title="src/processor.ts" theme={"system"}
const { preloadHeight, preloadedPools } = loadPools() // e.g. from a filesystem

export const processor = new EvmBatchProcessor()
  .setGateway('https://v2.archive.subsquid.io/network/ethereum-mainnet')
  .setRpcEndpoint('<eth_rpc_endpoint_url>')
  .setBlockRange({
    from: 12_369_621,
  })
  .setFields({
    log: {
      topics: true,
      data: true,
    },
    transaction: {
      hash: true,
    },
  })
  .addLog({
    address: [FACTORY_ADDRESS],
    topic0: [factoryAbi.events.PoolCreated.topic],
  })
  .addLog({
    range: {
      from: 12_369_621,
      to: preloadHeight,
    },
    address: preloadedPools,
    topic0: [poolAbi.events.Swap.topic],
    transaction: true,
  })
  .addLog({
    range: {
      from: preloadHeight + 1,
    },
    topic0: [poolAbi.events.Swap.topic],
    transaction: true,
  })
```

The list of deployments can be preloaded with a small auxiliary squid and stored e.g. in `./assets`. This squid should be re-ran every time the number of blocks for which the whole network data is retrieved (that is, `preloadedHeight+1` to current head) becomes unacceptably large.

This approach is implemented in the [squid indexing the thena.fi decentralized exchange](https://github.com/subsquid-labs/thena-squid).


# Proxy contracts
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/evm/proxy-contracts

Index EVM proxy contracts with Squid SDK — strategies for handling EIP-1967 transparent proxies and beacon proxies that delegate to implementation contracts.

# Indexing proxy contracts

*Proxy contracts* are smart contracts that use the [DELEGATECALL](https://www.ethervm.io/#F4) EVM instruction to forward calls to some of their methods to another contract (referred to as the *implementation contract*). This setup allows changing the code that runs on method calls without redeploying the contract and losing its [state](https://coinsbench.com/solidity-layout-and-access-of-storage-variables-simply-explained-1ce964d7c738). Proxy contracts are standardized in [ERC-1967](https://eips.ethereum.org/EIPS/eip-1967).

The easiest way to know if the contract is a proxy or not is to visit the "Contract" tab of its Etherscan page. Proxy contracts will typically have the "Read as Proxy" and "Write as Proxy" buttons available. Here is how it looks for the [USDC contract](https://etherscan.io/address/0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48):

Clicking on the "Read as Proxy" button reveals the address of the implementation contract, [FiatTokenV2\_1](https://etherscan.io/address/0xa2327a938febf5fec13bacfb16ae10ecbc4cbdcf) in this case.

A few scenarios are possible when indexing a proxy contract:

1. **Events and calls that need to be indexed are described in the implementation contract**. This is by far the most common use case. If that is what you need, simply use the implementation ABI to interface with the proxy contract. In our USDC example you would need to retrieve the ABI of the FiatTokenV2\_1 contract, e.g. by running
   ```bash theme={"system"}
   # 0xa232...bDCF is the implementation contract address
   npx squid-evm-typegen src/abi 0xa2327a938Febf5FEC13baCFb16Ae10EcBc4cbDCF#fiatToken
   ```
   That retrieves the ABI from Etherscan API and uses it to create [TypeScript wrapper classes](/en/sdk/squid-sdk/resources/tools/typegen/state-queries) for implementation functions and events at './src/abi/fiatToken.ts\`. Use these to subscribe to and decode the data of the proxy contract:
   ```ts title="./src/processor.ts" theme={"system"}
   import * as fiatToken from './abi/fiatToken'
   export const processor = new EvmBatchProcessor()
     .addLog({
       // 0xA0b8...eB48 is the proxy contract address
       address: ['0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48'],
       topic0: [fiatToken.events.Mint.topic],
     })
   ```
   ```ts title="./src/main.ts" theme={"system"}
     // ...
     let {minter, to, amount} = fiatToken.events.Mint.decode(log)
     // ...
   ```
   Complete example is available [here](https://github.com/subsquid-labs/evm-logs-example) (uses `Transfer`s instead of `Mint`s).

2. **Events and calls that need to be indexed are described in the proxy contract itself**. This typically occurs in indexers that track contract upgrades. In this case simply use the ABI of the proxy contract to both request and decode the data:
   ```bash theme={"system"}
   npx squid-evm-typegen src/abi 0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48#usdcProxy
   ```
   ```ts title="./src/processor.ts" theme={"system"}
   import * as usdcProxy from './abi/usdcProxy'
   export const processor = new EvmBatchProcessor()
     .addLog({
       address: ['0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48'],
       topic0: [usdcProxy.events.Upgraded.topic],
     })
   ```
   ```ts title="./src/main.ts" theme={"system"}
     // ...
     let {implementation} = usdcProxy.events.Upgraded.decode(log)
     // ...
   ```

3. **Events and call described in both contracts are needed**. If that is your use case, retrieve ABIs of both the proxy and the implementation and use both:
   ```ts title="./src/processor.ts" theme={"system"}
   import * as fiatToken from './abi/fiatToken'
   import * as usdcProxy from './abi/usdcProxy'
   export const processor = new EvmBatchProcessor()
     .addLog({
       address: ['0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48'],
       topic0: [
         fiatToken.events.Mint.topic,
         usdcProxy.events.Upgraded.topic,
       ],
     })
   ```


# External APIs and IPFS
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/external-api

Enrich blockchain data with external APIs and IPFS in your Squid SDK indexer — best practices for caching, retries, and avoiding rate limits.

Squid processor is a usual node.js process, so one can fetch data from external APIs or IPFS gateways within the `processor.run()` method. In squids deployed to [SQD Cloud](/en/cloud) one normally uses API calls in combination with API keys set via [secrets](/en/cloud/resources/env-variables).

### Example

For example, one can enrich the indexed transaction with historical price data using the Coingecko API.

```ts theme={"system"}
// ...
processor.run(new TypeormDatabase(), async (ctx) => {
  const burns: Burn[] = []
  for (let c of ctx.blocks) {
    for (let txn of c.transactions) {
      burns.push(new Burn({
        id: formatID(c.header.height, txn.hash),
        block: c.header.height,
        address: txn.from,
        value: txn.value,
        txHash: txn.hash,
        price: await getETHPriceByDate(c.header.timestamp)
      }))
    }
  }
})

async function getETHPriceByDate(timestamp: number): Promise<bigint> {
  const formatted = moment(new Date(timestamp).toISOString()).format("DD-MM-yyyy")
  const res = await axios.get(
    `https://api.coingecko.com/api/v3/coins/ethereum/history?date=${formatted}&localization=false`
  )
  return res.data.market_data.current_price.usd
}
```

## IPFS fetching

For reliable indexing of content stored on IPFS (e.g. NFT metadata) we recommend fetching from dedicated IPFS gateways, e.g. provided by [Filebase](https://filebase.com/dedicated-gateways/).

For a more elaborate example of with IPFS gateway and external API calls, inspect the [Step 3 of the BAYC tutorial](/en/sdk/squid-sdk/tutorials/bayc/step-three-adding-external-data).

### Example

```typescript theme={"system"}
// FIXME: replace with a dedicated gateway
export const BASE_URL = 'https://cloudflare-ipfs.com/ipfs/'

export const api = Axios.create({
  baseURL: BASE_URL,
  headers: {
    'Content-Type': 'application/json',
  },
  withCredentials: false,
  timeout: 5000,
  httpsAgent: new https.Agent({ keepAlive: true }),
})

export const fetchMetadata = async (
  ctx: DataHandlerHandlerContext<Store, typeof fieldSelection>,
  cid: string
): Promise<any | null> => {
  try {
    const { status, data } = await api.get(`${BASE_URL}/${cid}`)
    ctx.log.info(`[IPFS] ${status} CID: ${cid}`)
    if (status < 400) {
      return data
    }
  } catch (e) {
    ctx.log.warn(
      `[IPFS] ERROR CID: ${cid} TRY ${(e as Error).message}`
    )
  }
  return null
}

processor.run(new TypeormDatabase(), async (ctx) => {
  for (let c of ctx.blocks) {
    for (let log of c.logs) {
      // track and decode NFT events to get CID
      // use fetchMetadata() to fetch metadata
    }
  }
})
```


# Migrate from The Graph
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/migrate/migrate-subgraph

Migrate a subgraph from The Graph to a Squid SDK indexer — port mappings, schema, manifest, and unit tests for an SQD-powered drop-in replacement.

This guide walks through the steps to migrate a subgraph to SQD. In what follows we will convert the [Gravatar](https://github.com/graphprotocol/example-subgraph) subgraph into a squid and run it locally. Impatient readers may clone the squid from the [repo](https://github.com/subsquid-labs/gravatar-squid) and run it by following the instructions in README:

```bash theme={"system"}
git clone https://github.com/subsquid-labs/gravatar-squid.git
```

`EvmBatchProcessor` provided by the Squid SDK defines a single handler that indexes EVM logs and transaction data in batches. It differs from the programming model of subgraph mappings that defines a separate data handler for each EVM log topic to be indexed. Due to significantly less frequent database hits (once per batch compared to once per log) the batch-based handling model shows up to a 10x increase in the indexing speed.

At the same time, concepts of the [schema file](/en/sdk/squid-sdk/reference/schema-file), [code generation from the schema file](/en/sdk/squid-sdk/reference/schema-file/intro#typeorm-codegen) and [auto-generated GraphQL API](/en/sdk/squid-sdk/resources/serving-graphql) should be familiar to subgraph developers. In most cases the schema file of a subgraph can be imported into a squid as is.

There are some known limitations:

* Many-to-Many entity relations should be [modeled explicitly](/en/sdk/squid-sdk/reference/schema-file/entity-relations#many-to-many-relations) as two many-to-one relations

On top of the features provided by subgraphs, Squid SDK and SQD Cloud offer extra flexibility in developing tailored APIs and ETLs for on-chain data:

* Full control over the target database (Postgres), including custom migrations and ad-hoc queries in the handler
* Custom target databases and data formats (e.g. CSV)
* Arbitrary code execution in the data handler
* [Extension of the GraphQL API](/en/sdk/squid-sdk/reference/openreader-server/configuration/custom-resolvers) with arbitrary SQL
* [Secret environment variables](/en/cloud/resources/env-variables), allowing to seamlessly use private third-party JSON-RPC endpoints and integrate with external APIs
* [Deployment slots and tags](/en/cloud/resources/slots-and-tags)
* [API caching](/en/sdk/squid-sdk/reference/openreader-server/configuration/caching)

For a full feature set comparison, see [SQD vs The Graph](/en/sdk/squid-sdk/subsquid-vs-thegraph).

## Squid setup

### 1. Install Squid CLI

Instructions [here](/en/sdk/squid-sdk/squid-cli/installation).

### 2. Fetch the template

Create a squid from the minimalistic [`evm` template](https://github.com/subsquid-labs/squid-evm-template):

```bash theme={"system"}
sqd init my-new-squid -t evm
cd squid-evm-template
```

### 3. Copy the schema file and generate entities

The minimal template already contains a dummy `schema.graphql` file. We replace it with the subgraph schema as is:

```gql file=schema.graphl theme={"system"}
type Gravatar @entity {
  id: ID!
  owner: Bytes!
  displayName: String!
  imageUrl: String!
}
```

Next, we generate the entities from the schema using the [`squid-typeorm-codegen`](/en/sdk/squid-sdk/reference/schema-file/intro#typeorm-codegen) tool of the Squid SDK, then build the squid:

```bash theme={"system"}
npx squid-typeorm-codegen
npm run build
```

This command is equivalent to running `yarn codegen` in subgraph.

After that, start the local database and regenerate migrations based on the generated entities using the [`squid-typeorm-migration`](/en/sdk/squid-sdk/resources/persisting-data/typeorm) tool:

```bash theme={"system"}
docker compose up -d
```

```bash theme={"system"}
rm -r db/migrations
```

```bash theme={"system"}
npx squid-typeorm-migration generate
```

A database migration file for creating a table for `Gravatar` will appear in `db/migrations`. Apply it with

```bash theme={"system"}
npx squid-typeorm-migration apply
```

### 4. Generate typings from ABI

Copy `./abis/Gravity.json` from the subgraph project and paste it to `./abi` folder in the squid project.
To generate the typings, run:

```bash theme={"system"}
npx squid-evm-typegen ./src/abi ./abi/*.json --multicall
```

Alternatively, similar to `graph add <address> [<subgraph-manifest default: "./subgraph.yaml">]` command, to generate typings, run:

```bash theme={"system"}
npx squid-evm-typegen src/abi 0x2E645469f354BB4F5c8a05B3b30A929361cf77eC#Gravity --clean
```

This command runs the `evm-typegen` tool that fetches the contract ABI by the address and generates type-safe access classes in `src/abi/Gravity.ts`. The generated boilerplate code will be used to decode EVM logs and directly query the contract. It also contains topic definitions used in the next step.

### 5. Subscribe to EVM logs

While in The Graph data source is defined in the manifest file `subgraph.yaml`, in SQD subscriptions to EVM data, including logs, are performed at the processor object definition customarily located at `src/processor.ts`. The processor is configured directly by the code, unlike subgraphs which require handlers and events to be defined in the manifest file.

```ts file=src/processor.ts theme={"system"}
import { EvmBatchProcessor} from '@subsquid/evm-processor'

// generated by the evm-typegen tool
// the events object contains typings for all events defined in the ABI
import { events } from './abi/Gravity'

export const GRAVATAR_CONTRACT =
  '0x2E645469f354BB4F5c8a05B3b30A929361cf77eC'.toLowerCase()

export const processor = new EvmBatchProcessor()
  // change the gateway URL to run against other EVM networks, e.g.
  // 'https://v2.archive.subsquid.io/network/polygon-mainnet'
  // 'https://v2.archive.subsquid.io/network/binance-mainnet'
  .setGateway('https://v2.archive.subsquid.io/network/ethereum-mainnet')
  .setRpcEndpoint('<my_eth_rpc_url>')
  .setBlockRange({ from: 6175243 })
  .setFinalityConfirmation(75)
  .addLog({
    address: [ GRAVATAR_CONTRACT ],
    topic0: [
      events.NewGravatar.topic,
      events.UpdatedGravatar.topic,
    ],
  })
```

In the snippet above we tell the squid processor to fetch logs emitted by the contract `0x2E645469f354BB4F5c8a05B3b30A929361cf77eC` with topic0 within a specified list. The configuration also states that indexing should start from block `6175243`, the height at which the contract was deployed.

Check out the [EVM indexing](/en/sdk/squid-sdk) section for the list of supported networks and configuration details.

The above snippet is eqivalent to the following `subgraph.yaml`:

```yaml file=subgraph.yaml theme={"system"}
specVersion: 0.0.4
description: Gravatar for Ethereum
repository: https://github.com/graphprotocol/example-subgraph
schema:
  file: ./schema.graphql
dataSources:
  - kind: ethereum/contract
    name: Gravity
    network: mainnet
    source:
      address: '0x2E645469f354BB4F5c8a05B3b30A929361cf77eC'
      abi: Gravity
    mapping:
      kind: ethereum/events
      apiVersion: 0.0.5
      language: wasm/assemblyscript
      entities:
        - Gravatar
      abis:
        - name: Gravity
          file: ./abis/Gravity.json
      eventHandlers:
        - event: NewGravatar(uint256,address,string,string)
          handler: handleNewGravatar
        - event: UpdatedGravatar(uint256,address,string,string)
          handler: handleUpdatedGravatar
      file: ./src/mapping.ts
```

### 6. Transform and save the data

In Subgraph data is saved in the `mapping.ts` file. The mapping function will receive an `ethereum.Block` as its only argument. In SQD, we set up the processor in `processor.ts` and save the data in the `main.ts`.

We migrate the subgraph handlers that transform the event data into `Gravatar` objects. Instead of saving or updating gravatars one by one, `EvmBatchProcessor` receives an ordered batch of event items it is subscribed to. In our case we have only two kinds of logs -- emitted on gravatar creations and updates.

The entry point for transform code is `src/main.ts`. We start by appending an auxiliary data normalization function to the end of that file:

```ts title="src/main.ts" theme={"system"}
function extractData(evmLog: any): { id: bigint, owner: string, displayName: string, imageUrl: string} {
  if (evmLog.topics[0] === events.NewGravatar.topic) {
    return events.NewGravatar.decode(evmLog)
  }
  if (evmLog.topics[0] === events.UpdatedGravatar.topic) {
    return events.UpdatedGravatar.decode(evmLog)
  }
  throw new Error('Unsupported topic')
}
```

Next, we make a [batch handler](/en/sdk/squid-sdk/reference/processors/architecture#processorrun) collecting the updates from a single batch of EVM logs. To convert a `0x...` string into a byte array we use the `decodeHex` utility from Squid SDK.

```ts title="src/main.ts" theme={"system"}
import { TypeormDatabase } from '@subsquid/typeorm-store'
import { decodeHex } from '@subsquid/evm-processor'
import { events } from './abi/Gravity'
import { Gravatar } from './model'
import { processor, GRAVATAR_CONTRACT } from './processor'

processor.run(new TypeormDatabase({supportHotBlocks: true}), async (ctx) => {
  const gravatars: Map<string, Gravatar> = new Map()
  for (const c of ctx.blocks) {
    for (const e of c.logs) {
      if (!(e.address === GRAVATAR_CONTRACT &&
            (e.topics[0] === events.NewGravatar.topic ||
             e.topics[0] === events.UpdatedGravatar.topic))) continue
      const { id, owner, displayName, imageUrl } = extractData(e)
      let idString = '0x' + id.toString(16)
      gravatars.set(idString, new Gravatar({
        id: idString,
        owner: decodeHex(owner),
        displayName,
        imageUrl
      }))
    }
  }
  await ctx.store.upsert([...gravatars.values()])
})
```

The implementation is straightforward -- the newly created and/or updated gravatars are tracked by an in-memory map. The values are persisted in a single batch upsert once all items are processed.

### 7. Run the processor and GraphQL API

To start the indexing, (re)build the project and run the processor:

```bash theme={"system"}
npm run build
```

```bash theme={"system"}
node -r dotenv/config lib/main.js
```

The processor will output the sync progress and the ETA to reach the chain head. After it reaches the head it will continue indexing new blocks until stopped.

To start an API server (at port `4350` by default) with a GraphQL schema auto-generated from the schema file, run in a new terminal window

```bash theme={"system"}
npx squid-graphql-server
```

and inspect the auto-generated GraphQL API using an interactive playground at [http://localhost:4350/graphql](http://localhost:4350/graphql).

## What's Next?

* Compare your API to that of subgraph using the [compareGraphQL](https://github.com/subsquid-labs/compareGraphQL) script
* Have a closer look at [`EvmBatchProcessor`](/en/sdk/squid-sdk)
* Learn how to [deploy a squid to SQD Cloud](/en/cloud) for free
* Learn how to [index and query the contract state](/en/sdk/squid-sdk/resources/tools/typegen/state-queries)
* Inspect a more complex [Uniswap V3 squid](https://github.com/dariaag/uniswap-squid-arrow) which tracks Uniswap V3 trading data. It was migrated from the [Uniswap V3 subgraph](https://github.com/Uniswap/v3-subgraph). It takes only a few hours to sync from scratch on a local machine.


# ArrowSquid for EVM
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/migrate/migrate-to-arrowsquid

Step-by-step migration from FireSquid to ArrowSquid for EVM Squid SDK indexers — schema, processor, and storage code changes for the new architecture.

# Migrate to ArrowSquid (EVM)

This is an EVM guide. Substrate guide is available [here](/en/sdk/squid-sdk/resources/migrate/migrate-to-arrowsquid-on-substrate).

ArrowSquid refers to the versions `@subsquid/evm-processor@1.x` and `@subsquid/substrate-processor@3.x`. ArrowSquid is not compatible with the FireSquid [archive](/en/sdk/squid-sdk/glossary#archives) endpoints. Instead, it uses [SQD Network](/en/network) gateways (see the [Supported EVM networks](/en/data/evm) page).

The main feature introduced by the ArrowSquid update on EVM is the new ability of the [processor](/en/sdk/squid-sdk/overview) to ingest unfinalized blocks directly from a network node, instead of waiting for the archive to ingest and serve it first. The processor can now handle forks and rewrite the contents of its database if it happens to have indexed orphaned blocks. This allows SQD-based APIs to become near real-time and respond to the on-chain activity with subsecond latency.

Another major feature introduced by ArrowSquid is the support for transaction execution receipts, [traces](/en/sdk/squid-sdk/reference/processors/evm-batch/traces) and [state diffs](/en/sdk/squid-sdk/reference/processors/evm-batch/state-diffs). It enables a significantly more fine-grained control over the smart contract states, especially in the situations when the EVM log data is insufficient. For example, one can:

* Reliably index transaction data, taking into account the transaction status
* Keep track of internal calls
* Observe smart contract state changes even if they are caused by internal transactions
* Track smart contract creation and destruction

The `EvmBatchProcessor` configuration and data selection interfaces has been simplified and the way in which the data is fetched has been made more efficient.

End-to-end ArrowSquid examples can be found [in the SDK repo](https://github.com/subsquid/squid-sdk/tree/master/test/eth-usdc-transfers) and in the [EVM examples](/en/sdk/squid-sdk/examples) section.

Here is a step-by-step guide for migrating a squid built with an older SDK version to the post-ArrowSquid tooling.

## Step 1

Update all packages affected by the update:

```bash theme={"system"}
npm i @subsquid/evm-processor@next
npm i @subsquid/typeorm-store@next
```

If your squid uses [`file-store`](/en/sdk/squid-sdk/resources/persisting-data/file), please update any related packages to the `@next` version, too.

## Step 2

Replace the old `setDataSource()` processor configuration call with a combination of [setGateway()](/en/sdk/squid-sdk/reference/processors/evm-batch/general#set-gateway) and [setRpcEndpoint()](/en/sdk/squid-sdk/reference/processors/evm-batch/general#set-rpc-endpoint). Use a [public SQD Network gateway URL for your network](/en/data/evm). If your squid did not use an RPC endpoint before, find one for your network and set it with [setRpcEndpoint()](/en/sdk/squid-sdk/reference/processors/evm-batch/general#set-rpc-endpoint). Also [configure](/en/sdk/squid-sdk/reference/processors/evm-batch/general#set-finality-confirmation) the network-specific number of transaction confirmations sufficient for finality. For Ethereum mainnet your edit might look like this:

```diff theme={"system"}
 processor
-  .setDataSource({
-    archive: lookupArchive('eth-mainnet', {release: 'FireSquid'})
-  })
+  .setGateway('https://v2.archive.subsquid.io/network/ethereum-mainnet')
+  .setRpcEndpoint({
+    url: '<my_eth_rpc_url>',
+    rateLimit: 10
+  })
+  .setFinalityConfirmation(75)
```

We recommend using a private RPC endpoint for the best performance, e.g. from [BlastAPI](https://blastapi.io/). For squids deployed to [SQD Cloud](/en/cloud/overview) you may also consider using our [RPC addon](/en/cloud/resources/rpc-proxy).

Your squid will work without an RPC endpoint, but with a significantly increased chain latency (a few hours for most chains, roughly a day for BSC). If that works for you, you can use just the SQD Network gateway without setting an RPC here and skip [Step 7](#step-7) altogether.

## Step 3

Next, we have to account for the changes in signatures of [`addLog()`](/en/sdk/squid-sdk/reference/processors/evm-batch/logs) and [`addTransaction()`](/en/sdk/squid-sdk/reference/processors/evm-batch/transactions) processor methods. Previously, each call of these methods supplied its own fine-grained field selectors. In the new interface, these calls can only enable or disable the retrieval of related data (with boolean flags `transaction` for `addLog()` and `logs` and a few others for `addTransaction()`). Field selection is now done by the new [`setFields()`](/en/sdk/squid-sdk/reference/processors/evm-batch/field-selection) method on a per-item-type basis: once for all `log`s, once for all `transaction`s etc. The setting is processor-wide: for example, all `transaction`s returned by the processor will have the same set of available fields, regardless of whether they are taken from the batch context directly or are accessed from within a `log` item.

Begin migrating to the new interface by finding all calls to `addLog()` and combining all the `evmLog` field selectors into a single processor-wide field selector that requests all fields previously requested by individual selectors. Remove the `id`, `logIndex` (previously `index`) and `transactionIndex` fields: now they are always available and cannot be requested explicitly. When done, make a call to `setFields()` and supply the new field selector at the `log` field of its argument. For example, suppose the processor was initialized with the following three calls:

```typescript theme={"system"}
const processor = new EvmBatchProcessor()
  .addLog(CONTRACT_ADDRESS, {
    filter: [[ abi.events.Transfer.topic ]],
    data: {
      evmLog: {
        id: true
      }
    } as const
  })
  .addLog(CONTRACT_ADDRESS, {
    filter: [[ abi.events.Approval.topic ]],
    data: {
      evmLog: {
        id: true,
        data: true
      }
    } as const
  })
  .addLog(CONTRACT_ADDRESS, {
    filter: [[ abi.events.ApprovalForAll.topic ]],
    data: {
      evmLog: {
        topics: true
      }
    } as const
  })
```

then the new global selector should be added like this:

```typescript theme={"system"}
const processor = new EvmBatchProcessor()
  // ... addLog() calls ...
  .setFields({
    log: {
      data: true,
      topics: true
    }
  })
```

Be aware that this operation will not increase the amount of data retrieved from SQD Network, since previously such coalescence was done under the hood and all fields were retrieved by the processor anyway. In fact, the amount of data should decrease due to a more efficient transfer mechanism employed by ArrowSquid.

See the [Field selection](/en/sdk/squid-sdk/reference/processors/evm-batch/field-selection) page for full documentation on field selectors.

## Step 4

Repeat step 3 for the `transaction` field selector. Make sure to check any transaction field selections by `addLog()` calls in addition to these made by [`addTransaction()`](/en/sdk/squid-sdk/reference/processors/evm-batch/transactions). Remove the default fields `id` and `transactionIndex` (previously `index`) and add the final field selector to the `.setFields()` call. For example, suppose the processor was initialized like this:

```typescript theme={"system"}
const processor = new EvmBatchProcessor()
  .addLog(CONTRACT_ADDRESS, {
    filter: [[ abi.events.Transfer.topic ]],
    data: {
      evmLog: { /* ...some log field selections... */ },
      transaction: {
        hash: true
      }
    } as const
  })
  .addTransaction(CONTRACT_ADDRESS, {
    sighash: [ abi.functions.approve.sighash ],
    data: {
      transaction: {
        gas: true,
        gasPrice: true
      }
    } as const
  })
```

then the new global selector should be added like this:

```diff theme={"system"}
 const processor = new EvmBatchProcessor()
   // ... addLog() and addTransaction() calls ...
   .setFields({
     log: { /* ...some log field selections... */ },
+    transaction: {
+      hash: true,
+      gas: true,
+      gasPrice: true
+    }
   })
```

See the [Field selection](/en/sdk/squid-sdk/reference/processors/evm-batch/field-selection) page for full documentation on field selectors.

## Step 5

Replace the old calls to `addLog()` and `addTransaction()` with calls using the [new signatures](/en/sdk/squid-sdk/reference/processors/evm-batch).

<Warning>
  The meaning of passing `[]` as a set of parameter values has been changed in the ArrowSquid release: now it *selects no data*. Pass `undefined` for a wildcard selection:

  ```typescript theme={"system"}
  .addLog({address: []}) // selects no logs
  .addLog({}) // selects all logs
  ```

  ```typescript theme={"system"}
  .addTransaction({from: []}) // selects no transactions
  .addTransaction({}) // selects all transactions
  ```
</Warning>

Old data requests will be erased during the process. Make sure to request the appropriate data with the boolean flags (`transaction` for `addLog()` and `logs` for `addTransaction()`) while doing that.

For example, a processor originally initialized like this:

```typescript theme={"system"}
const processor = new EvmBatchProcessor()
  .addLog(CONTRACT_ADDRESS, {
    filter: [[ abi.events.Transfer.topic ]],
    data: {
      evmLog: {
        topics: true,
        data: true
      },
      transaction: {
        hash: true
      }
    } as const
  })
  .addTransaction(CONTRACT_ADDRESS, {
    sighash: [ abi.functions.approve.sighash ],
    data: {
      transaction: {
        gas: true,
        gasPrice: true
      }
    } as const
  })
```

should now be made with

```typescript theme={"system"}
const processor = new EvmBatchProcessor()
  .addLog({
    address: [ CONTRACT_ADDRESS ],
    topic0: [ abi.events.Transfer.topic ],
    transaction: true // IMPORTANT: set this to true whenever the old call defined options.data.transaction
  })
  .addTransaction({
    to: [ CONTRACT_ADDRESS ],
    sighash: [ abi.functions.approve.sighash ]
  })
  .setFields({
    log: {
      topics: true,
      data: true
    },
    transaction: {
      hash: true,
      gas: true,
      gasPrice: true
    }
  })
```

## Step 6

Finally, update the batch handler to use the new [batch context](/en/sdk/squid-sdk/reference/processors/architecture#batch-context). There are two ways to do that:

1. If you're in a hurry, use the [`transformContext.ts`](https://gist.github.com/belopash/aa6b67dc374add44b9bdff1c9c1eee17) module. Download it with
   ```bash theme={"system"}
   curl -o src/transformContext.ts https://gist.githubusercontent.com/belopash/aa6b67dc374add44b9bdff1c9c1eee17/raw/441d43a932591624822b5bfd51a23147b5cecac2/transformContext.ts
   ```
   then transform the new context to the old format at the beginning of the batch handler:
   ```typescript title="src/processor.ts" theme={"system"}
   import {transformContext} from './transformContext'

   // ...

   processor.run(db, async (newCtx: DataHandlerContext<Store, any>) => {
     let ctx = transformContext(newCtx)
     // the rest of the batch handler should work unchanged
   })
   ```

2. Alternatively, rewrite your batch handler using the [new batch context interface](/en/sdk/squid-sdk/reference/processors/architecture#batch-context). Consult the [block data](/en/sdk/squid-sdk/reference/processors/evm-batch/context-interfaces) page for EVM-specific details on the new context format.

## Step 7

Update your transactions/events decoding code. The big change here is that now decoders generated by `@subsquid/evm-typegen` return `bigint` where `ethers.BigNumber` was used before. Regenerate all TypeScript ABI wrappers as described in the [EVM typegen](/en/sdk/squid-sdk/resources/tools/typegen/state-queries) section, then find all places where `ethers.BigNumber`s returned by old decoders were handled in your code and rewrite it to use `bigint`s.

## Step 8

Iteratively reconcile any type errors arising when building your squid (e.g. with `npm run build`). In case you're using `tranformContext.ts` you may find the types it exports helpful. If you need to specify the field selection generic argument explicitly, get it as a `typeof` of the `setFields` argument value:

```ts theme={"system"}
import { OldBlockData } from './transformContext'

const fieldSelection = {
  log: {
    data: true
  },
  transaction: {
    hash: true,
  }
} as const

let processor = new EvmBatchProcessor()
  .setFields(fieldSelection)
  /* the rest of the processor configuration */

type MyBlockData = OldBlockData<typeof fieldSelection>
// ...
```

At this point your squid should be able to work with the ArrowSquid tooling. If it doesn't, read on.

## Troubleshooting

If these instructions did not work for you, please let us know at the [SquidDevs Telegram chat](https://t.me/HydraDevs).


# ArrowSquid for Substrate
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/migrate/migrate-to-arrowsquid-on-substrate

Step-by-step migration guide from FireSquid to ArrowSquid for Substrate-based Squid SDK indexers — schema, processor, and storage code changes.

# Migrate to ArrowSquid (Substrate)

This is a Substrate guide. EVM guide is available [here](/en/sdk/squid-sdk/resources/migrate/migrate-to-arrowsquid).

ArrowSquid refers to `@subsquid/substrate-processor` versions `3.x` and above. It is not compatible with the FireSquid [archive](/en/sdk/squid-sdk/glossary#archives) endpoints. Instead, the new version uses [SQD Network](/en/network) gateways. See the [Supported networks](/en/data/substrate) page.

The main feature introduced by the ArrowSquid update is the new ability of the [processor](/en/sdk/squid-sdk/overview) to ingest unfinalized blocks directly from a network node, instead of waiting for an archive to ingest and serve it first. The processor can now handle forks and rewrite the contents of its database if it happens to have indexed orphaned blocks. This allows SQD-based APIs to become near real-time and respond to the on-chain activity with subsecond latency.

Other new features include the new streamlined processor configuration interface and automatic retrieval of execution traces. On the implementation side, the way in which data is fetched has been made more efficient.

End-to-end Substrate ArrowSquid examples:

* [general Substrate indexing](https://github.com/subsquid-labs/squid-substrate-template);
* [indexing an Ink! contract](https://github.com/subsquid-labs/squid-ink-template/);
* [working with Frontier EVM pallet data](https://github.com/subsquid-labs/squid-frontier-evm-template).

Here is a step-by-step guide for migrating a squid built with an older SDK version to the post-ArrowSquid tooling.

## Step 1

Update all packages affected by the update:

```bash theme={"system"}
npm i @subsquid/substrate-processor@latest @subsquid/typeorm-store@latest
```

```bash theme={"system"}
npm i --save-dev @subsquid/substrate-typegen@latest
```

If your squid uses [`file-store`](/en/sdk/squid-sdk/resources/persisting-data/file), please update any related packages to the `@latest` version.

## Step 2

Replace the old `setDataSource()` processor configuration call with a combination of [setGateway()](/en/sdk/squid-sdk/reference/processors/substrate-batch/general#set-gateway) and [setRpcEndpoint()](/en/sdk/squid-sdk/reference/processors/substrate-batch/general#set-rpc-endpoint). Use a [public SQD Network gateway URL for your network](/en/data/substrate). If your squid did not use an RPC endpoint before, find one for your network and set it with [setRpcEndpoint()](/en/sdk/squid-sdk/reference/processors/substrate-batch/general#set-rpc-endpoint). For Aleph Zero your edit might look like this:

```diff theme={"system"}
 processor
-  .setDataSource({
-    archive: 'https://aleph-zero.archive.subsquid.io/graphql'
-  })
+  .setGateway('https://v2.archive.subsquid.io/network/aleph-zero')
+  .setRpcEndpoint({
+    url: 'https://aleph-zero-rpc.dwellir.com',
+    rateLimit: 10
+  })
```

We recommend using a private RPC endpoint for the best performance, e.g. from [Dwellir](https://www.dwellir.com). For squids deployed to [SQD Cloud](/en/cloud/overview) you may also consider using our [RPC addon](/en/cloud/resources/rpc-proxy).

Your squid will work with just an RPC endpoint, but it will sync significantly slower. With a SQD Network gateway available the processor will only use RPC to retrieve metadata **and** sync the few most recent blocks not yet made available by the gateway; without it, it will retrieve all data from the endpoint.

## Step 3

Next, we have to account for the changes in signatures of the data requesting processor methods

* [`addEvent()`](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#events),
* [`addCall()`](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#calls),
* [`addEvmLog()`](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#addevmlog),
* [`addEthereumTransaction()`](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#addethereumtransaction),
* [`addContractsContractEmitted()`](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#addcontractscontractemitted),
* [`addGearMessageQueued()`](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#addgearmessagequeued),
* [`addGearUserMessageSent()`](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#addgearusermessagesent),

Previously, each call of these methods supplied its own fine-grained data fields selector. In the new interface, these calls only request data items, either directly or by relation (for example with the `call` flag for event-requesting methods). Field selection is now done by the new `setFields()` method on a per-item-type basis: once for all `Call`s, once for all `Event`s etc. The setting is processor-wide: for example, all `Call`s returned by the processor will have the same set of available fields, regardless of whether they were requested directly or as related data.

Begin migrating to the new interface by finding all calls to these methods and combining all the field selectors into processor-wide `event`, `call` and `extrinsic` field selectors that request all fields previously requested by individual selectors. Note that `call.args` and `event.args` are now requested by default and can be omitted. When done, add a call to `setFields()` supplying it with the new field selectors.

The new field selector format is fully documented on the [Field selection](/en/sdk/squid-sdk/reference/processors/substrate-batch/field-selection) page.

<Info>
  Blanket field selections like `{data: {event: {extrinsic: true}}}` are not supported in ArrowSquid. If you used one of these, please find out which exact fields you use in the batch handler and specifically request them.
</Info>

<Warning>
  Do not include related data requests into the field selection. E.g.

  ```ts theme={"system"}
    .setFields({
      event: {
        call: true
      }
    })
  ```

  will at best fail to compile and at worst make your squid fail due to HTTP 400s from the network gateway.
</Warning>

For example, suppose the processor was initialized with the following three calls:

```typescript theme={"system"}
const processor = new SubstrateBatchProcessor()
  .addCall('Balances.transfer_keep_alive', {
    data: {
      call: {
        origin: true,
        args: true
      }
    }
  } as const)
  .addEvent('Balances.Transfer', {
    data: {
      event: {
        args: true,
        extrinsic: {
          hash: true,
          fee: true
        },
        call: {
          success: true,
          error: true
        }
      }
    }
  } as const)
  .addEvmLog(CONTRACT_ADDRESS,
    filter: [[ abi.events.SomeLog.topic ]],
    {
      data: {
        event: {
          extrinsic: {
            hash: true,
            tip: true
          }
        }
      }
    } as const
  )
```

then the new global selectors should be added like this:

```typescript theme={"system"}
const processor = new SubstrateBatchProcessor()
  // ... addXXX() calls ...
  .setFields({
    event: {},
    call: {
      origin: true,
      success: true,
      error: true
    },
    extrinsic: {
      hash: true,
      fee: true,
      tip: true
    }
  })
```

Be aware that this operation will not increase the amount of data retrieved from the SQD Network gateway, since previously such coalescence was done under the hood and all fields were retrieved by the processor anyway. In fact, the amount of data should decrease due to a more efficient transfer mechanism employed by ArrowSquid.

There are two old field requests that have no direct equivalent in the new interface:

<AccordionGroup>
  <Accordion title="call.parent">
    It is currently impossible to request just the parent call. Work around by requesting the full call stack with `stack: true` in the call-requesting configuration calls, then using `.parentCall` property or `getParentCall()` method of `Call` data items to get parent calls.
  </Accordion>

  <Accordion title="event.evmTxHash">
    Processor no longer makes EVM transaction hashes explicitly available. Currently the only way to get hashes of logs' parent txs is to figure out the subset of calls that emitted the required logs, request them explicitly with their related events, filter out `Ethereum.Executed` events and decode these. See [this issue](https://github.com/subsquid/squid-sdk/issues/211).
  </Accordion>
</AccordionGroup>

## Step 4

Replace the old calls to the data requesting processor methods with calls using the new signatures.

<Warning>
  The meaning of passing `[]` as a set of parameter values has been changed in the ArrowSquid release: now it *selects no data*. Pass `undefined` for a wildcard selection:

  ```typescript theme={"system"}
  .addEvent({name: []}) // selects no events
  .addEvent({}) // selects all events
  ```
</Warning>

Old data request calls will be erased during the process. Make sure to request the appropriate related data with the boolean flags (`call` for event-requesting methods, `events` for call-requesting methods and `extrinsic`, `stack` for both).

Interfaces of data request methods are documented on the Data requests reference page:

* [`addEvent()`](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#events),
* [`addCall()`](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#calls),
* [`addEvmLog()`](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#addevmlog),
* [`addEthereumTransaction()`](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#addethereumtransaction),
* [`addContractsContractEmitted()`](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#addcontractscontractemitted),
* [`addGearMessageQueued()`](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#addgearmessagequeued),
* [`addGearUserMessageSent()`](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#addgearusermessagesent).

Here is a fully updated initialization code for the example processor from step 3:

```typescript theme={"system"}
const processor = new SubstrateBatchProcessor()
  .addCall({
    name: [ 'Balances.transfer_keep_alive' ]
  })
  .addEvent({
    name: [ 'Balances.Transfer' ],
    call: true,
    extrinsic: true
  })
  .addEvmLog({
    address: [ CONTRACT_ADDRESS ],
    topic0: [ abi.events.SomeLog.topic ],
    extrinsic: true
  })
  .setFields({
    event: {},
    call: {
      origin: true,
      success: true,
      error: true
    },
    extrinsic: {
      hash: true,
      fee: true,
      tip: true
    }
  })

```

## Step 5

Finally, update the batch handler to use the new [batch context](/en/sdk/squid-sdk/reference/processors/architecture#batch-context). The main change here is that now `block.items` is split into three separate iterables: `block.calls`, `block.events` and `block.extrinsics`. There are two ways to migrate:

1. If you're in a hurry, use the `orderItems(block: Block)` function from [this snippet](https://gist.github.com/belopash/5d61dcce7739f60d55c4faaec0148282):
   ```typescript title="src/main.ts" theme={"system"}
   // ...
   // paste the gist here

   processor.run(db, async ctx => {
     // ...
     for (let block of ctx.blocks) {
       for (let item of orderItems(block)) {
         // item processing code should work unchanged
       }
     }
     // ...
   })
   ```

2. Alternatively, rewrite your batch handler using the [new batch context interface](/en/sdk/squid-sdk/reference/processors/architecture#batch-context).

See [Block data for Substrate](/en/sdk/squid-sdk/reference/processors/substrate-batch/context-interfaces) for the documentation on Substrate-specific part of batch context.

## Step 6

Rewrite your `typegen.json` in the new style. Here is an example:

```json theme={"system"}
{
  "outDir": "src/types",
  "specVersions": "https://v2.archive.subsquid.io/metadata/kusama",
  "pallets": {
    "Balances": {
      "events": [
        "Transfer"
      ],
      "calls": [],
      "storage": [],
      "constants": []
    }
  }
}
```

Note the changes:

1. Archive URL as `"specVersions"` is replaced with an URL of our new metadata service (`"https://v2.archive.subsquid.io/metadata/kusama"`)
2. Requests for data wrappers are now made on a per-pallet basis.

Check out the updated [Substrate typegen documentation page](/en/sdk/squid-sdk/tutorials/batch-processor-in-action). If you used any storage calls, consult [this documentation page](/en/sdk/squid-sdk/resources/tools/typegen/state-queries) for guidance.

Once you're done migrating `typegen.json`, regenerate the wrapper classes with

```bash theme={"system"}
npx squid-substrate-typegen typegen.json
```

## Step 7

Iteratively reconcile any type errors arising when building your squid (e.g. with `npm run build`). If you need to specify the field selection generic type argument explicitly, get it as a `typeof` of the `setFields` argument value:

```ts theme={"system"}
import {Block} from '@subsquid/substrate-processor'

const fieldSelection = {
  event: {},
  call: {
    origin: true,
    success: true,
    error: true
  },
  extrinsic: {
    hash: true,
    fee: true,
    tip: true
  }
} as const

type MyBlock = Block<typeof fieldSelection>
// ...
```

At this point your squid should be able to work with the ArrowSquid tooling. If it doesn't, read on.

## Troubleshooting

If these instructions did not work for you, please let us know at the [SquidDevs Telegram chat](https://t.me/HydraDevs).


# hasura-configuration tool v2
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/migrate/migrate-to-hasura-configuration-tool-v2

Migrate to the Hasura configuration tool v2 in Squid SDK — breaking changes, new metadata schema, and how to update your squid project.

# Migrating to Hasura configuration tool v2

Pre-2.0.0 [`@subsquid/hasura-configuration`](/en/sdk/squid-sdk/resources/tools/hasura-configuration) used a fixed naming schema for [relation fields](/en/sdk/squid-sdk/reference/schema-file/entity-relations):

* Forward relation fields were *always* named after the type they referred to. Type names were `snake_case`d. For example, any field referring to an entity called `BurnTest` was called `burn_test`.
* Inverse relation fields names were also determined by the relation type:
  * In one-to-one relation their names were `to_snake_case(typeName)`, where `typeName` is the name of the entity that the inverse field is typed with.
  * In one-to-many inverse relations they were named `${to_snake_case(typeName)}s`.

Any names given to the fields in the [schema](/en/sdk/squid-sdk/reference/schema-file) were ignored.

`@subsquid/hasura-configuration@2.0.0` introduces full support for in-schema field names. Now the fields will be called exactly as they are in the schema file. If these field names are different from what's described above, your API will have a breaking change. If you'd like to avoid it, please make sure that the fields in your `schema.graphql` are named exactly as described above.

To update the tool and your Hasura config:

```bash theme={"system"}
npm i @subsquid/hasura-configuration@latest
npx squid-hasura-configuration regenerate
```

If you deployed to the Cloud you can safely (barring the field names change) redeploy your squid in-place.

If you opted to not change your schema to accommodate the old relation field names, please follow this up by revising GraphQL queries in any of your client apps.


# Multichain
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/multichain

Combine data from multiple chains in a single Squid SDK indexer — multichain processor patterns, shared schema, and write-coordination strategies.

# Multichain indexing

Squids can extract data from multiple chains into a shared data sink. If the data is [stored to Postgres](/en/sdk/squid-sdk/resources/persisting-data/typeorm) it can then be served as a unified multichain [GraphQL API](/en/sdk/squid-sdk/resources/serving-graphql).

To do this, run one [processor](/en/sdk/squid-sdk/overview) per source network:

1. Make a separate entry point (`main.ts` or equivalent) for each processor. The resulting folder structure may look like this:

   <Tree>
     <Tree.Folder name="src">
       <Tree.Folder name="bsc">
         <Tree.File name="main.ts" />

         <Tree.File name="processor.ts" />
       </Tree.Folder>

       <Tree.Folder name="eth">
         <Tree.File name="main.ts" />

         <Tree.File name="processor.ts" />
       </Tree.Folder>
     </Tree.Folder>
   </Tree>

Alternatively, parameterize your processor using environment variables: you can [set these on a per-processor basis](/en/cloud/reference/manifest#processor) if you use a deployment manifest to run your squid.

2. Arrange for running the processors alongside each other conveniently:
   * Add `sqd` commands for running each processor to [`commands.json`](/en/sdk/squid-sdk/squid-cli/commands-json), e.g.
     ```json title="commands.json" theme={"system"}
     ...
       "process:eth": {
         "deps": ["build", "migration:apply"],
         "cmd": ["node", "lib/eth/main.js"]
       },
       "process:bsc": {
         "deps": ["build", "migration:apply"],
         "cmd": ["node", "lib/bsc/main.js"]
       },
     ...
     ```
     [Full example](https://github.com/subsquid-labs/multichain-transfers-example/blob/master/commands.json)
   * If you are going to use [`sqd run`](/en/sdk/squid-sdk/squid-cli/run) for local runs or deploy your squid to [SQD Cloud](/en/cloud), list your processors at the `deploy.processor` section of your [deployment manifest](/en/cloud/reference/manifest#processor):
     ```yaml theme={"system"}
     deploy:
       processor:
         - name: eth-processor
           cmd: [ "sqd", "process:prod:eth" ]
         - name: bsc-processor
           cmd: [ "sqd", "process:prod:bsc" ]
     ```
     Make sure to give each processor a unique name!

## On Postgres

Also ensure that

1. State schema name for each processor is unique

```ts title="src/bsc/main.ts" theme={"system"}
processor.run(
  new TypeormDatabase({
    stateSchema: 'bsc_processor'
  }),
  async ctx => { // ...
```

```ts title="src/eth/main.ts" theme={"system"}
processor.run(
  new TypeormDatabase({
    stateSchema: 'eth_processor'
  }),
  async (ctx) => { // ...
```

2. [Schema](/en/sdk/squid-sdk/reference/schema-file) and [GraphQL API](/en/sdk/squid-sdk/resources/serving-graphql) are shared among the processors.

### Handling concurrency

* Cross-chain data dependencies are to be avoided. With the [default isolation level](/en/sdk/squid-sdk/reference/store/typeorm#typeormdatabase-constructor-arguments) used by `TypeormDatabase`, `SERIALIZABLE`, one of the processors will crash with an error whenever two cross-dependent transactions are submitted to the database simultaneously. It will write the correct data when restarted, but such restarts can impact performance, especially in squids that use many (>5) processors.

* The alternative isolation level is `READ COMMITTED`. At this level data dependencies will not cause the processors to crash, but the execution is not guaranteed to be deterministic unless the sets of records that different processors read/write do not overlap.

* To avoid cross-chain data dependencies, use per-chain records for volatile data. E.g. if you track account balances across multiple chains you can avoid overlaps by storing the balance for each chain in a different table row.

  When you need to combine the records (e.g. get a total of all balaces across chains) use a [custom resolver](/en/sdk/squid-sdk/reference/openreader-server/configuration/custom-resolvers) to do it on the GraphQL server side.

* It is OK to use cross-chain [entities](/en/sdk/squid-sdk/reference/schema-file/entities) to simplify aggregation. Just don't store any data in them:
  ```graphql theme={"system"}
  type Account @entity {
    id: ID! # evm address
    balances: [Balance!]! @derivedFrom("field": "account")
  }

  type Balance @entity {
    id: ID! # chainId + evm address
    account: Account!
    value: BigInt!
  }
  ```

## On [file-store](/en/sdk/squid-sdk/resources/persisting-data/file)

Ensure that you use a unique target folder for each processor.

## Example

A complete example is available [here](https://github.com/subsquid-labs/squid-multichain-template).


# Saving to BigQuery
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/persisting-data/bigquery

Save Squid SDK indexed data to Google BigQuery via @subsquid/bigquery-store — schema sync, batch loading, and partitioning patterns for analytics.

# BigQuery support

Squids can store their data to BigQuery datasets using the `@subsquid/bigquery-store` package. Define and use the `Database` object as follows:

```ts title="src/main.ts" theme={"system"}
import {
  Column,
  Table,
  Types,
  Database
} from '@subsquid/bigquery-store'
import {BigQuery} from '@google-cloud/bigquery'

const db = new Database({
  bq: new BigQuery(),
  dataset: 'subsquid-datasets.test_dataset',
  tables: {
    TransfersTable: new Table(
      'transfers',
      {
        from: Column(Types.String()),
        to: Column(Types.String()),
        value: Column(Types.BigNumeric(38))
      }
    )
  }
})

processor.run(db, async ctx => {
  // ...
  let from: string = ...
  let to: string = ...
  let value: bigint | number = ...
  ctx.store.TransfersTable.insert({from, to, value})
})
```

Here,

* `bq` is a `BigQuery` instance. When created without arguments like this it'll look at the `GOOGLE_APPLICATION_CREDENTIALS` environment variable for a path to a JSON with authentication details.
* `dataset` is the path to the target dataset.

<Warning>
  The dataset must be created prior to running the processor.
</Warning>

* `tables` lists the tables that will be created and populated within the dataset. For every field of the `tables` object an eponymous field of the `ctx.store` object will be created; calling `insert()` or `insertMany()` on such a field will result in data being queued for writing to the corresponding dataset table. The actual writing will be done at the end of the batch in a single transaction, ensuring dataset integrity.

Tables are made out of statically typed columns. Available types are listed on the [reference page](/en/sdk/squid-sdk/reference/store/bigquery).

## Deploying to SQD Cloud

We discourage uploading any sensitive data with squid code when [deploying](/en/cloud) to SQD Cloud. To pass your [credentials JSON](https://cloud.google.com/docs/authentication/application-default-credentials#GAC) to your squid, create a [Cloud secret](/en/cloud/resources/env-variables#secrets) variable populated with its contents:

```bash theme={"system"}
sqd secrets set GAC_JSON_FILE < creds.json
```

Then in `src/main.ts` write the contents to a file:

```ts title=src/main.ts theme={"system"}
import fs from 'fs'

fs.writeFileSync('creds.json', process.env.GAC_JSON_FILE || '')
```

Set the `GOOGLE_APPLICATION_CREDENTIALS` variable and request the secret in the [deployment manifest](/en/cloud/reference/manifest):

```yaml title="squid.yaml" theme={"system"}
deploy:
  processor:
    env:
      GAC_JSON_FILE: ${{ secrets.GAC_JSON_FILE }}
      GOOGLE_APPLICATION_CREDENTIALS: creds.json
```

## Examples

An end-to-end example geared towards local runs can be found in [this repo](https://github.com/subsquid-labs/squid-bigquery-example). Look at [this branch](https://github.com/subsquid-labs/squid-bigquery-example/tree/cloud-secrets) for an example of a squid made for deployment to SQD Cloud.

## Troubleshooting

### Transaction is aborted due to concurrent update

This means that your project has an open [session](https://cloud.google.com/bigquery/docs/sessions-intro) that is updating some of the tables used by the squid.

Most commonly, the session is left by a squid itself after an unclean termination. You have two options:

1. If you are not sure if your squid is the only app that uses sessions to access your BigQuery project, find the faulty session manually and terminate it. See [Get a list of your active sessions](https://cloud.google.com/bigquery/docs/sessions-get-ids#list_active) and [Terminate a session by ID](https://cloud.google.com/bigquery/docs/sessions-terminating#terminate_a_session_by_id).

2. **DANGEROUS** If you are absolutely certain that the squid is the only app that uses sessions to access your BigQuery project, you can terminate all the dangling sessions by running

   ```sql theme={"system"}
   FOR session in (
     SELECT
       session_id,
       MAX(creation_time) AS last_modified_time,
     FROM `region-us`.INFORMATION_SCHEMA.SESSIONS_BY_PROJECT
     WHERE
       session_id IS NOT NULL
       AND is_active
     GROUP BY session_id
     ORDER BY last_modified_time DESC
   )
   DO
     CALL BQ.ABORT_SESSION(session.session_id);
   END FOR;
   ```

   Replace `region-us` with your dataset's region in the code above.

   You can also enable `abortAllProjectSessionsOnStartup` and supply `datasetRegion` in your database config to perform this operation at startup:

   ```ts theme={"system"}
   const db = new Database({
     // ...
     abortAllProjectSessionsOnStartup: true,
     datasetRegion: 'region-us'
   })
   ```

   This method **will** cause data loss if, at the moment when the squid starts, some other app happens to be writing data anywhere in the project using the sessions mechanism.

### Error 413 (Request Entity Too Large)

Squid produced too much data per batch per table and BigQuery refused to handle it. Begin by finding out which table causes the issue (e.g. by counting `insert()` calls), then enable pagination for that table:

```ts theme={"system"}
const db = new Database({
  bq: new BigQuery(), // set GOOGLE_APPLICATION_CREDENTIALS at .env
  dataset: `${projectId}.${datasetId}`,
  tables: {
    TransfersTable: new Table(
      'transfers',
      {
        from: Column(Types.String()),
        to: Column(Types.String()),
        value: Column(Types.BigNumeric(38))
      },
      5000 // <- page size for the insert operation
    )
  },
  ...
```


# Saving to filesystems
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/persisting-data/file

Save Squid SDK indexed data to filesystem-based datasets — CSV, JSON, JSONL, and Parquet output via the @subsquid/file-store package.

# Filesystem store

<Warning>
  Make sure you understand and use the [`chunkSizeMb` setting](#overview) and the [`setForceFlush()`](#setforceflush) store method. Failure to do so may cause the processor to output an empty dataset.
</Warning>

## Overview

Squid SDK provides append-only [`Store` interface](/en/sdk/squid-sdk/resources/persisting-data/overview) implementations for saving the data generated by squids to files. The store is designed primarily for offline analytics. It supports CSV and Parquet files, persisted either locally or to a S3 storage. See the list of [Packages](#packages) below.

The core component of all file-based stores is the `Database` class from `@subsquid/file-store`. To use it, construct an instance and pass it to [`processor.run()`](/en/sdk/squid-sdk/reference/processors/architecture#processorrun). This results in `ctx.store` exposing [table writers](#table-writer-interface) that accept rows of data to be stored.

File-based stores *always* partition data sets along the "block height" dimension, even when it is not in the schema. The number of blocks per partition is variable: a new partition is written when either

* the internal buffer (size governed by the `chunkSizeMb` [`Database` constructor option](#database-options)) of the store fills up, *or*
* after any call to [batch handler](/en/sdk/squid-sdk/reference/processors/architecture#processorrun) during which there was a call to [`setForceFlush()`](#setforceflush).

Same failover guarantees as with the [Postgres-based store](/en/sdk/squid-sdk/resources/persisting-data/typeorm) are provided: the processor will roll back to the last successful state after a restart.

#### Example

Save ERC20 `Transfer` events retrieved by [EVM processor](/en/sdk/squid-sdk) in `transfers.csv` files:

```typescript theme={"system"}
import {EvmBatchProcessor} from '@subsquid/evm-processor'
import * as erc20abi from './abi/erc20'
import {Database, LocalDest} from '@subsquid/file-store'
import {Column, Table, Types} from '@subsquid/file-store-csv'

const processor = /* processor definition */

const dbOptions = {
  tables: {
    TransfersTable: new Table('transfers.csv', {
      from: Column(Types.String()),
      to: Column(Types.String()),
      value: Column(Types.Numeric())
    })
  },
  dest: new LocalDest('./data'),
  chunkSizeMb: 10
}

processor.run(new Database(dbOptions), async (ctx) => {
  for (let c of ctx.blocks) {
    for (let log of c.logs) {
      if (/* the log item is a Transfer we're interested in */) {
        let { from, to, value } =
          erc20abi.events.Transfer.decode(log)
        ctx.store.TransfersTable.write({ from, to, value })
      }
    }
  }
})
```

The resulting `./data` folder may look like this:

```bash theme={"system"}
./data/
├── 0000000000-0007688959
│   └── transfers.csv
├── 0007688960-0007861589
│   └── transfers.csv
...
├── 0016753040-0016762029
│   └── transfers.csv
└── status.txt
```

Each of the folders here contains a little over 10 MBytes of data. `status.txt` contains the height of the last indexed block and its hash.

## Packages

`@subsquid/file-store` is the core package that contains the implementation of `Database` for filesystems. At least one file format add-on must be installed alongside it:

* **CSV**: Supported via [`@subsquid/file-store-csv`](/en/sdk/squid-sdk/reference/store/file/csv).
* **Parquet**: An [advanced format](https://parquet.apache.org) that works well for larger data sets. Supported via [`@subsquid/file-store-parquet`](/en/sdk/squid-sdk/reference/store/file/parquet).

Data in either of these formats can be written to

* **A local filesystem**: Supported by `@subsquid/file-store` out of the box.
* **A bucket in an Amazon S3-compatible cloud**: Supported via [`@subsquid/file-store-s3`](/en/sdk/squid-sdk/reference/store/file/s3-dest).

## `Database` Options

Constructor of the `Database` class from `file-store` accepts a configuration object as its only argument. Its format is as follows:

```typescript theme={"system"}
DatabaseOptions {
  tables: Record<string, Table>
  dest: Dest
  chunkSizeMb?: number
  hooks?: DatabaseHooks<Dest>
}
```

Here,

* `Table` is an interface for classes that make [table writers](#table-writer-interface), objects that convert in-memory tabular data into format-specific file contents. An implementation of `Table` is available for every file format supported by `file-store`. Consult [pages about specific output formats](/en/sdk/squid-sdk/reference/store/file) to find out how to define `Table`s.
* **`tables`** is a mapping from developer-defined string handles to `Table` instances. A table writer will be created for each `Table` in this mapping. It will be exposed at `ctx.store.<tableHandle>`.
* **`dest`** is an instance of `Dest`, an interface for objects that take the properly formatted file contents and write them onto a particular filesystem. An implementation of `Dest` is available for every filesystem supported by `file-store`. For local filesystems use the `LocalDest` class from the `@subsquid/file-store` package and supply `new LocalDest(outputDirectoryName)` here. For other targets consult [documentation pages specific to your filesystem choice](/en/sdk/squid-sdk/reference/store/file).
* **`chunkSizeMb`** governs the size of the internal data buffer. A dataset partition will be written as soon as this buffer fills up, or at the end of the batch if [setForceFlush()](#setforceflush) was called.
* **`hooks`** are useful for [persisting data between batches](#hooks).

## Table Writer Interface

For each `Table` supplied via the `tables` field of the constructor argument, `Database` adds a table writer property to [`ctx.store`](/en/sdk/squid-sdk/resources/persisting-data/overview). The writer is exposed at `ctx.store.<tableHandle>`, where `<tableHandle>` is the key of the `Table` instance in the `tables` mapping. It has the following methods:

**`ctx.store.<tableHandle>.write(record: T)`**\
**`ctx.store.<tableHandle>.writeMany(records: T[])`**

Here, `T` is a `Table` implementation-specific data row type. See the [documentation pages on specific file formats](/en/sdk/squid-sdk/reference/store/file) for details.

These synchronous methods add rows of data to an in-memory buffer and perform no actual filesystem writes. Instead, the write happens automatically when the internal buffer reaches [`chunkSizeMb`](#overview) or at the end of the batch during which [`setForceFlush()`](#setforceflush) was called. The methods return the table writer instance and can be chained.

For example, with a `Database` defined like this:

```ts theme={"system"}
const db = new Database({
  tables: {
    TransfersTable: new Table(/* table options */)
  },
  // ...dest and dataset partitioning options
})
```

the following calls become available in the [batch handler](/en/sdk/squid-sdk/reference/processors/architecture#processorrun):

```ts theme={"system"}
processor.run(db, async ctx => {
  let record = // row in a format specific to Table implementation
  ctx.store.TransfersTable.write(record)
  ctx.store.TransfersTable.writeMany([record])
})
```

## `setForceFlush()`

Both of the following calls

**`ctx.setForceFlush()`**

**`ctx.setForceFlush(true)`**

set the force flush flag within the store. If the flag is still set at the end of a [batch](/en/sdk/squid-sdk/reference/processors/architecture#processorrun), a dataset partition will be written regardless of how full the data buffer currently is. This is useful e.g. in ensuring that the partition size is not much greater than a constant value when writing data at a low rate:

```ts theme={"system"}
let blockCount = 0

processor.run(db, async ctx => {
  // ...data is transformed and queued for writing
  blockCount += ctx.blocks.length
  if (blockCount >= 500_000) {
    ctx.store.setForceFlush()
  }
})
```

Unset the flag with

**`ctx.setForceFlush(false)`**

## Hooks

By default, `Database` maintains a record of the syncing progress in the `status.txt` file. When the processor with a `Database` instance starts, it calls the `onStateRead()` function that reads the highest reached block from `status.txt` on the target filesystem and returns its hash and height. If the file does not exist, the function returns -1 for height and the syncing resumes starting at the next (zeroth/genesis) block.

Syncing status record is updated every time a new partition is written to the dataset: the processor calls `onFlush()` which overwrites `status.txt` with the new highest reached block.

As a result, the integrity of the data set is guaranteed given the blockchain history up to the point recorded in `status.txt`.

The functions `onStateRead()` and `onStateUpdate()` can be overridden using the **`hooks`** constructor argument field. To do that, set that field to

```typescript theme={"system"}
DatabaseHooks<Dest> {
  onStateRead(dest: Dest): Promise<HashAndHeight | undefined>
  onStateUpdate(dest: Dest, info: HashAndHeight): Promise<void>
}
```

Parameters:

* **dest**: the `Dest` object used by `Database`. Use it to access the filesystem.
* **info**: a `{height: number, hash: string}` object.

Overriding these functions can be useful for transferring some processor state between batches reliably. A basic example of using **`hooks`** can be found [here](https://github.com/subsquid/squid-file-store/blob/fcc8a63dc2deb8c3f843aa525f806ff9cdf778fb/test/src/main.ts#L36).


# Store interface
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/persisting-data/overview

Squid SDK Store interface — the persistence API used inside batch handlers to write indexed entities into Postgres, BigQuery, or files.

`Store` is a generic interface exposed by [`DataHandlerContext`](/en/sdk/squid-sdk/reference/processors/architecture#batch-context) at `.store`. It is used in the [batch handler](/en/sdk/squid-sdk/reference/processors/architecture#processorrun) to persist data. Its concrete type is inferred from the `Database` argument of the [`run()` processor method](/en/sdk/squid-sdk/reference/processors/architecture#processorrun):

```typescript theme={"system"}
run<Store>(
  db: Database<Store>,
  handler: (ctx: DataHandlerContext<Store, F extends FieldSelection>)
    => Promise<void>
): void
```

The `Database` implementation supplied here defines the store and the way the processor [persists its status](/en/sdk/squid-sdk/faq#how-do-squids-keep-track-of-their-sync-progress). Squid SDK supports `Database` implementations for:

* [`TypeORM`-compatible databases](/en/sdk/squid-sdk/resources/persisting-data/typeorm)
* [file-based datasets](/en/sdk/squid-sdk/resources/persisting-data/file)
* [Google BigQuery](/en/sdk/squid-sdk/resources/persisting-data/bigquery)

Support for more databases and analytics storage will be added in the future.

## Custom `Database`

A custom implementation of the `Database` interface is the recommended solution for squids with multiple data sinks and/or for non-transactional or non-relational databases. In such a case, the inferred `Store` facade exposed to the handlers may provide multiple targets for persisting the data. `Database.transact` should handle potential edge-cases when writes to either of the data sinks fail.

In order to implement a custom data sink adapter, it suffices to implement the `Database` interface:

```ts theme={"system"}
export type Database<S> = FinalDatabase<S> | HotDatabase<S>

export interface FinalDatabase<S> {
    supportsHotBlocks?: false
    connect(): Promise<HashAndHeight>
    transact(info: FinalTxInfo, cb: (store: S) => Promise<void>): Promise<void>
}

export interface HotDatabase<S> {
    supportsHotBlocks: true
    connect(): Promise<HotDatabaseState>
    transact(info: FinalTxInfo, cb: (store: S) => Promise<void>): Promise<void>
    transactHot(info: HotTxInfo, cb: (store: S, block: HashAndHeight) => Promise<void>): Promise<void>
}
```

Consult [this file](https://github.com/subsquid/squid-sdk/blob/master/util/util-internal-processor-tools/src/database.ts) for details.

The interface only defines how the processor advances with the indexing and connects to the data sink. The following is left as implementation details:

* persisting the indexing status
* opening and rolling back the transaction (if applicable)

Consult these implementations for details:

* (Transactional) Postgres-based [TypeormDatabase](https://github.com/subsquid/squid-sdk/blob/master/typeorm/typeorm-store/src/database.ts)
* (Non-transactional) [File-based database](https://github.com/subsquid/squid-csv-store/blob/main/src/database.ts)


# Saving to PostgreSQL
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/persisting-data/typeorm

Save Squid SDK indexed data to PostgreSQL via the typeorm-store package — auto-generated entity models, batched upserts, and migration tooling.

`TypeormDatabase` context store provides a wrapper over the [TypeORM `EntityManager`](https://typeorm.io/entity-manager-api) optimized for batch saving. It currently supports only Postgres-compatible databases and seamlessly integrates with entity classes generated from the [schema file](/en/sdk/squid-sdk/reference/schema-file).

Check out [this section of the reference page](/en/sdk/squid-sdk/reference/store/typeorm#database-connection-parameters) to learn how to specify the database connection parameters.

## Usage

```ts theme={"system"}
import { Store, TypeormDatabase } from '@subsquid/typeorm-store'
import { FooEntity } from './model'

const dbOptions = {/* ... constructor options ... */}

processor.run(new TypeormDatabase(dbOptions), async ctx => {
  // ...  
  await ctx.store.upsert([new FooEntity({ id: '1'}), new FooEntity({ id: '2'})])
})
```

Here,

* `FooEntity` represents a TypeORM entity class. In squids, these are typically generated from [schema files](/en/sdk/squid-sdk/reference/schema-file) with [squid-typeorm-codegen](/en/sdk/squid-sdk/reference/schema-file/intro#typeorm-codegen).
* `TypeormDatabase` constructor options govern the behavior of the class at the highest level. They are described in [this section of the reference](/en/sdk/squid-sdk/reference/store/typeorm#typeormdatabase-constructor-arguments).
* `ctx.store` is of type `Store`. See the [`Store` interface](/en/sdk/squid-sdk/reference/store/typeorm#store-interface) section of the reference page for more details.

## Database migrations

The Squid SDK manages the database schema with [TypeORM-based database migrations](https://typeorm.io/migrations) by means of the `squid-typeorm-migration(1)` tool.
The tool auto-generates the schema migrations from the TypeORM entities created by [codegen](/en/sdk/squid-sdk/reference/schema-file/intro#typeorm-codegen), so that custom migration scripts are rarely needed.

Here are some useful commands:

```bash theme={"system"}
npx squid-typeorm-migration apply # Apply the DB migrations
```

```bash theme={"system"}
npx squid-typeorm-migration generate # Generate a DB migration matching the TypeORM entities
```

```bash theme={"system"}
rm -r db/migrations # Clean the migrations folder
```

Inspect the full list of available options with

```bash theme={"system"}
npx squid-typeorm-migration --help
```

To generate or update the migrations after a schema change, follow the steps below.

### Updating after schema changes

In most cases the simplest way to update the schema is to drop the database and regenerate the migrations from scratch.

**1. Update schema.graphql**

**2. Regenerate the TypeORM entity classes**

```bash theme={"system"}
# generate entites code from the schema file
npx squid-typeorm-codegen
```

**3. Recreate the database**

```bash theme={"system"}
# drop the database
docker compose down
```

```bash theme={"system"}
# start a blank database
docker compose up -d
```

Note that without dropping the database the next step will generate a migration only for the schema difference.

**4. Build the squid code**

```bash theme={"system"}
npm run build
```

**5. Recreate the database migration**

```bash theme={"system"}
rm -r db/migrations
```

```bash theme={"system"}
npx squid-typeorm-migration generate
```

**6. Apply the database migration**

```bash theme={"system"}
npx squid-typeorm-migration apply
```

### Updating a deployed squid schema

In some rare cases it is possible to update the schema without dropping the database and restarting the squid from a blank state. The most important case is adding an index to an entity field. More complex changes are usually not feasible.

Updating a running squid requires that you add an incremental migration.

**1. Ensure that your local database is running and has the same schema as the database of your Cloud squid**

In most situations re-creating the database container and applying existing migrations should be enough:

```bash theme={"system"}
docker compose down
docker compose up -d
npx squid-typeorm-migration apply
```

**2. Update schema.graphql**

For example, [add an index](/en/sdk/squid-sdk/reference/schema-file/indexes-and-constraints)

**3. Regenerate the model classes and build the code**

```bash theme={"system"}
npx squid-typeorm-codegen
npm run build
```

**4. Add a new database migration**

```bash theme={"system"}
npx squid-typeorm-migration generate
```

This will create a new file in `db/migrations`. You may want to examine it before proceeding to the next step.

**5. Update the squid in Cloud**

If the squid is deployed to SQD Cloud, [update the deployed version](/en/sdk/squid-sdk/squid-cli/deploy).

If you're self-hosting it, update your remote codebase and run

```bash theme={"system"}
npx squid-typeorm-migration apply
```

### SQD Cloud deployment

By default, the TypeORM migrations are automatically applied by Cloud with the command `npx squid-typeorm-migration apply` before the squid services are started. For custom behavior, one can override the migration script using the optional `migrate:` section of [squid.yaml](/en/cloud/reference/manifest#deploy).

<Info>
  To force Cloud to reset the database and start with a blank state after a schema change, use the `--hard-reset` flag of [sqd deploy](/en/sdk/squid-sdk/squid-cli/deploy).
</Info>


# Self-hosting
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/self-hosting

Self-host Squid SDK indexers locally or on-premises with Docker — single-image setup for processor and GraphQL server with Postgres backing storage.

<Tip>
  Many tips from our [best practices guide](/en/cloud/resources/best-practices) apply to self-hosted squids, too. Check it out if you intend to use your squid in production.
</Tip>

To deploy a squid locally or on-premises, use the following Dockerfile template to build a single image for both `api` and `processor` services:

```dockerfile title="Dockerfile" theme={"system"}
FROM node:20-alpine AS node
FROM node AS node-with-gyp
FROM node-with-gyp AS builder
WORKDIR /squid
ADD package.json .
ADD package-lock.json .
# remove if needed
ADD assets assets
# remove if needed
ADD db db
# remove if needed
ADD schema.graphql .
RUN npm ci
ADD tsconfig.json .
ADD src src
RUN npm run build
FROM node-with-gyp AS deps
WORKDIR /squid
ADD package.json .
ADD package-lock.json .
RUN npm ci --production
FROM node AS squid
WORKDIR /squid
COPY --from=deps /squid/package.json .
COPY --from=deps /squid/package-lock.json .
COPY --from=deps /squid/node_modules node_modules
COPY --from=builder /squid/lib lib
# remove if no assets folder
COPY --from=builder /squid/assets assets
# remove if no db folder
COPY --from=builder /squid/db db
# remove if no schema.graphql is in the root
COPY --from=builder /squid/schema.graphql schema.graphql
# remove if no commands.json is in the root
ADD commands.json .
RUN echo -e "loglevel=silent\\nupdate-notifier=false" > /squid/.npmrc
RUN npm i -g @subsquid/commands && mv $(which squid-commands) /usr/local/bin/sqd
ENV PROCESSOR_PROMETHEUS_PORT 3000
```

Then build an image with

```bash theme={"system"}
docker buildx build . -t my-squid
```

or with

```bash theme={"system"}
docker build . -t my-squid
```

if you're using an older Docker version.

## Sample compose file

Once can the run the squid services with the freshly built image. Here is a sample `docker-compose` file:

```yaml theme={"system"}
services:
  db:
    image: postgres:15
    environment:
      POSTGRES_DB: squid
      POSTGRES_PASSWORD: postgres
    healthcheck:
      test: ["CMD-SHELL", "pg_isready", "-d", "squid"]
      interval: 5s
      timeout: 5s
      retries: 5
    # Uncomment for logging all SQL statements
    # command: ["postgres", "-c", "log_statement=all"]
  api:
    image: my-squid
    environment:
      - DB_NAME=squid
      - DB_PORT=5432
      - DB_HOST=db
      - DB_PASS=postgres
      - GQL_PORT=4350
    ports:
      # GraphQL endpoint at port 4350
      - "4350:4350"
    command: ["sqd", "serve:prod"]
    depends_on:
      db:
        condition: service_healthy
  processor:
    image: my-squid
    environment:
      - DB_NAME=squid
      - DB_PORT=5432
      - DB_HOST=db
      - DB_PASS=postgres
      # any other variables that your squid processor may be using
    ports:
      # prometheus metrics exposed at port 3000
      - "3000:3000"
    command: ["sqd", "process:prod"]
    depends_on:
      db:
        condition: service_healthy
```

Note that `sqd serve:prod` and `sqd process:prod` commands are defined in the [`commands.json` file](/en/sdk/squid-sdk/squid-cli/commands-json).


# Serving GraphQL
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/serving-graphql

Choose a GraphQL server for your Squid SDK indexer — OpenReader, Hasura, and PostGraphile compared, with deployment notes for SQD Cloud and self-host.

It is common (although not required) for squids to serve GraphQL APIs. Historically, the most common way to do that was to [persist the squid data to PostgreSQL](/en/sdk/squid-sdk/resources/persisting-data/typeorm), then attach [OpenReader](#openreader) to it. Although this is still supported, we encourage using [PostGraphile](#postgraphile) or [Hasura](#hasura) in new PostgreSQD-based projects. See [OpenReader's limitations](/en/sdk/squid-sdk/reference/openreader-server/overview#limitations) if you're curious about our motivation.

## PostGraphile

[PostGraphile](https://www.graphile.org/postgraphile/) is an open-source tool that builds powerful, extensible and performant GraphQL APIs from PostgreSQL schemas. Its pros include:

* aggregations;
* reliable support for subscriptions;
* capability for deep API customization;
* organization of API customization code into plugins.

The recommended way of integrating PostGraphile into squid projects is by making a dedicated entry point at `src/api.ts`. A complete example squid implementing this approach is available in [this repository](https://github.com/subsquid-labs/squid-postgraphile-example/).

With this entry point in place, we [create a `sqd` command](https://github.com/subsquid-labs/squid-postgraphile-example/blob/f1fd1691eb59da2c9d57c475a71d0ed44cfed891/commands.json#L58) for running PostGraphile with [`commands.json`](/en/sdk/squid-sdk/squid-cli/commands-json), then use it in the [`deploy.api` entry](https://github.com/subsquid-labs/squid-postgraphile-example/blob/f1fd1691eb59da2c9d57c475a71d0ed44cfed891/squid.yaml#L15) of [Squid manifest](/en/cloud/reference/manifest). Although none of this is required, this makes it easier to run the squid both locally (with [`sqd run`](/en/sdk/squid-sdk/squid-cli/run)) and in the [Cloud](/en/cloud).

As per usual with PostGraphile installations, you can freely extend it with plugins, including your own. Here is an [example plugin for serving the `_squidStatus` queries](https://github.com/subsquid-labs/squid-postgraphile-example/blob/f1fd1691eb59da2c9d57c475a71d0ed44cfed891/src/api.ts#L11) from the standard Squid SDK GraphQL server schema. A plugin for making PostGraphile API fully compatible with [old APIs](/en/sdk/squid-sdk/reference/openreader-server/api) served by OpenReader will be made available soon.

## Hasura

[Hasura](https://hasura.io) is a powerful open-source GraphQL engine. You can use it to:

* expose multiple data sources of different kinds (various databases, APIs etc) via a single API;
* reliably serve subscriptions;
* perform aggregations;
* deeply customize your CRUD API.

You can integrate Hasura with your squid in two ways:

1. **Use Hasura to gather data from multiple sources, including your squid.**

   For this scenario we recommend separating your Hasura instance from your squid, which should consist of just one service, [the processor](/en/sdk/squid-sdk/reference/processors/architecture), plus the database. Supply your database credentials to Hasura, then configure it to produce the desired API.

   If you run your squid in our [Cloud](/en/cloud) you can find database credentials in [the app](https://app.subsquid.io/squids):2. **Run a dedicated Hasura instance for serving the data just from your squid.**

   A complete example implementing this approach is available in [this repository](https://github.com/subsquid-labs/squid-hasura-example). Here's how it works:

   * Locally, Hasura runs in a [Docker container](https://github.com/subsquid-labs/squid-hasura-example/blob/70bb6d703dc90c1bb00b47f3fef7f388ab54e565/docker-compose.yml#L14C1-L28C20). In the Cloud it is managed via the [Hasura addon](/en/cloud/reference/hasura).
   * Hasura metadata is shared among all squid instances by means of the [Hasura configuration tool](/en/sdk/squid-sdk/resources/tools/hasura-configuration). The tool can automatically create an initial configuration based on your [TypeORM models](/en/sdk/squid-sdk/reference/schema-file/intro#typeorm-codegen), then persist any changes you might make with the web GUI and metadata exports.
   * Admin authentication secret is set via the `HASURA_GRAPHQL_ADMIN_SECRET`. The variable is set in `.env` locally and from a [secret](/en/cloud/resources/env-variables#secrets) in Cloud deployments.

   See the [configuration tool page](/en/sdk/squid-sdk/resources/tools/hasura-configuration) and the [repo readme](https://github.com/subsquid-labs/squid-hasura-example#readme) for more details.

## OpenReader

[OpenReader](/en/sdk/squid-sdk/reference/openreader-server) is a GraphQL server developed by the SQD team. Although still supported, it's not recommeded for new PostgreSQL-powered projects due to its [limitations](/en/sdk/squid-sdk/reference/openreader-server/overview#limitations), especially for APIs implementing GraphQL subscriptions.

The server uses the [schema file](/en/sdk/squid-sdk/reference/schema-file) to produce its [core API](/en/sdk/squid-sdk/reference/openreader-server/api) that can be extended with [custom resolvers](/en/sdk/squid-sdk/reference/openreader-server/configuration/custom-resolvers). Extra features include [DoS protection](/en/sdk/squid-sdk/reference/openreader-server/configuration/dos-protection) and [caching](/en/sdk/squid-sdk/reference/openreader-server/configuration/caching).


# Hasura configuration tool
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/tools/hasura-configuration

Configure Hasura for a Squid SDK indexer with the hasura-configuration tool — auto-generate metadata from your schema and apply it to a Hasura instance.

<Info>
  For info on migrating to `@subsquid/hasura-configuration>=2.0.0` see [this page](/en/sdk/squid-sdk/resources/migrate/migrate-to-hasura-configuration-tool-v2).
</Info>

[`@subsquid/hasura-configuration`](https://www.npmjs.com/package/@subsquid/hasura-configuration) is a tool for managing Hasura configuration in [PostgreSQL-powered squids](/en/sdk/squid-sdk/resources/persisting-data/typeorm). Install it with

```bash theme={"system"}
npm i @subsquid/hasura-configuration
```

Make sure that the following environment variables are set:

* `HASURA_GRAPHQL_ENDPOINT` for Hasura URL (defaults to `http://localhost:8080`).
* `HASURA_GRAPHQL_ADMIN_SECRET` for admin access (only required to use `squid-hasura-configuration apply`).
* If your Hasura instance(s) use a role other than `public` to serve the anonymous part of your API, also set `HASURA_GRAPHQL_UNAUTHORIZED_ROLE`.

## Generating the initial configuration

The tool uses your squid's [TypeORM models](/en/sdk/squid-sdk/reference/schema-file/intro#typeorm-codegen) as input when generating the initial configuration. Make sure they are up to date.

<Accordion title="Here's how">
  1. Finalize any edits to [`schema.graphql`](/en/sdk/squid-sdk/reference/schema-file)

  2. Update the TypeScript code of your models with
     ```bash theme={"system"}
     npx squid-typeorm-codegen
     ```

  3. Compile your models with
     ```bash theme={"system"}
     npm run build
     ```

  4. Regenerate your [DB migrations](/en/sdk/squid-sdk/resources/persisting-data/typeorm#database-migrations) with a clean database to make sure they match your updated schema.
     ```bash theme={"system"}
     docker compose down
     docker compose up -d
     rm -r db
     npx squid-typeorm-migration generate
     ```

  You can turn off your database after doing that - Hasura configuration tool does not use it to make its initial configuration
</Accordion>

When done, run

```bash theme={"system"}
npx squid-hasura-configuration regenerate
```

The generated configuration will be available at `hasura_metadata.json`. It enables:

* tracking all tables that correspond to [schema entities](/en/sdk/squid-sdk/reference/schema-file/entities);
* `SELECT` permissions for the `public` (or `$HASURA_GRAPHQL_UNAUTHORIZED_ROLE` if it is defined) role for all columns in these tables;
* tracking all [entity relationships](/en/sdk/squid-sdk/reference/schema-file/entity-relations).

## Applying the configuration

Make sure your database is up, your Hasura instance is connected to it and the schema is up to date. If necessary, apply the migrations:

```bash theme={"system"}
npx squid-typeorm-migration apply
```

When done, you can apply the generated config with

```bash theme={"system"}
npx squid-hasura-configuration apply
```

or import it using the *Settings > Metadata Actions > Import metadata* function of the web GUI.

## Persisting configuration changes

<Warning>
  Regenerating `hasura_metadata.json` removes any modifications you might have made via metadata exporting. So, it is advisable that you finalize your schema before you begin any manual API fine-tuning.
</Warning>

When running a squid with a dedicated Hasura instance you will notice that squid resetting operations (`docker compose down; docker compose up -d` and `sqd deploy -r`) restore your Hasura API to its non-configured state. As you develop your API further you may want to persist your changes. `squid-hasura-configuration` helps with that by being compatible with the *Settings > Metadata Actions > Import/Export metadata* functions of the web GUI.

Any extra configuration you may make via the web GUI or [Hasura metadata API](https://hasura.io/docs/2.0/api-reference/metadata-api/index) can be persisted by exporting the metadata to `hasura_metadata.json` via the *Export metadata* function, then applying it to blank Hasura instances with

```bash theme={"system"}
npx squid-hasura-configuration apply
```

## Example

See [this repo](https://github.com/subsquid-labs/squid-hasura-example).


# TypeORM migration generation
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/tools/migrations-gen

Generate Postgres migrations from your Squid SDK schema with sqd codegen — apply schema changes incrementally without dropping your indexed data.

The [`typeorm-migration`](https://github.com/subsquid/squid-sdk/tree/master/typeorm/typeorm-migration) tool is used to generate, apply and revert database migrations. It follows the conventions below.

* The migrations are generated in the `db/migrations` folder.
* The database connection is inferred from the `DB_XXX` environment variables.
* All entities should be exported from `lib/model` commonjs module, i.e. the entity classes must be compiled from TypeScript.

Here are some useful commands:

```bash theme={"system"}
npx squid-typeorm-migration apply # apply pending migrations
```

```bash theme={"system"}
npx squid-typeorm-migration generate # generate the migration for the schema defined in schema.graphql
```

```bash theme={"system"}
rm -r db/migrations # clean the db/migrations folder
```


# TypeORM model generation
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/tools/model-gen

Generate TypeORM model classes from a Squid SDK schema.graphql file with the model-gen tool — used by the typeorm-store and consumed by the processor.

TypeORM entities can be automatically generated from the [schema file](/en/sdk/squid-sdk/reference/schema-file) defined in `schema.graphql`.
The tool is called [`squid-typeorm-codegen(1)`](https://github.com/subsquid/squid-sdk/tree/master/typeorm/typeorm-codegen) and has no additonal options.

Install with

```sh theme={"system"}
npm i @subsquid/typeorm-codegen --save-dev
```

Invoke with

```sh theme={"system"}
npx squid-typeorm-codegen
```


# Squid generation tools
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/tools/squid-gen

Squid SDK code generation tools — squid-gen and abi2squid generate complete indexer projects from contract ABIs for any EVM chain.

<Warning>
  As of 2025-01-01 we no longer maintain the old `squid-gen` tool.
</Warning>

SQD provides [tools](https://github.com/subsquid/squid-gen) for generating ready-to-use squids that index events and function calls of smart contracts. EVM/Solidity and WASM/ink! smart contracts are supported. The tools can be configured to make squids that save data to a [PostgreSQL database](/en/sdk/squid-sdk/resources/persisting-data/typeorm) or to a [file-based dataset](/en/sdk/squid-sdk/resources/persisting-data/file). All that is required is NodeJS, [Squid CLI](/en/sdk/squid-sdk/squid-cli/installation) and, if your squid will be using a database, Docker.

Squid generation procedure is very similar for both contract types. Here are the steps:

1. Create a new blank squid with `sqd init` using a suitable template:
   ```bash theme={"system"}
   # for EVM/Solidity contracts
   sqd init my-squid -t abi
   # OR
   # for WASM/ink! contracts
   sqd init my-squid -t https://github.com/subsquid-labs/squid-ink-abi-template
   ```
   Enter the squid folder and install the dependencies:
   ```bash theme={"system"}
   cd my-squid
   npm ci
   ```

2. Write the [configuration](#configuration) of the future squid to `squidgen.yaml`. Retrieve any necessary contract ABIs and store them at `./abi`.

   Alternatively, skip to the next step and specify the configuration via CLI. **Note:** some features will not be available.

3. Generate and build the squid code:
   ```bash theme={"system"}
   npx squid-gen config squidgen.yaml
   ```
   ```bash theme={"system"}
   npm run build
   ```
   If you chose to configure the tool via CLI instead, do so now. Here's an example:
   ```bash theme={"system"}
   npx squid-gen-abi \
     --address 0x2E645469f354BB4F5c8a05B3b30A929361cf77eC \
     --archive https://v2.archive.subsquid.io/network/ethereum-mainnet \
     --event NewGravatar \
     --event UpdatedGravatar \
     --function '*' \
     --from 6000000
   ```
   See `npx squid-gen-abi --help` for all options.

4. Prepare your squid for launching. If it is using a database, start a PostgreSQL container, then regenerate and apply migrations:
   ```bash theme={"system"}
   docker compose up -d
   ```
   ```bash theme={"system"}
   npx squid-typeorm-migration generate
   ```
   ```bash theme={"system"}
   npx squid-typeorm-migration apply
   ```
   If it is storing its data to a dataset, [strip the project folder of database-related facilities](#strip-the-squid-folder-for-file-store) that are no longer needed.

5. Test the complete squid by running it locally. Start a [processor](/en/sdk/squid-sdk/overview) with
   ```bash theme={"system"}
   node -r dotenv/config lib/main.js
   ```
   If your squid will be serving GraphQL also run `npx squid-graphql-server` in a separate terminal. Make sure that the squid saves the requested data to its target:
   * if it is serving GraphQL, visit the local [GraphiQL playground](http://localhost:4350/graphql);
   * for PostgreSQL-based squids you can also connect to the database with `PGPASSWORD=postgres psql -U postgres -p 23798 -h localhost squid` and take a look at the contents;
   * if it is storing data to a file-based dataset, wait for the first filesystem sync then verify that all the expected files are present and contain the expected data. If your squid produces data at a low rate, you may have to tweak the [`chunkSizeMb` setting](/en/sdk/squid-sdk/resources/persisting-data/file#overview) and/or add a [`ctx.store.setForceFlush()`](/en/sdk/squid-sdk/resources/persisting-data/file#setforceflush) call to manually write dataset chunks at appropriate intervals.

At this point your squid is ready. You can run it on your own infrastructure or [deploy it to SQD Cloud](/en/cloud).

## Configuration

A valid config for the `squid-gen config` is a YAML file with the following sections:

* **archive** is an endpoint URL of a [SQD Network](/en/network/overview) gateway. Find an appropriate gateway at the [Supported networks](/en/data/evm) page or with [`sqd gateways`](/en/sdk/squid-sdk/squid-cli/gateways).

* **target** section describes how the scraped data should be stored. Set
  ```yaml theme={"system"}
  target:
    type: postgres
  ```
  to use a PostgreSQL database that can be presented to users as a GraphQL API or used as-is. Another option is to [store the data to a file-based dataset](#file-store-targets).

* **contracts** is a list of contracts to be indexed. Define the following fields for each contract:
  * **name**
  * **address**
  * **range** (optional): block range to be indexed. An object with `from` and `to` properties, each of which can be omitted. Defaults to indexing the whole chain.
  * **abi** (optional on EVM): path to the contract JSON ABI. If omitted for an EVM contract, the tool will attempt to fetch the ABI by address from the Etherscan API or a compatible alternative set by the `etherscanApi` root option.
  * **proxy** (EVM-only): when indexing a [proxy contract](https://eips.ethereum.org/EIPS/eip-1967) for events or calls defined in the implementation, set this option to its address and the `address` option to the address of the implementation contract. That way the tool will retrieve the ABI of the implementation and use it to index the output of the proxy.
  * **events** (optional): a list of events to be indexed or a boolean value. Set to `true` to index all events defined in the ABI. Defaults to `false`, meaning that no events are to be indexed.
  * **functions** (EVM-only, optional): a list of functions the calls of which are to be indexed or a boolean value. Set to `true` to index calls of all functions defined in the ABI. Defaults to `false`, meaning that no function calls are to be indexed.

* **etherscanApi** (EVM-only, optional): Etherscan API-compatible endpoint to fetch contract ABI by a known address. Default: [https://api.etherscan.io/](https://api.etherscan.io/).

## `file-store` targets

Currently the only [file-based data target type](/en/sdk/squid-sdk/resources/persisting-data/file) supported by `squid-gen` packages is [`parquet`](/en/sdk/squid-sdk/reference/store/file/parquet). When used, it requires that the `path` field is also set alongside `type`. A `path` can be a local path or an URL pointing to a folder within a bucket on an S3-compatible cloud service.

Support for `file-store` is in alpha stage. Known caveats:

* If a S3 URL is used, then the S3 region, endpoint and user credentials will be [taken from the default environment variables](/en/sdk/squid-sdk/reference/store/file/s3-dest). Fill your `.env` file and/or set your [SQD Cloud secrets](/en/cloud/resources/env-variables) accordingly.

* Unlike their PostgreSQL-powered equivalents, the squids that use `file-store` may not write their data often. You may have to configure the `chunkSizeMb` parameter of the `Database` class and/or call [`ctx.store.setForceFlush()`](/en/sdk/squid-sdk/resources/persisting-data/file#setforceflush) when appropriate to strike an acceptable balance between the lag of the indexed data and the number of files in the resulting dataset. See [Filesystem store overview](/en/sdk/squid-sdk/resources/persisting-data/file) for details.

* For `parquet` targets, the [`Decimal(38)`](/en/sdk/squid-sdk/reference/store/file/parquet#columns) column type is used by the code generator to represent `uint256`. This is done for compatibility reasons: very few tools seem to support reading wider decimals from Parquet files. If you're getting a lot of errors containing `value ... does not fit into Decimal(38, 0)`, consider replacing the `Decimal(38)` column type with `Decimal(78)` or `String()` at `src/table.ts`.

* At the moment, squids generated with file-based data targets contain a lot of facilities for managing the database and have to be [stripped](#strip-the-squid-folder-for-file-store) of them before use.

## Strip the squid folder for `file-store`

Steps to convert a squid made with a database-enabled template for use with `file-store`:

1. Remove unneeded files and packages.

```bash theme={"system"}
rm docker-compose.yml
npm uninstall @subsquid/graphql-server @subsquid/typeorm-migration @subsquid/typeorm-store @subsquid/util-internal-json pg typeorm @subsquid/typeorm-codegen
```

2. Replace `commands.json` with the one from the [file-store-parquet-example repo](https://github.com/subsquid-labs/file-store-parquet-example).

```bash theme={"system"}
curl -o commands.json https://raw.githubusercontent.com/subsquid-labs/file-store-parquet-example/main/commands.json
```

3. In `squid.yaml`, remove the `deploy.addons` section and replace the `deploy.api` section with

```bash theme={"system"}
  api:
    cmd: [ "sleep", "3600" ]
```

4. Install any required `file-store` packages.

```bash theme={"system"}
# if target.type was `parquet`
npm install @subsquid/file-store-parquet
# if target.path was an S3 URL
npm install @subsquid/file-store-s3
```

## Configuration examples

### EVM/Solidity

* Index `LiquidationCall` events of the [AAVE V2 Lending Pool contract](https://etherscan.io/address/0x7d2768dE32b0b80b7a3454c06BdAc94A69DDc7A9), starting from block 11362579 when it was deployed. Save the results to PostgreSQL. Use the ABI located at `./abi/aave-v2-pool.json`.
  ```yaml theme={"system"}
  archive: eth-mainnet
  target:
    type: postgres
  contracts:
    - name: aave-v2-pool
      address: "0x7d2768dE32b0b80b7a3454c06BdAc94A69DDc7A9"
      abi: ./abi/aave-v2-pool.json
      range:
        from: 11362579
      events:
        - LiquidationCall
  ```

* Index events and function calls by the [DPX contract](https://arbiscan.io/address/0x6c2c06790b3e3e3c38e12ee22f8183b37a13ee55) (a proxy) on Arbitrum, based on the ABI of the [implementation contract](https://arbiscan.io/address/0x3f770ac673856f105b586bb393d122721265ad46) retrieved from [Arbiscan](https://arbiscan.io) API. Save the results to Parquet files at './data'.
  ```yaml theme={"system"}
  archive: arbitrum
  target:
    type: parquet
    path: ./data
  contracts:
    - name: dpx
      address: "0x3f770Ac673856F105b586bb393d122721265aD46"
      proxy: "0x6C2C06790b3E3E3c38e12Ee22F8183b37a13EE55"
      events: true
      functions: true
  etherscanApi: https://api.arbiscan.io/
  ```
  **Note:** this example is known to run into the integer length issue described in the [`file-store` targets](#file-store-targets) section. One way to make it work is to widen all `Decimal` column types from 38 to 78 symbols:
  ```bash theme={"system"}
  sed -i -e 's/38/78/g' src/table.ts
  ```

* Index all events and function calls of the [Positions NFT](https://etherscan.io/address/0xc36442b4a4522e871399cd717abdd847ab11fe88) and [Factory](https://etherscan.io/address/0x1f98431c8ad98523631ae4a59f267346ea31f984) contracts of Uniswap, send the results to the `uniswap-data` folder of the `subsquid-testing-data` bucket.
  ```yaml theme={"system"}
  archive: eth-mainnet
  target:
    type: parquet
    path: s3://subsquid-testing-bucket/uniswap-data
  contracts:
    - name: positions
      address: "0xC36442b4a4522E871399CD717aBDD847Ab11FE88"
      events: true
      functions: true
    - name: factory
      address: "0x1F98431c8aD98523631AE4a59f267346ea31F984"
      events: true
      functions: true
  ```
  **Notes:**
  * This example is also susceptible to the integer length issue and will drop at least two events if used as-is, without widening the column types.
  * The generated squid requires some variables to be set to connect to S3. Here's an example of what `.env` may look like:
    ```bash theme={"system"}
    S3_REGION=us-east-1
    S3_ENDPOINT=https://s3.filebase.com
    S3_ACCESS_KEY_ID=myAccessKeyId
    S3_SECRET_ACCESS_KEY=mySecretAccessKey
    ```

### WASM/ink!

* Index `Transfer` events emitted by an ERC20 contract on Shibuya, save results to PostgreSQL. Do not forget to use the [ink-abi template](https://github.com/subsquid-labs/squid-ink-abi-template)!
  ```yaml theme={"system"}
  archive: shibuya
  target:
    type: postgres
  contracts:
    - name: testToken
      abi: "./abi/erc20.json"
      address: "0x5207202c27b646ceeb294ce516d4334edafbd771f869215cb070ba51dd7e2c72"
      events:
        - Transfer
  ```
  **Note:** you can get the ABI from the `squid-gen` repository:
  ```bash theme={"system"}
  curl -o abi/erc20.json https://raw.githubusercontent.com/subsquid/squid-gen/master/tests/ink-erc20/abi/erc20.json
  ```


# Data decoding
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/tools/typegen/decoding

Use generated typegen modules to decode EVM and Substrate data in Squid SDK — type-safe accessors for events, calls, and storage queries.

<Tabs>
  <Tab title="squid-evm-typegen">
    Decoding events:

    ```ts theme={"system"}
    import * as depositsAbi from './abi/deposits'

    processor.run(new TypeormDatabase(), async (ctx) => {
      for (let c of ctx.blocks) {
        for (let log of c.logs) {
          if (log.address === CONTRACT_ADDRESS &&
              log.topics[0] == depositsAbi.events.Deposit.topic) {

            // type-safe decoding of the Deposit event data
            const amt = depositsAbi.events.Deposit.decode(log).wad
          }
        }
      }
    })
    ```

    Similar to `events`, transaction access is provided by the `functions` object for each contract method defined in the ABI.
  </Tab>

  <Tab title="squid-substrate-typegen">
    To decode events and calls, first determine the appropriate runtime version with `.is()`, then decode them with `.decode()`:

    ```typescript theme={"system"}
    import {events, calls} from './types'

    processor.run(new TypeormDatabase(), async ctx => {
      for (let block of ctx.blocks) {
         for (let event of block.events) {
          if (event.name == events.balances.transfer.name) {
            let rec: {from: Bytes, to: Bytes, amount: bigint}
            if (events.balances.transfer.v1020.is(event)) {
              let [from, to, amount, fee] =
                events.balances.transfer.v1020.decode(event)
              rec = {from, to, amount}
            }
            // ... decode all runtime versions similarly
            // with events.balances.transfer.${ver}.is/.decode
          }
        }
        for (let call of block.calls) {
          if (call.name == calls.balances.forceTransfer.name) {
            let rec: {source: Bytes, dest: Bytes, value: bigint} | undefined
            if (calls.balances.forceTransfer.v1020.is(call)) {
              let res =
                calls.balances.forceTransfer.v1020.decode(call)
              assert(res.source.__kind === 'AccountId')
              assert(res.dest.__kind === 'AccountId')
              rec = {
                source: res.source.value,
                dest: res.dest.value,
                value: res.value
              }
            }
            // ... decode all runtime versions similarly
            // with calls.balances.forceTransfer.${ver}.is/.decode
        }
      }
    })
    ```
  </Tab>

  <Tab title="squid-ink-typegen">
    Here is an example of a utility module generated by the ink! typegen:

    ```typescript title="src/abi/erc20.ts" theme={"system"}
    const _abi = new Abi(metadata)

    export function decodeEvent(hex: string): Event {
      return _abi.decodeEvent(hex)
    }

    export function decodeMessage(hex: string): Message {
      return _abi.decodeMessage(hex)
    }

    export function decodeConstructor(hex: string): Constructor {
      return _abi.decodeConstructor(hex)
    }
    ```

    The usage in a batch handler is straightforward:

    ```ts theme={"system"}
    processor.run(new TypeormDatabase(), async ctx => {
      for (let block of ctx.blocks) {
        for (let event of block.events) {
          if (event.name==='Contracts.ContractEmitted' &&
              event.args.contract===CONTRACT_ADDRESS) {

            let event = erc20.decodeEvent(event.args.data)
            if (event.__kind==='Transfer') {
              // event is of type `Event_Transfer`
            }
          }
        }
      }
    })
    ```
  </Tab>
</Tabs>


# Generating utility modules
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/tools/typegen/generation

Generate task-specific utility modules with Squid SDK typegen — produce typed clients for EVM ABIs and Substrate metadata for type-safe indexing.

<Tabs>
  <Tab title="squid-evm-typegen">
    The `squid-evm-typegen(1)` tool generates TypeScript facades for EVM transactions, logs and `eth_call` queries.

    The generated facades are assumed to be used by squids indexing EVM data.

    The tool takes a JSON ABIs as an input. Those can be specified in three ways:

    1. as a plain JSON file(s):

       ```bash theme={"system"}
       npx squid-evm-typegen src/abi abi/erc20.json
       ```

       To include all files in `./abi` and add an interface for the Multicall contract, run

       ```bash theme={"system"}
       npx squid-evm-typegen ./src/abi ./abi/*.json --multicall
       ```

       You can get JSON ABIs for explorer (Etherscan, Bscscan etc) verified contract by visiting the contract page, going to the "Contract" tab and scrolling down to the "Contract ABI" section. *Do not use the "Export ABI" function!* Copy the contents to the clipboard and paste them to a new JSON file.

    2. **(requires an Etherscan API key)** as a contract address. One can pass multiple addresses at once.

       ```bash theme={"system"}
       npx squid-evm-typegen --etherscan-api-key <your_key> src/abi 0xBB9bc244D798123fDe783fCc1C72d3Bb8C189413
       ```

    <Info>
      Please check if your contract is a proxy when using this method. If it is,
      consult [this page](/en/sdk/squid-sdk/resources/evm/proxy-contracts) for guidance.
    </Info>

    3. as an arbitrary URL:

       ```bash theme={"system"}
       npx squid-evm-typegen src/abi https://example.com/erc721.json
       ```

    In all cases typegen will use `basename` of the ABI as the root name for the generated files. You can change the basename of generated files using the fragment `(#)` suffix.

    ```bash theme={"system"}
    squid-evm-typegen src/abi 0xBB9bc244D798123fDe783fCc1C72d3Bb8C189413#my-contract-name
    ```

    **Arguments:**

    |              |                                                                                         |
    | ------------ | --------------------------------------------------------------------------------------- |
    | `output-dir` | output directory for generated definitions                                              |
    | `abi`        | A contract address, an URL or a local path to the ABI file. Accepts multiple contracts. |

    **Options:**

    |                         |                                                                                                                                                                                                                                  |
    | ----------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
    | `--multicall`           | generate a facade for the MakerDAO multicall contract. May significantly improve the performance of contract state calls by [batching RPC requests](/en/sdk/squid-sdk/resources/tools/typegen/state-queries#batch-state-queries) |
    | `--etherscan-api <url>` | etherscan API to fetch contract ABI by a known address. By default, `https://api.etherscan.io/`                                                                                                                                  |
    | `--clean`               | delete output directory before the run                                                                                                                                                                                           |
    | `-h, --help`            | display help for command                                                                                                                                                                                                         |

    <Warning>
      The generated modules depend on `@subsquid/evm-abi`. Please add it to your
      `package.json` as a peer dependency if it's not there already: `bash npm i
              @subsquid/evm-abi `
    </Warning>

    ## Usage

    The generated utility modules have three intended uses:

    1. Constants: EVM log topic0 values and sighashes for transactions. Example:

       ```ts theme={"system"}
       // generated by evm-typegen
       import * as weth from "./abi/weth";

       const CONTRACT_ADDRESS =
         "0xC02aaA39b223FE8D0A0e5C4F27eAD9083C756Cc2".toLowerCase();

       const processor = new EvmBatchProcessor()
         .setGateway("https://v2.archive.subsquid.io/network/ethereum-mainnet")
         .addLog({
           address: [CONTRACT_ADDRESS],
           topic0: [weth.events.Deposit.topic, weth.events.Withdrawal.topic],
         });
       ```

    2. [Decoding of EVM logs and transactions](/en/sdk/squid-sdk/resources/tools/typegen/decoding)

    3. [Direct chain state queries](/en/sdk/squid-sdk/resources/tools/typegen/state-queries), including [queries to multicall](/en/sdk/squid-sdk/resources/tools/typegen/state-queries#batch-state-queries).
  </Tab>

  <Tab title="squid-substrate-typegen">
    The substrate typegen tool is a part of Squid SDK. It generates TypeScript wrappers for interfacing Substrate events and calls.

    Usage:

    ```bash theme={"system"}
    npx squid-substrate-typegen typegen.json
    ```

    If necessary, multiple config files can be supplied:

    ```bash theme={"system"}
    npx squid-substrate-typegen typegen0.json typegen1.json ...
    ```

    The structure of the `typegen.json` config file is best illustrated with an example:

    ```json theme={"system"}
    {
      "outDir": "src/types",
      "specVersions": "https://v2.archive.subsquid.io/metadata/kusama",
      "pallets": {
        // add one such section for each pallet
        "Balances": {
          "events": [
            // list of events to generate wrappers for
            "Transfer"
          ],
          "calls": [
            // list of calls to generate wrappers for
            "transfer_allow_death"
          ],
          "storage": ["Account"],
          "constants": ["ExistentialDeposit"]
        }
      }
    }
    ```

    The `specVersions` field is either

    * a metadata service endpoint URL, like
      ```
      https://v2.archive.subsquid.io/metadata/{network}
      ```
      or
    * a path to a [`jsonl`](https://jsonlines.org) file generated by [`substrate-metadata-explorer(1)`](https://github.com/subsquid/squid-sdk/tree/master/substrate/substrate-metadata-explorer).

    To generate all items defined by a given pallet, set any of the `events`, `calls`, `storage` or `constants` fields to `true`, e.g.

    ```json theme={"system"}
    {
      "outDir": "src/types",
      "specVersions": "kusamaVersions.jsonl",
      "pallets": {
        "Balances": {
          // generate wrappers for all Balances pallet constants
          "constants": true
        }
      }
    }
    ```

    <Info>
      In the rare cases when the typegen needs a types bundle, supply it alongside metadata:

      ```json theme={"system"}
      {
        "outDir": "src/types",
        "specVersions": "westendVersions.jsonl",
        "typesBundle": "westendTypes.json",
        ...
      }
      ```
    </Info>

    ## TypeScript wrappers

    Wrappers generated by the typegen command can be found in the specified `outDir` (`src/types` by convention). Assuming that this folder is imported as `types` (e.g. with `import * as types from './types'`), you'll be able to find the wrappers at:

    * for events: `types.events.${palletName}.${eventName}`
    * for calls: `types.calls.${palletName}.${callName}`
    * for storage items: `types.storage.${palletName}.${storageItemName}`
    * for constants: `types.constants.${palletName}.${constantName}`

    All identifiers (pallet name, call name etc) are lowerCamelCased. E.g. the constant `Balances.ExistentialDeposit` becomes `types.events.balances.existentialDeposit` and the call `Balances.set_balance` becomes `types.calls.setBalance`.

    ## Usage

    1. Item name constants (e.g. `events.balances.transfer.name`).

    2. Runtime versioning-aware [decoding](/en/sdk/squid-sdk/resources/tools/typegen/decoding).

    3. [Chain storage queries](/en/sdk/squid-sdk/resources/tools/typegen/state-queries)

    4. Runtime constants:
       ```typescript theme={"system"}
       import {constants} from './types'
       // ...
       processor.run(new TypeormDatabase(), async ctx => {
         for (let block of ctx.blocks) {
           if (constants.balances.existentialDeposit.v1020.is(block.header)) {
             let c = new constants.balances.existentialDeposit.v1020.get(block.header)
             ctx.log.info(`Balances.ExistentialDeposit (runtime version V1020): ${c}`)
           }
         }
       })
       ```
  </Tab>

  <Tab title="squid-ink-typegen">
    Use [`squid-ink-typegen`](https://github.com/subsquid/squid-sdk/tree/master/substrate/ink-typegen) to generate facade classes for decoding ink! smart contract data from JSON ABI metadata.

    Usage:

    ```bash theme={"system"}
    npx squid-ink-typegen --abi abi/erc20.json --output src/abi/erc20.ts
    ```

    The generated source code exposes the [decoding methods](/en/sdk/squid-sdk/resources/tools/typegen/decoding), some useful types and a class for performing [wrapped RPC queries](/en/sdk/squid-sdk/resources/tools/typegen/state-queries), using `@subsquid/ink-abi` under the hood
  </Tab>
</Tabs>


# Direct RPC queries
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/tools/typegen/state-queries

Query chain state directly via wrapped RPC calls in Squid SDK — typegen-generated state-call wrappers for fetching contract storage and balances.

<Tabs>
  <Tab title="squid-evm-typegen">
    In order to make on-chain calls, one must set up a JSON-RPC endpoint using [setRpcEndpoint()](/en/sdk/squid-sdk/reference/processors/evm-batch/general#set-rpc-endpoint). We recommend using a private endpoint from e.g. [BlastAPI](https://blastapi.io/) or the SQD Cloud's [RPC addon](/en/cloud/resources/rpc-proxy), and set it via an environment variable:

    ```ts theme={"system"}
    //...
      .setRpcEndpoint(process.env.RPC_ETH_HTTP)
    //...
    ```

    You can define the `RPC_ETH_HTTP` in three ways:

    * for local runs, simply update the local `.env` file;
    * for squids deployed to Cloud define it as a [secret](/en/cloud/resources/env-variables) on your Cloud account;
    * if you are using the [RPC addon](/en/cloud/resources/rpc-proxy), leave it to the Cloud to define it for you.

    ## `Contract` class

    The EVM contract state is accessed using the `Contract` class generated by `squid-evm-typegen`. It takes a handler context and a contract address as constructor arguments. The state is always accessed at the context block height unless explicitly defined in the constructor.

    For example, assume that we index an ERC721 contract. Typescript ABI module generated with `squid-evm-typegen` will contain the following class:

    ```typescript theme={"system"}
    export class Contract extends ContractBase {
      //...
      balanceOf(owner: string): Promise<ethers.BigNumber> {
        return this.call('balanceOf', [owner])
      }
      //...
    }
    ```

    Now suppose we want to query our contract from the batch handler. To create a `Contract` pass the context object and the current `Block` to its constructor, then query the contract state at that block:

    ```ts theme={"system"}
    // ...
    const CONTRACT_ADDRESS = '0xb654611f84a8dc429ba3cb4fda9fad236c505a1a'

    processor.run(new TypeormDatabase(), async ctx => {
      for (const block of ctx.blocks) { 
        const contract = new abi.Contract(ctx, block.header, CONTRACT_ADDRESS)
        // query the contract state at the current block
        const uri = await contract.balanceOf('0xd8da6bf26964af9d7eed9e03e53415d37aa96045')
        // ...
      }
    })
    ```

    ## Batch state queries

    The [MakerDAO Multicall contract](https://github.com/makerdao/multicall) was designed to batch multiple state queries into a single contract call. In the context of indexing, it normally significantly improves the indexing speed since JSON RPC calls are typically the bottleneck.

    Multicall contracts are deployed in many EVM chains, see the [contract repo](https://github.com/makerdao/multicall) for addresses. You can use any of them with a `multicall` Typescript module that is generated when running `squid-evm-typegen` with `--multicall` option. The module exports a `Multicall` class with this method:

    ```ts theme={"system"}
    tryAggregate<Args extends any[], R>(
      func: Func<Args, {}, R>,
      calls: [address: string, args: Args][],
      paging?: number
    ): Promise<MulticallResult<R>[]>
    ```

    The arguments are as follows:

    * `func`: the contract function to be called
    * `calls`: an array of tuples `[contractAddress: string, args]`. Each specified contract will be called with the specified arguments.
    * `paging` an (optional) argument for the maximal number of calls to be batched into a single JSON PRC request. Note that large page sizes may cause timeouts.

    A typical usage is as follows:

    ```ts title="src/main.ts" theme={"system"}
    // generated by evm-typegen
    import { functions } from './abi/mycontract'
    import { Multicall } from './abi/multicall'

    const MY_CONTRACT = '0xac5c7493036de60e63eb81c5e9a440b42f47ebf5'
    const MULTICALL_CONTRACT = '0x5ba1e12693dc8f9c48aad8770482f4739beed696'

    processor.run(new TypeormDatabase(), async (ctx) => {
      for (let c of ctx.blocks) {
        // some logic
      }
      const lastBlock = ctx.blocks[ctx.blocks.length - 1]
      const multicall = new Multicall(
        ctx,
        lastBlock,
        MULTICALL_CONTRACT
      )
      // call MY_CONTRACT.myContractMethod('foo') and MY_CONTRACT.myContractMethod('bar')
      const args = ['foo', 'bar']
      const results = await multicall.tryAggregate(
        functions.myContractMethod,
        args.map(a => [MY_CONTRACT, a]) as [string, any[]],
        100
      )

      results.forEach((res, i) => {
        if (res.success) {
          ctx.log.info(`Result for argument ${args[i]} is ${res.value}`)
        }
      })
    })
    ```
  </Tab>

  <Tab title="squid-substrate-typegen">
    It is sometimes impossible to extract the required data with only event and call data without querying the runtime state.
    The context exposes a lightweight gRPC client to the chain node accessible via `ctx._chain`.
    It exposes low-level methods for accessing the storage. However, the recommended way to query the storage is with type-safe wrappers generated with `squid-substrate-typegen`.

    Substrate typegen tool exposes storage access wrappers at `src/types/storage.ts`. The wrappers follow the [general naming pattern](/en/sdk/squid-sdk/resources/tools/typegen/generation#typescript-wrappers) used by Substrate typegen:

    ```
    storage.${palletName}.${storageName}
    ```

    with all identifiers lowerCamelCased. Each wrapper exposes a generated `get()` query method and, if available, methods for multi-key queries, listing keys, key-value pairs and retrieving the default value.

    Note that the generated getters **always query historical blockchain state at the height derived from their `block` argument**.

    Here is an example of one such wrapper:

    ```typescript title="src/types/balances/storage.ts" theme={"system"}
    import {sts, Block, Bytes, Option, Result, StorageType} from '../support'
    import * as v1050 from '../v1050'
    import * as v9420 from '../v9420'

    export const account = {
      v1050: new StorageType('Balances.Account', 'Default', [v1050.AccountId], v1050.AccountData) as AccountV1050,
      v9420: new StorageType('Balances.Account', 'Default', [v9420.AccountId32], v9420.AccountData) as AccountV9420,
    }

    export interface AccountV1050  {
      getDefault(block: Block): v1050.AccountData
      get(block: Block, key: v1050.AccountId): Promise<v1050.AccountData | undefined>
      getMany(block: Block, keys: v1050.AccountId[]): Promise<v1050.AccountData | undefined[]>
    }

    export interface AccountV9420  {
      getDefault(block: Block): v9420.AccountData
      get(block: Block, key: v9420.AccountId32): Promise<v9420.AccountData | undefined>
      getMany(block: Block, keys: v9420.AccountId32[]): Promise<v9420.AccountData | undefined[]>
      getKeys(block: Block): Promise<v9420.AccountId32[]>
      getKeys(block: Block, key: v9420.AccountId32): Promise<v9420.AccountId32[]>
      getKeysPaged(pageSize: number, block: Block): AsyncIterable<v9420.AccountId32[]>
      getKeysPaged(pageSize: number, block: Block, key: v9420.AccountId32): AsyncIterable<v9420.AccountId32[]>
      getPairs(block: Block): Promise<[k: v9420.AccountId32, v: v9420.AccountData | undefined][]>
      getPairs(block: Block, key: v9420.AccountId32): Promise<[k: v9420.AccountId32, v: v9420.AccountData | undefined][]>
      getPairsPaged(pageSize: number, block: Block): AsyncIterable<[k: v9420.AccountId32, v: v9420.AccountData | undefined][]>
      getPairsPaged(pageSize: number, block: Block, key: v9420.AccountId32): AsyncIterable<[k: v9420.AccountId32, v: v9420.AccountData | undefined][]>
    }
    ```

    The generated access interface provides methods for accessing:

    * the default storage value with `getDefault(block)`
    * a single storage value with `get(block, key)`
    * multiple values in a batch call with `getMany(block, keys[])`
    * all storage keys with `getKeys(block)`
    * all keys with a given prefix with `getKeys(block, keyPrefix)` (only if the storage keys are decodable)
    * paginated keys via `getKeysPaged(pageSize, block)` and `getKeysPaged(pageSize, block, keyPrefix)`
    * key-value pairs via `getPairs(block)` and `getPairs(block, keyPrefix)`
    * paginated key-value pairs via `getPairsPaged(pageSize, block)` and `getPairsPaged(pageSize, block, keyPrefix)`

    ### Example

    ```typescript title="src/main.ts" theme={"system"}
    import {storage} from './types'

    processor.run(new TypeormDatabase(), async ctx => {
      let aliceAddress = ss58.decode('5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY').bytes
      for (const blockData of ctx.blocks) {
        if (storage.balances.account.v1050.is(blockData.header)) {
          let aliceBalance = (await storage.balances.account.v1050.get(blockData.header, aliceAddress))?.free
          ctx.log.info(`Alice free account balance at block ${blockData.header.height}: ${aliceBalance}`)
        }
      }
    })
    ```
  </Tab>

  <Tab title="squid-ink-typegen">
    The generated `Contract` class provides facades for all state calls that don't mutate the contract state. The info about the state mutability is taken from the contract metadata.

    ```ts theme={"system"}
    // Generated code:
    export class Contract {
        // optional blockHash indicates the block at which the state is queried
        constructor(private ctx: ChainContext, private address: string, private blockHash?: string) { }

        total_supply(): Promise<Balance> {
            return this.stateCall('0xdb6375a8', [])
        }

        balance_of(owner: Uint8Array): Promise<bigint> {
            return this.stateCall('0x6568382f', [owner])
        }
    }
    ```

    ### Example

    ```ts theme={"system"}
    processor.run(new TypeormDatabase(), async ctx => {
      for (let block of ctx.blocks) {
        for (let event of block.events) {
           // query the balance at the current block
           const contract = new Contract(ctx, CONTRACT_ADDRESS, block.header.hash)
           let aliceAddress = ss58.decode('5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY').bytes
           let aliceBalance = await contract.balance_of(aliceAddress)
        }
      }
    })
    ```
  </Tab>
</Tabs>


# RPC ingestion and reorgs
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/unfinalized-blocks

How Squid SDK handles unfinalized blocks and reorgs — RPC ingestion, finality assumptions, and the rollback model for chain consensus changes.

Starting with the ArrowSquid release, the processor can ingest data either from a [SQD Network](/en/network) gateway or directly from an RPC endpoint. If both a network gateway and a RPC endpoint are provided, the processor will use network data until it reaches the highest block available there, then index the few remaining blocks using the RPC endpoint. This allows squids to combine low sync times with near real-time chain data access. It is, however, possible to use just the RPC endpoint (e.g. for [local development](/en/sdk/squid-sdk/tutorials/evm-local)). On [EVM](/en/sdk/squid-sdk/reference/processors/evm-batch) it is also possible to use just the gateway (e.g. for non-realtime analytics).

RPC ingestion can create a heavy load on node endpoints. The load is typically short in duration when a SQD Network gateway is used, and the total number of requests is low. However, the request rate may be sufficient to trigger HTTP 429 responses. Use private endpoints and rate limit your requests with the `rateLimit` option ([EVM](/en/sdk/squid-sdk/reference/processors/evm-batch/general#set-rpc-endpoint)/[Substrate](/en/sdk/squid-sdk/reference/processors/substrate-batch/general#set-rpc-endpoint)).

## Indexing unfinalized blocks

When ingesting from RPC, Squid SDK can index blocks before they are [finalized](https://info.etherscan.com/epoch-in-ethereum/), enabling real-time use cases. If a blockhain reorganization happens, [processor](/en/sdk/squid-sdk/reference/processors/architecture) will roll back any changes to the database made due to orphaned blocks, then re-run its batch handler on consensus blocks.

To take advantage of this feature, a squid processor must

1. have a [node RPC endpoint](/en/sdk/squid-sdk/reference/processors/evm-batch/general#set-rpc-endpoint) as one of its data sources and
2. use a [`Database`](/en/sdk/squid-sdk/resources/persisting-data/overview) with unfinalized blocks support (e.g. [`TypeormDatabase`](/en/sdk/squid-sdk/resources/persisting-data/typeorm)) in its [`processor.run()`](/en/sdk/squid-sdk/reference/processors/architecture#processorrun) call, and
3. not disable RPC ingestion explicitly with [`setRpcDataIngestionSettings({ disabled: true })`](/en/sdk/squid-sdk/reference/processors/evm-batch/general#set-rpc-data-ingestion-settings).

With this, the processor will consider some blocks to be "hot":

<Tabs>
  <Tab title="EVM">
    All blocks with fewer confirmations than the number set by the [`setFinalityConfirmations()`](/en/sdk/squid-sdk/reference/processors/evm-batch/general#set-finality-confirmation) setting
  </Tab>

  <Tab title="Substrate">
    All blocks above the latest finalized block provided by the [`chain_getFinalizedHead()`](https://polkadot.js.org/docs/substrate/rpc/#getfinalizedhead-blockhash) RPC method
  </Tab>
</Tabs>

The processor will periodically (interval setting for [EVM](/en/sdk/squid-sdk/reference/processors/evm-batch/general#set-rpc-data-ingestion-settings), [Substrate](/en/sdk/squid-sdk/reference/processors/substrate-batch/general#set-rpc-data-ingestion-settings)) poll the RPC endpoint for changes in consensus. When the consensus changes, it will re-run the [batch handler](/en/sdk/squid-sdk/reference/processors/architecture#processorrun) with the new consensus data and ask the `Database` to adjust its state. The `Database` then must roll back the changes made due to orphaned blocks and apply the new changes. With this, the state of the `Database` reflects the current blockchain consensus at all times.


# Solana Indexing CLI Cheatsheet
Source: https://docs.sqd.dev/en/sdk/squid-sdk/solana-indexing/how-to-start/cli-cheatsheet

Commonly used sqd CLI commands for Solana indexers — init, deploy, run, secrets, and gateways with examples for the Squid SDK Solana data source.

# CLI cheatsheet

This quide provides a quick reference to the commands needed to launch the squid and manage the database.

### Install dependencies

```sh theme={"system"}
npm i
```

### Compile the project

```sh theme={"system"}
npx tsc
```

### Launch Postgres database to store the data

```sh theme={"system"}
docker compose up -d
```

### Apply database migrations to create the target schema

```sh theme={"system"}
npx squid-typeorm-migration apply
```

### Run indexer

```sh theme={"system"}
node -r dotenv/config lib/main.js
```

### Check out the indexed swaps

```sh theme={"system"}
docker exec "$(basename "$(pwd)")-db-1" psql -U postgres \
  -c "SELECT slot, from_token, to_token, from_amount, to_amount FROM exchange ORDER BY id LIMIT 10"
```

Notice that this project doesn't utilize `sqd` commands.


# Indexing Orca Whirlpool
Source: https://docs.sqd.dev/en/sdk/squid-sdk/solana-indexing/how-to-start/indexing-orca

Tutorial: index Orca Whirlpool DEX transactions on Solana with Squid SDK — set up the processor, decode instructions, and persist to Postgres.

# Indexing the Orca DEX

In this step-by-step tutorial we will look into a squid that gets data about the [Orca Exchange](https://www.orca.so/) NFTs, their transfers and owners from the [Solana blockchain](https://solana.com).

Pre-requisites: Node.js **v20 or newer**, Git, Docker.

## Download the project

Begin by retrieving the template and installing the dependencies:

```bash theme={"system"}
git clone https://github.com/subsquid-labs/solana-example
cd solana-example
npm i
```

## Interfacing with the Whirlpool program

First, we inspect the data available for indexing. In Solana, most programs use the [Anchor framework](https://www.anchor-lang.com/). Anchor makes the metadata describing the shape of the instructions, transactions and contract variables available as an [Interface Definition Language](https://www.quicknode.com/guides/solana-development/anchor/what-is-an-idl) (IDL) JSON file. For many popular programs (including Whirlpool) IDL files are published on-chain. SQD provides a [tool](/en/sdk/squid-sdk/solana-indexing/sdk/typegen) for retrieving program IDLs and generating boilerplate ABI code for data decoding. This can be done with

```bash theme={"system"}
npx squid-solana-typegen src/abi whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc#whirlpool
```

Here, `src/abi` is the destination folder and the `whirlpool` suffix sets the base name for the generated file.

Checking out the generated `src/abi/whirlpool/instructions.ts` file. It exports an `instruction` instance for every instruction in the ABI. Here's how it is initialized for `swap`:

```typescript theme={"system"}
export const swap = instruction(
  {
    d8: '0xf8c69e91e17587c8',
  },
  {
    tokenProgram: 0,
    tokenAuthority: 1,
    whirlpool: 2,
    tokenOwnerAccountA: 3,
    tokenVaultA: 4,
    tokenOwnerAccountB: 5,
    tokenVaultB: 6,
    tickArray0: 7,
    tickArray1: 8,
    tickArray2: 9,
    oracle: 10,
  },
  struct({
    amount: u64,
    otherAmountThreshold: u64,
    sqrtPriceLimit: u128,
    amountSpecifiedIsInput: bool,
    aToB: bool,
  }),
)
```

Here, `d8` are the eight bytes that the relevant instruction data starts with.

## Configuring the data source

"Data source" is a component that defines what data should be retrieved and where to get it. To configure the data source to retrieve the data produced by the `swap` instruction of the Whirlpool program, we initialize it like this:

```typescript title="src/main.ts" theme={"system"}
// ...
import {run} from '@subsquid/batch-processor'
import {augmentBlock} from '@subsquid/solana-objects'
import {DataSourceBuilder, SolanaRpcClient} from '@subsquid/solana-stream'
import {TypeormDatabase} from '@subsquid/typeorm-store'
import assert from 'assert'
import * as tokenProgram from './abi/token-program'
import * as whirlpool from './abi/whirpool'
import {Exchange} from './model'

const dataSource = new DataSourceBuilder()
  // Provide SQD Network Gateway URL.
  .setGateway('https://v2.archive.subsquid.io/network/solana-mainnet')
  .setRpc(process.env.SOLANA_NODE == null ? undefined : {
    client: new SolanaRpcClient({
      url: process.env.SOLANA_NODE,
      // rateLimit: 100 // requests per sec
    }),
    strideConcurrency: 10 // max concurrent RPC connections
  })
  .setBlockRange({from: 259_984_950})
  .setFields({
    block: {
      timestamp: true
    },
    transaction: {
      signatures: true
    },
    instruction: {
      programId: true,
      accounts: true,
      data: true
    },
    tokenBalance: {
      preAmount: true,
      postAmount: true,
      preOwner: true,
      postOwner: true
    }
  })
  .addInstruction({
    // select instructions that:
    where: {
      // were executed by the Whirlpool program, and
      programId: [whirlpool.programId],
      // have the first eight bytes of .data equal to swap descriptor, and
      d8: [whirlpool.instructions.swap.d8],
      // limiting to USDC-SOL pair only
      ...whirlpool.instructions.swap.accountSelection({
        whirlpool: ['7qbRF6YsyGuLUVs6Y1q64bdVrfe4ZcUUz1JRdoVNUJnm']
      }),
      // were successfully committed
      isCommitted: true
    },
    // for each instruction data item selected above
    // make sure to also include:
    include: {
      // inner instructions
      innerInstructions: true,
      // transaction that executed the instruction
      transaction: true,
      // all token balance records of that transaction
      transactionTokenBalances: true,
    }
  }).build()
```

Here,

* `'https://v2.archive.subsquid.io/network/solana-mainnet'` is the address for the public SQD Network gateway for Solana mainnet. The only other Solana-compatible dataset currently available is Eclipse Testnet, with the gateway at `'https://v2.archive.subsquid.io/network/eclipse-testnet'`. Many other networks are available on EVM and Substrate - see the exhaustive [public networks list](/en/data/evm).
* `'process.env.SOLANA_NODE'` is an environment variable pointing at a public RPC endpoint we chose to use in this example. When an endpoint is available, the processor will begin ingesting data from it once it reaches the highest block available within SQD Network.
* `259_984_950` is first Solana block currently indexed by SQD.
* The argument of [`addInstruction()`](/en/sdk/squid-sdk/solana-indexing/sdk/solana-batch/instructions) is a set of filters that tells the processor to retrieve all data on the swap instruction of the Whirlpool program with discriminator matching the hash of the `<namespace>:<instruction>` of the swap instruction.
* The argument of [`setFields()`](/en/sdk/squid-sdk/solana-indexing/sdk/solana-batch/field-selection) specifies the exact fields we need for every data item type.

See [`SolanaDataSource` reference](/en/sdk/squid-sdk/solana-indexing/sdk/solana-batch) for more options.

With a data source it becomes possible to retrieve filtered blockchain data from [SQD Network](/en/network/overview), transform it and save the result to a destination of choice.

## Decoding the event data

The other part the squid processor (the ingester process of the indexer) is the callback function used to process batches of the filtered data, the [batch handler](/en/sdk/squid-sdk/reference/processors/architecture#processorrun). In Solana Squid SDK it is typically defined within a `run()` call, like this:

```typescript theme={"system"}
import {run} from '@subsquid/batch-processor'

run(dataSource, database, async ctx => {
    // data transformation and persistence code here
})
```

Here,

* `dataSource` is the data source object described in the previous section
* `database` is a [`Database` implementation](/en/sdk/squid-sdk/resources/persisting-data/overview) specific to the target data sink. We want to store the data in a PostgreSQL database and present with a GraphQL API, so we provide a [`TypeormDatabase`](/en/sdk/squid-sdk/resources/persisting-data/typeorm) object here.
* `ctx` is a [batch context](/en/sdk/squid-sdk/reference/processors/architecture#batch-context) object that exposes a batch of data (at `ctx.blocks`) and any data persistence facilities derived from `db` (at `ctx.store`). See [Block data for Solana](/en/sdk/squid-sdk/solana-indexing/sdk/solana-batch/context-interfaces) for details on how the data batches are presented.

Batch handler is where the raw on-chain data is decoded, transformed and persisted. This is the part we'll be concerned with for the rest of the tutorial.

We begin by defining a database and starting the data processing:

```typescript title="src/main.ts" theme={"system"}
const database = new TypeormDatabase()

// Now we are ready to start data processing
run(dataSource, database, async ctx => {
  // Block items that we get from `ctx.blocks` are flat JS objects.
  //
  // We can use `augmentBlock()` function from `@subsquid/solana-objects`
  // to enrich block items with references to related objects and
  // with convenient getters for derived data (e.g. `Instruction.d8`).
  let blocks = ctx.blocks.map(augmentBlock)

  let exchanges: Exchange[] = []

  for (let block of blocks) {
    for (let ins of block.instructions) {
      // https://read.cryptodatabytes.com/p/starter-guide-to-solana-data-analysis
      if (ins.programId === whirlpool.programId &&
          ins.d8 === whirlpool.instructions.swap.d8) {

        let exchange = new Exchange({
          id: ins.id,
          slot: block.header.slot,
          tx: ins.getTransaction().signatures[0],
          timestamp: new Date(block.header.timestamp * 1000)
        })

        assert(ins.inner.length == 2)
        let srcTransfer = tokenProgram.instructions.transfer.decode(ins.inner[0])
        let destTransfer = tokenProgram.instructions.transfer.decode(ins.inner[1])

        let srcBalance = ins.getTransaction().tokenBalances.find(tb => tb.account == srcTransfer.accounts.source)
        let destBalance = ins.getTransaction().tokenBalances.find(tb => tb.account === destTransfer.accounts.destination)

        let srcMint = ins.getTransaction().tokenBalances.find(tb => tb.account === srcTransfer.accounts.destination)?.preMint
        let destMint = ins.getTransaction().tokenBalances.find(tb => tb.account === destTransfer.accounts.source)?.preMint

        assert(srcMint)
        assert(destMint)

        exchange.fromToken = srcMint
        exchange.fromOwner = srcBalance?.preOwner || srcTransfer.accounts.source
        exchange.fromAmount = srcTransfer.data.amount

        exchange.toToken = destMint
        exchange.toOwner = destBalance?.postOwner || destBalance?.preOwner || destTransfer.accounts.destination
        exchange.toAmount = destTransfer.data.amount

        exchanges.push(exchange)
      }
    }
  }

  await ctx.store.insert(exchanges)
})
```

This goes through all the instructions in the block, verifies that they indeed are `swap` instruction from the Whirlpool program and decodes the data of each inner instruction.
Then it retrieves the transaction from the decoded inner instruction and source and destination accounts.
The decoding is done with the `tokenProgram.instructions.transfer.decode` function from the Typescript ABI provided in the project.

At this point the squid is ready for its first test run. Execute

```bash theme={"system"}
npx tsc
docker compose up -d
npx squid-typeorm-migration apply
node -r dotenv/config lib/main.js
```

You can verify that the data is being stored in the database by running

```bash theme={"system"}
docker exec "$(basename "$(pwd)")-db-1" psql -U postgres -c "SELECT * FROM exchange"
```

The full code can be found [here](https://github.com/subsquid-labs/solana-example).


# Balances
Source: https://docs.sqd.dev/en/sdk/squid-sdk/solana-indexing/sdk/solana-batch/balances

Track Solana account balance changes with addBalance() in Squid SDK — filter by account, delta, and pre/post balance for indexing on Solana.

#### `addBalance(options)`

This allows for tracking SOL account balances. `options` has the following structure:

```typescript theme={"system"}
{
  // data requests
  where?: {
    account?: string[]
  }

  // related data retrieval
  include?: {
    transaction?: boolean
    transactionInstructions?: boolean
  }

  range?: {
    from: number
    to?: number
  }
}
```

The data requests here are:

* `account`: the set of accounts to track. Leave undefined to subscribe to balance updates of all accounts in the whole network.

Related data retrieval flags:

* `transaction = true`: retrieve the transaction that gave rise to the balance update
* `transactionInstructions = true`: retrieve all instructions executed by the parent transaction

The related data will be added to the appropriate iterables within the [block data](/en/sdk/squid-sdk/solana-indexing/sdk/solana-batch/context-interfaces). You can also call `augmentBlock()` from `@subsquid/solana-objects` on the block data to populate the convenience reference fields like `instruction.inner`.

Selection of the exact fields to be retrieved for each balance item and the related data is done with the `setFields()` method documented on the [Field selection](/en/sdk/squid-sdk/solana-indexing/sdk/solana-batch/field-selection) page.


# Block data for Solana
Source: https://docs.sqd.dev/en/sdk/squid-sdk/solana-indexing/sdk/solana-batch/context-interfaces

Block data interfaces for Solana indexing with Squid SDK — type definitions for blocks, transactions, instructions, balances, rewards, and logs.

In Solana Squid SDK, the data is processed by repeatedly calling the user-defined [batch handler](/en/sdk/squid-sdk/reference/processors/architecture#processorrun) function on batches of on-chain data. The sole argument of the batch handler is its context `ctx`, and `ctx.blocks` is an array of `Block` objects containing the data to be processed, aligned at the block level.

For `SolanaBatchProcessor` the `Block` interface is defined as follows:

```ts theme={"system"}
export interface Block<F extends FieldSelection = {}> {
  header: BlockHeader<F>;
  instructions: Instruction<F>[];
  transactions: Transaction<F>[];
  logs: LogMessage<F>[];
  balances: Balance<F>[];
  tokenBalances: TokenBalance<F>[];
  rewards: Reward<F>[];
}
```

`F` here is the type of the argument of the [`setFields()`](/en/sdk/squid-sdk/solana-indexing/sdk/solana-batch/field-selection) processor method.

`BlockData.header` contains the block header data. The rest of the fields are iterables containing the six kinds of blockchain data. Canonical ordering within each iterable depends on the data kind:

* `transactions` are ordered in the same way as they are within blocks;
* `instructions` follow the order of transactions that gave rise to them;
* `tokenBalances` are ordered in a deterministic but otherwise unspecified way.

The exact fields available in each data item type are inferred from the `setFields()` call argument. They are documented on the [field selection](/en/sdk/squid-sdk/solana-indexing/sdk/solana-batch/field-selection) page:

* [`Instruction` section](/en/sdk/squid-sdk/solana-indexing/sdk/solana-batch/field-selection#instruction);
* [`Transaction` section](/en/sdk/squid-sdk/solana-indexing/sdk/solana-batch/field-selection#transaction);
* [`LogMessage` section](/en/sdk/squid-sdk/solana-indexing/sdk/solana-batch/field-selection#logmessage);
* [`Balance` section](/en/sdk/squid-sdk/solana-indexing/sdk/solana-batch/field-selection#balance);
* [`TokenBalance` section](/en/sdk/squid-sdk/solana-indexing/sdk/solana-batch/field-selection#tokenbalance);
* [`Reward` section](/en/sdk/squid-sdk/solana-indexing/sdk/solana-batch/field-selection#reward);
* [`BlockHeader` section](/en/sdk/squid-sdk/solana-indexing/sdk/solana-batch/field-selection#block-header).

## Example

The handler below simply outputs all the log items emitted by the contract `0x2E645469f354BB4F5c8a05B3b30A929361cf77eC` in [real time](/en/sdk/squid-sdk/resources/unfinalized-blocks):

```ts theme={"system"}
import { TypeormDatabase } from "@subsquid/typeorm-store";
import { EvmBatchProcessor } from "@subsquid/evm-processor";

const CONTRACT_ADDRESS =
  "0x2E645469f354BB4F5c8a05B3b30A929361cf77eC".toLowerCase();

const processor = new EvmBatchProcessor()
  .setGateway("https://v2.archive.subsquid.io/network/ethereum-mainnet")
  .setRpcEndpoint("<my_eth_rpc_url>")
  .setFinalityConfirmation(75)
  .setBlockRange({ from: 17000000 })
  .addLog({
    address: [CONTRACT_ADDRESS],
  })
  .setFields({
    // could be omitted: this call does not change the defaults
    log: {
      topics: true,
      data: true,
    },
  });

processor.run(new TypeormDatabase(), async (ctx) => {
  for (let c of ctx.blocks) {
    for (let log of c.logs) {
      if (log.address === CONTRACT_ADDRESS) {
        ctx.log.info(log, `Log:`);
      }
    }
  }
});
```

One can experiment with the [`setFields()`](/en/sdk/squid-sdk/reference/processors/evm-batch/field-selection) argument and see how the output changes.

For more elaborate examples, check [Solana Examples](/en/sdk/squid-sdk/examples). -->


# Solana Field Selection
Source: https://docs.sqd.dev/en/sdk/squid-sdk/solana-indexing/sdk/solana-batch/field-selection

Fine-tune Solana data requests with setFields() in Squid SDK — choose only the block, transaction, instruction, balance, reward, and log fields needed.

#### `setFields(options)`

Set the fields to be retrieved for data items of each supported type. The `options` object has the following structure:

```ts theme={"system"}
{
  instruction?:  // field selector for instructions
  transaction?:  // field selector for transactions
  log?:          // field selector for log messages
  balance?:      // field selector for balances
  tokenBalance?: // field selector for token balances
  reward?:       // field selector for reward
  block?:        // field selector for block headers
}
```

Every field selector is a collection of boolean fields, typically mapping one-to-one to the fields of data items within the batch context [iterables](/en/sdk/squid-sdk/solana-indexing/sdk/solana-batch/context-interfaces). Defining a field of a field selector of a given type and setting it to true will cause the processor to populate the corresponding field of all data items of that type. Here is a definition of a processor that requests `signatures` and `err` fields for transactions and the `data` field for instructions:

```ts theme={"system"}
const dataSource = new DataSourceBuilder()
  .setFields({
    transaction: {
      signatures: true,
      err: true
    },
    instruction: {
      data: true
    }
  })
```

Same fields will be available for all data items of any given type, including the items accessed via nested references. Suppose we used the processor defined above to subscribe to some transactions as well as some instructions, and for each instruction we requested a parent transaction:

```ts theme={"system"}
dataSource
  .addInstruction({
    where: {
      // some instruction data requests
    },
    include: {
      transaction: true
    }
  })
  .addTransaction({
    where: {
      // some transaction data requests
    },
    include: {
      instruction: true
    }
  })
  .build()
```

After populating the convenience reference fields with `augmentBlock()` from `@subsquid/solana-objects`, `signatures` and `err` fields would be available both within the transaction items of the `transactions` iterable of [block data](/en/sdk/squid-sdk/solana-indexing/sdk/solana-batch/context-interfaces) and within the transaction items that provide parent transaction information for the instructions matching the `addInstruction()` data request:

```ts theme={"system"}
run(dataSource, database, async (ctx) => {
  let blocks = ctx.blocks.map(augmentBlock)
  for (let block of blocks) {
    for (let txn of block.transactions) {
      let sig = txn.signature; // OK
    }
    for (let ins of block.instructions) {
      if (/* ins matches the data request */) {
        let parentTxSig = log.transaction.signature; // also OK!
      }
    }
  }
})
```

Some data fields, like `signatures` for transactions, are enabled by default but can be disabled by setting a field of a field selector to `false`. For example, this code will not compile:

```ts theme={"system"}
const dataSource = new DataSourceBuilder()
  .setFields({
    transaction: {
      signatures: false,
    }
  })
  .build()

run(dataSource, database, async (ctx) => {
  for (let block of ctx.blocks) {
    for (let txn of block.transactions) {
      let signatures = txn.signatures; // ERROR: no such field
    }
  }
})
```

Disabling unused fields will improve sync performance, as the disabled fields will not be fetched from the SQD Network gateway.

## Data item types and field selectors

<Tip>
  Most IDEs support smart suggestions to show the possible field selectors. For VS Code, press `Ctrl+Space`.
</Tip>

<Info>
  All addresses and pubkeys are represented as base58-encoded strings.
</Info>

Here we describe the data item types as functions of the field selectors. Unless otherwise mentioned, each data item type field maps to the eponymous field of its corresponding field selector. Item fields are divided into three categories:

* Fields that are always added regardless of the `setFields()` call.
* Fields that are enabled by default and can be disabled by `setFields()`. E.g. a `signatures` field will be fetched for transactions by default, but can be disabled by setting `signatures: false` within the `log` field selector.
* Fields that can be requested by `setFields()`.

### Instruction

`Instruction` data items may have the following fields:

```ts theme={"system"}
Instruction {
  // independent of field selectors
  transactionIndex: number
  instructionAddress: number[]

  // can be disabled with field selectors
  programId: string
  accounts: string[]
  data: string
  isCommitted: boolean

  // can be enabled with field selectors
  computeUnitsConsumed?: bigint
  error?: string
  hasDroppedLogMessages: boolean
}
```

Here, *instruction address* is an array of tree indices addressing the instruction in the call tree:

* Top level instructions get addresses `[0]`, `[1]`, ..., `[N-1]`, where N is the number of top level instructions in the transaction.
* An address of length 2 indicates an inner instruction directly called by one of the top level instructions. For example, `[3, 0]` is the first inner instruction called by the fourth top level instruction.
* Addresses of length 3 or more indicate inner instructions invoked by other inner instructions.

### Transaction

`Transaction` data items may have the following fields:

```ts theme={"system"}
Transaction {
  // independent of field selectors
  transactionIndex: number
 
  // can be disabled with field selectors
  signatures: string[]
  err: null | object

  // can be requested with field selectors
  version: 'legacy' | number
  accountKeys: string[]
  addressTableLookups: AddressTableLookup[]
  numReadonlySignedAccounts: number
  numReadonlyUnsignedAccounts: number
  numRequiredSignatures: number
  recentBlockhash: string
  computeUnitsConsumed: bigint
  fee: bigint
  loadedAddresses: {
    readonly: string[]
    writable: string[]
  } // request the whole struct with loadedAddresses: true
  hasDroppedLogMessages: boolean
}
```

### LogMessage

`LogMessage` data items may have the following fields:

```ts theme={"system"}
LogMessage {
  // independent of field selectors
  transactionIndex: number
  logIndex: number
  instructionAddress: number[]

  // can be disabled with field selectors
  programId: string
  kind: 'log' | 'data' | 'other'
  message: string
}
```

### Balance

`Balance` data items may have the following fields:

```ts theme={"system"}
Balance{
  // independent of field selectors
  transactionIndex: number
  account: string[]

  // can be disabled with field selectors
  pre: bigint
  post: bigint
}
```

### TokenBalance

Field selection for token balances data items is more nuanced, as depending on the subtype of the token balance some fields may be `undefined`. `PostTokenBalance` and `PreTokenBalance` both represent token balances, however `PreTokenBalance` will have `postProgramId, postMint, postDecimals, postOwner and postAmount` as `undefined`.

`PostTokenBalance` will have `preProgramId, preMint, preDecimals, preOwner and preAmount` as `undefined`.

`TokenBalance` data items may have the following fields:

```ts theme={"system"}
TokenBalance {
  // independent of field selectors
  transactionIndex: number
  account: string

  // can be disabled with field selectors
  preMint: string
  preDecimals: number
  preOwner?: string
  preAmount: bigint
  postMint: string
  postDecimals: number
  postOwner?: string
  postAmount: bigint

  // can be enabled by field selectors
  postProgramId?: string
  preProgramId?: string
}
```

### Reward

`Reward` data items may have the following fields:

```ts theme={"system"}
Reward{
  // independent of field selectors
  pubkey: string

  // can be disabled with field selectors
  lamports: bigint
  rewardType?: string

  // can be enabled by field selectors
  postBalance: bigint
  commission?: number
}
```

### Block header

`BlockHeader` data items may have the following fields:

```ts theme={"system"}
BlockHeader{
  // independent of field selectors
  hash: string
  height: number
  parentHash: string

  // can be disabled with field selectors
  slot: number
  parentSlot: number
  timestamp: number
}
```

## A complete example

```ts theme={"system"}
import { run } from "@subsquid/batch-processor";
import { augmentBlock } from "@subsquid/solana-objects";
import { DataSourceBuilder, SolanaRpcClient } from "@subsquid/solana-stream";
import { TypeormDatabase } from "@subsquid/typeorm-store";
import * as whirlpool from "./abi/whirpool";

const dataSource = new DataSourceBuilder()
  .setGateway("https://v2.archive.subsquid.io/network/solana-mainnet")
  .setRpc({
     client: new SolanaRpcClient({
       url: process.env.SOLANA_NODE,
     }),
     strideConcurrency: 10,
  })
  .setBlockRange({ from: 259_984_950 });
  .setFields({
    block: {
      timestamp: false
    },
    transaction: {
      signatures: false,
      version: true,
      fee: true
    },
    instruction: {
      accounts: false,
      error: true
    },
    tokenBalance: {
      postProgramId: true
    }
  })
  .addInstruction({
    // select instructions that
    where: {
      // were executed by the Whirlpool program
      programId: [whirlpool.programId],
      // have the first 8 bytes of .data equal to swap descriptor
      d8: [whirlpool.instructions.swap.d8],
      // were successfully committed
      isCommitted: true,
    },
    include: {
      // inner instructions
      innerInstructions: true,
      // transaction that executed the given instruction
      transaction: true,
      // all token balance records of executed transaction
      transactionTokenBalances: true
    },
  })
  .build();

processor.run(new TypeormDatabase(), async (ctx) => {
  // Simply output all the items in the batch.
  // It is guaranteed to have all the data matching the data requests,
  // but not guaranteed to not have any other data.
  ctx.log.info(ctx.blocks, "Got blocks");
})
```


# Solana Batch General Settings
Source: https://docs.sqd.dev/en/sdk/squid-sdk/solana-indexing/sdk/solana-batch/general

General settings for Squid SDK SolanaBatchProcessor — configure data source, batch size, finality, prometheus metrics, and runtime options for indexers.

<Tip>
  The method documentation is also available inline and can be accessed via suggestions in most IDEs.
</Tip>

The following setters configure the global settings of `DataSourceBuilder` for Solana Procesor. They return the modified instance and can be chained.

One or both of [`setGateway()`](#set-gateway) or [`setRpcEndpoint()`](#set-rpc) must be called to make the data source usable. `setRpcEndpoint()` is required if you need real-time data; `setGateway()` is required if you want to get historical data from SQD Network at high speed. All other methods are optional.

* If you add both a SQD Network gateway and an RPC endpoint, the processor will obtain as much data as is currently available from the gateway, then switch to ingesting recent data via RPC.
* If you only add a SQD Network gateway, your data will be being several thousands of blocks behind the chain head most of the time.

### `setGateway(url: string | GatewaySettings)`

Use a [SQD Network](/en/network) gateway. The argument is either a string URL of the gateway or

```ts theme={"system"}
{
  url: string // gateway URL
  requestTimeout?: number // in milliseconds
}
```

### `setRpc(settings?: RpcSettings)`

Adds a RPC data source. If added, it will be used for [RPC ingestion](/en/sdk/squid-sdk/resources/unfinalized-blocks). The argument format is:

```ts theme={"system"}
type RpcSettings = {
  client: SolanaRpcClient;
  strideSize?: number; // `getBlock` batch call size, default 5
  strideConcurrency?: number; // num of concurrent connections, default 10
  concurrentFetchThreshold?: number; // min distance from head that triggers a fetch, default 50
};
```

`SolanaRpcClient` class is exported by `@subsquid/solana-stream`. Its constructor arg type is

```ts theme={"system"}
{
  url: string; // http, https, ws and wss are supported
  capacity?: number; // num of concurrent connections, default 10
  rateLimit?: number; // requests per second, default is no limit
  requestTimeout?: number; // in milliseconds, default 30_000
  retryAttempts?: number, // num of retries on failed RPC calls, default 0
  retrySchedule?: number, // retry pauses in ms
  maxBatchCallSize?: number; // default 100
  headers?: Record<string, string>, // http headers
  log?: Logger | null // customize or disable RPC client logs
}
```

### `setBlockRange({from: number, to?: number})`

Limits the range of blocks to be processed. When the upper bound is specified, processor will terminate with exit code 0 once it reaches it.

Note that block ranges can also be specified separately for each data request. This method sets global bounds for all block ranges in the configuration.

### `includeAllBlocks(range?: {from: number, to?: number})`

By default, processor will fetch only blocks which contain requested items. This method modifies such behavior to fetch all chain blocks. Optionally a range of blocks can be specified for which the setting should be effective.


# Instructions
Source: https://docs.sqd.dev/en/sdk/squid-sdk/solana-indexing/sdk/solana-batch/instructions

Track Solana instruction execution with addInstruction() in Squid SDK — filter by program ID, accounts, and inner instructions for efficient indexing.

#### `addInstruction(options)`

This allows for tracking program instructions. `options` has the following structure:

```typescript theme={"system"}
{
  // data requests
  where?: {
    programId?: string[]

    d1?: string[]
    d2?: string[]
    d3?: string[]
    d4?: string[]
    d8?: string[]

    a0?: string[]
    a1?: string[]
    a2?: string[]
    a3?: string[]
    a4?: string[]
    a5?: string[]
    a6?: string[]
    a7?: string[]
    a8?: string[]
    a9?: string[]

    isCommitted?: boolean
  }

  // related data retrieval
  include?: {
    transaction?: boolean
    transactionTokenBalances?: boolean
    logs?: boolean
    innerInstructions?: boolean
  }

  range?: {
    from: number
    to?: number
  }
}
```

The data requests here are:

* `programId`: the set of program addresses to track. Leave undefined to subscribe to instructions data of all programs from the whole network.
* `d1` through `d8`: sets of 1, 2, 3, 4 and 8-byte instruction discriminators, correspondingly.
* `a0` through `a9`: sets of base58-encoded account inputs to the instruction, at positions 0-9 correspondingly.
* `isCommitted`: `true` to request only instructions that did not revert.

Related data retrieval flags:

* `transaction = true`: retrieve the transaction that gave rise to instruction
* `transactionTokenBalances = true`: retrieve all token balance records in the parent instruction
* `logs = true`: retrieve all logs emitted due to the instruction
* `innerInstructions = true`: retrieve the data for all instruction calls due to the matching instruction

The related data will be added to the appropriate iterables within the [block data](/en/sdk/squid-sdk/solana-indexing/sdk/solana-batch/context-interfaces). You can also call `augmentBlock()` from `@subsquid/solana-objects` on the block data to populate the convenience reference fields like `instruction.inner`.

Selection of the exact fields to be retrieved for each instruction item and the related data is done with the `setFields()` method documented on the [Field selection](/en/sdk/squid-sdk/solana-indexing/sdk/solana-batch/field-selection) page.


# Log messages
Source: https://docs.sqd.dev/en/sdk/squid-sdk/solana-indexing/sdk/solana-batch/logs

Subscribe to Solana program log messages with addLog() in Squid SDK — filter by program ID and message content for efficient log-based indexing.

#### `addLog(options)`

Get log messages emitted by some *or all* programs in the network. `options` has the following structure:

```typescript theme={"system"}
{
  // data requests
  where?: {
    programId?: string[]
    kind?: ('log' | 'data' | 'other')[]
  }

  // related data retrieval
  include?: {
    transaction?: boolean
    instruction?: boolean
  }

  range?: {
    from: number,
    to?: number
  }
}
```

Data requests:

* `programId`: the set of addresses of programs emitting the logs. Leave it undefined to subscribe to logs from all programs in the network.
* `kind`: the set of values of `kind`.

With `transaction = true` the processor will retrieve all parent transactions and add them to the `transactions` iterable within the [block data](/en/sdk/squid-sdk/solana-indexing/sdk/solana-batch/context-interfaces). You can also call `augmentBlock()` from `@subsquid/solana-objects` on the block data to populate the convenience reference fields like `log.transaction`.

Note that logs can also be requested by the other `SolanaDataSource` methods as related data.

Selection of the exact fields to be retrieved for each log and its optional parent transaction is done with the `setFields()` method documented on the [Field selection](/en/sdk/squid-sdk/solana-indexing/sdk/solana-batch/field-selection) page.

## Examples

Fetch all event logs emitted by Orca Whirlpool.

```ts theme={"system"}
const dataSource = new DataSourceBuilder()
  .setGateway('https://v2.archive.subsquid.io/network/solana-mainnet')
  .addLog({
    where: {
      programId: [PYTH_PUSH_ORACLE_PROGRAM_ID]
    },
    include: {
      instruction: true
    },
    range: {
      from: 241_000_000
    }
  })
  .build()
```


# Rewards
Source: https://docs.sqd.dev/en/sdk/squid-sdk/solana-indexing/sdk/solana-batch/rewards

Track Solana validator and stake account rewards with addReward() in Squid SDK — filter by reward type for indexing staking and inflation flows.

#### `addReward(options)`

This allows for tracking rewards. `options` has the following structure:

```typescript theme={"system"}
{
  // data requests
  where?: {
    pubkey?: string[]
  }

  range?: {
    from: number
    to?: number
  }
}
```

`pubkey` here is the set public keys of reward receivers. Leave undefined to subscribe to all rewards.

Selection of the exact fields to be retrieved for each reward item is done with the `setFields()` method documented on the [Field selection](/en/sdk/squid-sdk/solana-indexing/sdk/solana-batch/field-selection) page.


# Token balances
Source: https://docs.sqd.dev/en/sdk/squid-sdk/solana-indexing/sdk/solana-batch/token-balances

Track Solana SPL token balance changes with addTokenBalance() in Squid SDK — filter by mint, owner, and program for efficient SPL token indexing.

#### `addTokenBalance(options)`

This allows for tracking token balances. `options` has the following structure:

```typescript theme={"system"}
{
  // data requests
  where?: {
    account?: string[]
    preProgramId?: string[]
    postProgramId?: string[]
    preMint?: string[]
    postMint?: string[]
    preOwner?: string[]
    postOwner?: string[]
  }

  // related data retrieval
  include?: {
    transaction?: boolean
    transactionInstructions?: boolean
  }

  range?: {
    from: number
    to?: number
  }
}
```

The data requests here are:

* `account`: the set of accounts to track. Leave undefined to subscribe to balance updates of all accounts across the network.
* `preProgramId` : pubkeys of the Token program that owns the account.
* `postProgramId`: pubkeys of the Token program that owns the account.
* `preMint`: pubkeys of the token's mint prior to execution.
* `postMint`: pubkeys of the token's mint after execution.
* `preOwner`: pubkeys of token balance's owner after execution.
* `postOwner`: pubkeys of token balance's owner after execution.

All account/pubkeys should be base58-encoded strings.

Related data retrieval flags:

* `transaction = true`: retrieve the transaction that gave rise to the balance update
* `transactionInstructions = true`: retrieve all instructions executed by the parent transaction

The related data will be added to the appropriate iterables within the [block data](/en/sdk/squid-sdk/solana-indexing/sdk/solana-batch/context-interfaces). You can also call `augmentBlock()` from `@subsquid/solana-objects` on the block data to populate the convenience reference fields like `balance.transaction`.

Selection of the exact fields to be retrieved for each token balance item and the related data is done with the `setFields()` method documented on the [Field selection](/en/sdk/squid-sdk/solana-indexing/sdk/solana-batch/field-selection) page.


# Solana Transactions Indexing
Source: https://docs.sqd.dev/en/sdk/squid-sdk/solana-indexing/sdk/solana-batch/transactions

Subscribe to Solana transaction data with addTransaction() in Squid SDK — filter by signer, account, and program ID for efficient on-chain indexing.

#### `addTransaction(options)`

Get some *or all* transactions on the network. `options` has the following structure:

```typescript theme={"system"}
{
  // data requests
  where?: {
    feePayer?: string[]
  }

  // related data retrieval
  include?: {
    instructions?: boolean
    logs?: boolean
  }

  range?: {
    from: number
    to?: number
  }
}
```

Data requests:

* `feePayer` sets the addresses of the fee payers. Leave it undefined to subscribe to all transactions.

Enabling the `instructions` and/or `logs` flags will cause the processor to retrieve [instructions](/en/sdk/squid-sdk/solana-indexing/sdk/solana-batch/instructions) and [logs](/en/sdk/squid-sdk/solana-indexing/sdk/solana-batch/logs) that occurred as a result of each selected transaction. The data will be added to the appropriate iterables within the [block data](/en/sdk/squid-sdk/solana-indexing/sdk/solana-batch/context-interfaces). You can also call `augmentBlock()` from `@subsquid/solana-objects` on the block data to populate the convenience reference fields like `transaction.logs`.

Note that transactions can also be requested by the other `SolanaDataSource` methods as related data.

Selection of the exact fields to be retrieved for each transaction and the optional related data items is done with the `setFields()` method documented on the [Field selection](/en/sdk/squid-sdk/solana-indexing/sdk/solana-batch/field-selection) page.

## Examples

Request all transactions with fee payer `rec5EKMGg6MxZYaMdyBfgwp4d5rB9T1VQH5pJv5LtFJ` and include logs and instructions:

```ts theme={"system"}
processor
  .addTransaction({
    where: {
      feePayer: ['rec5EKMGg6MxZYaMdyBfgwp4d5rB9T1VQH5pJv5LtFJ'],
    },
    include: {
      logs: true,
      instructions: true
    }
  })
  .build()
```


# Typegen
Source: https://docs.sqd.dev/en/sdk/squid-sdk/solana-indexing/sdk/typegen

Solana typegen tool for Squid SDK — generates type-safe instruction and event decoders from Anchor IDL files for indexing Solana programs.

A *typegen* is a tool for generating utility code for technology-specific operations such as decoding.

Solana typegen:

* decodes instruction and log message data based on the IDL
* exposes useful constants such as program IDs and instruction discriminators
* provides functions that simplify selecting data items based on accounts

Install it with

```bash theme={"system"}
npm install @subsquid/solana-typegen
```

<Tip>
  Replace `npx squid-solana-typegen` with `npx @subsquid/solana-typegen` if you don't want to install the package.
</Tip>

The `squid-solana-typegen` tool generates TypeScript facades for Solana instructions and logs. It takes JSON IDLs as inputs. The IDLs can be specified in three ways:

1. as a plain JSON file(s):

   ```bash theme={"system"}
   npx squid-solana-typegen src/abi whirlpool.json
   ```

   If you use this option, you can also place your JSON IDLs to the `idl` folder and run

   ```bash theme={"system"}
   npx squid-solana-typegen src/abi ./idl/*
   ```

2. load IDL from a Solana node and generate types:

   ```bash theme={"system"}
   npx squid-solana-typegen src/abi whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc#whirlpool
   ```

## Usage

The generated utility modules have three intended uses:

1. Constants: Solana instruction discriminators:

   ```ts theme={"system"}
   // generated by evm-typegen
   import * as whirlpool from "./abi/whirlpool";

   const dataSource = new DataSourceBuilder()
     .setGateway("https://v2.archive.subsquid.io/network/solana-mainnet")
     .addInstruction({
       where: {
         programId: [whirlpool.programId],
         d8: [whirlpool.instructions.swap.d8]
       }
     })
     .build();
   ```

2. Decoding of Solana instructions

   ```ts theme={"system"}
   // generated by evm-typegen
   import * as whirlpool from "./abi/whirlpool";

   // ...

   for (let block of blocks) {
     for (let ins of block.instructions) {
       if (ins.programId === whirlpool.programId &&
           ins.d8 === whirlpool.instructions.swap.d8) {

         let decodedSwap = whirlpool.instructions.swap.decode(ins)
         let { accounts, data } = decodedSwap
       }
     }
   }
   ```

3. Simplifying the account-specific data requests

   ```ts theme={"system"}
   .addInstruction({
     where: {
       programId: [whirlpool.programId],
       d8: [whirlpool.instructions.swap.d8],
       // select instructions for the USDC-SOL pair only
       ...whirlpool.instructions.swap.accountSelection({
         whirlpool: ['7qbRF6YsyGuLUVs6Y1q64bdVrfe4ZcUUz1JRdoVNUJnm']
       })
     }
   })
   ```


# sqd auth
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/auth

Log in to SQD Cloud with the sqd auth CLI command — exchange your account credentials for an API token to authenticate sqd deploy and other commands.

Log in to the Cloud

* [`sqd auth`](#sqd-auth-1)

## `sqd auth`

Log in to the Cloud

```
USAGE
  $ sqd auth -k <value> [--interactive]

FLAGS
  -k, --key=<value>       (required) Cloud auth key. Log in to https://app.subsquid.io to create or update your key.
      --[no-]interactive  Disable interactive mode
```

*See code: [src/commands/auth.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/auth.ts)*


# sqd autocomplete
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/autocomplete

Display sqd CLI shell autocomplete installation instructions — supported shells (bash, zsh, fish, PowerShell) and how to enable command completion.

Display autocomplete installation instructions.


# commands.json
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/commands-json

Define custom sqd subcommands and aliases via commands.json in your Squid SDK project — extend the workflow with reusable scripts for build and deploy.

The `sqd` tool automatically discovers and loads any extra commands defined in the `commands.json` file. Here is a sample file demonstrating the available features:

```json theme={"system"}
{ // comments are ok
  "$schema": "https://subsquid.io/schemas/commands.json",
  "commands": {
    "clean": {
      "description": "delete all build artifacts",
      "cmd": ["rm", "-rf", "lib"]
    },
    "build": {
      "description": "build the project",
      "deps": ["clean"], // commands to execute before
      "cmd": ["tsc"]
    },
    "typegen": {
      "hidden": true, // Don't show in the overview listing
      "workdir": "abi", // change working dir
      "command": [
        "squid-evm-typegen", // node_modules/.bin is in the PATH
        "../src/abi",
        {"glob": "*.json"} // cross-platform glob expansion
      ],
      "env": { // additional environment variables
        "DEBUG": "*"
      }
    }
  }
}
```

This functionality is managed by the [`@subsquid/commands`](https://github.com/subsquid/squid-sdk/tree/master/util/commands) package.

All [squid templates](/en/sdk/squid-sdk/how-to-start/squid-development#templates) include such a file with a predefined set of useful shortcuts. See [Cheatsheet](/en/sdk/squid-sdk/how-to-start/cli-cheatsheet).


# sqd deploy
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/deploy

Deploy new or update an existing squid deployment in the Cloud. Squid name and also optionally slot and/or tag are taken from the provided deployment manifest.

Deploy new or update an existing squid deployment in the Cloud. Squid name and also optionally slot and/or tag are taken from the provided deployment manifest.

* [`sqd deploy SOURCE`](#sqd-deploy-source)

## `sqd deploy SOURCE`

Deploy new or update an existing squid in the Cloud

```
USAGE
  $ sqd deploy SOURCE [--interactive]
    [-r [<org>/]<name>(@<slot>|:<tag>) | -o <code> | -n <name> | [-s <slot>] | [-t <tag>]]
    [-m <manifest_path>] [--hard-reset] [--stream-logs] [--add-tag <value>]
    [--allow-update] [--allow-tag-reassign] [--allow-manifest-override]

ARGUMENTS
  SOURCE  [default: .] Squid source. Could be:
          - a relative or absolute path to a local folder (e.g. ".")
          - a URL to a .tar.gz archive
          - a github URL to a git repo with a branch or commit tag

FLAGS
  -m, --manifest=<manifest_path>  [default: squid.yaml] Specify the relative local path
                                  to a squid manifest file in the squid working directory
      --add-tag=<value>           Add a tag to the deployed squid
      --allow-manifest-override   Allow overriding the manifest during deployment
      --allow-tag-reassign        Allow reassigning an existing tag
      --allow-update              Allow updating an existing squid
      --hard-reset                Perform a hard reset before deploying. This will drop
                                  and re-create all squid resources, including the
                                  database, causing a short API downtime
      --[no-]interactive          Disable interactive mode
      --[no-]stream-logs          Attach and stream squid logs after the deployment

SQUID FLAGS
  -n, --name=<name>                               Name of the squid
  -r, --reference=[<org>/]<name>(@<slot>|:<tag>)  Fully qualified reference of the squid.
                                                  It can include the organization, name,
                                                  slot, or tag
  -s, --slot=<slot>                               Slot of the squid
  -t, --tag=<tag>                                 Tag of the squid

ORG FLAGS
  -o, --org=<code>  Code of the organization

DESCRIPTION
  Deploy new or update an existing squid in the Cloud

EXAMPLES

  // Create a new squid with name provided in the manifest file
  $ sqd deploy .

  // Create a new squid deployment and override it's name to "my-squid-override"
  $ sqd deploy . -n my-squid-override

  // Update the "my-squid" squid with slot "asmzf5"
  $ sqd deploy . -n my-squid -s asmzf5

  // Use a manifest file located in ./path-to-the-squid/squid.prod.yaml
  $ sqd deploy ./path-to-the-squid -m squid.prod.yaml

  // Full paths are also fine
  $ sqd deploy /Users/dev/path-to-the-squid -m /Users/dev/path-to-the-squid/squid.prod.yaml
```

*See code: [src/commands/deploy.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/deploy.ts)*


# sqd gateways
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/gateways

Explore SQD Network gateways and data sources for a squid with the sqd gateways CLI command — list endpoints, statuses, and chains supported.

Explore data sources for a squid

* [`sqd gateways list`](#sqd-gateways-list)

## `sqd gateways list`

List available gateways

```
USAGE
  $ sqd gateways list [--interactive] [-t <evm|substrate>] [-n <regex>] [-c <number>]

FLAGS
  -c, --chain=<number>        Filter by chain ID or SS58 prefix
  -n, --name=<regex>          Filter by network name
  -t, --type=<evm|substrate>  Filter by network type
      --[no-]interactive      Disable interactive mode

ALIASES
  $ sqd gateways ls
```

*See code: [src/commands/gateways/ls.ts](https://github.com/subsquid/squid-cli/blob/master/src/commands/gateways/ls.ts)*


# sqd init
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/init

Initialize a new Squid SDK project from a template or GitHub repo with sqd init — choose EVM, Substrate, Solana, Tron, or Fuel starters and scaffold files.

Setup a new squid project from a template or github repo

* [`sqd init NAME`](#sqd-init-name)

## `sqd init NAME`

Setup a new squid project from a template or github repo

```
USAGE
  $ sqd init NAME [--interactive] [-t <value>] [-d <value>] [-r]

ARGUMENTS
  NAME  The squid name. It must contain only alphanumeric or dash ("-") symbols and must not start with "-".

FLAGS
  -d, --dir=<value>
      The target location for the squid. If omitted, a new folder NAME is created.

  -r, --remove
      Clean up the target directory if it exists

  -t, --template=<value>
      A template for the squid. Accepts:
      - a github repository URL containing a valid squid.yaml manifest in the root folder
      or one of the pre-defined aliases:
      - evm  A minimal squid template for indexing EVM data.
      - abi  A template to auto-generate a squid indexing events and txs from a contract ABI
      - multichain  A template for indexing data from multiple chains
      - gravatar  A sample EVM squid indexing the Gravatar smart contract on Ethereum.
      - substrate  A template squid for indexing Substrate-based chains.
      - ink  A template for indexing Ink! smart contracts
      - ink-abi  A template to auto-generate a squid from an ink! contract ABI
      - frontier-evm  A template for indexing Frontier EVM chains, like Moonbeam and Astar.

  --[no-]interactive
      Disable interactive mode
```

*See code: [src/commands/init.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/init.ts)*


# Installation
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/installation

Install the sqd CLI in your local environment — npm install -g, standalone binaries for macOS/Linux/Windows, and shell autocomplete configuration.

Squid CLI is a command line tool for

* scaffolding new squids from templates
* running SDK tools and scripts defined in `commands.json` in a cross-platform way
* managing squid deployments in [SQD Cloud](/en/cloud) (former Aquarium)

The CLI is distributed as a [`npm` package](https://www.npmjs.com/package/@subsquid/cli).

To install Squid CLI, follow the steps below.

<Steps>
  <Step title="0. Install and setup Squid CLI">
    First, install the latest version of Squid CLI as a global `npm` package:

    ```bash theme={"system"}
    npm i -g @subsquid/cli@latest
    ```

    Check the version:

    ```bash theme={"system"}
    sqd --version
    ```

    Make sure the output looks like `@subsquid/cli@<version>`.

    <Info>
      The next steps are **optional** for building and running squids. A key is required to enable the CLI commands managing the [SQD Cloud](/en/cloud) deployments.
    </Info>
  </Step>

  <Step title="1. Obtain a SQD Cloud deployment key">
    Sign in to [Cloud](https://app.subsquid.io/), and obtain (or refresh) the deployment key page by clicking at the profile picture > "Deployment key":

    <Frame>
      <img alt="SQD Cloud deployment key page" />
    </Frame>
  </Step>

  <Step title="2. Authenticate Squid CLI">
    Open a terminal window and run

    ```bash theme={"system"}
    sqd auth -k <DEPLOYMENT_KEY>
    ```
  </Step>

  <Step title="3. Explore with `--help`">
    Use `sqd --help` to get a list of the available command and `sqd <command> --help` to get help on the available options for a specific command, e.g.

    ```bash theme={"system"}
    sqd deploy --help
    ```
  </Step>
</Steps>


# sqd list
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/list

List squids deployed to SQD Cloud with the sqd ls command — running, stopped, and archived deployments per organization with their slots and tags.

List squids deployed to the Cloud

* [`sqd list`](#sqd-list-1)

## `sqd list`

List squids deployed to the Cloud

```
USAGE
  $ sqd list [--interactive] [--truncate]
    [-r [<org>/]<name>(@<slot>|:<tag>) | -o <code> | -n <name> | [-s <slot>] | [-t <tag>]]

FLAGS
  --[no-]interactive  Disable interactive mode
  --[no-]truncate     Truncate data in columns: false by default

SQUID FLAGS
  -n, --name=<name>                               Name of the squid
  -r, --reference=[<org>/]<name>(@<slot>|:<tag>)  Fully qualified reference
                                                  of the squid. It can include
                                                  the organization, name, slot,
                                                  or tag
  -s, --slot=<slot>                               Slot of the squid
  -t, --tag=<tag>                                 Tag of the squid

ORG FLAGS
  -o, --org=<code>  Code of the organization

ALIASES
  $ sqd ls
```

*See code: [src/commands/ls.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/ls.ts)*


# sqd logs
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/logs

Fetch live and historical logs from a squid deployed to SQD Cloud with the sqd logs command — filter by container, time range, slot, and tag.

Fetch logs from a squid deployed to the Cloud

* [`sqd logs`](#sqd-logs-1)

## `sqd logs`

Fetch logs from a squid deployed to the Cloud

```
USAGE
  $ sqd logs [--interactive] [--since <value>] [--search <value>] [-f | -p <value>]
    [-r [<org>/]<name>(@<slot>|:<tag>) | -o <code> | [-s <slot> -n <name>] | [-t <tag> ]]
    [-c processor|query-node|api|db-migrate|db...]
    [-l error|debug|info|warning...] [--since <value>]

FLAGS
  -c, --container=<option>...  Container name
                               <options: processor|query-node|api|db-migrate|db>
  -f, --follow                 Follow
  -l, --level=<option>...      Log level
                               <options: error|debug|info|warning>
  -p, --pageSize=<value>       [default: 100] Logs page size
      --[no-]interactive       Disable interactive mode
      --search=<value>         Filter by content
      --since=<value>          [default: 1d] Filter by date/interval

SQUID FLAGS
  -n, --name=<name>                               Name of the squid
  -r, --reference=[<org>/]<name>(@<slot>|:<tag>)  Fully qualified reference of the squid.
                                                  It can include the organization, name,
                                                  slot, or tag
  -s, --slot=<slot>                               Slot of the squid
  -t, --tag=<tag>                                 Tag of the squid

ORG FLAGS
  -o, --org=<code>  Code of the organization
```

Notes:

* `--since` accepts the notation of the [`ms` library](https://www.npmjs.com/package/ms): 1d, 10h, 1m.
* With `--level=error` `sqd` will fetch logs emitted by three different calls to the [SDK Logger](/en/sdk/squid-sdk/reference/logger), namely
  * `ctx.log.error`
  * `ctx.log.fatal`
  * `ctx.log.trace`

*See code: [src/commands/logs.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/logs.ts)*


# sqd remove
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/remove

Remove a squid deployment from SQD Cloud with the sqd rm command — delete a specific slot or tag and free up resources from your organization quota.

Remove a squid deployed to the Cloud

* [`sqd remove`](#sqd-remove-1)

## `sqd remove`

Remove a squid deployed to the Cloud

```
USAGE
  $ sqd remove [--interactive] [-f]
    [-r [<org>/]<name>(@<slot>|:<tag>) | -o <code> | [-s <slot> -n <name>] | [-t <tag> ]]

FLAGS
  -f, --force             Does not prompt before removing a squid or its version
      --[no-]interactive  Disable interactive mode

SQUID FLAGS
  -n, --name=<name>                               Name of the squid
  -r, --reference=[<org>/]<name>(@<slot>|:<tag>)  Fully qualified reference of the squid. It can include the organization, name, slot, or tag
  -s, --slot=<slot>                               Slot of the squid
  -t, --tag=<tag>                                 Tag of the squid

ORG FLAGS
  -o, --org=<code>  Code of the organization

ALIASES
  $ sqd rm
```

*See code: [src/commands/rm.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/rm.ts)*


# sqd restart
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/restart

Restart a squid deployment on SQD Cloud with the sqd restart command — recycle the processor container without redeploying source code or migrations.

Restart a squid deployed to the Cloud

* [`sqd restart`](#sqd-restart-1)

## `sqd restart`

Restart a squid deployed to the Cloud

```
USAGE
  $ sqd restart [--interactive]
    [-r [<org>/]<name>(@<slot>|:<tag>) | -o <code> | [-s <slot> -n <name>] | [-t <tag> ]]

FLAGS
  --[no-]interactive  Disable interactive mode

SQUID FLAGS
  -n, --name=<name>                               Name of the squid
  -r, --reference=[<org>/]<name>(@<slot>|:<tag>)  Fully qualified reference of the squid.
                                                  It can include the organization, name,
                                                  slot, or tag
  -s, --slot=<slot>                               Slot of the squid
  -t, --tag=<tag>                                 Tag of the squid

ORG FLAGS
  -o, --org=<code>  Code of the organization
```

*See code: [src/commands/restart.ts](https://github.com/subsquid/squid-cli/blob/master/src/commands/restart.ts)*


# sqd run
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/run

Run a squid locally with the sqd run command — load environment variables and execute the processor and GraphQL server using the deployment manifest.

Run a squid locally according to the [deployment manifest](/en/cloud/reference/manifest).

* [sqd run PATH](#sqd-run-path)

Notes:

* The command is especially useful for running [multichain squids](/en/sdk/squid-sdk/resources/multichain), as it runs all services in the same terminal and handles failures gracefully.

## `sqd run PATH`

Run a squid project locally

```
USAGE
  $ sqd run PATH [--interactive] [-m <value>] [-f <value>] [-i <value>... | -e <value>...] [-r <value>]

FLAGS
  -e, --exclude=<value>...  Do not run specified services
  -f, --envFile=<value>     [default: .env] Relative path to an additional environment file
  -i, --include=<value>...  Run only specified services
  -m, --manifest=<value>    [default: squid.yaml] Relative path to a squid manifest file
  -r, --retries=<value>     [default: 5] Attempts to restart failed or stopped services
      --[no-]interactive    Disable interactive mode
```

*See code: [src/commands/run.ts](https://github.com/subsquid/squid-cli/blob/master/src/commands/run.ts)*


# sqd secrets
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/secrets

Manage SQD Cloud organization secrets with the sqd secrets CLI — set, list, and delete API keys, RPC endpoints, and database credentials.

Manage account secrets

The secrets are [exposed as a context](/en/cloud/resources/env-variables#secrets), and are accessible to all the squids deployed by the current SQD Cloud [organization](/en/cloud/resources/organizations).

* [`sqd secrets list`](#sqd-secrets-list)
* [`sqd secrets remove NAME`](#sqd-secrets-remove-name)
* [`sqd secrets set NAME VALUE`](#sqd-secrets-set-name-value)

## `sqd secrets list`

List organization secrets in the Cloud

```
USAGE
  $ sqd secrets list [--interactive] [-o <value>]

FLAGS
  -o, --org=<value>       Organization
      --[no-]interactive  Disable interactive mode

ALIASES
  $ sqd secrets ls
```

*See code: [src/commands/secrets/ls.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/secrets/ls.ts)*

## `sqd secrets remove NAME`

Delete an organization secret in the Cloud

```
USAGE
  $ sqd secrets remove NAME [--interactive] [-o <value>]

ARGUMENTS
  NAME  The secret name

FLAGS
  -o, --org=<value>       Organization
      --[no-]interactive  Disable interactive mode

ALIASES
  $ sqd secrets rm
```

*See code: [src/commands/secrets/rm.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/secrets/rm.ts)*

## `sqd secrets set NAME VALUE`

Add or update an organization secret in the Cloud. If value is not specified, it is read from standard input. The secret will be exposed as an environment variable with the given name to all the squids in the organization. NOTE: The changes take affect only after a squid is restarted or updated.

```
USAGE
  $ sqd secrets set NAME VALUE [--interactive] [-o <value>]

ARGUMENTS
  NAME   The secret name
  VALUE  The secret value

FLAGS
  -o, --org=<value>       Organization
      --[no-]interactive  Disable interactive mode
```

*See code: [src/commands/secrets/set.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/secrets/set.ts)*


# sqd tags
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/tags

Manage version tags for squids deployed to SQD Cloud with the sqd tags CLI — promote, alias, and roll back releases following the slots-and-tags model.

Manage squid deployements' tags. See the [slots and tags guide](/en/cloud/resources/slots-and-tags).

* [`sqd tags add`](#sqd-tags-add)
* [`sqd tags remove`](#sqd-tags-remove)

## `sqd tags add`

Add a tag to a squid deployment / slot

```
USAGE
  $ sqd tags add TAG [--interactive] [--allow-tag-reassign]
    [-r [<org>/]<name>(@<slot>|:<tag>) | -o <code> | [-s <slot> -n <name>] | [-t <tag> ]]

ARGUMENTS
  TAG  New tag to assign

FLAGS
  --allow-tag-reassign  Allow reassigning an existing tag
  --[no-]interactive    Disable interactive mode

SQUID FLAGS
  -n, --name=<name>                               Name of the squid
  -r, --reference=[<org>/]<name>(@<slot>|:<tag>)  Fully qualified reference of the squid.
                                                  It can include the organization, name,
                                                  slot, or tag
  -s, --slot=<slot>                               Slot of the squid
  -t, --tag=<tag>                                 Tag of the squid

ORG FLAGS
  -o, --org=<code>  Code of the organization
```

*See code: [src/commands/tags/add.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/tags/add.ts)*

## `sqd tags remove`

Remove a tag from a squid deployment / slot

```
USAGE
  $ sqd tags remove TAG [--interactive]
    [-r [<org>/]<name>(@<slot>|:<tag>) | -o <code> | [-s <slot> -n <name>] | [-t <tag> ]]

ARGUMENTS
  TAG  New tag to assign

FLAGS
  --[no-]interactive  Disable interactive mode

SQUID FLAGS
  -n, --name=<name>                               Name of the squid
  -r, --reference=[<org>/]<name>(@<slot>|:<tag>)  Fully qualified reference of the squid.
                                                  It can include the organization, name,
                                                  slot, or tag
  -s, --slot=<slot>                               Slot of the squid
  -t, --tag=<tag>                                 Tag of the squid

ORG FLAGS
  -o, --org=<code>  Code of the organization
```

*See code: [src/commands/tags/remove.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/tags/remove.ts)*


# sqd whoami
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/whoami

Show the SQD Cloud user account currently authenticated with the sqd CLI — display username, organization, and roles for the active session.

Show the user details for the current Cloud account

* [`sqd whoami`](#sqd-whoami-1)

## `sqd whoami`

Show the user details for the current Cloud account

```
USAGE
  $ sqd whoami [--interactive]

FLAGS
  --[no-]interactive  Disable interactive mode
```

*See code: [src/commands/whoami.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/whoami.ts)*


# SQD Firehose
Source: https://docs.sqd.dev/en/sdk/squid-sdk/subgraphs-support

Run subgraphs on SQD Network via Firehose — a high-throughput SQD-native gRPC sync layer that replaces RPC-based ingestion for The Graph subgraphs.

# Run subgraphs without full nodes

<Warning>
  This tutorial uses alpha-quality software. If you encouter any issues while using it please let us know at the [SquidDevs Telegram chat](https://t.me/HydraDevs).
</Warning>

<Info>
  **Dependencies**: Docker, Git, NodeJS, Yarn.
</Info>

Developing and running [subgraphs](https://thegraph.com/docs/en/glossary/) is hard, as one has to run a full archival node, and for many networks it is not feasible to run a full archival node to at all.

**SQD Firehose** is an open-source lightweight adapter run as a side-car to a graph indexer node, ingesting and filtering the data directly from SQD Network instead of an RPC endpoint. However, since the network does not provide the real-time blocks, the most recent and unfinalized blocks are (optionally) ingested from a complementary RPC endpoint in a seamless way.

Currently it is only possible to run subgraphs against a production-ready [permissioned SQD Network instance](/en/network/overview#open-private-network) (formerly known as Subsquid Archives). Running against the decentralized SQD Network is scheduled to be supported in Q2 2024 (see [SQD Network Overview](/en/network/overview) to learn more about the difference).

The easiest way to run a subgraph with SQD Firehose to use our [graph-node-setup](https://github.com/subsquid-labs/graph-node-setup) repo. Here's how:

1. Clone the repo and install the dependencies:

   ```bash theme={"system"}
   git clone https://github.com/subsquid-labs/graph-node-setup
   cd graph-node-setup
   npm ci
   ```

2. Interactively configure the environment with

   ```bash theme={"system"}
   npm run configure
   ```

   <Frame>
     <img alt="Configuration interface" />
   </Frame>

   You will be asked to select a network. You can pick any network from our [supported EVM networks](/en/data/evm); networks that are not currently [supported by TheGraph](https://thegraph.com/docs/en/developing/supported-networks/) will be available their under SQD names.

   Optionally you can also provide an RPC endpoint. If you do, it will be used to sync a few thousands of blocks at the chain end, so it does not have to be a paid one. However, `firehose-grpc` does not limit its request rate yet, so using a public RPC might result in a cooldown.

   If you do not provide an RPC endpoint, your subgraph deployments will be a few thousands of blocks behind the chain head.

   <Warning>
     Running against the decentralized testnet is not currently supported. This feature will be re-enabled in Q2 2024.
   </Warning>

3. Download and deploy your subgraph of choice! For example, if you configured the environment to use Ethereum mainnet (`eth-mainnet`), you can deploy the well known Gravatar subgraph:

   ```bash theme={"system"}
   git clone https://github.com/graphprotocol/example-subgraph
   cd example-subgraph

   # the repo is a bit outdated, giving it a deps update
   rm yarn.lock
   npx --yes npm-check-updates --upgrade
   yarn install

   # generate classes for the smart contract
   # and events used in the subgraph
   npm run codegen

   # create and deploy the subgraph
   npm run create-local
   npm run deploy-local
   ```

   GraphiQL playground will be available at [http://127.0.0.1:8000/subgraphs/name/example/graphql](http://127.0.0.1:8000/subgraphs/name/example/graphql).

## Troubleshooting

Do not hesitate to let us know about any issues (whether listed here or not) at the [SquidDevs Telegram chat](https://t.me/HydraDevs).

* If your subgraph is not syncing and you're getting
  ```
  thread 'tokio-runtime-worker' panicked at 'called `Option::unwrap()` on a `None` value', src/ds_rpc.rs:556:80
  ```
  errors in the `graph-node-setup-firehose` container logs, that likely means that the chain RPC is not fully Ethereum-compatible and a workaround is not yet implemented in `firehose-grpc`. You can still sync your subgraph with RPC ingestion disabled.


# SQD vs The Graph
Source: https://docs.sqd.dev/en/sdk/squid-sdk/subsquid-vs-thegraph

Compare SQD and The Graph — feature parity, indexing throughput, query performance, decentralization model, and pricing for blockchain indexers.

[The Graph](https://thegraph.com) is an indexing protocol and framework for EVM chains. In The Graph, indexing projects are called subgraphs. A Graph indexing node is a black-box that executes subgraphs compiled into WASM. The data is sourced directly from an archival blockchain node, and the processed data is stored in a built-in Postgres database.

On the contrary, SQD employs a radically open modular architecture with:

* a separated layer for efficient data extraction and batch queries ([SQD Network](/en/network))
* a client-side libraries for data transformation and presentation.

The data transformation and presentation is enabled by a growing list of tools and SDKs that consume the raw data from SQD Network:

* Typescript based [Squid SDK](/en/sdk/squid-sdk) offers pluggable data targets for both online and offline use-cases
* Python-based [DipDup SDK](https://dipdup.io/docs/quickstart-evm?ref=blog.subsquid.io)
* Subgraphs (via [SQD Firehose adapter](/en/sdk/squid-sdk/subgraphs-support))
* ApeWorx (via [ape-subsquid plugin](/en/sdk/squid-sdk/apeworx))

## Feature matrix

|                                         |                                                                     SQD Network + Squid SDK                                                                     |             The Graph             |
| :-------------------------------------: | :-------------------------------------------------------------------------------------------------------------------------------------------------------------: | :-------------------------------: |
|           Programming language          |                                                                            Typescript                                                                           | AssemblyScript (compiled to WASM) |
|              Indexing speed             |                                                                           \~1k-50k bps                                                                          |           \~100-150 bps           |
|           ABI-based generator           |                                                                               Yes                                                                               |                Yes                |
| Real-time indexing (unfinalized blocks) |                                                                               Yes                                                                               |                 No                |
|              Off-chain data             |                                                                               Yes                                                                               |                 No                |
|               Data targets              |                                                                           Customizable                                                                          |           Postgres-only           |
|        Customizable DB migrations       |                                                                               Yes                                                                               |                 No                |
|        Factory contract indexing        |                                                                        Yes, via wildcards                                                                       |                Yes                |
|         Multi-contract indexing         |                                                                               Yes                                                                               |              Limited              |
|          Analytic data targets          |                                                                      BigQuery, Parquet, CSV                                                                     |                 No                |
|               Local setup               |                                                                               Easy                                                                              |     Requires an archival node     |
|               GraphQL API               |                                                                 Generated from `schema.graphql`                                                                 |  Generated from `schema.graphql`  |
|      Custom resolvers and mutations     |                                                                               Yes                                                                               |                 No                |
|              Subscriptions              |                                                                               Yes                                                                               |           Via middleware          |
|              Hosted service             |                                                                               Yes                                                                               |         Yes (to be sunset)        |
|           Secret env variables          |                                                                               Yes                                                                               |                 No                |
|                 Payment                 |                                                          [Fiat, subscription-based](/en/cloud/pricing)                                                          |         GRT, pay-per-query        |
|             Decentralization            | Decentralized data sourcing via [SQD Network](/en/network), with opt-in decentralized data targets (Kwil DB, Ceramic) and processing (via Lava, in development) |         The Graph network         |

## Architecture difference

By design, The Graph indexing node is a black-box that executes subgraphs compiled into WASM. The data is sourced directly from the archival node and local IPFS, and the processed data is stored in a built-in Postgres database. The data stored in the database is considered to be "sealed", so that no external process can modify or read the data except through the GraphQL interface.


# Squid SDK Troubleshooting
Source: https://docs.sqd.dev/en/sdk/squid-sdk/troubleshooting

Common issues, errors, and fixes for Squid SDK developers — schema migration failures, processor crashes, sync delays, and TypeScript build errors.

Many issues can be resolved by following the [best practices guide](/en/cloud/resources/best-practices) of SQD Cloud.

## Processor

<AccordionGroup>
  <Accordion title="`Error: data out-of-bounds` `ethers` errors on EVM">
    * Usually this means that you're using the decoder on the wrong data. Make sure that the decoder receives *only* the data you intend it to.

      Example: suppose you want to add the processing of a `Mint` event to a squid that is currently processing only `Transfer` events. You change the processor configuration to get the `Mint` events for you, but you forget to sort the events in the batch handler and a data item with a `Mint` event finds its way into a decoder of `Transfer`s.

    * Another common source of this error is faulty RPC endpoints. If your EVM processor crashes during RPC ingestion on a log with `'0x'` in its data field, try switching to another RPC provider or, if you are developing locally, to another Ethereum network emulator.
  </Accordion>

  <Accordion title="`BAD_DATA` when using a Multicall contract">
    This error can occur for a variety of reasons, but one common cause is choosing a Multicall deployment that is newer than the oldest blocks that have to be indexed. When [batch state queries](/en/sdk/squid-sdk/resources/tools/typegen/state-queries#batch-state-queries) are performed on historical chain state older than the Multicall deployment, EVM detects that and refuses to run.

    Solutions:

    1. Use an older Multicall deployment.
    2. Delay your chain state queries until a later block.

    These issues are explored in [Part 4 of the BAYC tutorial](/en/sdk/squid-sdk/tutorials/bayc/step-four-optimizations).
  </Accordion>
</AccordionGroup>

## Data sinks

<AccordionGroup>
  <Accordion title="`QueryFailedError: relation &#x22;...&#x22; does not exist`">
    It is likely that the generated migrations in the `db/migrations` folder are outdated and do not match the schema file.
    Recreate the migrations from scratch as detailed on [this page](/en/sdk/squid-sdk/resources/persisting-data/typeorm#updating-after-schema-changes).
  </Accordion>

  <Accordion title="`Query runner already released. Cannot run queries anymore`, or `too late to perform db updates, make sure you haven't forgot to await on db query`">
    If your squid [saves its data to a database](/en/sdk/squid-sdk/resources/persisting-data/typeorm), all operations with `ctx.store` are asynchronous. Make sure you `await` on all `store` operations like `upsert`, `update`, `find`, `save` etc.

    You may find the [require-await](https://eslint.org/docs/latest/rules/require-await) eslint rule to be helpful.
  </Accordion>

  <Accordion title="`QueryFailedError: invalid byte sequence for encoding &#x22;UTF8&#x22;: 0x00`">
    PostgreSQL doesn't support storing `NULL (\0x00)` characters in text fields. Usually the error occurs when a raw bytes string (like `UIntArray` or `Bytes`) is inserted into a `String` field. If this is the case, use hex encoding, e.g. using [`util-internal-hex`](https://github.com/subsquid/squid/tree/master/util/util-internal-hex) library. For addresses, use the [`ss58` encoding library](https://github.com/subsquid/squid/tree/master/ss58).
  </Accordion>
</AccordionGroup>

## GraphQL

<AccordionGroup>
  <Accordion title="API queries are too slow">
    * Make sure all the necessary fields are [indexed](/en/sdk/squid-sdk/reference/schema-file/indexes-and-constraints). The best way to do that is to perform the [query optimization procedure](/en/cloud/resources/query-optimization).
    * Annotate the schema and [set reasonable limits](/en/sdk/squid-sdk/reference/openreader-server/configuration/dos-protection) for the incoming queries to protect against DoS attacks
  </Accordion>

  <Accordion title="`response might exceed the size limit`">
    Make sure the input query has limits set or the entities are decorated with `@cardinality`. We recommend using `XXXConnection` queries for pagination. For configuring limits and max response sizes, see [DoS protection](/en/sdk/squid-sdk/reference/openreader-server/configuration/dos-protection).
  </Accordion>
</AccordionGroup>


# DipDup
Source: https://docs.sqd.dev/en/sdk/alternative-clients/dipdup

Python indexing framework that can use SQD Network as a data source

[DipDup](https://dipdup.io) is a Python indexing framework that can use [SQD Network](/en/network/overview) as a data source. It offers:

* SQLite, PostgreSQL and TimescaleDB data sinks
* GraphQL APIs based on Hasura

Development workflow uses the `dipdup` tool to generate a stub project. Once done with that, all you have to do is to define the data schema and the handlers. Take a look at their [quickstart](https://dipdup.io/docs/quickstart-evm) for more details.

With its handler-based architecture and the choice of Python as the transform logic language, DipDup is easier to develop for than [Squid SDK](/en/sdk/squid-sdk/overview), but has higher requirements on database IO bandwith and CPU. The IO bandwidth issue is partially solved by DipDup's caching layer used for database access.

## Getting Started

Visit the [DipDup documentation](https://dipdup.io/docs/) to get started with building Python-based indexers using SQD Network.


# Migrate SDK squids to Portal
Source: https://docs.sqd.dev/en/sdk/migration/evm-gateway-to-portal

Replace gateway data sources with Portal for improved performance and stability

<Card title="← Back to Portal Setup" icon="arrow-left" href="/en/portal/migration">
  Return to the Portal setup overview to explore other deployment options.
</Card>

This guide walks you through migrating SDK-based squids from gateways of the open private version of SQD Network to portals of the permissionless SQD Networkas your primary data source.

## Prerequisites

* An existing Squid SDK-based squid using `@subsquid/evm-processor` with `.setGateway()`
* Access to an SQD portal URL (see Step 2 for options)

## Migration steps

<Steps>
  <Step title="Install portal-api version">
    Navigate to your squid's folder and install the `portal-api` version of `@subsquid/evm-processor`:

    ```bash theme={"system"}
    npm i @subsquid/evm-processor@portal-api
    ```
  </Step>

  <Step title="Obtain a portal URL">
    Obtain a portal URL for your dataset. Choose the option that matches your setup:

    <Tabs>
      <Tab title="SQD Cloud user">
        If you're an existing user of [the SQD Cloud](https://app.subsquid.io):

        1. [Enable the preview mode](https://app.subsquid.io/preview)
        2. Navigate to [the portals page](https://app.subsquid.io/portal)
        3. Click on the tile of your network and follow the instructions

        <Check>
          Once configured, you can run your portal-powered squid both locally and in the Cloud. You can skip Step 3, it's already configured automatically.
        </Check>
      </Tab>

      <Tab title="Self-hosted portal">
        If you want to use a private portal:

        1. [Set up your portal](/en/portal/self-hosting)
        2. Get your portal's [dataset-specific API URL](/en/portal/self-hosting#using-your-portal) for the network you're interested in
      </Tab>

      <Tab title="Public portal (testing)">
        If you don't have an SQD Cloud account and want to test a portal, use the following URL template:

        ```bash theme={"system"}
        https://portal.sqd.dev/datasets/<dataset-slug>
        ```

        Replace `<dataset-slug>` with the last path segment of the gateway URL for your network found on the [EVM Networks listing](/en/data/evm).

        **Example:** The URL for the Ethereum dataset is:

        ```bash theme={"system"}
        https://portal.sqd.dev/datasets/ethereum-mainnet
        ```
      </Tab>
    </Tabs>
  </Step>

  <Step title="Update processor configuration">
    Configure your squid to ingest data from the portal by replacing the `.setGateway()` call in your processor configuration (typically in `src/processor.ts`) with `.setPortal()`.

    <Tabs>
      <Tab title="Local or self-hosted portal">
        Replace `.setGateway()` with `.setPortal()` using your portal URL directly:

        ```typescript src/processor.ts theme={"system"}
        export const processor = new EvmBatchProcessor()
          .setPortal('<portal URL for your dataset>')
        ```
      </Tab>

      <Tab title="Cloud deployment">
        If you're deploying to [the Cloud](/en/cloud/), source the portal URL from an environment variable configured in your manifest:

        ```typescript src/processor.ts theme={"system"}
        import { assertNotNull } from '@subsquid/util-internal'

        export const processor = new EvmBatchProcessor()
          .setPortal(
            assertNotNull(
              process.env.PORTAL_URL,
              'Required env variable PORTAL_URL is missing'
            )
          )
        ```
      </Tab>
    </Tabs>

    <Warning>
      If your squid uses an RPC endpoint, keep that configuration for now. Portals do not currently support real-time data ingestion, but this capability is coming soon.
    </Warning>
  </Step>
</Steps>

<Check>
  Your squid is now configured to source data from Portal. Test it locally to verify everything works correctly.
</Check>


# ApeWorx plugin
Source: https://docs.sqd.dev/en/sdk/squid-sdk/apeworx

Use SQD Network as a data source for ApeWorx — Python plugin for retrieving blocks, events, and transactions in bulk via the subsquid plugin.

# ApeWorx SQD plugin

<Info>
  The `subsquid` ApeWorx plugin is currently in beta. Please report any bugs or suggestions to [Squid Devs](https://t.me/HydraDevs) or open an issue at the [GitHub repo](https://github.com/subsquid/ape-subsquid/) of the plugin.
</Info>

[ApeWorx](https://apeworx.io) is a modular Web3 development framework for Python programmers. Among other things, it is capable of [retrieving blockchain data in bulk](https://docs.apeworx.io/ape/stable/userguides/data.html). The data can come from various sources, including SQD Network.

The network provides free access to blocks and event logs data. On long block ranges (>1k blocks) data retrieval is orders of magnitude (>10x) faster compared to RPC-based data sources.

## Basic usage

In an existing [ApeWorx installation](https://docs.apeworx.io/ape/stable/userguides/quickstart.html#installation), run the following to install the `subsquid` plugin:

```bash theme={"system"}
ape plugins install "ape-subsquid@git+https://github.com/subsquid/ape-subsquid.git@main"
```

To source data from SQD Network, start `ape console` as usual, e.g. with

```bash theme={"system"}
ape console --network ethereum:mainnet:geth
```

then pass

```python theme={"system"}
engine_to_use='subsquid'
```

to your data query methods. You can speed up the following calls:

1. Bulk block data retrieval with `chain.blocks.query`:
   ```python theme={"system"}
   df = chain.blocks.query(
       '*',
       start_block=18_000_000,
       stop_block=19_000_000,
       engine_to_use='subsquid'
   )
   ```
   This query retrieves data on 1M blocks in about 11 minutes.

2. Contract events retrieval:
   ```python theme={"system"}
   contract = Contract('0xdac17f958d2ee523a2206206994597c13d831ec7', abi='usdt.json')
   df = contract.Transfer.query(
       '*',
       start_block=18_000_000,
       stop_block=18_100_000,
       engine_to_use='subsquid'
   )
   ```
   This query retrieves 1.6M events emitted over 100k block in about 17 minutes.

<Warning>
  At the moment, all SQD Network datasets are updated only once every several thousands of blocks. The current dataset height can be retrieved with `get_network_height()`:

  ```python theme={"system"}
  from ape_subsquid import get_network_height

  get_network_height() # currently 19212115 while the chain is at 19213330
  ```

  Queries that request blocks above the current dataset height **will fail** with an `ape_subsquid.exceptions.DataRangeIsNotAvailable` exception. That includes queries without an explicitly set `stop_block`. If you don't need the recent data, you can explicitly request the data up to the block height of the SQD dataset, e.g.

  ```python theme={"system"}
  subsquid_height = get_network_height()
  df = chain.blocks.query(
      '*',
      start_block=19_000_000,
      stop_block=subsquid_height,
      engine_to_use='subsquid'
  )
  ```

  To get the latest data, retrieve the tail with the default engine and append it to the dataframe:

  ```python theme={"system"}
  taildf = chain.blocks.query('*', start_block=subsquid_height+1)
  df = pd.concat([df, taildf], ignore_index=True, copy=False)
  ```
</Warning>

<Info>
  When working with block ranges much longer than 1M blocks, the plugin may occasionally fail due to HTTP 503 errors rarely returned by the network. If you encounter this issue, split your block range into sub-1M blocks intervals and retrieve the data for each interval separately, retrying when the queries throw exceptions.
</Info>

## Networks support

The following networks available via plugins from `ape plugins list --all` are supported:

* `arbitrum`
* `avalanche`
* `base`
* `bsc`
* `ethereum`
* `fantom`
* `optimism`
* `polygon`
* `polygon-zkevm`

## Extra functionality

### Low-level queries

The plugin supports the following [low-level queries](https://docs.apeworx.io/ape/stable/methoddocs/api.html#module-ape.api.query):

* [`BlockQuery`](https://docs.apeworx.io/ape/stable/methoddocs/api.html#ape.api.query.BlockQuery)
* [`ContractEventQuery`](https://docs.apeworx.io/ape/stable/methoddocs/api.html#ape.api.query.ContractEventQuery)
* [`ContractCreationQuery`](https://docs.apeworx.io/ape/stable/methoddocs/api.html#ape.api.query.ContractCreationQuery) (only for [networks with traces support](/en/data/evm))
* [`AccountTransactionQuery`](https://docs.apeworx.io/ape/stable/methoddocs/api.html#ape.api.query.AccountTransactionQuery)

The following queries are **NOT** supported:

* [`BlockTransactionQuery`](https://docs.apeworx.io/ape/stable/methoddocs/api.html#ape.api.query.BlockTransactionQuery): uses hashes instead of numbers to identify blocks. This is not supported by SQD Network.
* [`ContractMethodQuery`](https://docs.apeworx.io/ape/stable/methoddocs/api.html#ape.api.query.ContractMethodQuery): not used by any high-level methods at the moment. If you need it, let us know at the [Squid Devs channel](https://t.me/HydraDevs).

### Account history retrieval

The plugin can be used to retrieve historical data for individual accounts, although this is much slower than working with specialized APIs such as Etherscan.

```python theme={"system"}
accountHistory = chain.history._get_account_history('vitalik.eth')
accountHistory.query('value', engine_to_use='subsquid')
```


# Squid SDK Examples
Source: https://docs.sqd.dev/en/sdk/squid-sdk/examples

A library of example Squid SDK indexers — DEX swaps, NFTs, stablecoins, multichain, ink! contracts — to learn from and bootstrap your own project.

<Columns>
  <Card title="Subsquid Labs on GitHub" icon="github" href="https://github.com/subsquid-labs">
    Example squid repositories.
  </Card>

  <Card title="Examples on docs.sqd.ai" icon="list" href="https://docs.sqd.ai/sdk/examples">
    Tag-filtered list of example squids.
  </Card>
</Columns>


# Squid SDK FAQ
Source: https://docs.sqd.dev/en/sdk/squid-sdk/faq

Frequently asked questions about Squid SDK — common errors, processor configuration, GraphQL setup, and migration guidance for indexer developers.

<AccordionGroup>
  <Accordion title="What are some real-world applications for which Squid SDK was a good fit?">
    Here is an incomplete list:

    * DeFi dashboards, tracking addresses and internal transactions
    * NFT marketplaces, with a dynamic sets of NFT contracts to watch
    * Historical price feeds, tracking Uniswap trades and Chainlink oracle contracts
    * Mining smart contract deployments and the bytecode
    * Real-time bots (\<1sec delay) triggered by on-chain activity
  </Accordion>

  <Accordion title="How does Squid SDK handle unfinalized blocks?">
    The SQD Network only serves finalized blocks and is typically \~1000 blocks behind the tip. The most recent blocks, as well as the unfinalized blocks are seamlessly handled by the SDK from a complementary RPC data source, set by the `chain` config. Potential chain reorgs are automatically handled under the hood. See [Indexing unfinalized blocks](/en/sdk/squid-sdk/resources/unfinalized-blocks) for details.
  </Accordion>

  <Accordion title="What is the latency for the data served by the squid?">
    Since the ArrowSquid release, the Squid SDK has the option to ingest unfinalized blocks directly from an RPC endpoint, making the indexing real-time.
  </Accordion>

  <Accordion title="How do I enable GraphQL subscriptions for local runs?">
    Add `--subscription` flag to the `serve` command defined in `commands.json`. See [Subscriptions](/en/sdk/squid-sdk/reference/openreader-server/configuration/subscriptions) for details.
  </Accordion>

  <Accordion title="How do squids keep track of their sync progress?">
    Depends on the data sink used. Squid processors that use [`TypeormDatabase`](/en/sdk/squid-sdk/resources/persisting-data/typeorm) keep their state in a [schema](https://www.postgresql.org/docs/current/sql-createschema.html), not in a table. By default the schema is called `squid_processor` (name must be overridden in [multiprocessor squids](/en/sdk/squid-sdk/resources/multichain)). You can view it with

    ```sql theme={"system"}
    select * from squid_processor.status;
    ```

    and manually drop it with

    ```sql theme={"system"}
    drop schema squid_processor cascade;
    ```

    to reset the processor status.

    Squids that store their data in [file-based datasets](/en/sdk/squid-sdk/resources/persisting-data/file) store their status in `status.txt` by default. This can be overridden by defining custom [database hooks](/en/sdk/squid-sdk/resources/persisting-data/file#hooks).
  </Accordion>

  <Accordion title="Is there a healthcheck endpoint for the indexer?">
    Yes, the processor exposes the key prometheus metrics at the `${process.env.PROMETHEUS_PORT}/metric` endpoint. The squids deployed to the SQD Cloud also publicly explose the metrics, see [Monitoring in the Cloud](/en/cloud/resources/monitoring)
  </Accordion>

  <Accordion title="Do squids have a debug mode?">
    Yes. To see all debug messages, set the `SQD_DEBUG` anv variable to `*`:

    ```bash title=".env" theme={"system"}
    SQD_DEBUG=*
    ```

    You can also enable debug messages just for a specific namespace. For example,

    ```bash title=".env" theme={"system"}
    SQD_DEBUG=sqd:processor:archive
    ```

    will show only the messages related to your squid's queries to the SQD Network.
  </Accordion>
</AccordionGroup>


# Fuel Indexing CLI Cheatsheet
Source: https://docs.sqd.dev/en/sdk/squid-sdk/fuel-indexing/cli-cheatsheet

Commonly used sqd CLI commands for Fuel indexers — init, deploy, run, secrets, and gateways with examples for the Squid SDK Fuel data source.

# CLI cheatsheet

This guide provides a quick reference to the commands needed to launch the squid and manage the database.

### Install dependencies

```sh theme={"system"}
npm i
```

### Compile the project

```sh theme={"system"}
npx tsc
```

### Launch Postgres database to store the data

```sh theme={"system"}
docker compose up -d
```

### Apply database migrations to create the target schema

```sh theme={"system"}
npx squid-typeorm-migration apply
```

### Run indexer

```sh theme={"system"}
node -r dotenv/config lib/main.js
```

### Check out the indexed swaps

```sh theme={"system"}
docker exec "$(basename "$(pwd)")-db-1" psql -U postgres \
  -c "SELECT id, logs_count, found_at FROM contract ORDER BY logs_count desc LIMIT 10"
```

You can use the `sqd` utility to shorten these commands and manage their interrelations. [Learn more here](/en/sdk/squid-sdk/squid-cli/commands-json).


# Fuel Datasource General Settings
Source: https://docs.sqd.dev/en/sdk/squid-sdk/fuel-indexing/fuel-datasource/general

General settings for Squid SDK Fuel datasource — configure data source, batch size, finality, prometheus metrics, and runtime options for indexers.

<Tip>
  The method documentation is also available inline and can be accessed via suggestions in most IDEs.
</Tip>

The following setters configure the global settings of `DataSourceBuilder` for Fuel Procesor. They return the modified instance and can be chained.

The only required configuration method is [`setGateway()`](#set-gateway). If you need real-time data, please also use [`setGraphql()`](#set-graphql).

* If you add both a SQD Network gateway and an RPC endpoint, the processor will obtain as much data as is currently available from the gateway, then switch to ingesting recent data via RPC.
* If you only add a SQD Network gateway, your data will be being several thousands of blocks behind the chain head most of the time.

### `setGateway(url: string | GatewaySettings)`

Use a [SQD Network](/en/network) gateway. The argument is either a string URL of the gateway or

```ts theme={"system"}
{
  url: string // gateway URL
  requestTimeout?: number // in milliseconds
}
```

### `setGraphql(settings?: GraphqlSettings)`

We must use regular GraphQL endpoint to get through the last mile and stay on top of the chain. This is a limitation, and we promise to lift it in the future.

```ts theme={"system"}
type GraphqlSettings = {
  url: string // e.g. https://mainnet.fuel.network/v1/graphql
  strideSize?: number; // `getBlock` batch call size, default 5
  strideConcurrency?: number; // num of concurrent connections, default 10
};
```

### `setBlockRange({from: number, to?: number})`

Limits the range of blocks to be processed. When the upper bound is specified, processor will terminate with exit code 0 once it reaches it.

Note that block ranges can also be specified separately for each data request. This method sets global bounds for all block ranges in the configuration.

### `includeAllBlocks(range?: {from: number, to?: number})`

By default, processor will fetch only blocks which contain requested items. This method modifies such behavior to fetch all chain blocks. Optionally a range of blocks can be specified for which the setting should be effective.


# Receipts
Source: https://docs.sqd.dev/en/sdk/squid-sdk/fuel-indexing/fuel-datasource/receipts

Subscribe to Fuel script execution receipts with addReceipt() in Squid SDK — filter by contract, asset, and receipt type for indexing on Fuel Network.

#### `addReceipt(options)`

Get some *or all* transactions on the network. `options` has the following structure:

```typescript theme={"system"}
{
  // data requests
  type?: ReceiptType[]
  contract?: string[]

  // related data retrieval
  transaction?: boolean

  range?: {
    from: number
    to?: number
  }
}
```

Data requests:

* `type` sets the type of the receipt. Receipt type has the following options: `'CALL' | 'RETURN' | 'RETURN_DATA' | 'PANIC' | 'REVERT' | 'LOG' | 'LOG_DATA' | 'TRANSFER' | 'TRANSFER_OUT' | 'SCRIPT_RESULT' | 'MESSAGE_OUT' | 'MINT' | 'BURN'`. Leave it undefined to subscribe to all receipts.
* `contract` sets the contract addresses to track. Leave it undefined to subscribe to all receipts.

Enabling the `transaction` flag will cause the processor to retrieve transactions that gave rise to the matching receipts. The data will be added to the appropriate iterables within the [block data](/en/sdk/squid-sdk/fuel-indexing/fuel-datasource/context-interfaces). You can also call `augmentBlock()` from `@subsquid/fuel-objects` on the block data to populate the convenience reference fields like `receipt.transaction`.

Note that receipts can also be requested by the other `FuelDataSource` methods as related data.

Selection of the exact fields to be retrieved for each transaction and the optional related data items is done with the `setFields()` method documented on the [Field selection](/en/sdk/squid-sdk/fuel-indexing/fuel-datasource/field-selection) page.

## Examples

Request all receipts of the `LOG_DATA` type and include parent transactions:

```ts theme={"system"}
processor
  .addReceipt({
    type: ["LOG_DATA"],
    transaction: true,
  })
  .build();
```


# Indexing Fuel Network data
Source: https://docs.sqd.dev/en/sdk/squid-sdk/fuel-indexing/indexing-receipts

Index Fuel Network data with Squid SDK — process receipts and transactions through batch handlers and persist into Postgres or files.

# Indexing transaction receipts

In this step-by-step tutorial we will look into a squid that indexes [Fuel Network](https://fuel.network/) data.

Pre-requisites: Node.js v20 or newer, Git, Docker.

## Download the project

Begin by retrieving the template and installing the dependencies:

```bash theme={"system"}
git clone https://github.com/subsquid-labs/fuel-example
cd fuel-example
npm i
```

## Configuring the data source

"Data source" is a component that defines what data should be retrieved and where to get it. To configure the data source to retrieve the data produced by the `receipt` field of the fuel transaction, we initialize it like this:

```typescript title="src/main.ts" theme={"system"}
const dataSource = new DataSourceBuilder()
  .setGateway('https://v2.archive.subsquid.io/network/fuel-mainnet')
  .setGraphql({
    url: 'https://mainnet.fuel.network/v1/graphql',
    strideConcurrency: 3,
    strideSize: 50
  })
  .setFields({
    receipt: {
      contract: true,
      receiptType: true
    }
  })
  .addReceipt({
    type: ['LOG_DATA']
  })
  .build()
```

Here,

* `https://v2.archive.subsquid.io/network/fuel-mainnet` is the address for the public SQD Network gateway for Fuel. Check out the exhaustive [public gateways list](/en/data/other).
* The argument of [`addReceipt()`](/en/sdk/squid-sdk/fuel-indexing/fuel-datasource/receipts) is a set of filters that tells the processor to retrieve all receipts of type `LOG`.
* The argument of [`setFields()`](/en/sdk/squid-sdk/fuel-indexing/fuel-datasource/field-selection) specifies the exact fields we need for every data item type. In this case we request `contract` and `receiptType` for receipt data items.

See also [`FuelDataSource` reference](/en/sdk/squid-sdk/fuel-indexing/fuel-datasource) and the comments in `main.ts` of the `fuel-example` repo.

With a data source it becomes possible to retrieve filtered blockchain data from [SQD Network](/en/network/overview), transform it and save the result to a destination of choice.

## Decoding the event data

The other part of the squid processor (the ingester process of the indexer) is the callback function used to process batches of the filtered data, the [batch handler](/en/sdk/squid-sdk/reference/processors/architecture#processorrun). In Fuel Squid SDK it is typically defined within a `run()` call, like this:

```typescript theme={"system"}
import {run} from '@subsquid/batch-processor'

run(dataSource, database, async ctx => {
  // data transformation and persistence code here
})
```

Here,

* `dataSource` is the data source object described in the previous section
* `database` is a [`Database` implementation](/en/sdk/squid-sdk/resources/persisting-data/overview) specific to the target data sink. We want to store the data in a PostgreSQL database and present with a GraphQL API, so we provide a [`TypeormDatabase`](/en/sdk/squid-sdk/resources/persisting-data/typeorm) object here.
* `ctx` is a [batch context](/en/sdk/squid-sdk/reference/processors/architecture#batch-context) object that exposes a batch of data (at `ctx.blocks`) and any data persistence facilities derived from `db` (at `ctx.store`). See [Block data for Fuel](/en/sdk/squid-sdk/fuel-indexing/fuel-datasource/context-interfaces) for details on how the data batches are presented.

Batch handler is where the raw on-chain data is decoded, transformed and persisted. This is the part we'll be concerned with for the rest of the tutorial.

We begin by defining a database and starting the data processing:

```typescript title="src/main.ts" theme={"system"}
run(dataSource, database, async ctx => {
  // Block items that we get from `ctx.blocks` are flat JS objects.
  //
  // We can use `augmentBlock()` function from `@subsquid/fuel-objects`
  // to enrich block items with references to related objects.
  let contracts: Map<String, Contract> = new Map()

  let blocks = ctx.blocks.map(augmentBlock)

  for (let block of blocks) {
    for (let receipt of block.receipts) {
      if (receipt.receiptType == 'LOG_DATA' && receipt.contract != null) {
        let contract = contracts.get(receipt.contract)
        if (!contract) {
          contract = await ctx.store.findOne(Contract, {where: {id: receipt.contract}})
          if (!contract) {
            contract = new Contract({
              id: receipt.contract,
              logsCount: 0,
              foundAt: block.header.height
            })
          }
        }
        contract.logsCount += 1
        contracts.set(contract.id, contract)
      }
    }
  }

  ctx.store.upsert([...contracts.values()])
})
```

This goes through all the receipts in the block, verifies that they have type `LOG_DATA"`, reads the contract field from the receipt and saves it to the database.

At this point the squid is ready for its first test run. Execute

```bash theme={"system"}
npx tsc
docker compose up -d
npx squid-typeorm-migration apply
node -r dotenv/config lib/main.js
```

You can verify that the data is being stored in the database by running

```bash theme={"system"}
docker exec "$(basename "$(pwd)")-db-1" psql -U postgres -c "SELECT * FROM contract"
```

Full code can be found [here](https://github.com/subsquid-labs/fuel-example).


# Squid SDK Glossary
Source: https://docs.sqd.dev/en/sdk/squid-sdk/glossary

Definitions of key Squid SDK terms — archives, gateways, FireSquid, processors, batch handlers, store, schema, and the SQD Network terminology.

### Archives

Deprecated term used for [SQD Network](/en/network) and for the data sourcing service of the deprecated FireSquid SDK version. Occasionally refers to a chain-specific endpoint available from either source (e.g. "an Ethereum archive"). The new terminology is:

* "Archives" as an abstract collection of services for some networks is replaced by "[SQD Network](/en/network)" (when referring to data location) or "SQD Network gateway" (when referring to the service)
* "public Archives" are replaced by the [open private version](/en/network/overview#open-private-network) of SQD Network
* "an archive" for a particular network is replaced by "a SQD Network gateway"

List of gateways for the open private SQD Network is available in [these docs](/en/data/evm) and via [`sqd gateways`](/en/sdk/squid-sdk/squid-cli/gateways).

**Not to be confused with [archive blockchain nodes](https://ethereum.org/developers/docs/nodes-and-clients/archive-nodes)**.

### `archive-registry`

The deprecated NPM package `@subsquid/archive-registry` that was used to look up squid data sources by network aliases (with `lookupArchive()` and a small CLI). We now recommend using raw gateway URLs instead of `lookupArchive()` calls in processor configuration. The exploratory CLI is replaced by [`sqd gateways`](/en/sdk/squid-sdk/squid-cli/gateways); list of available network-specific gateways is also [available in these docs](/en/data/evm).

### Block

An atomic state transition of a blockchain. Typically an ordered collection of transactions.

### Call

On [Substrate](#substrate), a call is a sub-routine changing the runtime state. An extrinsic consists of a root call which in turn may have sub-calls, thus calls executed by an extrinsic have parent-child relationship. For example, `util.batch` extrinsic has a single root call and multiple child calls. SQD processor is call-based rather than extrinsic based, as normally one is interested in specific calls changing the substrate state, no matter if it was part of a batch extrinsic, or it was wrapped in a sudo or proxy call.

### Cloud (former Aquarium)

A [cloud service](/en/cloud) for deploying [squids](#squid) in a serverless fashion maintained by Subsquid Labs GmbH.

### Contracts pallet

[`Contracts`](https://substrate-developer-hub.github.io/substrate-how-to-guides/docs/pallet-design/add-contracts-pallet/) is a [pallet](#pallet) developed by Parity to execute WASM-based smart contracts.

### ETL

Stands for Extract-Transform-Load. A pipeline to extract data from the source, enrich, transform, normalize and load into the target data store.

### Event

An operation performed during a blockchain state transition that results in emission of a structured log message. Subsequently, the message can be retrieved from blockchain nodes.

### EVM

Stands for the Ethereum Virtual Machine. An instruction set and the runtime specification originally developed for the Ethereum network which was later adopted by many other chains (e.g. BSC, Polygon).

### Extrinsic

On [Substrate](#substrate), a generalized transaction externally submitted to a Substrate runtime for execution. There are [technical nuances](https://substrate.stackexchange.com/questions/2248/is-a-transaction-an-extrinsic) differentiating transactions and extrinsics.

### FRAME pallets

A collection of standard [Substrate](#substrate) pallets maintained by Parity. Currently kept in the [Substrate Repo](https://github.com/paritytech/substrate/tree/master/frame).

### Frontier pallet

[Frontier](https://github.com/polkadot-evm/frontier) is a [Substrate](#substrate) [pallet](#pallet) implementing the [Ethereum Virtual Machine](#evm). In particular, Substrate chains with Frontier pallet support EVM-based smart contracts.

### GraphQL

A declarative query language and an API specification language developed by Facebook as an alternative to REST. See the official [GraphQL docs](https://graphql.org/) for more details.

### Hot blocks

An alternative term for [unfinalized blocks](/en/sdk/squid-sdk/resources/unfinalized-blocks) that emphasizes the fact that the data extracted from these blocks is volatile. Sometimes used in Squid SDK code and comments.

### ink!

An SDK (software development kit) and a smart-contract language for developing WASM-based smart contracts, maintained by Parity. The contracts developed with ink! are compiled into a WASM blob compatible with the API exposed by the [`Contracts` pallet](#contracts-pallet). More details [here](https://use.ink).

### OpenReader

1. SQD's own open source [GraphQL server](/en/sdk/squid-sdk/reference/openreader-server/overview), built in-house. No longer recommended for new projects running PostgreSQL due to its [limitations](/en/sdk/squid-sdk/reference/openreader-server/overview#limitations). See [Serving GraphQL](/en/sdk/squid-sdk/resources/serving-graphql) to learn more.

2. The GraphQL [schema generation library](https://github.com/subsquid/squid-sdk/tree/master/graphql/openreader) at the heart of 1. Implements [OpenCRUD](https://www.opencrud.org/).

### Pallet

A portable module that can be added to a [Substrate](#substrate) runtime. Typically, contains a self-contained implementation of a business logic that can be re-used across multiple chains.

### Schema file

A file describing the target data schema for a squid, normally called `schema.graphql`. The file uses a GraphQL dialect to define entities, properties and relations. See [details here](/en/sdk/squid-sdk/reference/schema-file).

### State

A key-value map defining the internal worldview of an EVM contract of a Substrate runtime at a specific point in time. The consensus algorithm ensures that the honest majority of the nodes agree on the runtime state.

### Storage

On [Substrate](#substrate), a persistent key-value database kept by the chain nodes. It is used to access the current and historical [state](#state). See details on the [Substrate docs page](https://docs.substrate.io/fundamentals/state-transitions-and-storage/)

### Squid

A project consisting of an [ETL](#etl) for extracting and transforming on-chain data (squid processor), and optionally an API to present the data (squid API).

### Squid processor

The [ETL](#etl) part of the squid. Extracts on-chain data from an [SQD Network](/en/network) gateway and/or directly from chain RPC, then transforms and optionally enriches it with external data. Saves the result into a target [data sink](/en/sdk/squid-sdk/reference/store).

### Squid API

The data presentation part of the squid. Typically, it's an [OpenReader](#openreader) GraphQL API auto-generated from a schema file.

### Substrate

[Substrate](https://substrate.io) is a framework for developing blockchain runtimes. Used to develop the Polkadot, Kusama chains and all the parachains.

### Substrate Runtime

The code that defines the state transition logic of a blockchain, and by extention its business logic. See details on the [Substrate docs page](https://docs.substrate.io/fundamentals/architecture/).

### Squid SDK

[Squid SDK](/en/sdk/squid-sdk/overview) is a collection of open-source libraries for building [squids](#squid).

### Typegen

A tool generating strongly typed data access classes from a metadata in some format. Squid SDK includes typegen tools:

* for accessing EVM smart contract data based on the contract ABI
* for accessing event, extrinsics and storage data based on [Substrate](#substrate) metadata
* for accessing [ink!](#ink) smart contract data based on the contract metadata

See [this section](/en/sdk/squid-sdk/resources/tools/typegen/generation) for documentation on these tools.

### WASM

A portable binary code format and an execution environment specification. WASM programs enjoy deterministic outputs and near-native execution speeds, which makes WASM an attractive alternative to EVM.


# sqd CLI cheatsheet
Source: https://docs.sqd.dev/en/sdk/squid-sdk/how-to-start/cli-cheatsheet

Cheatsheet of commonly used sqd CLI commands — init, deploy, logs, secrets, gateways — with examples for everyday Squid SDK development workflows.

# Squid CLI cheatsheet

The [`sqd` CLI tool](/en/sdk/squid-sdk/squid-cli) has [built-in aliasing](/en/sdk/squid-sdk/squid-cli/commands-json) that picks up the commands defined in `commands.json` in the project root. In all [squid templates](/en/sdk/squid-sdk/how-to-start/squid-development#templates) this file is pre-populated with some handy scripts briefly described below.

One can always inspect the available commands defined in `commands.json` with

```
sqd --help
```

The commands defined by `commands.json` will appear in the `SQUID COMMANDS` help sections.

Before using the `sqd` CLI tool, make sure all the project dependencies are installed:

```sh theme={"system"}
npm i
```

### Building the squid

```sh theme={"system"}
sqd build             Build the squid project
sqd clean             Delete all build artifacts
```

### Running the squid

<Info>
  Both `sqd up` and `sqd down` assume that the `docker compose` command is supported and the `docker` deamon is running. Modify the definitions in `commands.json` accordingly if `docker-compose` should be used instead.
</Info>

```
sqd up                  Start a local PG database
sqd down                Drop the local PG database
sqd run [PATH]          Run all the services defined in squid.yaml locally
sqd serve               Start the GraphQL server
sqd serve:prod          Start the GraphQL API server with caching and limits
```

### DB migrations

Read [TypeORM Migration generation](/en/sdk/squid-sdk/resources/tools/migrations-gen) for details.

```
sqd migration:apply             apply pending migrations
sqd migration:generate          generate the migration for the schema defined in schema.graphql
sqd migration:clean             clean the db/migrations folder 
```

### Code generation

Consult [TypeORM Model generation](/en/sdk/squid-sdk/resources/tools/model-gen) for TypeORM model generation details, and [Type-safe decoding](https://docs.subsquid.io/sdk/resources/tools/typegen/) for type generation.

<Info>
  Depending on the template, `sqd typegen` is aliased to a different typegen tool specific to the chain type and thus has different usage. Consult `sqd typegen --help` for details.
</Info>

```
sqd codegen        Generate TypeORM entities from schema.graphql
sqd typegen        Generate data access classes for an ABI file(s) in the ./abi folder
```


# Environment set up
Source: https://docs.sqd.dev/en/sdk/squid-sdk/how-to-start/development-environment-set-up

Prepare your development environment for Squid SDK — install Node, Docker, sqd CLI, and required tools for building TypeScript blockchain indexers.

### Node.js

To install Node.js:

<Tabs>
  <Tab title="Linux">
    Use the package manager of your distro or the [official binaries](https://nodejs.org/en/download).
  </Tab>

  <Tab title="Mac">
    Use the [official binaries](https://nodejs.org/en/download) or [Homebrew](https://nodejs.org/en/download/package-manager/#alternatives-2).
  </Tab>

  <Tab title="Windows">
    The best bet is to leverage WSL2 and follow [this guide](https://docs.microsoft.com/en-us/windows/dev-environment/javascript/nodejs-on-wsl). [Official installer](https://nodejs.org/en/download) also works.
  </Tab>
</Tabs>

Make sure that your Node.js installation is v16 or newer. To check an existing installation, run:

```bash theme={"system"}
node --version
```

### Squid CLI

Follow [these instructions](/en/sdk/squid-sdk/squid-cli/installation).

### Git

Squid CLI uses [Git](https://git-scm.com) to retrieve templates. To install it:

<Tabs>
  <Tab title="Linux">
    Use the package manager of your distro.
  </Tab>

  <Tab title="Mac">
    Use the [official installer](https://sourceforge.net/projects/git-osx-installer/) or any one of the [alternative approaches](https://git-scm.com/download/mac).
  </Tab>

  <Tab title="Windows">
    Use [Git for Windows](https://git-scm.com/download/win).
  </Tab>
</Tabs>

### Docker

Most squids [use a database](/en/sdk/squid-sdk/resources/persisting-data/typeorm) to store the processed data. Install Docker to manage local squid databases with convenience.

<Tabs>
  <Tab title="Linux">
    * [Here’s an instruction](https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository) on how to install Docker on Ubuntu
    * On other distros consider using their package managers
  </Tab>

  <Tab title="Mac">
    Install the [Desktop version](https://docs.docker.com/desktop/mac/install/).
  </Tab>

  <Tab title="Windows">
    Install the [Desktop version](https://docs.docker.com/desktop/windows/install/).
  </Tab>
</Tabs>


# Project structure
Source: https://docs.sqd.dev/en/sdk/squid-sdk/how-to-start/layout

Squid SDK project folder layout — entry points, schema file, processor source, migrations, manifest, environment, and conventions for clean structure.

# Squid project structure

All files and folders except `package.json` are optional.

* `package.json` -- Configuration file for dependencies and the build script (invoked with `npm run build`). Hard requirement for deploying to [SQD Cloud](/en/cloud).
* `package-lock.json` OR `yarn.lock` OR `pnpm-lock.yaml` -- Dependencies shrinkwrap. Required for [Cloud](/en/cloud) deployment, except those that [override the dependencies installation command](/en/cloud/reference/manifest#cmd).
* `tsconfig.json` -- Configuration of `tsc`. Required for most squids.
* [Deployment manifest](/en/cloud/reference/manifest) (`squid.yaml` by default) -- Definitions of squid services used for running it locally with [`sqd run`](/en/sdk/squid-sdk/squid-cli/run) and deploying to [SQD Cloud](/en/cloud).
* `.squidignore` -- Files and patterns to be excluded when sending the squid code to the [Cloud](/en/cloud). When not supplied, some files will still be omitted: see the [reference page](/en/cloud/reference/squidignore) for details.
* `schema.graphql` -- [The schema definition file](/en/sdk/squid-sdk/reference/schema-file). Required if your squid [stores its data in PostgreSQL](/en/sdk/squid-sdk/resources/persisting-data/typeorm).
* `/src` -- The TypeScript source code folder for the squid processor.
  * `/src/main.ts` -- The entry point of the squid processor process. Typically, contains a `processor.run()` call.
  * `/src/processor.ts` -- Processor object ([EVM](/en/sdk/squid-sdk/reference/processors/evm-batch) or [Substrate](/en/sdk/squid-sdk/reference/processors/substrate-batch)) definition and configuration.
  * `/src/model/generated` -- The folder for the TypeORM entities generated from `schema.graphql`.
  * `/src/model` -- The module exporting the entity classes.
  * `/src/server-extension/resolvers` -- A folder for [user-defined GraphQL resolvers](/en/sdk/squid-sdk/reference/openreader-server/configuration/custom-resolvers) used by [OpenReader](/en/sdk/squid-sdk/reference/openreader-server).
  * `/src/types` -- A folder for types generated by the Substrate [typegen](/en/sdk/squid-sdk/resources/tools/typegen) tool for use in data decoding.
  * `/src/abi` -- A folder for modules generated by the EVM [typegen](/en/sdk/squid-sdk/resources/tools/typegen) tool containing type definitions and data decoding boilerplate code.
* `/db` -- The designated folder with the [database migrations](/en/sdk/squid-sdk/resources/persisting-data/typeorm).
* `/lib` -- The output folder for the compiled squid code.
* `/assets` -- A designated folder for custom user-provided files (e.g. static data files to seed the squid processor with).
* `/abi` -- A designated folder for JSON ABI files used as input by the EVM [typegen](/en/sdk/squid-sdk/resources/tools/typegen).
* `docker-compose.yml` -- A Docker compose file for local runs. Has a Postgres service definition by default.
* `.env` -- Defines environment variables used by `docker-compose.yml` and when the squid is run locally.
* `typegen.json` -- The config file for the Substrate [typegen](/en/sdk/squid-sdk/resources/tools/typegen) tool.
* `commands.json` -- [User-defined scripts](/en/sdk/squid-sdk/squid-cli/commands-json) picked up by [Squid CLI](/en/sdk/squid-sdk/squid-cli/commands-json). See also the [CLI cheatsheet](/en/sdk/squid-sdk/how-to-start/cli-cheatsheet).


# The frontier package
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/frontier

Index EVMs running on Substrate (Frontier) with Squid SDK — use the @subsquid/frontier package to decode Frontier-EVM events and contract calls.

# `@subsquid/frontier`

The way the Frontier EVM pallet exposes EVM logs and transaction may change due to runtime upgrades. [`@subsquid/frontier`](https://github.com/subsquid/squid-sdk/tree/master/substrate/frontier) provides helper methods that are aware of the upgrades:

#### `getEvmLog(event: Event): EvmLog`

Extract the EVM log data from `EVM.Log` event.

#### `getTransaction(call: Call): LegacyTransaction | EIP2930Transaction | EIP1559Transaction`

Extract the transaction data from `Ethereum.transact` call with additional fields depending on the EVM transaction type.

#### `getTransactionResult(ethereumExecuted: Event): {from: string, to: string, transactionHash: string, status: 'Succeed' | 'Error' | 'Revert' | 'Fatal', statusReason: string}`

Extract transaction result from an `Ethereum.Executed` event.

See also the [Frontier EVM guide](/en/sdk/squid-sdk/resources/substrate/frontier-evm).

#### Example

```typescript theme={"system"}
const processor = new SubstrateBatchProcessor()
  .setGateway('https://v2.archive.subsquid.io/network/astar-substrate')
  .setRpcEndpoint('https://astar-rpc.dwellir.com')
  .addEthereumTransaction({})
  .addEvmLog({})

processor.run(new TypeormDatabase(), async ctx => {
  for (const block of ctx.blocks) {
    for (const event of block.events) {
      if (event.name === 'EVM.Log') {
        // no need to supply any extra data to determine
        // the runtime version: event has all necessary references
        const {address, data, topics} = getEvmLog(event)

        // process evm log data
      }
    }
    for (const call of block.calls) {
      if (call.name==='Ethereum.transact') {
        const txn = getTransaction(call)
        // process evm txn data
      }   
    }
  }
})
```


# Squid SDK Logger
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/logger

Native logger module of Squid SDK — pino-based structured logging with log levels, JSON output, and integration with batch processor lifecycle.

A [`Logger`](https://github.com/subsquid/squid/tree/master/util/logger) interface is injected into the [handler context](/en/sdk/squid-sdk/reference/processors/architecture#batch-context) with `ctx.log`. It is bound to the namespace `sqd:processor:mapping`. The context logger is a recommended way of logging for squid processors.

`Logger` exposes the following logging levels, in order of increasing severity:

* `TRACE`
* `DEBUG`
* `INFO`
* `WARN`
* `ERROR`
* `FATAL`

By default, the logging level is set to `INFO`.

And here is an example:

```typescript theme={"system"}
processor.run(new TypeormDatabase(), async (ctx) => {
  ctx.log.trace("Trace Log example");
  ctx.log.debug("Debug Log example");
  ctx.log.info("Info Log example");
  ctx.log.warn("Warn Log example");
  ctx.log.error("Error Log example");
  ctx.log.fatal("Fatal Log example")
});
```

## Overriding the log level

The log level can be overridden by setting a matching namespace selector to one of the `SQD_TRACE`, ..., `SQD_FATAL` env variables. In particular, to set the handler logs level to `DEBUG` set the environment variable `SQD_DEBUG` to `sqd:processor:mapping`:

```bash theme={"system"}
SQD_DEBUG=sqd:processor:mapping
```

The namespace selector supports wildcards, so one can also enable internal debug logs of `@subsquid/substrate-processor` with

```
SQD_DEBUG=sqd:processor*
```

since all processor context loggers inherit the processor-level namespace `sqd:processor`.

## Accessing logs of a deployed Squid

Processor logs can be inspected once the squid is deployed to Cloud:

```bash theme={"system"}
sqd logs -n <name> -s <slot> -f --level=<level>
```

or

```bash theme={"system"}
sqd logs -n <name> -t <tag> -f --level=<level>
```

<Accordion title="For older version-based deployments...">
  ...the slot string is `v${version}`, so use

  ```bash theme={"system"}
  sqd logs -n <name> -s v<version> -f --level=<level>
  ```

  Check out the [Slots and tags guide](/en/cloud/resources/slots-and-tags) to learn more.
</Accordion>

The available levels are:

* `info`
* `warning`
* `debug`
* `error` - will fetch messages emitted by
  * `ctx.log.error`
  * `ctx.log.fatal`
  * `ctx.log.trace`

See [CLI Reference](/en/sdk/squid-sdk/squid-cli/logs) or `sqd logs --help` for a full list of log options supported by SQD Cloud.


# AND/OR filters
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/openreader-server/api/and-or-filters

Combine GraphQL query filters with AND/OR operators in OpenReader — basic logic combinators for refining entity queries served by Squid SDK.

## Overview

Our GraphQL implementation offers a vast selection of tools to filter and section results. One of these is the `where` clause, very common in most database query languages and [explained here](/en/sdk/squid-sdk/reference/openreader-server/api/queries#filter-query-results--search-queries) in detail.

In our GraphQL server implementation, we included logical operators to be used in the `where` clause, allowing to group multiple parameters in the same `where` argument using the `AND` and `OR` operators to filter results based on more than one criteria.

Note that the [newer](/en/sdk/squid-sdk/reference/openreader-server/overview#supported-queries) and [more advanced](/en/sdk/squid-sdk/reference/openreader-server/api/paginate-query-results) `{entityName}sConnection` queries support exactly the same format of the `where` argument as the older `{entityName}s` queries used in the examples provided here.

### Example of an `OR` clause:

Fetch a list of `accounts` that either have a balance bigger than a certain amount, or have a specific id.

```graphql theme={"system"}
query {
  accounts(
    orderBy: balance_DESC, 
    where: {
      OR: [
        {balance_gte: "240000000000000000"}
        {id_eq: "CksmaBx9rKUG9a7eXwc5c965cJ3QiiC8ELFsLtJMYZYuRWs"}
      ]
    }
  ) {
    balance
    id
  }
}

```

### Example of `AND` clause:

Fetch a list of `accounts` that have a balance between two specific amounts:

```graphql theme={"system"}
query {
  accounts(
    orderBy: balance_DESC, 
    where: {
      AND: [
        {balance_lte: "240000000000000000"}
        {balance_gte: "100000000000000"}
      ]
    }
  ) {
    balance
    id
  }
}

```


# Cross-relation queries
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/openreader-server/api/cross-relation-field-queries

Filter Squid SDK GraphQL queries by fields of related entities — cross-relation predicates in OpenReader for traversing nested data in queries.

# Cross-relation field queries

## Introduction

The [previous section](/en/sdk/squid-sdk/reference/openreader-server/api/nested-field-queries) has already demonstrated that queries can return not just scalars such as a String, but also fields that refer to object or entity types. What's even more interesting is that queries can leverage fields of related objects to filter results.

Let's take this sample schema with two entity types and a one-to-many relationship between them:

```graphql title="schema.graphql" theme={"system"}
type Account @entity {
    id: ID!
    wallet: String!
    balance: Int!
    history: [HistoricalBalance!] @derivedFrom(field: "account")
}

type HistoricalBalance @entity {
    "Unique identifier"
    id: ID!
    
    "Related account"
    account: Account!
    
    "Balance"
    balance: Int!
}
```

With the functionality offered by cross-relation field queries, we could ask for `Account`s that have at least some `historicalBalance`s with a `balance` smaller than a certain threshold:

```graphql theme={"system"}
query MyQuery {
  accounts(where: {historicalBalances_some: {balance_lt: "10000000000"}}) {
    id
  }
}
```

This allows to query based not just on the entity itself, but on the related entities as well, which is intuitively a very powerful feature.

`*_some` is not the only operator available for making cross-relation field queries. A short description of each such operator is provided in the sections below.

## The `*_every` filter

Returns entities for which **all** of the nested entities linked via the related field satisfy the condition. Example:

```graphql title="schema.graphql" theme={"system"}
query MyQuery {
  accounts(where: {historicalBalances_every: {balance_lt: "10000000000"}}) {
    id
  }
}
```

This query will return all `Account`s where **each and every one** of the `HistoricalBalance` entities related to them have a `balance` smaller than the threshold. It is sufficient for a single `HistoricalBalance` to have a `balance` larger than the set value to make sure that the related `Account` is not returned in the query.

## The `*_none`  filter

Returns entities for which **none** of the nested entities linked via the related field satisfy the condition. Example:

```graphql theme={"system"}
query MyQuery {
  accounts(where: {historicalBalances_none: {balance_lt: "10000000000"}}) {
    id
  }
}
```

The query will return all `Account`s in which not a single related `HistoricalBalance` has a `balance` smaller than the set threshold.

## The `*_some` filter

Returns entities for which **at least one** of the nested entities linked via the related field satisfies the condition. Example:

```graphql theme={"system"}
query MyQuery {
  accounts(where: {historicalBalances_some: {balance_lt: "10000000000"}}) {
    id
  }
}
```

All `Account`s that have at least some `historicalBalance`s with a `balance` smaller than `10000000000` will be returned. This means that a single `HistoricalBalance` satisfying the condition is sufficient for the related `Account` to become a part of the results.

## `{entityName}sConnection` queries

Same as always, the `where` argument works for these queries in exactly the same way as it does for `{entityName}s` queries used in examples above. For example this query

```graphql theme={"system"}
query MyQuery {
  accountsConnection(orderBy: id_ASC, where: {historicalBalances_some: {balance_lt: "10000000000"}}) {
    edges {
      node {
        id
      }
    }
  }
}
```

will return (in an appropriately shaped response) IDs for all `Accounts` that have at least some `historicalBalance`s with a `balance` smaller than `10000000000`.


# OpenReader GraphQL Intro
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/openreader-server/api/intro

Introduction to GraphQL queries served by OpenReader, the Squid SDK GraphQL server — entity queries, filters, sorting, and pagination basics.

<Info>
  At the moment, [Squid SDK GraphQL server](/en/sdk/squid-sdk/reference/openreader-server) can only be used with squids that use Postgres as their target database.
</Info>

GraphQL is an API query language, and a server-side runtime for executing queries using a custom type system. Head over to the [official documentation website](https://graphql.org/learn/) for more info.

A GraphQL API served by the [GraphQL server](/en/sdk/squid-sdk/reference/openreader-server) has two components:

1. Core API is defined by the [schema file](/en/sdk/squid-sdk/reference/schema-file).
2. Extensions added via [custom resolvers](/en/sdk/squid-sdk/reference/openreader-server/configuration/custom-resolvers).

In this section we cover the core GraphQL API, with short explanations on how to perform GraphQL queries, how to paginate and sort results. This functionality is supported via [OpenReader](https://github.com/subsquid/squid-sdk/tree/master/graphql/openreader), SQD's own implementation of [OpenCRUD](https://www.opencrud.org).


# JSON queries
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/openreader-server/api/json-queries

Query entities with object-typed and JSON fields in OpenReader — Squid SDK GraphQL syntax for filtering and selecting nested JSON content.

The possibility of defining JSON objects as fields of a type in a GraphQL schema has been explained in the [schema reference](/en/sdk/squid-sdk/reference/schema-file).

This guide is focusing on how to query such objects and how to fully leverage their potential. Let's take the example of this (non-crypto related, for once😁) schema:

```graphql title="schema.graphql" theme={"system"}
type Entity @entity {
    id: ID!
    a: A
}

type A {
    a: String
    b: B
}

type B {
    a: A
    b: String
    e: Entity
}
```

It's composed of one entity and two JSON objects definitions, used in a "nested" way.

Let's now look at a simple query:

```graphql theme={"system"}
query {
    entities(orderBy: id_ASC) {
        id
        a { a }
    }
}
```

This will return a result such as this one (imagining this data exists in the database):

```graphql theme={"system"}
{
  entities: [
    {id: '1', a: {a: 'a'}},
    {id: '2', a: {a: 'A'}},
    {id: '3', a: {a: null}},
    {id: '4', a: null}
  ]
}
```

Simply enough, the first two objects have an object of type `A` with some content inside, the third one has an object, but its `a` field is `null` and the fourth one simply does not have an `A` object at all.


# Nested field queries
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/openreader-server/api/nested-field-queries

Query Squid SDK entities related to other entities via nested fields in OpenReader — traverse one-to-many and many-to-many relations in GraphQL.

With OpenReader, fields of an Entity that contain fields themselves are shown as nested fields and it is possible to filter these as well. GraphQL queries can traverse related objects and their fields, letting clients fetch lots of related data in one request, instead of making several roundtrips as one would need in a classic REST architecture.

As an example, this query searches for all `accounts` whose balance is bigger than a threshold value, fetching the `id` and `balance` simple fields, as well as the `historicalBalances` **nested field**.

```graphql theme={"system"}
query {
  accounts(orderBy: balance_ASC, where: {balance_gte: "250000000000000000"}) {
    id
    balance
    historicalBalances {
      balance
      date
      id
    }
  }
}

```

A nested field is a list (one account can have multiple `historicalBalances`) of objects with fields of their own. These objects can be filtered, too.

In the following query the `historicalBalances` are filtered in order to only return the balances created after a certain date:

```graphql theme={"system"}
query {
  accounts(orderBy: balance_ASC, where: {balance_gte: "250000000000000000"}) {
    id
    balance
    historicalBalances(where: {date_lte: "2020-10-31T11:59:59.000Z"}, orderBy: balance_DESC) {
      balance
      date
      id
    }
  }
}

```

Note that the [newer](/en/sdk/squid-sdk/reference/openreader-server/overview#supported-queries) and [more advanced](/en/sdk/squid-sdk/reference/openreader-server/api/paginate-query-results) `{entityName}sConnection` queries support exactly the same format of the `where` argument as the older `{entityName}s` queries used in the examples provided here.


# OpenReader Pagination
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/openreader-server/api/paginate-query-results

Paginate large GraphQL query results in OpenReader — first/after, last/before cursor pagination, and offset-based syntax for Squid SDK APIs.

# Paginate query results

There are multiple ways to obtain this behavior, let's take a look at a couple of them.

## Cursor based pagination

Cursors are used to traverse across entities of an entity set. They work by returning a pointer ("cursor") to a specific entity which can then be used to fetch the next batch. The batch will start with the entity after the one the cursor points to. For cursor-based pagination, OpenReader follows the [Relay Cursor Connections spec](https://relay.dev/graphql/connections.htm).

Currently, only forward pagination is supported. If your use case requires bidirectional pagination please let us know at our [Telegram channel](https://t.me/HydraDevs).

In SQD GraphQL server, cursor based pagination is implemented with `{entityName}sConnection` queries available for every entity in the input schema. These queries require an explicitly supplied [`orderBy` argument](/en/sdk/squid-sdk/reference/openreader-server/api/sorting), and *the field that is used for ordering must also be requested by the query itself*. Check out [this section](/en/sdk/squid-sdk/reference/openreader-server/api/paginate-query-results#important-note-on-orderby) for a valid query template.

Example: this query fetches a list of videos where `isExplicit` is true and gets their count.

```graphql theme={"system"}
query {
  videosConnection(orderBy: id_ASC, where: { isExplicit_eq: true }) {
    totalCount
    edges {
      node {
        id
        title
      }
    }
  }
}
```

### **Operator `first`**

The `first` operator is used to fetch a specified number of entities from the beginning of the output.

Example: Fetch the first 5 videos:

```graphql theme={"system"}
query Query1 {
  videosConnection(orderBy: id_ASC, first: 5) {
    edges {
      node {
        id
        title
      }
    }
  }
}
```

### **PageInfo object**

`PageInfo` is a "virtual" entity that can be requested from any `{entityName}sConnection` query (see below). It returns the relevant cursors and some page information:

```graphql theme={"system"}
pageInfo {
  startCursor
  endCursor
  hasNextPage
  hasPreviousPage
}
```

### **Operator `after`**

Example: Fetch the first 10 channels, ordered by `createdAt`. Then, in a second query, fetch the next 10 channels:

```graphql theme={"system"}
query FirstBatchQ {
  channelsConnection(first: 10, orderBy: createdAt_ASC) {
    pageInfo {
      endCursor
      hasNextPage
    }
    edges {
      node {
        id
        handle
        createdAt
      }
    }
  }
}

query SecondBatchQ {
  channelsConnection(after: <endCursor>, orderBy: createdAt_ASC) {
    pageInfo {
      endCursor
      hasNextPage
    }
    edges {
      node {
        id
        handle
        createdAt
      }
    }
  }
}
```

### **Important Note on `orderBy`**

The field chosen to `orderBy` needs to be present in the query itself. For example, any `after` query must follow this template:

```graphql theme={"system"}
query QueryName {
  <entityName>sConnection(after: <endCursor>, orderBy: <fieldNameToOrderBy>_ASC) {
    pageInfo {
      endCursor
      hasNextPage
      ...<any other page info fields>...
    }
    edges {
      node {
        <fieldNameToOrderBy>
        ...<any other fields of interest>...
      }
    }
  }
}
```

Otherwise, the returned result wouldn't be ordered correctly.

### Examples

An interactive example of using cursor-based pagination can be found in [this repo](https://github.com/subsquid-labs/cursor-pagination-client-example).

## Paginating with `{entityName}s` queries

### Arguments `limit` and `offset`

In a list of entities returned by a query, the `limit` argument specifies how many should be retained, while the `offset` argument specifies how many should be skipped first. Default values are `50` for `limit` and `0` for `offset`.

### **Limit results**

Example: Fetch the first 5 channels:

```graphql theme={"system"}
query {
  channels(limit: 5) {
    id
    handle
  }
}
```

### **Limit results from an offset**

Example: Fetch 5 channels from the list of all channels, starting with the 6th one:

```graphql theme={"system"}
query {
  channels(limit: 5, offset: 5) {
    id
    handle
  }
}
```


# Entity queries
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/openreader-server/api/queries

Run basic GraphQL entity queries against a Squid SDK GraphQL API — single-entity, list, and connection-style queries with filtering and pagination.

# Entity Queries

## Introduction

OpenReader auto-generates queries from the `schema.graphql` file. All entities defined in the schema can be queried over the GraphQL endpoint.

## Exploring queries

Let’s take a look at the different queries you can run using the GraphQL server. We’ll use examples based on a typical channel/video schema.

### Simple entity queries

#### **Fetch list of entities**

Fetch a list of channels:

```graphql theme={"system"}
query {
  channels {
    id
    handle
  }
}
```

or, using a [newer](/en/sdk/squid-sdk/reference/openreader-server/overview#supported-queries) and [more advanced](/en/sdk/squid-sdk/reference/openreader-server/api/paginate-query-results) `{entityName}sConnection` query

```graphql theme={"system"}
query {
  channelsConnection(orderBy: id_ASC) {
    edges {
      node {
        id
        handle
      }
    }
  }
}
```

#### **Fetch an entity using its unique fields**

Fetch a channel by a unique id or handle:

```graphql theme={"system"}
query Query1 {
  channelByUniqueInput(where: { id: "1" }) {
    id
    handle
  }
}

query Query2 {
  channelByUniqueInput(where: { handle: "Joy Channel" }) {
    id
    handle
  }
}
```

### Filter query results / search queries

#### **The `where` argument**

You can use the `where` argument in your queries to filter results based on some field’s values. You can even use multiple filters in the same `where` clause using the `AND` or the `OR` operators.

For example, to fetch data for a channel named `Joy Channel`:

```graphql theme={"system"}
query {
  channels(where: { handle_eq: "Joy Channel" }) {
    id
    handle
  }
}
```

Note that `{entityName}sConnection` queries support exactly the same format of the `where` argument:

```graphql theme={"system"}
query {
  channelsConnection(orderBy: id_ASC, where: { handle_eq: "Joy Channel"}) {
    edges {
      node {
        id
        handle
      }
    }
  }
}
```

#### **Supported Scalar Types**

SQD supports the following scalar types:

* String
* Int
* Float
* BigInt
* Boolean
* Bytes
* DateTime

#### **Equality Operators (`_eq`)**

`_eq` is supported by all the scalar types.

The following are examples of using this operator on different types:

* Fetch a list of videos where `title` is "Bitcoin"
* Fetch a list of videos where `isExplicit` is "true"
* Fetch a list of videos `publishedOn` is "2021-01-05"

```graphql theme={"system"}
query Query1 {
  videos(where: { title_eq: "Bitcoin" }) {
    id
    title
  }
}

query Query2 {
  videos(where: { isExplicit_eq: true }) {
    id
    title
  }
}

query Query3 {
  videos(where: { publishedOn_eq: "2021-01-05" }) {
    id
    title
  }
}
```

#### **Greater than or less than operators (`gt`, `lt`, `gte`, `lte`)**

The `_gt` (greater than), `_lt` (less than), `_gte` (greater than or equal to), `_lte` (less than or equal to) operators are available on `Int, BigInt, Float, DateTime` types.

The following are examples of using these operators on different types:

* Fetch a list of videos published before "2021-01-05"
* Fetch a list of channels before block "999"

```graphql theme={"system"}
query Query1 {
  videos(where: { publishedOn_gte: "2021-01-05" }) {
    id
    title
  }
}

query Query2 {
  channels(where: { block_lte: "999" }) {
    id
    handle
  }
}
```

#### **Text search or pattern matching operators (`_contains`, `_startsWith`, `_endsWith`)**

The `_contains`, `_startsWith`, `_endsWith` operators are used for pattern matching on string fields.

Example:

```graphql theme={"system"}
query Query1 {
  videos(where: { title_contains: "Bitcoin" }) {
    id
    title
  }
}

query Query2 {
  videos(where: { title_endsWith: "cryptocurrency" }) {
    id
    title
  }
}
```


# Union type resolution
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/openreader-server/api/resolve-union-types-interfaces

Resolve GraphQL union types and interfaces in Squid SDK OpenReader — use the __typename meta field to discriminate between concrete entity types.

Use cases for [Union types](/en/sdk/squid-sdk/reference/schema-file/unions-and-typed-json) have been discussed in the [schema reference](/en/sdk/squid-sdk/reference/schema-file). Here, we discuss how to query union types.

Let's take this modified schema from the [Substrate tutorial](/en/sdk/squid-sdk/tutorials/substrate):

```graphql title="schema.graphql" theme={"system"}
type Account @entity {
  id: ID! #Account address
  events: [Event]
}

type WorkReport {
  id: ID! #event id
  addedFiles: [[String]]
  deletedFiles: [[String]]
  extrinsicId: String
  blockHash: String!
}

type JoinGroup {
  id: ID!
  owner: String!
  extrinsicId: String
  blockHash: String!
}

type StorageOrder {
  id: ID!
  fileCid: String!
  extrinsicId: String
  blockHash: String!
}

union Event = WorkReport | JoinGroup | StorageOrder
```

Here, an `Event` will have different fields depending on the underlying type. This query demonstrates how to request different fields for each of these types:

```graphql theme={"system"}
query MyQuery {
  accounts {
    events {
      __typename
      ... on WorkReport {
        id
        blockHash
        extrinsicId
        deletedFiles
      }
      ... on JoinGroup {
        id
        blockHash
        extrinsicId
      }
      ... on StorageOrder {
        id
        blockHash
        extrinsicId
      }
    }
    id
  }
}
```

The special `__typename` field allows users to discern the returned object type without relying on comparing the sets of regular fields. For example, in the output of the query above `JoinGroup` and `StorageOrder` events can only be distingushed by looking at the `__typename` field. Here is a possible output to illustrate:

```json theme={"system"}
{
  "data": {
    "accounts": [
      {
        "events": [
          {
            "__typename": "WorkReport",
            "id": "0000584321-000001-01cdb",
            "blockHash": "0x01cdb3cb6fa00f62fd20220104f1d740a53518b63517419da8a89325d065562b",
            "extrinsicId": "0000584321-000001-01cdb",
            "deletedFiles": []
          }
        ],
        "id": "cTKmzHG3RHa1yhujyZpPnNL17p8a48Av3JFwDjpttLcxeSo26"
      },
      {
        "events": [
          {
            "__typename": "JoinGroup",
            "id": "0000584598-000010-d06ec",
            "blockHash": "0xd06ec6716e96108e24987ef03d23c857ef3b467dd057d7a32c4e123fe5a8df36",
            "extrinsicId": "0000584598-000004-d06ec"
          }
        ],
        "id": "cTKqevWRdvbNNAQ3hLxhsNYhQ8pf5YGkYnnVjgjLNiVr4kd7a"
      },
      {
        "events": [
          {
            "__typename": "StorageOrder",
            "id": "0000584627-000013-1fa19",
            "blockHash": "0x1fa19ae98731afad853ffd491fcbc0c3dcda6b8b7f5a2d56ac6c4c1eb9e4f95e",
            "extrinsicId": "0000584627-000005-1fa19"
          }
        ],
        "id": "cTGYF8jvcpnRmgNopqT4nVs5rWHEviAAdRdfNrZE8NFz2Av7B"
      }
    ]
  }
}
```


# Sorting
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/openreader-server/api/sorting

Sort GraphQL query results in OpenReader with the orderBy argument — sort by entity fields, nested fields, and apply multiple sort keys at once.

## Sort order

The sort order (ascending vs. descending) is set by specifying an `ASC` or `DESC` suffix for the column name in the `orderBy` input object, e.g. `title_DESC`.

### **Sorting entities**

Example: Fetch a list of videos sorted by their titles in an ascending order:

```graphql theme={"system"}
query {
  videos(orderBy: title_ASC) {
    id
    title
  }
}
```

or

```graphql theme={"system"}
query {
  videos(orderBy: [title_ASC]) {
    id
    title
  }
}
```

### **Sorting entities by multiple fields**

The `orderBy` argument takes an array of fields to allow sorting by multiple columns.

Example: Fetch a list of videos that is sorted by their titles (ascending) and then on their published date (descending):

```graphql theme={"system"}
query {
  videos(orderBy: [title_ASC, publishedOn_DESC]) {
    id
    title
    publishedOn
  }
}
```


# Access control
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/openreader-server/configuration/authorization

Configure authentication and authorization in OpenReader — protect Squid SDK GraphQL APIs with API keys, signed JWTs, and per-entity rules.

To implement access control, define the following function in the designated `src/server-extension/check` module:

```typescript theme={"system"}
import {
  RequestCheckContext
} from '../../node_modules/@subsquid/graphql-server/src/check'

export async function requestCheck(
  req: RequestCheckContext
): Promise<boolean | string> {
  ...
}
```

Once defined, this function will be called every time a request arrives. Then,

* if the function returns `true`, the request is processed as usual;
* if the function returns `false`, the server responds with `'{"errors":[{"message":"not allowed"}]}'`;
* if the function returns an `errorString`, the server responds with `` `{{"errors":[{"message":"${errorString}"}]}` ``.

The request information such as HTTP headers and GraphQL selections is available in the context. This makes it possible to authenticate the user that sent the query and either allow or deny access. The decision may take the query contents into account, allowing for some authorization granularity.

## `RequestCheckContext`

The context type has the following interface:

```typescript theme={"system"}
RequestCheckContext {
  http: {uri: string, method: string, headers: HttpHeaders}
  operation: OperationDefinitionNode
  operationName: string | null
  schema: GraphQLSchema
  context: Record<string, any>
  model: Model
}
```

Here,

* `http` field contains the low level HTTP info. Information on headers is stored in a `Map` from lowercase header names to values. For example, `req.http.headers.get('authorization')` is the value of the authorization header.
* `operation` is the root [`OperationDefinitionNode`](https://graphql-js.org/api/interface/OperationDefinitionNode) of the tree describing the query. Useful if the authorization decision depends on the query contents.
* `operationName` is the query name.
* `schema` is a [`GraphQLSchema`](https://graphql-js.org/api/class/GraphQLSchema) object.
* `context` holds a [`PoolOpenreaderContext`](https://github.com/subsquid/squid-sdk/blob/master/graphql/openreader/src/db.ts) at `context.openreader`. It can be used to access the database, though this is highly discouraged: the interfaces involved are considered to be internal and are subject to change without notice.
* `model` is an Openreader data [`Model`](https://github.com/subsquid/squid-sdk/blob/master/graphql/openreader/src/model.ts).

## Sending user data to resolvers

Authentication data such as user name can be passed from `requestCheck()` to a [custom resolver](/en/sdk/squid-sdk/reference/openreader-server/configuration/custom-resolvers) through Openreader context:

```typescript theme={"system"}
export async function requestCheck(req: RequestCheckContext): Promise<boolean | string> {
  ...
  // obtain user name e.g. by decoding the authentication header
  let user = ...
  // save user name to Openreader context
  req.context.openreader.user = user
  ...
}
```

A custom resolver that retrieves it may look like this:

```typescript theme={"system"}
@Resolver()
export class UserCommentResolver {
  constructor(private tx: () => Promise<EntityManager>) {}

  @Query(() => [UserCommentCountQueryResult])
  async countUserComments(
    @Ctx() ctx: any
  ): Promise<UserCommentCountQueryResult[]> {
    let user = ctx.openreader.user
    let manager = await this.tx()
    let result: UserCommentCountQueryResult[] =
      await manager
        .getRepository(UserComment)
        .query(`
          SELECT COUNT(*) as total
          FROM user_comment
          WHERE "user" = '${user}'
        `)
    return result
  }

  @Mutation(() => Boolean)
  async addComment(
    @Arg('text') comment: string,
    @Ctx() ctx: any
  ): Promise<Boolean> {
    let user = ctx.openreader.user
    let manager = await this.tx()
    await manager.save(new UserComment({
      id: `${user}-${comment}`,
      user,
      comment
    }))
    return true
  }
}
```

See full code in [this branch](https://github.com/subsquid-labs/access-control-example/tree/interacting-with-resolver).

This approach does not work with [subscriptions](/en/sdk/squid-sdk/reference/openreader-server/configuration/subscriptions).

## Examples

A simple strategy that authorizes anyone with a `12345` token to perform any query can be implemented with

```typescript title="src/server-extension/check.ts" theme={"system"}
import {
  RequestCheckContext
} from '../../node_modules/@subsquid/graphql-server/src/check'

export async function requestCheck(
  req: RequestCheckContext
): Promise<boolean | string> {
  return req.http.headers.get('authorization')==='Bearer 12345'
}
```

A more elaborate example with two users authorized to perform different query sets is available in [this repo](https://github.com/subsquid-labs/access-control-example). Another great example of using `requestCheck()` for authorization can be [spotted in the wild](https://github.com/reef-defi/reef-subsquid-processor/tree/master/src/server-extension) in the code of a squid used by [Reef](https://reef.io).


# Caching
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/openreader-server/configuration/caching

Enable response caching in OpenReader for faster Squid SDK GraphQL queries — TTL-based cache with cache-busting on schema or data updates.

The GraphQL API server provided by `@subsquid/graphql-server` supports caching via additional flags. It is done on a per-query basis. The whole response is cached for a specified amount of time (`maxAge`).

To enable caching when deploying to SQD Cloud, add the caching flags to the `serve:prod` command definition at [`commands.json`](/en/sdk/squid-sdk/squid-cli/commands-json), then use that command to run the server in the [deployment manifest](/en/cloud/reference/manifest#deploy). Cloud currently supports only in-memory cache.
For example, snippets below will deploy a GraphQL API server with a `100Mb` in-memory cache and invalidation time of `5` seconds:

```json title="commands.json" theme={"system"}
...
  "serve:prod": {
    "description": "Start the GraphQL API server with caching and limits",
    "cmd": [ "squid-graphql-server",
      "--dumb-cache", "in-memory",
      "--dumb-cache-ttl", "5000",
      "--dumb-cache-size", "100",
      "--dumb-cache-max-age", "5000" ]
  }
...
```

```yaml title="squid.yaml" theme={"system"}
# ...
deploy:
  # other services ...
  api:
    cmd: [ "sqd", "serve:prod" ]
```

Caching flags list is available via `npx squid-graphql-server --help`. Here are some more details on them:

### `--dumb-cache <cache-type>`

Enables cache, either `in-memory` or `redis`. For `redis`, a Redis connection string must be set by a variable `REDIS_URL`. SQD Cloud deployments currently support only `in-memory` cache.

### `--dumb-cache-size <mb>`

Cache max size. Applies only to in-memory cache.

### `--dumb-cache-max-age <ms>`

A globally set max age in milliseconds. The cached queries are invalidated after that period of time.

### `--dumb-cache-ttl <ms>`

Time-to-live for in-memory cache entries. Applies only to in-memory cache. The entries are eligible for eviction from the cache if not updated for longer than the time-to-live time.


# Custom API extensions
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/openreader-server/configuration/custom-resolvers

Extend a Squid SDK GraphQL API with custom resolvers — add aggregations, computed fields, and ad-hoc queries beyond the schema-derived entities.

# Custom GraphQL resolvers

One can extend the GraphQL API generated by OpenReader with custom queries. To do that, one can define GraphQL [query resolvers](https://www.apollographql.com/docs/apollo-server/data/resolvers/) in the designated module `src/server-extension/resolvers`. **Note that all resolver classes (including any additional types) must be exported by `src/server-extension/resolvers/index.ts`.**

A custom resolver should import [TypeGraphQL](https://typegraphql.com) types and [use annotations](https://typegraphql.com/docs/resolvers.html) provided by the library to define query arguments and return types. If your squid lacks a `type-graphql` dependency, add it with:

```bash theme={"system"}
npm i type-graphql
```

Custom resolvers are normally used in combination with [TypeORM EntityManager](https://typeorm.io/entity-manager-api) for accessing the API server target database. It is automatically injected when defined as a single constructor argument of the resolver.

## Examples

#### Simple entity counter

```typescript theme={"system"}
import { Query, Resolver } from 'type-graphql'
import type { EntityManager } from 'typeorm'
import { Burn } from '../model'

@Resolver()
export class CountResolver {
  constructor(private tx: () => Promise<EntityManager>) {}

  @Query(() => Number)
  async totalBurns(): Promise<number> {
    const manager = await this.tx()
    return await manager.getRepository(Burn).count()
  }
}
```

This example is designed to work with the `evm` template:

1. grab a test squid as described [here](/en/sdk/squid-sdk/how-to-start/squid-development);
2. install `type-graphql`;
3. save the example code to `src/server-extension/resolver.ts`;
4. re-export `CountResolver` at `src/server-extension/resolvers/index.ts`:
   ```ts theme={"system"}
   export { CountResolver } from '../resolver'
   ```
5. rebuild the squid with `npm run build`;
6. (re)start the GraphQL server with `npx squid-graphql-server`.

`totalBurns` selection will appear in the [GraphiQL playground](http://localhost:4350/graphql).

#### Custom SQL query

```typescript theme={"system"}
import { Arg, Field, ObjectType, Query, Resolver } from 'type-graphql'
import type { EntityManager } from 'typeorm'
import { MyEntity } from '../model'

// Define custom GraphQL ObjectType of the query result
@ObjectType()
export class MyQueryResult {
  @Field(() => Number, { nullable: false })
  total!: number

  @Field(() => Number, { nullable: false })
  max!: number

  constructor(props: Partial<MyQueryResult>) {
    Object.assign(this, props);
  }
}

@Resolver()
export class MyResolver {
  // Set by depenency injection
  constructor(private tx: () => Promise<EntityManager>) {}

  @Query(() => [MyQueryResult])
  async myQuery(): Promise<MyQueryResult[]> {
    const manager = await this.tx()
    // execute custom SQL query
    const result: = await manager.getRepository(MyEntity).query(
      `SELECT 
        COUNT(x) as total, 
        MAX(y) as max
      FROM my_entity 
      GROUP BY month`)
    return result
  }
}
```

#### More examples

Some great examples of `@subsquid/graphql-server`-based custom resolvers can be spotted in the wild in the [Rubick repo](https://github.com/kodadot/rubick/tree/main/src/server-extension/resolvers) by [KodaDot](https://github.com/kodadot).

For more examples of resolvers, see [TypeGraphQL examples repo](https://github.com/MichalLytek/type-graphql/tree/master/examples).

## Logging

To keep logging consistent across the entire GraphQL server, use `@subsquid/logger`:

```ts theme={"system"}
import {createLogger} from '@subsquid/logger'

// using a custom namespace ':my-resolver' for resolver logs
const LOG = createLogger('sqd:graphql-server:my-resolver')
LOG.info('created a dedicated logger for my-resolver')
```

`LOG` here is a [logger object](/en/sdk/squid-sdk/reference/logger) identical to `ctx.log` interface-wise.

## Interaction with global settings

* `--max-response-size` used for [DoS protection](/en/sdk/squid-sdk/reference/openreader-server/configuration/dos-protection) is ignored in custom resolvers.
* [Caching](/en/sdk/squid-sdk/reference/openreader-server/configuration/caching) works on custom queries in exactly the same way as it does on the schema-derived queries.

## Troubleshooting

#### `Reflect.getMetadata is not a function`

Add `import 'reflect-metadata'` on top of your custom resolver module and install the package if necessary.


# DoS protection
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/openreader-server/configuration/dos-protection

Protect your Squid SDK GraphQL API from denial-of-service queries — depth, complexity, and rate-limit configuration in OpenReader.

The squid [GraphQL API server](https://github.com/subsquid/squid-sdk/tree/master/graphql/graphql-server) accepts the following optional start arguments to fend off heavy queries.

To enable the protection, add the corresponding flags to `serve` and `serve:prod` command definitions at [`commands.json`](/en/sdk/squid-sdk/squid-cli/commands-json):

```json theme={"system"}
...
  "serve": {
    "description": "Start the GraphQL API server",
    "cmd": ["squid-graphql-server", "--max-root-fields", "10", "--max-response-size", "1000"]
  },
  "serve:prod": {
    "description": "Start the GraphQL API server in prod",
    "cmd": ["squid-graphql-server", "--max-root-fields", "10", "--max-response-size", "1000", "--dumb-cache", "in-memory"]
  },
...
```

When deploying to [SQD Cloud](/en/cloud), make sure to use `serve:prod` to start the GraphQL `api` service in the [deployment manifest](/en/cloud/reference/manifest#deploy):

```yaml title="squid.yaml" theme={"system"}
# ...
deploy:
  # other services ...
  api:
    cmd: [ "sqd", "serve:prod" ]
```

**`--max-request-size <kb>`**

The argument limits the size of a request in kilobytes. It is set to `256kb` by default.

**`--max-root-fields <count>`**

The maximal allowed number of root-level queries in a single GraphQL request.

**`--max-response-size <nodes>`**

This option limits the *estimated* query response size and makes server return an error if it exceeds the provided value. Note that the estimated size depends only on the decorators in `schema.graphql` and the requested fields.

The estimate is the product of the cardinality of the entity list and the response item weight.

The cardinality is estimated as the minimum of

* the `limit` argument of the query (`Infinity` if not provided)
* `@cardinality` value defined in `schema.graphql` (if the requested entity type is decorated in the schema file, `Infinity` otherwise)
* the size of the argument list of the `_eq` and `id_in` filters in the `where` clause (if applicable)

In particular, if there are no `@cardinality` decorators in `schema.graphql`, the client queries must explicitly provide limits or where filters to pass through.

The response item weight is calculated recursively:

* `byteWeight` for each scalar field or `1` if it's not decorated
* for non-scalar fields, the estimated weight times the estimated cardinality (if it's a list)
* each non-leaf node in the query AST tree adds a weight of `1`

In a nutshell, assuming that the schema file is properly decorated with `@cardinality` and `@byteWeight`, the estimated response size should roughly be at the same scale as the byte size of the query result.

**`--subscription-max-response-size <nodes>`**

Same as `--max-response-size` but for live query [subscriptions](/en/sdk/squid-sdk/reference/openreader-server/configuration/subscriptions).

#### Example

Assume the schema is defined as follows, and the server is launched with `--max-response-size 1000`.

```ts title="schema.graphql" theme={"system"}
type Foo @entity {
    id: ID!
    // a large string 
    bigField: String! @byteWeight(value: 1000.0)
    bar: Bar!
}

type Bar @entity {
    id: ID!
    // bar.foos typically contain about 100 items
    foos: [Foo!]! @derivedFrom(field: "bar") @cardinality(value: 100)
    bazs: [Baz!]! @derivedFrom(field: "bar")
}

// there are around 100 entities of type Baz
type Baz @entity @cardinality(value: 100) {
    id: ID!
    bar: Bar!
}
```

The following queries will be bounced:

```graphql theme={"system"}
query A {
    bars {
        id
    }
}

query B {
    bars(limit: 1001) {
        id
    }
}

query C {
    bars(limit: 100) {
        id 
        foos(limit: 10) {
            id
        }
    }
}

query D {
    bars(limit: 10) {
        id 
        foos {
            id
        }
    }
}

query E {
    foos(id_eq: "1") {
        id
        bigField
    }
}
```

* The estimated cardinality of query A is `Infinity`
* The estimated cardinality of query B `1001` and so the expected size exceeds the limit
* The estimated cardinality of query C is `100` while the item size is `13`, so the size is estimated to `1300`.
* The estimated cardinality of query D is `10` while the item size is `103`, so the size is estimated to `1030`.
* The estimated cardinality of query E is `1` while the item size is `1001` (due to `bigField` having weight `1000`).

At the same time, the following queries will go through:

```graphql theme={"system"}
query A {
    bars(limit: 100) {
        id
    }
}

query B {
    bars(limit: 3) {
        id
        foos {
            id
        }
        bazs {
            id
        }
    }
}

query C {
    bars {
        id
        bar {
            foos(where: { id_in ["1", "2" ]}) {
                id
            }
            id
        }
    }
}
```


# Subscriptions
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/openreader-server/configuration/subscriptions

Push real-time entity updates over a websocket in OpenReader subscriptions — Squid SDK GraphQL subscriptions for live API consumers.

<Warning>
  RAM usage of subscriptions scales poorly under high load, making the feature unsuitable for most production uses. There are currently no plans to fix this issue.
</Warning>

OpenReader supports [GraphQL subscriptions](https://www.apollographql.com/docs/react/data/subscriptions/) via live queries. To use these, a client opens a websocket connection to the server and sends a `subscription` query there. The query body is then repeatedly executed (every 5 seconds by default) and the results are sent to the client whenever they change.

To enable subscriptions, add the additional `--subscriptions` flag to the `squid-graphql-server` startup command. The poll interval is configured with the `--subscription-poll-interval` flag. For details and a full list of available options, run

```bash theme={"system"}
npx squid-graphql-server --help
```

For each entity types, the following queries are supported for subscriptions:

* `${EntityName}ById` -- query a single entity
* `${EntityName}s` -- query multiple entities with a `where` filter
  Note that despite being [deprecated](/en/sdk/squid-sdk/reference/openreader-server/overview#supported-queries) from the regular query set, `${EntityName}s` queries will continue to be available for subscriptions going forward.

## Local runs

To enable subscriptions, add the `--subscriptions` flag to `serve` and `serve:prod` commands at [`commands.json`](/en/sdk/squid-sdk/squid-cli/commands-json):

```json title="commands.json" theme={"system"}
...
  "serve": {
    "description": "Start the GraphQL API server",
    "cmd": ["squid-graphql-server", "--subscriptions"]
  },
  "serve:prod": {
    "description": "Start the GraphQL API server in prod",
    "cmd": ["squid-graphql-server", "--subscriptions", "--dumb-cache", "in-memory"]
  },
...
```

A `ws` endpoint will be available the usual `localhost:<GQL_PORT>/graphql` URL.

## SQD Cloud deployments

For SQD Cloud deployments, make sure to use the updated `serve:prod` command in the [deployment manifest](/en/cloud/reference/manifest#deploy):

```yaml title="squid.yaml" theme={"system"}
# ...
deploy:
  # other services ...
  api:
    cmd: [ "sqd", "serve:prod" ]
```

The subscription `wss` endpoint will be available at the canonical API endpoint `wss://{org}.subsquid.io/{name}/v/v{version}/graphql`.

## Example

Let's take the following simple schema with account and transfer entities:

```graphql file=schema.graphql theme={"system"}
type Account @entity {
  "Account address"
  id: ID!
  transfersTo: [Transfer!] @derivedFrom(field: "to")
  transfersFrom: [Transfer!] @derivedFrom(field: "from")
}

type Transfer @entity {
  id: ID!
  timestamp: DateTime! @index
  from: Account!
  to: Account!
  amount: BigInt! @index
}
```

After modifying `commands.json` start the GraphQL server with subscriptions with

```bash theme={"system"}
npx squid-graphql-server
```

The following sample [script](https://github.com/subsquid/squid-substrate-template/blob/main/scripts/sub-client.js) will subscribe to the most recent transfers (by `timestamp`).

```typescript theme={"system"}
const WebSocket = require('ws')
const { createClient } = require('graphql-ws');

const port = process.env.GQL_PORT || 4350
const host = process.env.GQL_HOST || 'localhost'
const proto = process.env.GQL_PROTO || 'ws'


const client = createClient({
  webSocketImpl: WebSocket,
  url: `${proto}://${host}:${port}/graphql`,
});

client.subscribe(
  {
    query: `
    subscription {
        transfers(limit: 5, orderBy: timestamp_DESC) {
            amount
            blockNumber
            from {
              id
            }
            to {
              id
            }
        }
    }  
    `,
  },
  {
    next: (data) => {
      console.log(`New transfers: ${JSON.stringify(data)}`);
    },
    error: (error) => {
      console.error('error', error);
    },
    complete: () => {
      console.log('done!');
    },
  }
);
```


# OpenReader Server Overview
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/openreader-server/overview

OpenReader is the open source GraphQL server bundled with Squid SDK — generates an OpenCRUD-style API from your schema.graphql with caching and DoS protection.

<Info>
  OpenReader is no longer recommended for use in new squid projects [relying on PostgreSQL](/en/sdk/squid-sdk/resources/persisting-data/typeorm). See [Serving GraphQL](/en/sdk/squid-sdk/resources/serving-graphql) to learn about the new options and the [Limitations](#limitations) section to understand our motivation.
</Info>

OpenReader is a server that presents data of PostgreSQL-powered squids as a GraphQL API. It relies on the [eponymous library](https://github.com/subsquid/squid-sdk/tree/master/graphql/openreader) lib of the Squid SDK for schema generation. [Schema file](/en/sdk/squid-sdk/reference/schema-file) is used as an input; the resulting API supports [OpenCRUD](https://www.opencrud.org/) queries for the entities defined in the schema.

To start the API server based on `schema.graphql` install `@subsquid/graphql-server` and run the following in the squid project root:

```bash theme={"system"}
npx squid-graphql-server
```

The `squid-graphql-server` executable supports multiple optional flags to enable [caching](/en/sdk/squid-sdk/reference/openreader-server/configuration/caching), [subscriptions](/en/sdk/squid-sdk/reference/openreader-server/configuration/subscriptions), [DoS protection](/en/sdk/squid-sdk/reference/openreader-server/configuration/dos-protection) etc. Its features are covered in the next sections.

The API server listens at port defined by the `GQL_PORT` environment variable (defaults to `4350`). The database connection is configured with the env variables `DB_NAME`, `DB_USER`, `DB_PASS`, `DB_HOST`, `DB_PORT`.

In [SQD Cloud](/en/cloud), OpenReader is usually ran as the `api:` service in the `deploy:` section of the [Deployment manifest](/en/cloud/reference/manifest).

## Supported queries

The details of the supported OpenReader queries can be found in a separate section [Core API](/en/sdk/squid-sdk/reference/openreader-server/api). Here is a brief overview of the queries generated by OpenReader for each entity defined in the schema file:

* the squid last processed block is available with `squidStatus { height }` query
* a "get one by ID" query with the name `{entityName}ById` for each [entity](/en/sdk/squid-sdk/reference/schema-file/entities) defined in the schema file
* a "get one" query for [`@unique` fields](/en/sdk/squid-sdk/reference/schema-file/indexes-and-constraints), with the name `{entityName}ByUniqueInput`
* Entity queries named `{entityName}sConnection`. Each query supports rich filtering support, including [field-level filters](/en/sdk/squid-sdk/reference/openreader-server/api/queries), composite [`AND` and `OR` filters](/en/sdk/squid-sdk/reference/openreader-server/api/and-or-filters), [nested queries](/en/sdk/squid-sdk/reference/openreader-server/api/nested-field-queries), [cross-relation queries](/en/sdk/squid-sdk/reference/openreader-server/api/cross-relation-field-queries) and [Relay-compatible](https://relay.dev/graphql/connections.htm) cursor-based [pagination](/en/sdk/squid-sdk/reference/openreader-server/api/paginate-query-results).
* [Subsriptions](/en/sdk/squid-sdk/reference/openreader-server/configuration/subscriptions) via live queries
* (Deprecated in favor of Relay connections) Lookup queries with the name `{entityName}s`.

[Union and typed JSON types](/en/sdk/squid-sdk/reference/schema-file/unions-and-typed-json) are mapped into [GraphQL Union Types](https://graphql.org/learn/schema/#union-types) with a [proper type resolution](/en/sdk/squid-sdk/reference/openreader-server/api/resolve-union-types-interfaces) with `__typename`.

## Built-in custom scalar types

The OpenReader GraphQL API defines the following custom scalar types:

* `DateTime` entity field values are presented in the ISO format
* `Bytes` entity field values are presented as hex-encoded strings prefixed with `0x`
* `BigInt` entity field values are presented as strings

## Limitations

* RAM usage of [subscriptions](/en/sdk/squid-sdk/reference/openreader-server/configuration/subscriptions) scales poorly under high load, making the feature unsuitable for most production uses. There are currently no plans to fix this issue.
* Setting up custom resolvers for subscriptions is unreasonably hard.
* `@subsquid/graphql-server` depends on the deprecated Apollo Server v3.


# Processor architecture
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/processors/architecture

Anatomy of Squid SDK processors — data source, query builder, batch handler, and Store roles in the processor lifecycle and the SQD Network protocol.

The processor service is a Node.js process responsible for data ingestion, transformation and data persisting into the target database. By [convention](/en/sdk/squid-sdk/how-to-start/layout), the processor entry point is at `src/main.ts`. It is run as

```bash theme={"system"}
node lib/main.js
```

For local runs, one normally additionally exports environment variables from `.env` using `dotenv`:

```bash theme={"system"}
node -r dotenv/config lib/main.js
```

## Processor choice

The Squid SDK currently offers specialized processor classes for EVM (`EvmBatchProcessor`) and Substrate networks (`SubstrateBatchProcessor`). More networks will be supported in the future. By convention, the processor object is defined at `src/processor.ts`.

Navigate to a dedicated section for each processor class:

* [`EvmBatchProcessor`](/en/sdk/squid-sdk/reference/processors/evm-batch)
* [`SubstrateBatchProcessor`](/en/sdk/squid-sdk/reference/processors/substrate-batch)

## Configuration

A processor instance should be configured to define the block range to be indexed, and the selectors of data to be fetched from [SQD Network](/en/network) and/or a node RPC endpoint.

## `processor.run()`

The actual data processing is done by the `run()` method called on a processor instance (typically at `src/main.ts`). The method has the following signature:

```ts theme={"system"}
run<Store>(
  db: Database<Store>,
  batchHander: (
    context: DataHandlerContext<Store, F extends FieldSelection>
  ) => Promise<void>
): void
```

The `db` parameter defines the target [data sink](/en/sdk/squid-sdk/resources/persisting-data), and `batchHandler` is an `async` `void` function defining the data transformation and persistence logic. It repeatedly receives batches of SQD Network data stored in `context.blocks`, transforms them and persists the results to the target database using the `context.store` interface (more on `context` in the next section).

To jump straight to examples, see [EVM Processor in action](/en/sdk/squid-sdk/tutorials/batch-processor-in-action) and [Substrate Processor in action](/en/sdk/squid-sdk/tutorials/batch-processor-in-action).

## Batch context

Batch handler takes a single argument of `DataHandlerContext` type:

```ts theme={"system"}
export interface DataHandlerContext<Store, F extends FieldSelection> {
    _chain: Chain
    log: Logger
    store: Store
    blocks: BlockData<F>[]
    isHead: boolean
}
```

Here, `F` is the type of the argument of the `setFields()` ([EVM](/en/sdk/squid-sdk/reference/processors/evm-batch/field-selection), [Substrate](/en/sdk/squid-sdk/reference/processors/substrate-batch/field-selection)) processor configuration method. `Store` type is inferred from the `Database` instance passed into the `run()` method.

#### `ctx._chain`

Internal handle for direct access to the underlying chain state via RPC calls. Rarely used directly, but rather by the facade access classes generated by the [typegen tools](/en/sdk/squid-sdk/glossary#typegen).

#### `ctx.log`

The native logger handle. See [Logging](/en/sdk/squid-sdk/reference/logger).

#### `ctx.store`

Interface for the target data sink. See [Persisting data](/en/sdk/squid-sdk/resources/persisting-data).

#### `ctx.blocks`

On-chain data items are grouped into blocks, with each block containing a header and iterables for all supported data item types. Boundary blocks are always included into the `ctx.blocks` iterable with valid headers, even when they do not contain any requested data. It follows that batch context *always* contains at least one block.

The set of iterables depends on the processor type (docs for [EVM](/en/sdk/squid-sdk/reference/processors/evm-batch/context-interfaces)/[Substrate](/en/sdk/squid-sdk/reference/processors/substrate-batch/context-interfaces)). Depending on the data item type, items within the iterables can be canonically ordered by how the data is recorded on-chain (e.g. transactions are ordered but traces are not). The shape of item objects is determined by the processor configuration done via the `.setFields()` method.

An idiomatic use of the context API is to iterate first over blocks and then over each iterable of each block:

<Tabs>
  <Tab title="EVM">
    ```ts theme={"system"}
    processor.run(new TypeormDatabase(), async (ctx) => {
      for (let block of ctx.blocks) {
        for (let log of block.logs) {
          // filter and process logs
        }
        for (let txn of block.transactions) {
          // filter and process transactions
        }
        for (let stDiff of block.stateDiffs) {
          // filter and process state diffs
        }
        for (let traces of block.traces) {
          // filter and process execution traces
        }
      }
    })
    ```
  </Tab>

  <Tab title="Substrate">
    ```ts theme={"system"}
    processor.run(new TypeormDatabase(), async (ctx) => {
      for (let block of ctx.blocks) {
        for (let event of block.events) {
          // filter and process events
        }
        for (let call of block.calls) {
          // filter and process calls
        }
        for (let extrinsic of block.extrinsics) {
          // filter and process extrinsics
        }
      }
    })
    ```
  </Tab>
</Tabs>

The canonical ordering of `ctx.blocks` enables efficient in-memory data processing. For example, multiple updates of the same entity can be compressed into a single database transaction.

Please be aware that the processor cannot ensure that data not meeting its filters will be excluded from iterables. It only guarantees the inclusion of data that matches the filters. Therefore, it is necessary to filter the data in the batch handler prior to processing.

#### `ctx.isHead`

Is `true` if the processor has reached the chain head. The last block `ctx.blocks` is then the current chain tip.


# Entities
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/schema-file/entities

Define high-level Squid SDK entities in schema.graphql — entity types, scalar fields, derived fields, and indexes that drive code generation.

Entities are defined by root-level GraphQL types decorated with `@entity`. Names and properties of entities are expected to be camelCased. They are converted into snake\_case for use as the corresponding database table and column names. The primary key column is always mapped to the entity field of a special `ID` type mapped as string (`varchar`). Non-nullable fields are marked with an exclamation mark (`!`) and are nullable otherwise.

The following [scalar types](https://graphql.org/learn/schema/#scalar-types) are supported by the `schema.graphql` dialect:

* `String` (mapped to `text`)
* `Int` (mapped to `int4`)
* `Float` (mapped to `numeric`, ts type `number`)
* `Boolean` (mapped to `bool`)
* `DateTime` (mapped to `timestamptz`, ts type `Date`)
* `BigInt` (mapped to `numeric`, ts type `bigint`)
* `BigDecimal` (mapped to `numeric`, ts type `BigDecimal` of [`@subsquid/big-decimal`](https://www.npmjs.com/package/@subsquid/big-decimal))
* `Bytes` (mapped to `bytea`, ts type `UInt8Array`)
* `JSON` (mapped to `jsonb`, ts type `unknown`)
* Enums (mapped to `text`)
* User-defined scalars (non-entity types). Such properties are mapped as `jsonb` columns.

**Example**

```graphql theme={"system"}
type Scalar @entity {
  id: ID!
  boolean: Boolean
  string: String
  enum: Enum
  bigint: BigInt
  dateTime: DateTime
  bytes: Bytes
  json: JSON
  deep: DeepScalar
}
        
type DeepScalar {
  bigint: BigInt
  dateTime: DateTime
  bytes: Bytes
  boolean: Boolean
}
        
enum Enum {
  A B C
}
```

## Arrays

An entity field can be an array of any scalar type except `BigInt` and `BigDecimal`. It will be mapped to the corresponding Postgres array type. Array elements may be defined as nullable or non-nullable.

**Example**

```graphql theme={"system"}
type Lists @entity {
  id: ID!
  intArray: [Int!]!
  enumArray: [Enum!]
  datetimeArray: [DateTime!]
  bytesArray: [Bytes!]
  listOfListsOfInt: [[Int]]
  listOfJsonObjects: [Foo!]
}
        
enum Enum {
  A B C D E F
}
        
type Foo {
  foo: Int
  bar: Int
}
```


# Entity relations
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/schema-file/entity-relations

Define one-to-many and many-to-many entity relations in Squid SDK schema files — @derivedFrom, @relation, and TypeORM-friendly conventions.

# Entity relations and inverse lookups

The term "entity relation" refers to the situation when an entity instance contains an instance of another entity within one of its fields. Type-wise this means that some entity (called the *owning entity*) has a field of a type that is some other, *non-owning* entity. Within the database, this is implemented as an (automatically indexed) foreign key column within the table mapped to the owning entity. A `fieldName` entity-typed field will map to a column named `field_name_id`.

[One-to-one](https://github.com/typeorm/typeorm/blob/master/docs/one-to-one-relations.md) and [one-to-many](https://github.com/typeorm/typeorm/blob/master/docs/many-to-one-one-to-many-relations.md) relations are supported by Typeorm. The "many" side of the one-to-many relations is always the owning side. Many-to-many relations are modeled as [two one-to-many relations with an explicit join table](#many-to-many-relations).

An entity relation is always unidirectional, but it is possible to request the data on the owning entity from the non-owning one. To do so, define a field decorated `@derivedFrom` in the schema. Doing so will cause the Typeorm code generated by [`squid-typeorm-codegen`](/en/sdk/squid-sdk/resources/persisting-data/typeorm) and the GraphQL API served by [`squid-graphql-server`](/en/sdk/squid-sdk/reference/openreader-server/overview) to show a virtual (that is, **not mapping to a database column**) field populated via inverse lookup queries.

The following examples illustrate the concepts.

## One-to-one relations

```graphql theme={"system"}
type Account @entity {
  id: ID!
  balance: BigInt!
  user: User @derivedFrom(field: "account")
}

type User @entity {
  id: ID!
  account: Account! @unique
  username: String!
  creation: DateTime!
}
```

The `User` entity references `Account` and owns the one-to-one relation. This is implemented as follows:

* On the database side: the `account` property of the `User` entity maps to the `account_id` foreign key column of the `user` table referencing the `account` table.
* On the TypeORM side: the `account` property of the `User` entity gets decorated with `@OneToOne` and `@JoinColumn`.
* On the GraphQL side: sub-selection of the `account` property is made available in `user`-related queries. Sub-selection of the `user` property is made available in `account`-related queries.

<Info>
  Unlike for the many-to-one case, the codegen will not add a virtual reverse lookup property to the TypeORM code for one-to-one relations. You can add it manually:

  ```typescript title="src/model/generated/account.model.ts" theme={"system"}
  import {OneToOne as OneToOne_} from "typeorm"

  @Entity_()
  export class Account {
    // ...
    @OneToOne_(() => User, e => e.account)
    user: User
  }
  ```

  If you are using this feature, please let us know at [the SquidDevs Telegram channel](https://t.me/HydraDevs).
</Info>

## Many-to-one/One-to-many relations

```graphql theme={"system"}
type Account @entity {
  "Account address"
  id: ID!
  transfersTo: [Transfer!] @derivedFrom(field: "to")
  transfersFrom: [Transfer!] @derivedFrom(field: "from")
}

type Transfer @entity {
  id: ID!
  to: Account!
  from: Account!
  amount: BigInt! 
}
```

Here `Transfer` defines owns the two relations and `Account` defines the corresponding inverse lookup properties. This is implemented as follows:

* On the database side: the `from` and `to` properties of the `Transfer` entity map to `from_id` and `to_id` foreign key columns of the `transfer` table referencing the `account` table.
* On the TypeORM side: properties `to` and `from` of the `Transfer` entity class get decorated with `@ManyToOne`. Properties `transfersTo` and `transfersFrom` decorated with `@OneToMany` get added to the `Account` entity class.
* On the GraphQL side: sub-selection of all relation-defined properties is made available in the schema.

## Many-to-many relations

Many-to-many entity relations should be modeled as two one-to-many relations with an explicitly defined join table.
Here is an example:

```graphql theme={"system"}
# an explicit join table 
type TradeToken @entity {
  id: ID! # This is required, even if useless
  trade: Trade!
  token: Token! 
}

type Token @entity {
  id: ID!
  symbol: String!
  trades: [TradeToken!]! @derivedFrom(field: "token")    
}

type Trade @entity {
  id: ID!
  tokens: [TradeToken!]! @derivedFrom(field: "trade")
}
```


# Indexes and constraints
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/schema-file/indexes-and-constraints

Annotate Squid SDK schema fields with @index and @unique to speed up GraphQL queries — match indexes to your most common entity filter patterns.

# Indexes and unique constraints

<Warning>
  The lack of indices is the most common cause of slow API queries
</Warning>

It is crucial to add database indexes to the entity fields on which one expects filtering and ordering. To add an index to a column, the corresponding entity field must be decorated with `@index`. The corresponding entity field will be decorated with [TypeORM `@Index()`](https://typeorm.io/indices#column-indices).

One can additionally decorate the field with `@unique` to enforce uniqueness. It corresponds to the [`@Index({ unique: true })`](https://typeorm.io/indices#unique-indices) TypeORM decorator.

### Example

```graphql theme={"system"}
type Transfer @entity {
  id: ID!
  to: Account!
  amount: BigInt! @index
  fee: BigInt! @index @unique
}
```

## Multi-column indices

Multi-column indices are defined on the entity level, with an optional `unique` constraint.

### Example

```graphql theme={"system"}
type Foo @entity @index(fields: ["foo", "bar"]) @index(fields: ["bar", "baz"]) 
  {
  id: ID!
  bar: Int!
  baz: [Enum!]
  foo: String!

type Extrinsic @entity @index(fields: ["hash", "block"], unique: true)  {
  id: ID!
  hash: String! @unique
  block: String!
}
```


# Interfaces
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/schema-file/interfaces

Define queryable GraphQL interfaces in Squid SDK schema files — share fields across entity types and run polymorphic queries via OpenReader.

The schema file supports [GraphQL Interfaces](https://graphql.org/learn/schema/#interfaces) for modelling complex types sharing common traits. Interfaces are annotated with `@query` at the type level and do not affect the database schema, only enriching the GraphQL API queries with [inline fragments](https://graphql.org/learn/queries/#inline-fragments).

Currently, only [OpenReader](/en/sdk/squid-sdk/reference/openreader-server) supports GraphQL interfaces defined in the schema file.

### Examples

```graphql theme={"system"}
interface MyEntity @query {
  id: ID!
  name: String
  ref: Ref
}

type Ref @entity {
  id: ID!
  name: String
  foo: Foo! @unique
  bar: Bar! @unique
} 

type Foo implements MyEntity @entity {
  id: ID!
  name: String
  ref: Ref @derivedFrom(field: "foo")
  foo: Int
}

type Bar implements MyEntity @entity {
  id: ID!
  name: String
  ref: Ref @derivedFrom(field: "bar")
  bar: Int
}

type Baz implements MyEntity @entity {
  id: ID!
  name: String
  ref: Ref
  baz: Int
}
```

The `MyEntity` interface above enables `myEntities` and `myEntitiesConnection` [GraphQL API queries](/en/sdk/squid-sdk/reference/openreader-server/api) with inline fragments and the `_type`, `__typename` [meta fields](https://graphql.org/learn/queries/#meta-fields):

```graphql theme={"system"}
query {
  myEntities(orderBy: [_type_DESC, id_ASC]) {
    id
    name
    ref {
        id
        name
    }
    __typename
    ... on Foo { foo }
    ... on Bar { bar }
    ... on Baz { baz }
  }
}
```


# Schema file and codegen
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/schema-file/intro

Introduction to the Squid SDK schema.graphql file and codegen tool — define entities, run sqd codegen, and generate TypeORM models for storage.

The schema file `schema.graphql` uses a GraphQL dialect to model the target entities and entity relations. The tooling around the schema file is then used to:

* Generate TypeORM entities (with `squid-typeorm-codegen(1)`, see below)
* Generate the database schema from the TypeORM entities (see [db migrations](/en/sdk/squid-sdk/resources/persisting-data/typeorm))
* Optionally, the schema can be used to present the target data with a [GraphQL API](/en/sdk/squid-sdk/resources/serving-graphql).

The schema file format is loosely compatible with the [subgraph schema](https://thegraph.com/docs/en/developing/creating-a-subgraph/) file, see [Migrate from subgraph](/en/sdk/squid-sdk/resources/migrate/migrate-subgraph) section for details.

## TypeORM codegen

The [`squid-typeorm-codegen(1)`](https://github.com/subsquid/squid-sdk/tree/master/typeorm/typeorm-codegen) tool is used to generate [TypeORM entity](https://typeorm.io/) classes from the schema defined in `schema.graphql`. Invoke it with

```bash theme={"system"}
npx squid-typeorm-codegen
```

By default the entity classes are generated in `src/model/generated`.

### Example

A `Foo` entity defined in the schema file:

```graphql title="schema.graphql" theme={"system"}
type Foo @entity {
  id: ID!
  bar: String
  baz: BigInt!
}
```

The generated `Foo` entity with TypeORM decorators:

```ts title="src/model/generated/foo.ts" theme={"system"}
import {Entity as Entity_, Column as Column_, PrimaryColumn as PrimaryColumn_} from "typeorm"
import * as marshal from "./marshal"

@Entity_()
export class Foo {
    constructor(props?: Partial<Foo>) {
        Object.assign(this, props)
    }

    @PrimaryColumn_()
    id!: string

    @Column_("text", {nullable: true})
    bar!: string | undefined | null

    @Column_("numeric", {transformer: marshal.bigintTransformer, nullable: false})
    baz!: bigint
}
```


# Unions and typed JSON
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/schema-file/unions-and-typed-json

Model Squid SDK union types and typed JSON fields in schema.graphql — discriminate variants and store structured JSON in Postgres-backed entities.

Complex scalar types can be modelled using a typed JSON fields together with union types, making safe union types.

## Typed JSON

It is possible to define explicit types for JSON fields. The generated entity classes and the GraphQL API will respect the type definition of the field, enforcing the data integrity.

**Example**

```graphql theme={"system"}
type Entity @entity {
  a: A
}

type A {
  a: String
  b: B
  c: JSON
}

type B {
  a: A
  b: String
  e: Entity
}
```

## Union types

One can leverage union types supported both by [Typescript](https://www.typescriptlang.org/docs/handbook/2/everyday-types.html#union-types) and [GraphQL](https://graphql.org/learn/schema/#union-types).  The union operator for `schema.graphql` supports only non-entity types, including typed JSON types described above. JSON types, however, are allowed to reference an entity type.

**Example**

```graphql theme={"system"}
type User @entity {
  id: ID!
  login: String!
}
        
type Farmer {
  user: User!
  crop: Int
}

type Degen {
  user: User!
  bag: String
}
        
union Owner = Farmer | Degen
        
type NFT @entity {
  name: String!
  owner: Owner!
}
```


# bigquery-store
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/store/bigquery

Reference for the @subsquid/bigquery-store package — write Squid SDK indexed data into Google BigQuery datasets with batch loading and schema sync.

# `@subsquid/bigquery-store`

See also the [BigQuery guide](/en/sdk/squid-sdk/resources/persisting-data/bigquery).

## Column types

|           Column type          |             Value type            |                                                     Dataset column type                                                     |
| :----------------------------: | :-------------------------------: | :-------------------------------------------------------------------------------------------------------------------------: |
|           `String()`           |              `string`             |                [STRING](https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types#string_type)               |
|   `Numeric(precision, scale)`  | <code>number \&#124 bigint</code> |   [NUMERIC(P\[, S\])](https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types#parameterized_decimal_type)  |
| `BigNumeric(precision, scale)` | <code>number \&#124 bigint</code> | [BIGNUMERIC(P\[, S\])](https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types#parameterized_decimal_type) |
|            `Bool()`            |             `boolean`             |                [BOOL](https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types#boolean_type)                |
|          `Timestamp()`         |               `Date`              |             [TIMESTAMP](https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types#timestamp_type)            |
|           `Float64()`          |              `number`             |           [FLOAT64](https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types#floating_point_types)          |
|            `Int64()`           | <code>number \&#124 bigint</code> |               [INT64](https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types#integer_types)               |


# CSV support
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/store/file/csv

Write Squid SDK indexed data as CSV files via the Table class — schema, encoding, and per-batch flushing for analytics-friendly file output.

# CSV format support

## `Table` Implementation

The `@subsquid/file-store-csv` package provides a `Table` implementation for writing to CSV files. Use it by [supplying one or more of its instances via the `tables` field of the `Database` constructor argument](/en/sdk/squid-sdk/resources/persisting-data/file#database-options). Constructor of the `Table` implementation accepts the following arguments:

* **`fileName: string`**: the name of the output file in every dataset partition folder.
* **`schema: {[column: string]: ColumnData}`**: a mapping from CSV column names to [`ColumnData` objects](#columns). A mapping of the same keys to data values is the row type used by the [table writer](/en/sdk/squid-sdk/resources/persisting-data/file#table-writer-interface).
* **`options?: TableOptions`**: see [`Table` Options](#table-options).

## Columns

`ColumnData` objects determine how the in-memory data representation of each table column should be serialized. They are made with the `Column` factory function that accepts a column data type and an optional `{nullable?: boolean}` `options` object as arguments.

Column types can be obtained by making the function calls listed below from the `Types` submodule. They determine the type that the [table writer](/en/sdk/squid-sdk/resources/persisting-data/file#table-writer-interface) will expect to find at the corresponding field of data row objects.

|            Column type            | Type of the data row field |
| :-------------------------------: | :------------------------: |
|          `Types.String()`         |          `string`          |
|         `Types.Numeric()`         |    `number` or `bigint`    |
|         `Types.Boolean()`         |          `boolean`         |
| `Types.DateTime(format?: string)` |           `Date`           |
|         `Types.JSON<T>()`         |             `T`            |

`Types.DateTime` accepts an optional [strftime](https://pubs.opengroup.org/onlinepubs/009695399/functions/strftime.html)-compatible format string. If it is omitted, the dates will be serialized to [ISO strings](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date/toISOString).

The type `T` supplied to the `Types.JSON()` generic function must be an object with string keys (extend `{[k: string]: any}`).

## `Table` Options

As its optional final argument, the constructor of `Table` accepts an object that defines table options:

```typescript theme={"system"}
TableOptions {
  dialect?: Dialect
  header?: boolean
}
```

Here,

* **`dialect`** determines the details of the CSV formatting (see the details below, default: `dialects.excel`)
* **`header`** determines whether a CSV header should be added (default: `true`)

`Dialect` type is defined as follows:

```typescript theme={"system"}
Dialect {
  delimiter: string
  escapeChar?: string
  quoteChar: string
  quoting: Quote
  lineterminator: string
}
```

where

```typescript theme={"system"}
enum Quote {
  ALL,        // Put all values in quotes.
  MINIMAL,    // Only quote strings with special characters.
              // A special character is one of the following:
              // delimiter, lineterminator, quoteChar.
  NONNUMERIC, // Quote strings, booleans, DateTimes and JSONs.
  NONE        // Do not quote values.
}
```

is the enum determining how the formatted values should be quoted. The quote character is escaped for all values of `quoting`; `Quote.NONE` additionally escapes the rest of the special characters and the escape character.

Two dialect presets are available via the `dialects` object exported by `@subsquid/file-store-csv`:

```typescript theme={"system"}
export let dialects = {
  excel: {
    delimiter: ',',
    quoteChar: '"',
    quoting: Quote.MINIMAL,
    lineterminator: '\r\n'
  },
  excelTab: {
    delimiter: '\t',
    quoteChar: '"',
    quoting: Quote.MINIMAL,
    lineterminator: '\r\n'
  }
}
```

## Example

This saves ERC20 `Transfer` events captured by the processor to TSV (tab-separated values) files. Full squid code is available in [this repo](https://github.com/subsquid-labs/file-store-csv-example).

```typescript theme={"system"}
import {Database, LocalDest} from '@subsquid/file-store'
import {
  Column,
  Table,
  Types,
  dialects
} from '@subsquid/file-store-csv'
...

const dbOptions = {
  tables: {
    TransfersTable: new Table(
      'transfers.tsv',
      {
        from: Column(Types.String()),
        to: Column(Types.String()),
        value: Column(Types.Numeric())
      },
      {
        dialect: dialects.excelTab,
        header: true
      }
    )
  },
  dest: new LocalDest('./data'),
  chunkSizeMb: 10
}

processor.run(new Database(dbOptions), async (ctx) => {
  ...
  let from: string = ...
  let to: string = ...
  let value: bigint = ...
  ctx.store.TransfersTable.write({ from, to, value })
  ...
})
```


# JSON support
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/store/file/json

Write Squid SDK indexed data as JSON or JSONL files via the Table class — newline-delimited and array variants with schema validation.

# JSON format support

## `Table` Implementation

The `@subsquid/file-store-json` package provides a `Table` implementation for writing to JSON and [JSONL](https://jsonlines.org) files. Use it by [supplying one or more of its instances via the `tables` field of the `Database` constructor argument](/en/sdk/squid-sdk/resources/persisting-data/file#database-options). The `Table` uses a constructor with the following signature:

```typescript theme={"system"}
Table<S extends Record<string, any>>(fileName: string, options?: {lines?: boolean})
```

Here,

* **`S`** is a Typescript type describing the schema of the table data.
* **`fileName: string`** is the name of the output file in every dataset partition folder.
* **`options?: {lines?: boolean}`** are table options. At the moment the only available setting is whether to use JSONL instead of a plain JSON array (default: false).

## Example

This saves ERC20 `Transfer` events captured by the processor to a JSONL file where each line is a JSON serialization of a `{from: string, to: string, value: number}` object. Full squid code is available in [this repo](https://github.com/subsquid-labs/file-store-json-example).

```typescript theme={"system"}
import {Database} from '@subsquid/file-store'
import {Table} from '@subsquid/file-store-json'

...

const dbOptions = {
  tables: {
    TransfersTable: new Table<{
      from: string,
      to: string,
      value: bigint
    }>('transfers.jsonl', { lines: true })
  },
  dest: new LocalDest('./data'),
  chunkSizeMb: 10
}

processor.run(new Database(dbOptions), async (ctx) => {
  ...
  let from: string = ...
  let to: string = ...
  let value: bigint = ...
  ctx.store.TransfersTable.write({ from, to, value })
  ...
})
```


# Parquet support
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/store/file/parquet

Write Squid SDK indexed data as Apache Parquet files via the Table class — columnar storage, optional compression, and schema-driven file output.

# Parquet format support

<Info>
  Support for the Parquet format is currently experimental. Contact us at the [SquidDevs Telegram channel](https://t.me/HydraDevs) for support.
</Info>

## `Table` Implementation

[Apache Parquet](https://parquet.apache.org) is an advanced format for storing tabular data in files. It divides table columns into [column chunks](https://parquet.apache.org/docs/concepts/). Each column chunk is stored contiguously, allowing efficient partial reads of column subsets. Column chunks can also be compressed with row-specific compression algorithms, further enhancing the performance. Retrieval relies on metadata appended to the end of a Parquet file. [Metadata standard](https://parquet.apache.org/docs/file-format/metadata/) of Apache Parquet is extremely powerful, enabling all sorts of [extensions](https://parquet.apache.org/docs/file-format/extensibility/). Among other things, metadata contains the schema of the data, making the format self-describing.

The `@subsquid/file-store-parquet` package provides a `Table` implementation for writing to Parquet files. Use it by [supplying one or more of its instances via the `tables` field of the `Database` constructor argument](/en/sdk/squid-sdk/resources/persisting-data/file#database-options). Constructor of the `Table` implementation accepts the following arguments:

* **`fileName: string`**: the name of the output file in every dataset partition folder.
* **`schema: {[column: string]: ColumnData}`**: a mapping from Parquet column names to [`ColumnData` objects](#columns). A mapping of the same keys to data values is the row type used by the [table writer](/en/sdk/squid-sdk/resources/persisting-data/file#table-writer-interface).
* **`options?: TableOptions`**: see [`Table` Options](#table-options).

## Columns

`ColumnData` objects define storage options for each table column. They are made with the `Column` factory function that accepts a column data type and an optional `options: ColumnOptions` object.

Column types can be obtained by making the function calls listed below from the `Types` submodule. They determine the [Parquet type](https://parquet.apache.org/docs/file-format/types/) that will be used to store the data and the type that the [table writer](/en/sdk/squid-sdk/resources/persisting-data/file#table-writer-interface) will expect to find at the corresponding field of data row objects.

|                  Column type                 |                                     Logical type                                     |                  Primitive type                 |                                    Valid data row object field contents                                    |
| :------------------------------------------: | :----------------------------------------------------------------------------------: | :---------------------------------------------: | :--------------------------------------------------------------------------------------------------------: |
|               `Types.String()`               |                                variable length string                                |                   `BYTE_ARRAY`                  |                                           `string` of any length                                           |
|          `Types.Binary` `(length?)`          |                          variable or fixed length byte array                         |    `BYTE_ARRAY` or `FIXED_LEN_` `BYTE_ARRAY`    |              `Uint8Array` of length equal to `length` if it is set or of any length otherwise              |
|                `Types.Int8()`                |                                 8-bit signed integer                                 |                     `INT32`                     |                                          `number` from -128 to 127                                         |
|                `Types.Int16()`               |                                 16-bit signed integer                                |                     `INT32`                     |                                        `number` from -32768 to 32767                                       |
|                `Types.Int32()`               |                                 32-bit signed integer                                |                     `INT32`                     |                                   `number` from -2147483648 to 2147483647                                  |
|                `Types.Int64()`               |                                 64-bit signed integer                                |                     `INT64`                     |                    `bigint` or `number` from -9223372036854775808 to 9223372036854775807                   |
|                `Types.Uint8()`               |                                8-bit unsigned integer                                |                     `INT32`                     |                                           `number` from 0 to 255                                           |
|               `Types.Uint16()`               |                                16-bit unsigned integer                               |                     `INT32`                     |                                          `number` from 0 to 65535                                          |
|               `Types.Uint32()`               |                                32-bit unsigned integer                               |                     `INT32`                     |                                        `number` from 0 to 4294967295                                       |
|               `Types.Uint64()`               |                                64-bit unsigned integer                               |                     `INT64`                     |                             `bigint` or `number` from 0 to 18446744073709551615                            |
|                `Types.Float()`               |                             32-bit floating point number                             |                     `FLOAT`                     |                                             non-`Nan` `number`                                             |
|               `Types.Double()`               |                             64-bit floating point number                             |                     `DOUBLE`                    |                                             non-`Nan` `number`                                             |
|               `Types.Boolean()`              |                                     boolean value                                    |                    `BOOLEAN`                    |                                                  `boolean`                                                 |
|              `Types.Timestamp()`             |                            UNIX timestamp in milliseconds                            |                     `INT64`                     |                                                   `Date`                                                   |
|    `Types.Decimal` `(precision, scale=0)`    | decimal with `precision` digits and `scale` digits to the right of the decimal point | `INT32` or `INT64` or `FIXED_LEN_` `BYTE_ARRAY` | `number` or `bigint` or [`BigDecimal`](https://github.com/subsquid/squid-sdk/tree/master/util/big-decimal) |
|  `Types.List` `(itemType, {nullable=false})` |        a list filled with optionally nullable items of `itemType` column type        |                        -                        |                                   `Array` of items satisfying `itemType`                                   |
| `Types.JSON<T extends {[k: string]: any}>()` |                                JSON object of type `T`                               |                   `BYTE_ARRAY`                  |                                            `Object` of type `T`                                            |
| `Types.BSON<T extends {[k: string]: any}>()` |                                BSON object of type `T`                               |                   `BYTE_ARRAY`                  |                                            `Object` of type `T`                                            |

<Tip>
  The widest decimals that [PyArrow](https://arrow.apache.org/docs/python/index.html) can read are `Types.Decimal(76)`.
</Tip>

The following column options are available:

```typescript theme={"system"}
ColumnOptions {
  nullable?: boolean
  compression?: Compression
  encoding?: Encoding
}
```

See the [Encoding and Compression](#encoding-and-compression) section for details.

## `Table` Options

As its optional final argument, the constructor of `Table` accepts an object that defines table options:

```typescript theme={"system"}
TableOptions {
  compression?: Compression
  rowGroupSize?: number
  pageSize?: number
}
```

Here,

* **`compression`** determines the file-wide compression algorithm. Per-column settings override this. See [Encoding and Compression](#encoding-and-compression) for the list of available algorithms. Default: `Compression.UNCOMPRESSED`.
* **`rowGroupSize`** determines the approximate uncompressed size of the row group in bytes. Default: `32 * 1024 * 1024`.
* **`pageSize`** determines the approximate uncompressed page size in bytes. Default: `8 * 1024`.

When `pageSize` is less than `rowGroupSize` times the number of columns, the latter setting will be ignored. In this case each row group will contain exactly one roughly `pageSize`d page for each column.

## Encoding and Compression

[Encodings](https://parquet.apache.org/docs/file-format/data-pages/encodings/) are set at a per-column basis. At the moment the default and the only supported value is `'PLAIN'`.

[Compression](https://github.com/apache/parquet-format/blob/master/Compression.md) can be set at a per-file or a per-column basis. Available values are

* `'UNCOMPRESSED'` (default)
* `'GZIP'`
* `'LZO'`
* `'BROTLI'`
* `'LZ4'`

## Example

This saves ERC20 `Transfer` events captured by the processor to a Parquet file. All columns except for `from` are `GZIP`ped. Row groups are set to be roughly 30000 bytes in size each. Each row group contains roughly ten \~1000 bytes-long pages per column. Full squid code is available in [this repo](https://github.com/subsquid-labs/file-store-parquet-example).

```typescript theme={"system"}
import {Database, LocalDest} from '@subsquid/file-store'
import {
  Column,
  Table,
  Types
} from '@subsquid/file-store-parquet'

...

const dbOptions = {
  tables: {
    TransfersTable: new Table(
      'transfers.parquet',
      {
        from: Column(
          Types.String(),
          {
            compression: 'UNCOMPRESSED'
          }
        ),
        to: Column(Types.String()),
        value: Column(Types.Uint64())
      },
      {
        compression: 'GZIP',
        rowGroupSize: 300000,
        pageSize: 1000
      }
    )
  },
  dest: new LocalDest('./data'),
  chunkSizeMb: 10
}

processor.run(new Database(dbOptions), async (ctx) => {
  ...
  let from: string = ...
  let to: string = ...
  let value: bigint = ...
  ctx.store.TransfersTable.write({ from, to, value })
  ...
})
```


# S3 support
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/store/file/s3-dest

Upload Squid SDK file-store output to S3-compatible buckets via the Dest class — Squid CSV, JSON, and Parquet streaming to AWS S3 or MinIO.

# S3 destination support

## Overview

Writing to Amazon S3-compatible file storage services such as [AWS](https://aws.amazon.com) and [Filebase](https://filebase.com) is supported via the `S3Dest` class from the `@subsquid/file-store-s3` package. Use it by [setting the `dest` field of the `Database` constructor argument](/en/sdk/squid-sdk/resources/persisting-data/file#database-options) to its instance. Constructor of `S3Dest` accepts the following arguments:

* **`url: string`**: S3 URL in the `s3://bucket/path` format.
* **`optionsOrClient?: S3Client | S3ClientConfig`**: an optional [S3 client](https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/Package/-aws-sdk-client-s3/Class/S3Client/) or [client config](https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-s3/interfaces/s3clientconfig.html). By default, a simple config parameterized by environment variables is used:
  ```typescript theme={"system"}
  {
    region: process.env.S3_REGION,
    endpoint: process.env.S3_ENDPOINT,
    credentials: {
      accessKeyId: assertNotNull(process.env.S3_ACCESS_KEY_ID),
      secretAccessKey: assertNotNull(process.env.S3_SECRET_ACCESS_KEY),
    },
  }
  ```

## Example

This saves the processor data in the `transfers-data` folder of the `subsquid-testing-bucket` bucket at the [Filebase](https://filebase.com) service. The service only has one region and one endpoint, and here they are hardcoded to reduce the number of required envirionment variables and illustrate how connection parameters can be supplied programmatically. Full squid code is available in [this repo](https://github.com/subsquid-labs/file-store-s3-example).

```typescript theme={"system"}
import {Database} from '@subsquid/file-store'
import {S3Dest} from '@subsquid/file-store-s3'
import {assertNotNull} from '@subsquid/util-internal' // pulled by @subsquid/file-store-s3

...

const dbOptions = {
  ...
  dest: new S3Dest(
    's3://subsquid-testing-bucket/transfers-data',
    {
      region: 'us-east-1',
      endpoint: 'https://s3.filebase.com',
      credentials: {
        accessKeyId: assertNotNull(process.env.S3_ACCESS_KEY_ID),
        secretAccessKey: assertNotNull(process.env.S3_SECRET_ACCESS_KEY)
      }
    }
  ),
  ...
}

processor.run(new Database(dbOptions), async (ctx) => {
  ...
}
```


# typeorm-store
Source: https://docs.sqd.dev/en/sdk/squid-sdk/reference/store/typeorm

Reference for @subsquid/typeorm-store — the optimized TypeORM-based Postgres store that powers Squid SDK indexers with batched upserts and inserts.

# `@subsquid/typeorm-store`

This page describes the interface of the classes from the `@subsquid/typeorm-store` NPM package. If you're looking for a guide on saving squid data to databases and the related workflows, check out the [Saving to PostgreSQL](/en/sdk/squid-sdk/resources/persisting-data/typeorm) page.

## `TypeormDatabase` constructor arguments

The argument of the `TypeormDatabase` class constructor may have the following fields:

* `stateSchema: string`: the name of the [database schema](https://www.postgresql.org/docs/current/sql-createschema.html) that the processor uses to persist its status (hash + height of the highest reached block). Useful for making sure that each processor uses its own state schema when running multiple processors against the same database (e.g. in a multichain setting). Default: `'squid_processor'`.
* `isolationLevel: 'SERIALIZABLE' | 'READ COMMITTED' | 'REPEATABLE READ'`: sets the [transaction isolation level](https://www.postgresql.org/docs/current/transaction-iso.html) of processor transactions. Default: `'SERIALIZABLE'`.
* `supportHotBlocks: boolean`: controls the support for hot blocks. Necessary in all squids that must be able to handle short-lived [blockchain forks](https://en.wikipedia.org/wiki/Fork_\(blockchain\)). That includes all squids that index chain data in near-real time using RPC endpoints. Default: `true`.
* `projectDir: string`: the folder where `TypeormDatabase` will look for the TypeORM model definition (at `lib/model`) and for migrations (at `db/migrations`). Default: `process.cwd()`.

## `Store` interface

### Batch access methods

#### **`upsert(e: E | E[])`**

Upsert a single or multiple entities to the database. **Does not cascade the upsert to the relations.**

```ts theme={"system"}
await ctx.store.upsert([new User({id: 'Bob'}), new User({id: 'Alice'}))])
```

#### **`save(e: E | E[])`**

Deprecated alias for [`upsert()`](#upsert).

#### **`insert(e: E | E[])`**

Inserts a given entity or entities into the database. Does not check if the entity(s) exist in the database and will fail if a duplicate is inserted. Executes a primitive INSERT operation **without cascading to the relations**.

```ts theme={"system"}
await ctx.store.insert([new User({ id: "Bob" }), new User({ id: "Alice" })]);
```

#### **`remove(e: E | E[] | EntityClass<E>, id?: string | string[])`**

Deletes a given entity or entities from the database. Accepts either an object or an entity ID(s). **Does not cascade the deletion**.

```ts theme={"system"}
await ctx.store.remove(User, ["Alice", "Bob"]);
```

### TypeORM methods

For details see [TypeORM EntityManager reference](https://typeorm.io/entity-manager-api).

#### **`get`**

Get an entity by ID.

```ts theme={"system"}
await ctx.store.get(User, "Bob");
```

#### **`count`**

Count the number of entities matching a where filter.

```ts theme={"system"}
await ctx.store.count(User, {
  where: {
    firstName: "Timber",
  },
});
```

#### **`countBy`**

Count the number of entities matching a filter.

```ts theme={"system"}
await ctx.store.countBy(User, { firstName: "Timber" })
```

#### **`find`**

Return a list matching a where filter.

```ts theme={"system"}
await ctx.store.find(User, {
  where: {
    firstName: "Timber",
  },
});
```

#### **`findBy`**

Return a list matching a filter.

```ts theme={"system"}
let accounts = await ctx.store.findBy(Account, {id: In([...accountIds])})
```

#### **`findOne`**

Return the first entity matching a where filter.

```ts theme={"system"}
const timber = await ctx.store.findOne(User, {
  where: {
    firstName: "Timber",
  },
});
```

#### **`findOneBy`**

Return the first entity matching a filter.

```ts theme={"system"}
const timber = await ctx.store.findOneBy(User, { firstName: "Timber" })
```

#### **`findOneOrFail`**

Throws if nothing is found.

```ts theme={"system"}
const timber = await ctx.store.findOneOrFail(User, {
  where: {
    firstName: "Timber",
  },
});
```

#### **`findOneByOrFail`**

Throws if nothing is found.

```ts theme={"system"}
const timber = await ctx.store.findOneByOrFail(User, { firstName: "Timber" })
```

### Find Operators

`find()` and `findXXX()` methods support the following operators:

* `In` (contains in array)
* `Not`
* `LessThan`
* `LessThanOrEqual`
* `MoreThan`
* `MoreThanOrEqual`
* `Like`
* `ILike`
* `Between`
* `Any`
* `IsNull`
* `Raw` (raw SQL fragments)

See the details and examples in the [TypeORM `FindOption` docs](https://typeorm.io/find-options#advanced-options).

#### Example

```ts theme={"system"}
let accounts = await ctx.store.findBy(Account, {id: In([...accountIds])})
```

### Joining relations

To load an entity with relations, use `relations` field on the `find` options and specify which relations should be joined:

```ts theme={"system"}
await ctx.store.find(User, {
  relations: {
    project: true,
  },
  where: {
    project: {
      name: "TypeORM",
      initials: "TORM",
    },
  },
});
```

See the [TypeORM docs](https://typeorm.io/find-options) sections for details.

## Database connection parameters

Database credentials must be supplied via the environment variables:

* `DB_HOST` (default `localhost`)
* `DB_PORT` (default `5432`)
* `DB_NAME` (default `postgres`)
* `DB_USER` (default `postgres`)
* `DB_PASS` (default `postgres`)
* `DB_SSL` (default `false`)
* `DB_SSL_REJECT_UNAUTHORIZED` (default `true`)
* `DB_URL` (default `undefined`, see the [DB\_URL section](#db_url))

<Info>
  When deploying to [Cloud](/en/cloud) with the [Postgres addon](/en/cloud/reference/pg) enabled in the [manifest](/en/cloud/reference/manifest), any user-supplied values are overwritten for most of these variables. See [Variable shadowing](/en/cloud/reference/pg#variable-shadowing).
</Info>

`typorm-store` also supports the following variables for connecting to databases that require client-side SSL:

* `DB_SSL_CA` - the root certificate in plain text
* `DB_SSL_CA_FILE` - path to a root certificate file
* `DB_SSL_CERT` - client certificate in plain text
* `DB_SSL_CERT_FILE` - path to client certificate in plain text
* `DB_SSL_KEY` - client key in plain text
* `DB_SSL_KEY_FILE` - path to client key in plain text

<Tip>
  In case you're deploying to [Cloud](/en/cloud) you can set [secrets](/en/cloud/resources/env-variables#secrets) to the contents of any given file via stdin:

  ```bash theme={"system"}
  sqd secrets set DB_SSL_CA < ca.crt
  ```
</Tip>

### `DB_URL`

When set, `DB_URL` takes precedence over all individual variables. Its format is as follows:

```
postgres[ql]://[username[:password]@][host[:port]]/database[?parameter_list]
```

where `parameter_list` is an `&`-separated list of assignments of SSL connection parameters:

* `ssl=(0|1|true|false)`
* `sslmode=(disabled|no-verify|prefer|require|verify-ca|verify-full)`
* `sslcert=<path_to_cert_file>`
* `sslkey=<path_to_key_file>`
* `sslrootcert=<path_to_root_cert_file>`

When any value is omitted from the URL, the value of the corresponding individual `DB_*` variable will be used instead. If that is not set, the default will be used.


# Batch processing
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/batch-processing

Batch-based data transformation model in Squid SDK — why processors handle ranges of blocks at once for higher indexing throughput than per-block models.

Batch data processing model employed by Squid SDK relies on the following principles:

* Minimize the number of database hits by grouping multiple single-row transactions into multi-row batch transactions.
* Transform the data in memory using vectorized operators.
* Use the [MakerDAO `Multicall` contract](/en/sdk/squid-sdk/resources/tools/typegen/state-queries#batch-state-queries) to batch EVM state queries.
* Use [`XXX.getMany()`](/en/sdk/squid-sdk/resources/tools/typegen/state-queries) to batch Substrate state queries.

In practice, batching is a more flexible (compared to handler parallelization) way to speed up the inherently sequential indexing of on-chain transactions and logs.

To illustrate, assume the processor must infer the current state of an on-chain record. It is configured to listen to the two on-chain events, `Create` and `Update`, that are emitted once the record is created or updated. The [data batch](/en/sdk/squid-sdk/reference/processors/architecture#ctxblocks) received by the processor is then an array of event items, i.e.

```ts theme={"system"}
[
    Create({id: 1, name: 'Alice'}), 
    Update({id: 1, name: 'Bob'}),
    Create({id: 2, name: 'Mikee'}), 
    Update({id: 1, name: 'Carol'}), 
    Update({id: 2, name: 'Mike'})
]
```

Following the principles above, a processor would update the intermediary entity states in memory, persisting only the final state:

```ts theme={"system"}
[{id: 1, name: 'Carol'}, {id: 2, name: 'Mike'}]
```

in a single transaction.

Let's see the batch processing principles in action.

## Patterns

An idiomatic use of [`processor.run()`](/en/sdk/squid-sdk/reference/processors/architecture#processorrun) is as follows:

```ts theme={"system"}
processor.run(new TypeormDatabase(), async (ctx) => {
    // a decoded and normalized ctx.blocks data
    // (just the logs in this example)
    const logDataBatch = []

    for (const block of ctx.blocks) {
      for (const log of c.logs) {
        // transform and normalize the raw logs data
        // based on the onchain data
        logDataBatch.push(decodeAndTransformToMyData(log))
      }
    }

    // the set of my target entity IDs to be updated/created
    const myEntityIds = new Set()
    for (const d of logDataBatch) {
        // the business logic mapping 
        // the on-chain data with the target
        // entity ID
        myEntityIds.add(extractEntityId(d))
    }

    // load the enities by the list of IDs and 
    // put into an ID map
    const myEntities: Map<string, MyEntity> = new Map(
       // batch-load using IN operator
      (await ctx.store.findBy(MyEntity, { id: In([...myEntityIds]) }))
        // put the result into the ID map
        .map((entity) => [entity.id, entity])
    );

    // calculate the updated state of the entities
    for (const d of logDataBatch) {
        const myEntity = myEntities.get(extractEntityId(d))
        if (myEntity == null) {
            // create a new instance with d and
            // add to myEntities map
        } else {
            // update myEntity using d
        }
    }

    // batch-update all entities in the map
    await ctx.store.save([...myEntities.values()])
});
```

For a full implementation of the above pattern, see [EVM squid example](https://github.com/subsquid-labs/evm-logs-example/tree/master/src) or [Substrate squid example](https://github.com/subsquid/squid-substrate-examples/tree/master/1-events).

## Anti-patterns

Avoid loading or persisting single entities unless strictly necessary. For example, here is a possible antipattern for the [Gravatar example](https://github.com/subsquid/gravatar-squid):

```ts theme={"system"}
processor.run(new TypeormDatabase(), async (ctx) => {
  for (const c of ctx.blocks) {
    for (const log of c.logs) {
      // making sure that we process only the relevant logs
      if (log.address !== GRAVATAR_CONTRACT ||
          (log.topics[0] !== events.NewGravatar.topic &&
           log.topics[0] !== events.UpdatedGravatar.topic)) continue
      const { id, owner, displayName, imageUrl } = extractData(log)
      // ANTIPATTERN!!!
      // Doing an upsert per event drastically decreases the indexing speed
      await ctx.store.save(Gravatar, new Gravatar({
        id: id.toHexString(),
        owner: decodeHex(owner),
        displayName,
        imageUrl
      }))
    }
  }
})
```

Instead, use an in-memory cache, and batch upserts:

```ts theme={"system"}
processor.run(new TypeormDatabase(), async (ctx) => {
  const gravatars: Map<string, Gravatar> = new Map();
  for (const c of ctx.blocks) {
    for (const log of c.logs) {
      if (log.address !== GRAVATAR_CONTRACT ||
          (log.topics[0] !== events.NewGravatar.topic &&
           log.topics[0] !== events.UpdatedGravatar.topic)) continue
      const { id, owner, displayName, imageUrl } = extractData(log)
      gravatars.set(id.toHexString(), new Gravatar({
        id: id.toHexString(),
        owner: decodeHex(owner),
        displayName,
        imageUrl
      }))
    }
  }
  await ctx.store.save([...gravatars.values()])
})
```

## Migrate from handlers

Batch-based processing can be used as a drop-in replacement for the handler-based mappings employed by e.g. subgraphs. While the handler-based processing is significantly slower due to excessive database lookups and writes, it may be a good intermediary step while [migrating an existing subgraph to Squid SDK](/en/sdk/squid-sdk/resources/migrate/migrate-subgraph).

One can simply re-use the existing handlers while looping over the `ctx` items:

```ts theme={"system"}
processor.run(new TypeormDatabase(), async (ctx) => {
  for (const c of ctx.blocks) {
    for (const log of c.logs) {
      switch (log.topics[0]) {
        case abi.events.FooEvent.topic:
          await handleFooEvent(ctx, log)
          continue
        case abi.events.BarEvent.topic:
          await handleFooEvent(ctx, log)
          continue
        default:
          continue
      }
    }
    for (const txn of c.transactions) {
      // 0x + 4 bytes
      const sighash = txn.input.slice(0, 10)
      switch (sighash) {
        case '0xa9059cbb': // transfer(address,uint256) sighash
          await handleTransferTx(ctx, txn)
          continue
        case abi.functions.approve.sighash:
          await handleApproveTx(ctx, txn)
          continue
        default:
          continue
      }
    }
  }
})
```

## Block hooks

Similarly, one can implement pre- and post- block hooks:

```ts theme={"system"}
processor.run(new TypeormDatabase(), async (ctx) => {
  for (const c of ctx.blocks) {
    await preBlockHook(ctx, c)
    for (const log of c.logs) {
      // some logic
    }
    for (const txn of c.transactions) {
      // some more logic
    }
    await postBlockHook(ctx, c)
  }
})
```


# External APIs and IPFS
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/external-api

Enrich blockchain data with external APIs and IPFS in your Squid SDK indexer — best practices for caching, retries, and avoiding rate limits.

Squid processor is a usual node.js process, so one can fetch data from external APIs or IPFS gateways within the `processor.run()` method. In squids deployed to [SQD Cloud](/en/cloud) one normally uses API calls in combination with API keys set via [secrets](/en/cloud/resources/env-variables).

### Example

For example, one can enrich the indexed transaction with historical price data using the Coingecko API.

```ts theme={"system"}
// ...
processor.run(new TypeormDatabase(), async (ctx) => {
  const burns: Burn[] = []
  for (let c of ctx.blocks) {
    for (let txn of c.transactions) {
      burns.push(new Burn({
        id: formatID(c.header.height, txn.hash),
        block: c.header.height,
        address: txn.from,
        value: txn.value,
        txHash: txn.hash,
        price: await getETHPriceByDate(c.header.timestamp)
      }))
    }
  }
})

async function getETHPriceByDate(timestamp: number): Promise<bigint> {
  const formatted = moment(new Date(timestamp).toISOString()).format("DD-MM-yyyy")
  const res = await axios.get(
    `https://api.coingecko.com/api/v3/coins/ethereum/history?date=${formatted}&localization=false`
  )
  return res.data.market_data.current_price.usd
}
```

## IPFS fetching

For reliable indexing of content stored on IPFS (e.g. NFT metadata) we recommend fetching from dedicated IPFS gateways, e.g. provided by [Filebase](https://filebase.com/dedicated-gateways/).

For a more elaborate example of with IPFS gateway and external API calls, inspect the [Step 3 of the BAYC tutorial](/en/sdk/squid-sdk/tutorials/bayc/step-three-adding-external-data).

### Example

```typescript theme={"system"}
// FIXME: replace with a dedicated gateway
export const BASE_URL = 'https://cloudflare-ipfs.com/ipfs/'

export const api = Axios.create({
  baseURL: BASE_URL,
  headers: {
    'Content-Type': 'application/json',
  },
  withCredentials: false,
  timeout: 5000,
  httpsAgent: new https.Agent({ keepAlive: true }),
})

export const fetchMetadata = async (
  ctx: DataHandlerHandlerContext<Store, typeof fieldSelection>,
  cid: string
): Promise<any | null> => {
  try {
    const { status, data } = await api.get(`${BASE_URL}/${cid}`)
    ctx.log.info(`[IPFS] ${status} CID: ${cid}`)
    if (status < 400) {
      return data
    }
  } catch (e) {
    ctx.log.warn(
      `[IPFS] ERROR CID: ${cid} TRY ${(e as Error).message}`
    )
  }
  return null
}

processor.run(new TypeormDatabase(), async (ctx) => {
  for (let c of ctx.blocks) {
    for (let log of c.logs) {
      // track and decode NFT events to get CID
      // use fetchMetadata() to fetch metadata
    }
  }
})
```


# Migrate from The Graph
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/migrate/migrate-subgraph

Migrate a subgraph from The Graph to a Squid SDK indexer — port mappings, schema, manifest, and unit tests for an SQD-powered drop-in replacement.

This guide walks through the steps to migrate a subgraph to SQD. In what follows we will convert the [Gravatar](https://github.com/graphprotocol/example-subgraph) subgraph into a squid and run it locally. Impatient readers may clone the squid from the [repo](https://github.com/subsquid-labs/gravatar-squid) and run it by following the instructions in README:

```bash theme={"system"}
git clone https://github.com/subsquid-labs/gravatar-squid.git
```

`EvmBatchProcessor` provided by the Squid SDK defines a single handler that indexes EVM logs and transaction data in batches. It differs from the programming model of subgraph mappings that defines a separate data handler for each EVM log topic to be indexed. Due to significantly less frequent database hits (once per batch compared to once per log) the batch-based handling model shows up to a 10x increase in the indexing speed.

At the same time, concepts of the [schema file](/en/sdk/squid-sdk/reference/schema-file), [code generation from the schema file](/en/sdk/squid-sdk/reference/schema-file/intro#typeorm-codegen) and [auto-generated GraphQL API](/en/sdk/squid-sdk/resources/serving-graphql) should be familiar to subgraph developers. In most cases the schema file of a subgraph can be imported into a squid as is.

There are some known limitations:

* Many-to-Many entity relations should be [modeled explicitly](/en/sdk/squid-sdk/reference/schema-file/entity-relations#many-to-many-relations) as two many-to-one relations

On top of the features provided by subgraphs, Squid SDK and SQD Cloud offer extra flexibility in developing tailored APIs and ETLs for on-chain data:

* Full control over the target database (Postgres), including custom migrations and ad-hoc queries in the handler
* Custom target databases and data formats (e.g. CSV)
* Arbitrary code execution in the data handler
* [Extension of the GraphQL API](/en/sdk/squid-sdk/reference/openreader-server/configuration/custom-resolvers) with arbitrary SQL
* [Secret environment variables](/en/cloud/resources/env-variables), allowing to seamlessly use private third-party JSON-RPC endpoints and integrate with external APIs
* [Deployment slots and tags](/en/cloud/resources/slots-and-tags)
* [API caching](/en/sdk/squid-sdk/reference/openreader-server/configuration/caching)

For a full feature set comparison, see [SQD vs The Graph](/en/sdk/squid-sdk/subsquid-vs-thegraph).

## Squid setup

### 1. Install Squid CLI

Instructions [here](/en/sdk/squid-sdk/squid-cli/installation).

### 2. Fetch the template

Create a squid from the minimalistic [`evm` template](https://github.com/subsquid-labs/squid-evm-template):

```bash theme={"system"}
sqd init my-new-squid -t evm
cd squid-evm-template
```

### 3. Copy the schema file and generate entities

The minimal template already contains a dummy `schema.graphql` file. We replace it with the subgraph schema as is:

```gql file=schema.graphl theme={"system"}
type Gravatar @entity {
  id: ID!
  owner: Bytes!
  displayName: String!
  imageUrl: String!
}
```

Next, we generate the entities from the schema using the [`squid-typeorm-codegen`](/en/sdk/squid-sdk/reference/schema-file/intro#typeorm-codegen) tool of the Squid SDK, then build the squid:

```bash theme={"system"}
npx squid-typeorm-codegen
npm run build
```

This command is equivalent to running `yarn codegen` in subgraph.

After that, start the local database and regenerate migrations based on the generated entities using the [`squid-typeorm-migration`](/en/sdk/squid-sdk/resources/persisting-data/typeorm) tool:

```bash theme={"system"}
docker compose up -d
```

```bash theme={"system"}
rm -r db/migrations
```

```bash theme={"system"}
npx squid-typeorm-migration generate
```

A database migration file for creating a table for `Gravatar` will appear in `db/migrations`. Apply it with

```bash theme={"system"}
npx squid-typeorm-migration apply
```

### 4. Generate typings from ABI

Copy `./abis/Gravity.json` from the subgraph project and paste it to `./abi` folder in the squid project.
To generate the typings, run:

```bash theme={"system"}
npx squid-evm-typegen ./src/abi ./abi/*.json --multicall
```

Alternatively, similar to `graph add <address> [<subgraph-manifest default: "./subgraph.yaml">]` command, to generate typings, run:

```bash theme={"system"}
npx squid-evm-typegen src/abi 0x2E645469f354BB4F5c8a05B3b30A929361cf77eC#Gravity --clean
```

This command runs the `evm-typegen` tool that fetches the contract ABI by the address and generates type-safe access classes in `src/abi/Gravity.ts`. The generated boilerplate code will be used to decode EVM logs and directly query the contract. It also contains topic definitions used in the next step.

### 5. Subscribe to EVM logs

While in The Graph data source is defined in the manifest file `subgraph.yaml`, in SQD subscriptions to EVM data, including logs, are performed at the processor object definition customarily located at `src/processor.ts`. The processor is configured directly by the code, unlike subgraphs which require handlers and events to be defined in the manifest file.

```ts file=src/processor.ts theme={"system"}
import { EvmBatchProcessor} from '@subsquid/evm-processor'

// generated by the evm-typegen tool
// the events object contains typings for all events defined in the ABI
import { events } from './abi/Gravity'

export const GRAVATAR_CONTRACT =
  '0x2E645469f354BB4F5c8a05B3b30A929361cf77eC'.toLowerCase()

export const processor = new EvmBatchProcessor()
  // change the gateway URL to run against other EVM networks, e.g.
  // 'https://v2.archive.subsquid.io/network/polygon-mainnet'
  // 'https://v2.archive.subsquid.io/network/binance-mainnet'
  .setGateway('https://v2.archive.subsquid.io/network/ethereum-mainnet')
  .setRpcEndpoint('<my_eth_rpc_url>')
  .setBlockRange({ from: 6175243 })
  .setFinalityConfirmation(75)
  .addLog({
    address: [ GRAVATAR_CONTRACT ],
    topic0: [
      events.NewGravatar.topic,
      events.UpdatedGravatar.topic,
    ],
  })
```

In the snippet above we tell the squid processor to fetch logs emitted by the contract `0x2E645469f354BB4F5c8a05B3b30A929361cf77eC` with topic0 within a specified list. The configuration also states that indexing should start from block `6175243`, the height at which the contract was deployed.

Check out the [EVM indexing](/en/sdk/squid-sdk) section for the list of supported networks and configuration details.

The above snippet is eqivalent to the following `subgraph.yaml`:

```yaml file=subgraph.yaml theme={"system"}
specVersion: 0.0.4
description: Gravatar for Ethereum
repository: https://github.com/graphprotocol/example-subgraph
schema:
  file: ./schema.graphql
dataSources:
  - kind: ethereum/contract
    name: Gravity
    network: mainnet
    source:
      address: '0x2E645469f354BB4F5c8a05B3b30A929361cf77eC'
      abi: Gravity
    mapping:
      kind: ethereum/events
      apiVersion: 0.0.5
      language: wasm/assemblyscript
      entities:
        - Gravatar
      abis:
        - name: Gravity
          file: ./abis/Gravity.json
      eventHandlers:
        - event: NewGravatar(uint256,address,string,string)
          handler: handleNewGravatar
        - event: UpdatedGravatar(uint256,address,string,string)
          handler: handleUpdatedGravatar
      file: ./src/mapping.ts
```

### 6. Transform and save the data

In Subgraph data is saved in the `mapping.ts` file. The mapping function will receive an `ethereum.Block` as its only argument. In SQD, we set up the processor in `processor.ts` and save the data in the `main.ts`.

We migrate the subgraph handlers that transform the event data into `Gravatar` objects. Instead of saving or updating gravatars one by one, `EvmBatchProcessor` receives an ordered batch of event items it is subscribed to. In our case we have only two kinds of logs -- emitted on gravatar creations and updates.

The entry point for transform code is `src/main.ts`. We start by appending an auxiliary data normalization function to the end of that file:

```ts title="src/main.ts" theme={"system"}
function extractData(evmLog: any): { id: bigint, owner: string, displayName: string, imageUrl: string} {
  if (evmLog.topics[0] === events.NewGravatar.topic) {
    return events.NewGravatar.decode(evmLog)
  }
  if (evmLog.topics[0] === events.UpdatedGravatar.topic) {
    return events.UpdatedGravatar.decode(evmLog)
  }
  throw new Error('Unsupported topic')
}
```

Next, we make a [batch handler](/en/sdk/squid-sdk/reference/processors/architecture#processorrun) collecting the updates from a single batch of EVM logs. To convert a `0x...` string into a byte array we use the `decodeHex` utility from Squid SDK.

```ts title="src/main.ts" theme={"system"}
import { TypeormDatabase } from '@subsquid/typeorm-store'
import { decodeHex } from '@subsquid/evm-processor'
import { events } from './abi/Gravity'
import { Gravatar } from './model'
import { processor, GRAVATAR_CONTRACT } from './processor'

processor.run(new TypeormDatabase({supportHotBlocks: true}), async (ctx) => {
  const gravatars: Map<string, Gravatar> = new Map()
  for (const c of ctx.blocks) {
    for (const e of c.logs) {
      if (!(e.address === GRAVATAR_CONTRACT &&
            (e.topics[0] === events.NewGravatar.topic ||
             e.topics[0] === events.UpdatedGravatar.topic))) continue
      const { id, owner, displayName, imageUrl } = extractData(e)
      let idString = '0x' + id.toString(16)
      gravatars.set(idString, new Gravatar({
        id: idString,
        owner: decodeHex(owner),
        displayName,
        imageUrl
      }))
    }
  }
  await ctx.store.upsert([...gravatars.values()])
})
```

The implementation is straightforward -- the newly created and/or updated gravatars are tracked by an in-memory map. The values are persisted in a single batch upsert once all items are processed.

### 7. Run the processor and GraphQL API

To start the indexing, (re)build the project and run the processor:

```bash theme={"system"}
npm run build
```

```bash theme={"system"}
node -r dotenv/config lib/main.js
```

The processor will output the sync progress and the ETA to reach the chain head. After it reaches the head it will continue indexing new blocks until stopped.

To start an API server (at port `4350` by default) with a GraphQL schema auto-generated from the schema file, run in a new terminal window

```bash theme={"system"}
npx squid-graphql-server
```

and inspect the auto-generated GraphQL API using an interactive playground at [http://localhost:4350/graphql](http://localhost:4350/graphql).

## What's Next?

* Compare your API to that of subgraph using the [compareGraphQL](https://github.com/subsquid-labs/compareGraphQL) script
* Have a closer look at [`EvmBatchProcessor`](/en/sdk/squid-sdk)
* Learn how to [deploy a squid to SQD Cloud](/en/cloud) for free
* Learn how to [index and query the contract state](/en/sdk/squid-sdk/resources/tools/typegen/state-queries)
* Inspect a more complex [Uniswap V3 squid](https://github.com/dariaag/uniswap-squid-arrow) which tracks Uniswap V3 trading data. It was migrated from the [Uniswap V3 subgraph](https://github.com/Uniswap/v3-subgraph). It takes only a few hours to sync from scratch on a local machine.


# ArrowSquid for EVM
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/migrate/migrate-to-arrowsquid

Step-by-step migration from FireSquid to ArrowSquid for EVM Squid SDK indexers — schema, processor, and storage code changes for the new architecture.

# Migrate to ArrowSquid (EVM)

This is an EVM guide. Substrate guide is available [here](/en/sdk/squid-sdk/resources/migrate/migrate-to-arrowsquid-on-substrate).

ArrowSquid refers to the versions `@subsquid/evm-processor@1.x` and `@subsquid/substrate-processor@3.x`. ArrowSquid is not compatible with the FireSquid [archive](/en/sdk/squid-sdk/glossary#archives) endpoints. Instead, it uses [SQD Network](/en/network) gateways (see the [Supported EVM networks](/en/data/evm) page).

The main feature introduced by the ArrowSquid update on EVM is the new ability of the [processor](/en/sdk/squid-sdk/overview) to ingest unfinalized blocks directly from a network node, instead of waiting for the archive to ingest and serve it first. The processor can now handle forks and rewrite the contents of its database if it happens to have indexed orphaned blocks. This allows SQD-based APIs to become near real-time and respond to the on-chain activity with subsecond latency.

Another major feature introduced by ArrowSquid is the support for transaction execution receipts, [traces](/en/sdk/squid-sdk/reference/processors/evm-batch/traces) and [state diffs](/en/sdk/squid-sdk/reference/processors/evm-batch/state-diffs). It enables a significantly more fine-grained control over the smart contract states, especially in the situations when the EVM log data is insufficient. For example, one can:

* Reliably index transaction data, taking into account the transaction status
* Keep track of internal calls
* Observe smart contract state changes even if they are caused by internal transactions
* Track smart contract creation and destruction

The `EvmBatchProcessor` configuration and data selection interfaces has been simplified and the way in which the data is fetched has been made more efficient.

End-to-end ArrowSquid examples can be found [in the SDK repo](https://github.com/subsquid/squid-sdk/tree/master/test/eth-usdc-transfers) and in the [EVM examples](/en/sdk/squid-sdk/examples) section.

Here is a step-by-step guide for migrating a squid built with an older SDK version to the post-ArrowSquid tooling.

## Step 1

Update all packages affected by the update:

```bash theme={"system"}
npm i @subsquid/evm-processor@next
npm i @subsquid/typeorm-store@next
```

If your squid uses [`file-store`](/en/sdk/squid-sdk/resources/persisting-data/file), please update any related packages to the `@next` version, too.

## Step 2

Replace the old `setDataSource()` processor configuration call with a combination of [setGateway()](/en/sdk/squid-sdk/reference/processors/evm-batch/general#set-gateway) and [setRpcEndpoint()](/en/sdk/squid-sdk/reference/processors/evm-batch/general#set-rpc-endpoint). Use a [public SQD Network gateway URL for your network](/en/data/evm). If your squid did not use an RPC endpoint before, find one for your network and set it with [setRpcEndpoint()](/en/sdk/squid-sdk/reference/processors/evm-batch/general#set-rpc-endpoint). Also [configure](/en/sdk/squid-sdk/reference/processors/evm-batch/general#set-finality-confirmation) the network-specific number of transaction confirmations sufficient for finality. For Ethereum mainnet your edit might look like this:

```diff theme={"system"}
 processor
-  .setDataSource({
-    archive: lookupArchive('eth-mainnet', {release: 'FireSquid'})
-  })
+  .setGateway('https://v2.archive.subsquid.io/network/ethereum-mainnet')
+  .setRpcEndpoint({
+    url: '<my_eth_rpc_url>',
+    rateLimit: 10
+  })
+  .setFinalityConfirmation(75)
```

We recommend using a private RPC endpoint for the best performance, e.g. from [BlastAPI](https://blastapi.io/). For squids deployed to [SQD Cloud](/en/cloud/overview) you may also consider using our [RPC addon](/en/cloud/resources/rpc-proxy).

Your squid will work without an RPC endpoint, but with a significantly increased chain latency (a few hours for most chains, roughly a day for BSC). If that works for you, you can use just the SQD Network gateway without setting an RPC here and skip [Step 7](#step-7) altogether.

## Step 3

Next, we have to account for the changes in signatures of [`addLog()`](/en/sdk/squid-sdk/reference/processors/evm-batch/logs) and [`addTransaction()`](/en/sdk/squid-sdk/reference/processors/evm-batch/transactions) processor methods. Previously, each call of these methods supplied its own fine-grained field selectors. In the new interface, these calls can only enable or disable the retrieval of related data (with boolean flags `transaction` for `addLog()` and `logs` and a few others for `addTransaction()`). Field selection is now done by the new [`setFields()`](/en/sdk/squid-sdk/reference/processors/evm-batch/field-selection) method on a per-item-type basis: once for all `log`s, once for all `transaction`s etc. The setting is processor-wide: for example, all `transaction`s returned by the processor will have the same set of available fields, regardless of whether they are taken from the batch context directly or are accessed from within a `log` item.

Begin migrating to the new interface by finding all calls to `addLog()` and combining all the `evmLog` field selectors into a single processor-wide field selector that requests all fields previously requested by individual selectors. Remove the `id`, `logIndex` (previously `index`) and `transactionIndex` fields: now they are always available and cannot be requested explicitly. When done, make a call to `setFields()` and supply the new field selector at the `log` field of its argument. For example, suppose the processor was initialized with the following three calls:

```typescript theme={"system"}
const processor = new EvmBatchProcessor()
  .addLog(CONTRACT_ADDRESS, {
    filter: [[ abi.events.Transfer.topic ]],
    data: {
      evmLog: {
        id: true
      }
    } as const
  })
  .addLog(CONTRACT_ADDRESS, {
    filter: [[ abi.events.Approval.topic ]],
    data: {
      evmLog: {
        id: true,
        data: true
      }
    } as const
  })
  .addLog(CONTRACT_ADDRESS, {
    filter: [[ abi.events.ApprovalForAll.topic ]],
    data: {
      evmLog: {
        topics: true
      }
    } as const
  })
```

then the new global selector should be added like this:

```typescript theme={"system"}
const processor = new EvmBatchProcessor()
  // ... addLog() calls ...
  .setFields({
    log: {
      data: true,
      topics: true
    }
  })
```

Be aware that this operation will not increase the amount of data retrieved from SQD Network, since previously such coalescence was done under the hood and all fields were retrieved by the processor anyway. In fact, the amount of data should decrease due to a more efficient transfer mechanism employed by ArrowSquid.

See the [Field selection](/en/sdk/squid-sdk/reference/processors/evm-batch/field-selection) page for full documentation on field selectors.

## Step 4

Repeat step 3 for the `transaction` field selector. Make sure to check any transaction field selections by `addLog()` calls in addition to these made by [`addTransaction()`](/en/sdk/squid-sdk/reference/processors/evm-batch/transactions). Remove the default fields `id` and `transactionIndex` (previously `index`) and add the final field selector to the `.setFields()` call. For example, suppose the processor was initialized like this:

```typescript theme={"system"}
const processor = new EvmBatchProcessor()
  .addLog(CONTRACT_ADDRESS, {
    filter: [[ abi.events.Transfer.topic ]],
    data: {
      evmLog: { /* ...some log field selections... */ },
      transaction: {
        hash: true
      }
    } as const
  })
  .addTransaction(CONTRACT_ADDRESS, {
    sighash: [ abi.functions.approve.sighash ],
    data: {
      transaction: {
        gas: true,
        gasPrice: true
      }
    } as const
  })
```

then the new global selector should be added like this:

```diff theme={"system"}
 const processor = new EvmBatchProcessor()
   // ... addLog() and addTransaction() calls ...
   .setFields({
     log: { /* ...some log field selections... */ },
+    transaction: {
+      hash: true,
+      gas: true,
+      gasPrice: true
+    }
   })
```

See the [Field selection](/en/sdk/squid-sdk/reference/processors/evm-batch/field-selection) page for full documentation on field selectors.

## Step 5

Replace the old calls to `addLog()` and `addTransaction()` with calls using the [new signatures](/en/sdk/squid-sdk/reference/processors/evm-batch).

<Warning>
  The meaning of passing `[]` as a set of parameter values has been changed in the ArrowSquid release: now it *selects no data*. Pass `undefined` for a wildcard selection:

  ```typescript theme={"system"}
  .addLog({address: []}) // selects no logs
  .addLog({}) // selects all logs
  ```

  ```typescript theme={"system"}
  .addTransaction({from: []}) // selects no transactions
  .addTransaction({}) // selects all transactions
  ```
</Warning>

Old data requests will be erased during the process. Make sure to request the appropriate data with the boolean flags (`transaction` for `addLog()` and `logs` for `addTransaction()`) while doing that.

For example, a processor originally initialized like this:

```typescript theme={"system"}
const processor = new EvmBatchProcessor()
  .addLog(CONTRACT_ADDRESS, {
    filter: [[ abi.events.Transfer.topic ]],
    data: {
      evmLog: {
        topics: true,
        data: true
      },
      transaction: {
        hash: true
      }
    } as const
  })
  .addTransaction(CONTRACT_ADDRESS, {
    sighash: [ abi.functions.approve.sighash ],
    data: {
      transaction: {
        gas: true,
        gasPrice: true
      }
    } as const
  })
```

should now be made with

```typescript theme={"system"}
const processor = new EvmBatchProcessor()
  .addLog({
    address: [ CONTRACT_ADDRESS ],
    topic0: [ abi.events.Transfer.topic ],
    transaction: true // IMPORTANT: set this to true whenever the old call defined options.data.transaction
  })
  .addTransaction({
    to: [ CONTRACT_ADDRESS ],
    sighash: [ abi.functions.approve.sighash ]
  })
  .setFields({
    log: {
      topics: true,
      data: true
    },
    transaction: {
      hash: true,
      gas: true,
      gasPrice: true
    }
  })
```

## Step 6

Finally, update the batch handler to use the new [batch context](/en/sdk/squid-sdk/reference/processors/architecture#batch-context). There are two ways to do that:

1. If you're in a hurry, use the [`transformContext.ts`](https://gist.github.com/belopash/aa6b67dc374add44b9bdff1c9c1eee17) module. Download it with
   ```bash theme={"system"}
   curl -o src/transformContext.ts https://gist.githubusercontent.com/belopash/aa6b67dc374add44b9bdff1c9c1eee17/raw/441d43a932591624822b5bfd51a23147b5cecac2/transformContext.ts
   ```
   then transform the new context to the old format at the beginning of the batch handler:
   ```typescript title="src/processor.ts" theme={"system"}
   import {transformContext} from './transformContext'

   // ...

   processor.run(db, async (newCtx: DataHandlerContext<Store, any>) => {
     let ctx = transformContext(newCtx)
     // the rest of the batch handler should work unchanged
   })
   ```

2. Alternatively, rewrite your batch handler using the [new batch context interface](/en/sdk/squid-sdk/reference/processors/architecture#batch-context). Consult the [block data](/en/sdk/squid-sdk/reference/processors/evm-batch/context-interfaces) page for EVM-specific details on the new context format.

## Step 7

Update your transactions/events decoding code. The big change here is that now decoders generated by `@subsquid/evm-typegen` return `bigint` where `ethers.BigNumber` was used before. Regenerate all TypeScript ABI wrappers as described in the [EVM typegen](/en/sdk/squid-sdk/resources/tools/typegen/state-queries) section, then find all places where `ethers.BigNumber`s returned by old decoders were handled in your code and rewrite it to use `bigint`s.

## Step 8

Iteratively reconcile any type errors arising when building your squid (e.g. with `npm run build`). In case you're using `tranformContext.ts` you may find the types it exports helpful. If you need to specify the field selection generic argument explicitly, get it as a `typeof` of the `setFields` argument value:

```ts theme={"system"}
import { OldBlockData } from './transformContext'

const fieldSelection = {
  log: {
    data: true
  },
  transaction: {
    hash: true,
  }
} as const

let processor = new EvmBatchProcessor()
  .setFields(fieldSelection)
  /* the rest of the processor configuration */

type MyBlockData = OldBlockData<typeof fieldSelection>
// ...
```

At this point your squid should be able to work with the ArrowSquid tooling. If it doesn't, read on.

## Troubleshooting

If these instructions did not work for you, please let us know at the [SquidDevs Telegram chat](https://t.me/HydraDevs).


# ArrowSquid for Substrate
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/migrate/migrate-to-arrowsquid-on-substrate

Step-by-step migration guide from FireSquid to ArrowSquid for Substrate-based Squid SDK indexers — schema, processor, and storage code changes.

# Migrate to ArrowSquid (Substrate)

This is a Substrate guide. EVM guide is available [here](/en/sdk/squid-sdk/resources/migrate/migrate-to-arrowsquid).

ArrowSquid refers to `@subsquid/substrate-processor` versions `3.x` and above. It is not compatible with the FireSquid [archive](/en/sdk/squid-sdk/glossary#archives) endpoints. Instead, the new version uses [SQD Network](/en/network) gateways. See the [Supported networks](/en/data/substrate) page.

The main feature introduced by the ArrowSquid update is the new ability of the [processor](/en/sdk/squid-sdk/overview) to ingest unfinalized blocks directly from a network node, instead of waiting for an archive to ingest and serve it first. The processor can now handle forks and rewrite the contents of its database if it happens to have indexed orphaned blocks. This allows SQD-based APIs to become near real-time and respond to the on-chain activity with subsecond latency.

Other new features include the new streamlined processor configuration interface and automatic retrieval of execution traces. On the implementation side, the way in which data is fetched has been made more efficient.

End-to-end Substrate ArrowSquid examples:

* [general Substrate indexing](https://github.com/subsquid-labs/squid-substrate-template);
* [indexing an Ink! contract](https://github.com/subsquid-labs/squid-ink-template/);
* [working with Frontier EVM pallet data](https://github.com/subsquid-labs/squid-frontier-evm-template).

Here is a step-by-step guide for migrating a squid built with an older SDK version to the post-ArrowSquid tooling.

## Step 1

Update all packages affected by the update:

```bash theme={"system"}
npm i @subsquid/substrate-processor@latest @subsquid/typeorm-store@latest
```

```bash theme={"system"}
npm i --save-dev @subsquid/substrate-typegen@latest
```

If your squid uses [`file-store`](/en/sdk/squid-sdk/resources/persisting-data/file), please update any related packages to the `@latest` version.

## Step 2

Replace the old `setDataSource()` processor configuration call with a combination of [setGateway()](/en/sdk/squid-sdk/reference/processors/substrate-batch/general#set-gateway) and [setRpcEndpoint()](/en/sdk/squid-sdk/reference/processors/substrate-batch/general#set-rpc-endpoint). Use a [public SQD Network gateway URL for your network](/en/data/substrate). If your squid did not use an RPC endpoint before, find one for your network and set it with [setRpcEndpoint()](/en/sdk/squid-sdk/reference/processors/substrate-batch/general#set-rpc-endpoint). For Aleph Zero your edit might look like this:

```diff theme={"system"}
 processor
-  .setDataSource({
-    archive: 'https://aleph-zero.archive.subsquid.io/graphql'
-  })
+  .setGateway('https://v2.archive.subsquid.io/network/aleph-zero')
+  .setRpcEndpoint({
+    url: 'https://aleph-zero-rpc.dwellir.com',
+    rateLimit: 10
+  })
```

We recommend using a private RPC endpoint for the best performance, e.g. from [Dwellir](https://www.dwellir.com). For squids deployed to [SQD Cloud](/en/cloud/overview) you may also consider using our [RPC addon](/en/cloud/resources/rpc-proxy).

Your squid will work with just an RPC endpoint, but it will sync significantly slower. With a SQD Network gateway available the processor will only use RPC to retrieve metadata **and** sync the few most recent blocks not yet made available by the gateway; without it, it will retrieve all data from the endpoint.

## Step 3

Next, we have to account for the changes in signatures of the data requesting processor methods

* [`addEvent()`](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#events),
* [`addCall()`](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#calls),
* [`addEvmLog()`](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#addevmlog),
* [`addEthereumTransaction()`](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#addethereumtransaction),
* [`addContractsContractEmitted()`](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#addcontractscontractemitted),
* [`addGearMessageQueued()`](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#addgearmessagequeued),
* [`addGearUserMessageSent()`](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#addgearusermessagesent),

Previously, each call of these methods supplied its own fine-grained data fields selector. In the new interface, these calls only request data items, either directly or by relation (for example with the `call` flag for event-requesting methods). Field selection is now done by the new `setFields()` method on a per-item-type basis: once for all `Call`s, once for all `Event`s etc. The setting is processor-wide: for example, all `Call`s returned by the processor will have the same set of available fields, regardless of whether they were requested directly or as related data.

Begin migrating to the new interface by finding all calls to these methods and combining all the field selectors into processor-wide `event`, `call` and `extrinsic` field selectors that request all fields previously requested by individual selectors. Note that `call.args` and `event.args` are now requested by default and can be omitted. When done, add a call to `setFields()` supplying it with the new field selectors.

The new field selector format is fully documented on the [Field selection](/en/sdk/squid-sdk/reference/processors/substrate-batch/field-selection) page.

<Info>
  Blanket field selections like `{data: {event: {extrinsic: true}}}` are not supported in ArrowSquid. If you used one of these, please find out which exact fields you use in the batch handler and specifically request them.
</Info>

<Warning>
  Do not include related data requests into the field selection. E.g.

  ```ts theme={"system"}
    .setFields({
      event: {
        call: true
      }
    })
  ```

  will at best fail to compile and at worst make your squid fail due to HTTP 400s from the network gateway.
</Warning>

For example, suppose the processor was initialized with the following three calls:

```typescript theme={"system"}
const processor = new SubstrateBatchProcessor()
  .addCall('Balances.transfer_keep_alive', {
    data: {
      call: {
        origin: true,
        args: true
      }
    }
  } as const)
  .addEvent('Balances.Transfer', {
    data: {
      event: {
        args: true,
        extrinsic: {
          hash: true,
          fee: true
        },
        call: {
          success: true,
          error: true
        }
      }
    }
  } as const)
  .addEvmLog(CONTRACT_ADDRESS,
    filter: [[ abi.events.SomeLog.topic ]],
    {
      data: {
        event: {
          extrinsic: {
            hash: true,
            tip: true
          }
        }
      }
    } as const
  )
```

then the new global selectors should be added like this:

```typescript theme={"system"}
const processor = new SubstrateBatchProcessor()
  // ... addXXX() calls ...
  .setFields({
    event: {},
    call: {
      origin: true,
      success: true,
      error: true
    },
    extrinsic: {
      hash: true,
      fee: true,
      tip: true
    }
  })
```

Be aware that this operation will not increase the amount of data retrieved from the SQD Network gateway, since previously such coalescence was done under the hood and all fields were retrieved by the processor anyway. In fact, the amount of data should decrease due to a more efficient transfer mechanism employed by ArrowSquid.

There are two old field requests that have no direct equivalent in the new interface:

<AccordionGroup>
  <Accordion title="call.parent">
    It is currently impossible to request just the parent call. Work around by requesting the full call stack with `stack: true` in the call-requesting configuration calls, then using `.parentCall` property or `getParentCall()` method of `Call` data items to get parent calls.
  </Accordion>

  <Accordion title="event.evmTxHash">
    Processor no longer makes EVM transaction hashes explicitly available. Currently the only way to get hashes of logs' parent txs is to figure out the subset of calls that emitted the required logs, request them explicitly with their related events, filter out `Ethereum.Executed` events and decode these. See [this issue](https://github.com/subsquid/squid-sdk/issues/211).
  </Accordion>
</AccordionGroup>

## Step 4

Replace the old calls to the data requesting processor methods with calls using the new signatures.

<Warning>
  The meaning of passing `[]` as a set of parameter values has been changed in the ArrowSquid release: now it *selects no data*. Pass `undefined` for a wildcard selection:

  ```typescript theme={"system"}
  .addEvent({name: []}) // selects no events
  .addEvent({}) // selects all events
  ```
</Warning>

Old data request calls will be erased during the process. Make sure to request the appropriate related data with the boolean flags (`call` for event-requesting methods, `events` for call-requesting methods and `extrinsic`, `stack` for both).

Interfaces of data request methods are documented on the Data requests reference page:

* [`addEvent()`](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#events),
* [`addCall()`](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#calls),
* [`addEvmLog()`](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#addevmlog),
* [`addEthereumTransaction()`](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#addethereumtransaction),
* [`addContractsContractEmitted()`](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#addcontractscontractemitted),
* [`addGearMessageQueued()`](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#addgearmessagequeued),
* [`addGearUserMessageSent()`](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#addgearusermessagesent).

Here is a fully updated initialization code for the example processor from step 3:

```typescript theme={"system"}
const processor = new SubstrateBatchProcessor()
  .addCall({
    name: [ 'Balances.transfer_keep_alive' ]
  })
  .addEvent({
    name: [ 'Balances.Transfer' ],
    call: true,
    extrinsic: true
  })
  .addEvmLog({
    address: [ CONTRACT_ADDRESS ],
    topic0: [ abi.events.SomeLog.topic ],
    extrinsic: true
  })
  .setFields({
    event: {},
    call: {
      origin: true,
      success: true,
      error: true
    },
    extrinsic: {
      hash: true,
      fee: true,
      tip: true
    }
  })

```

## Step 5

Finally, update the batch handler to use the new [batch context](/en/sdk/squid-sdk/reference/processors/architecture#batch-context). The main change here is that now `block.items` is split into three separate iterables: `block.calls`, `block.events` and `block.extrinsics`. There are two ways to migrate:

1. If you're in a hurry, use the `orderItems(block: Block)` function from [this snippet](https://gist.github.com/belopash/5d61dcce7739f60d55c4faaec0148282):
   ```typescript title="src/main.ts" theme={"system"}
   // ...
   // paste the gist here

   processor.run(db, async ctx => {
     // ...
     for (let block of ctx.blocks) {
       for (let item of orderItems(block)) {
         // item processing code should work unchanged
       }
     }
     // ...
   })
   ```

2. Alternatively, rewrite your batch handler using the [new batch context interface](/en/sdk/squid-sdk/reference/processors/architecture#batch-context).

See [Block data for Substrate](/en/sdk/squid-sdk/reference/processors/substrate-batch/context-interfaces) for the documentation on Substrate-specific part of batch context.

## Step 6

Rewrite your `typegen.json` in the new style. Here is an example:

```json theme={"system"}
{
  "outDir": "src/types",
  "specVersions": "https://v2.archive.subsquid.io/metadata/kusama",
  "pallets": {
    "Balances": {
      "events": [
        "Transfer"
      ],
      "calls": [],
      "storage": [],
      "constants": []
    }
  }
}
```

Note the changes:

1. Archive URL as `"specVersions"` is replaced with an URL of our new metadata service (`"https://v2.archive.subsquid.io/metadata/kusama"`)
2. Requests for data wrappers are now made on a per-pallet basis.

Check out the updated [Substrate typegen documentation page](/en/sdk/squid-sdk/tutorials/batch-processor-in-action). If you used any storage calls, consult [this documentation page](/en/sdk/squid-sdk/resources/tools/typegen/state-queries) for guidance.

Once you're done migrating `typegen.json`, regenerate the wrapper classes with

```bash theme={"system"}
npx squid-substrate-typegen typegen.json
```

## Step 7

Iteratively reconcile any type errors arising when building your squid (e.g. with `npm run build`). If you need to specify the field selection generic type argument explicitly, get it as a `typeof` of the `setFields` argument value:

```ts theme={"system"}
import {Block} from '@subsquid/substrate-processor'

const fieldSelection = {
  event: {},
  call: {
    origin: true,
    success: true,
    error: true
  },
  extrinsic: {
    hash: true,
    fee: true,
    tip: true
  }
} as const

type MyBlock = Block<typeof fieldSelection>
// ...
```

At this point your squid should be able to work with the ArrowSquid tooling. If it doesn't, read on.

## Troubleshooting

If these instructions did not work for you, please let us know at the [SquidDevs Telegram chat](https://t.me/HydraDevs).


# hasura-configuration tool v2
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/migrate/migrate-to-hasura-configuration-tool-v2

Migrate to the Hasura configuration tool v2 in Squid SDK — breaking changes, new metadata schema, and how to update your squid project.

# Migrating to Hasura configuration tool v2

Pre-2.0.0 [`@subsquid/hasura-configuration`](/en/sdk/squid-sdk/resources/tools/hasura-configuration) used a fixed naming schema for [relation fields](/en/sdk/squid-sdk/reference/schema-file/entity-relations):

* Forward relation fields were *always* named after the type they referred to. Type names were `snake_case`d. For example, any field referring to an entity called `BurnTest` was called `burn_test`.
* Inverse relation fields names were also determined by the relation type:
  * In one-to-one relation their names were `to_snake_case(typeName)`, where `typeName` is the name of the entity that the inverse field is typed with.
  * In one-to-many inverse relations they were named `${to_snake_case(typeName)}s`.

Any names given to the fields in the [schema](/en/sdk/squid-sdk/reference/schema-file) were ignored.

`@subsquid/hasura-configuration@2.0.0` introduces full support for in-schema field names. Now the fields will be called exactly as they are in the schema file. If these field names are different from what's described above, your API will have a breaking change. If you'd like to avoid it, please make sure that the fields in your `schema.graphql` are named exactly as described above.

To update the tool and your Hasura config:

```bash theme={"system"}
npm i @subsquid/hasura-configuration@latest
npx squid-hasura-configuration regenerate
```

If you deployed to the Cloud you can safely (barring the field names change) redeploy your squid in-place.

If you opted to not change your schema to accommodate the old relation field names, please follow this up by revising GraphQL queries in any of your client apps.


# Multichain
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/multichain

Combine data from multiple chains in a single Squid SDK indexer — multichain processor patterns, shared schema, and write-coordination strategies.

# Multichain indexing

Squids can extract data from multiple chains into a shared data sink. If the data is [stored to Postgres](/en/sdk/squid-sdk/resources/persisting-data/typeorm) it can then be served as a unified multichain [GraphQL API](/en/sdk/squid-sdk/resources/serving-graphql).

To do this, run one [processor](/en/sdk/squid-sdk/overview) per source network:

1. Make a separate entry point (`main.ts` or equivalent) for each processor. The resulting folder structure may look like this:

   <Tree>
     <Tree.Folder name="src">
       <Tree.Folder name="bsc">
         <Tree.File name="main.ts" />

         <Tree.File name="processor.ts" />
       </Tree.Folder>

       <Tree.Folder name="eth">
         <Tree.File name="main.ts" />

         <Tree.File name="processor.ts" />
       </Tree.Folder>
     </Tree.Folder>
   </Tree>

Alternatively, parameterize your processor using environment variables: you can [set these on a per-processor basis](/en/cloud/reference/manifest#processor) if you use a deployment manifest to run your squid.

2. Arrange for running the processors alongside each other conveniently:
   * Add `sqd` commands for running each processor to [`commands.json`](/en/sdk/squid-sdk/squid-cli/commands-json), e.g.
     ```json title="commands.json" theme={"system"}
     ...
       "process:eth": {
         "deps": ["build", "migration:apply"],
         "cmd": ["node", "lib/eth/main.js"]
       },
       "process:bsc": {
         "deps": ["build", "migration:apply"],
         "cmd": ["node", "lib/bsc/main.js"]
       },
     ...
     ```
     [Full example](https://github.com/subsquid-labs/multichain-transfers-example/blob/master/commands.json)
   * If you are going to use [`sqd run`](/en/sdk/squid-sdk/squid-cli/run) for local runs or deploy your squid to [SQD Cloud](/en/cloud), list your processors at the `deploy.processor` section of your [deployment manifest](/en/cloud/reference/manifest#processor):
     ```yaml theme={"system"}
     deploy:
       processor:
         - name: eth-processor
           cmd: [ "sqd", "process:prod:eth" ]
         - name: bsc-processor
           cmd: [ "sqd", "process:prod:bsc" ]
     ```
     Make sure to give each processor a unique name!

## On Postgres

Also ensure that

1. State schema name for each processor is unique

```ts title="src/bsc/main.ts" theme={"system"}
processor.run(
  new TypeormDatabase({
    stateSchema: 'bsc_processor'
  }),
  async ctx => { // ...
```

```ts title="src/eth/main.ts" theme={"system"}
processor.run(
  new TypeormDatabase({
    stateSchema: 'eth_processor'
  }),
  async (ctx) => { // ...
```

2. [Schema](/en/sdk/squid-sdk/reference/schema-file) and [GraphQL API](/en/sdk/squid-sdk/resources/serving-graphql) are shared among the processors.

### Handling concurrency

* Cross-chain data dependencies are to be avoided. With the [default isolation level](/en/sdk/squid-sdk/reference/store/typeorm#typeormdatabase-constructor-arguments) used by `TypeormDatabase`, `SERIALIZABLE`, one of the processors will crash with an error whenever two cross-dependent transactions are submitted to the database simultaneously. It will write the correct data when restarted, but such restarts can impact performance, especially in squids that use many (>5) processors.

* The alternative isolation level is `READ COMMITTED`. At this level data dependencies will not cause the processors to crash, but the execution is not guaranteed to be deterministic unless the sets of records that different processors read/write do not overlap.

* To avoid cross-chain data dependencies, use per-chain records for volatile data. E.g. if you track account balances across multiple chains you can avoid overlaps by storing the balance for each chain in a different table row.

  When you need to combine the records (e.g. get a total of all balaces across chains) use a [custom resolver](/en/sdk/squid-sdk/reference/openreader-server/configuration/custom-resolvers) to do it on the GraphQL server side.

* It is OK to use cross-chain [entities](/en/sdk/squid-sdk/reference/schema-file/entities) to simplify aggregation. Just don't store any data in them:
  ```graphql theme={"system"}
  type Account @entity {
    id: ID! # evm address
    balances: [Balance!]! @derivedFrom("field": "account")
  }

  type Balance @entity {
    id: ID! # chainId + evm address
    account: Account!
    value: BigInt!
  }
  ```

## On [file-store](/en/sdk/squid-sdk/resources/persisting-data/file)

Ensure that you use a unique target folder for each processor.

## Example

A complete example is available [here](https://github.com/subsquid-labs/squid-multichain-template).


# Saving to BigQuery
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/persisting-data/bigquery

Save Squid SDK indexed data to Google BigQuery via @subsquid/bigquery-store — schema sync, batch loading, and partitioning patterns for analytics.

# BigQuery support

Squids can store their data to BigQuery datasets using the `@subsquid/bigquery-store` package. Define and use the `Database` object as follows:

```ts title="src/main.ts" theme={"system"}
import {
  Column,
  Table,
  Types,
  Database
} from '@subsquid/bigquery-store'
import {BigQuery} from '@google-cloud/bigquery'

const db = new Database({
  bq: new BigQuery(),
  dataset: 'subsquid-datasets.test_dataset',
  tables: {
    TransfersTable: new Table(
      'transfers',
      {
        from: Column(Types.String()),
        to: Column(Types.String()),
        value: Column(Types.BigNumeric(38))
      }
    )
  }
})

processor.run(db, async ctx => {
  // ...
  let from: string = ...
  let to: string = ...
  let value: bigint | number = ...
  ctx.store.TransfersTable.insert({from, to, value})
})
```

Here,

* `bq` is a `BigQuery` instance. When created without arguments like this it'll look at the `GOOGLE_APPLICATION_CREDENTIALS` environment variable for a path to a JSON with authentication details.
* `dataset` is the path to the target dataset.

<Warning>
  The dataset must be created prior to running the processor.
</Warning>

* `tables` lists the tables that will be created and populated within the dataset. For every field of the `tables` object an eponymous field of the `ctx.store` object will be created; calling `insert()` or `insertMany()` on such a field will result in data being queued for writing to the corresponding dataset table. The actual writing will be done at the end of the batch in a single transaction, ensuring dataset integrity.

Tables are made out of statically typed columns. Available types are listed on the [reference page](/en/sdk/squid-sdk/reference/store/bigquery).

## Deploying to SQD Cloud

We discourage uploading any sensitive data with squid code when [deploying](/en/cloud) to SQD Cloud. To pass your [credentials JSON](https://cloud.google.com/docs/authentication/application-default-credentials#GAC) to your squid, create a [Cloud secret](/en/cloud/resources/env-variables#secrets) variable populated with its contents:

```bash theme={"system"}
sqd secrets set GAC_JSON_FILE < creds.json
```

Then in `src/main.ts` write the contents to a file:

```ts title=src/main.ts theme={"system"}
import fs from 'fs'

fs.writeFileSync('creds.json', process.env.GAC_JSON_FILE || '')
```

Set the `GOOGLE_APPLICATION_CREDENTIALS` variable and request the secret in the [deployment manifest](/en/cloud/reference/manifest):

```yaml title="squid.yaml" theme={"system"}
deploy:
  processor:
    env:
      GAC_JSON_FILE: ${{ secrets.GAC_JSON_FILE }}
      GOOGLE_APPLICATION_CREDENTIALS: creds.json
```

## Examples

An end-to-end example geared towards local runs can be found in [this repo](https://github.com/subsquid-labs/squid-bigquery-example). Look at [this branch](https://github.com/subsquid-labs/squid-bigquery-example/tree/cloud-secrets) for an example of a squid made for deployment to SQD Cloud.

## Troubleshooting

### Transaction is aborted due to concurrent update

This means that your project has an open [session](https://cloud.google.com/bigquery/docs/sessions-intro) that is updating some of the tables used by the squid.

Most commonly, the session is left by a squid itself after an unclean termination. You have two options:

1. If you are not sure if your squid is the only app that uses sessions to access your BigQuery project, find the faulty session manually and terminate it. See [Get a list of your active sessions](https://cloud.google.com/bigquery/docs/sessions-get-ids#list_active) and [Terminate a session by ID](https://cloud.google.com/bigquery/docs/sessions-terminating#terminate_a_session_by_id).

2. **DANGEROUS** If you are absolutely certain that the squid is the only app that uses sessions to access your BigQuery project, you can terminate all the dangling sessions by running

   ```sql theme={"system"}
   FOR session in (
     SELECT
       session_id,
       MAX(creation_time) AS last_modified_time,
     FROM `region-us`.INFORMATION_SCHEMA.SESSIONS_BY_PROJECT
     WHERE
       session_id IS NOT NULL
       AND is_active
     GROUP BY session_id
     ORDER BY last_modified_time DESC
   )
   DO
     CALL BQ.ABORT_SESSION(session.session_id);
   END FOR;
   ```

   Replace `region-us` with your dataset's region in the code above.

   You can also enable `abortAllProjectSessionsOnStartup` and supply `datasetRegion` in your database config to perform this operation at startup:

   ```ts theme={"system"}
   const db = new Database({
     // ...
     abortAllProjectSessionsOnStartup: true,
     datasetRegion: 'region-us'
   })
   ```

   This method **will** cause data loss if, at the moment when the squid starts, some other app happens to be writing data anywhere in the project using the sessions mechanism.

### Error 413 (Request Entity Too Large)

Squid produced too much data per batch per table and BigQuery refused to handle it. Begin by finding out which table causes the issue (e.g. by counting `insert()` calls), then enable pagination for that table:

```ts theme={"system"}
const db = new Database({
  bq: new BigQuery(), // set GOOGLE_APPLICATION_CREDENTIALS at .env
  dataset: `${projectId}.${datasetId}`,
  tables: {
    TransfersTable: new Table(
      'transfers',
      {
        from: Column(Types.String()),
        to: Column(Types.String()),
        value: Column(Types.BigNumeric(38))
      },
      5000 // <- page size for the insert operation
    )
  },
  ...
```


# Saving to filesystems
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/persisting-data/file

Save Squid SDK indexed data to filesystem-based datasets — CSV, JSON, JSONL, and Parquet output via the @subsquid/file-store package.

# Filesystem store

<Warning>
  Make sure you understand and use the [`chunkSizeMb` setting](#overview) and the [`setForceFlush()`](#setforceflush) store method. Failure to do so may cause the processor to output an empty dataset.
</Warning>

## Overview

Squid SDK provides append-only [`Store` interface](/en/sdk/squid-sdk/resources/persisting-data/overview) implementations for saving the data generated by squids to files. The store is designed primarily for offline analytics. It supports CSV and Parquet files, persisted either locally or to a S3 storage. See the list of [Packages](#packages) below.

The core component of all file-based stores is the `Database` class from `@subsquid/file-store`. To use it, construct an instance and pass it to [`processor.run()`](/en/sdk/squid-sdk/reference/processors/architecture#processorrun). This results in `ctx.store` exposing [table writers](#table-writer-interface) that accept rows of data to be stored.

File-based stores *always* partition data sets along the "block height" dimension, even when it is not in the schema. The number of blocks per partition is variable: a new partition is written when either

* the internal buffer (size governed by the `chunkSizeMb` [`Database` constructor option](#database-options)) of the store fills up, *or*
* after any call to [batch handler](/en/sdk/squid-sdk/reference/processors/architecture#processorrun) during which there was a call to [`setForceFlush()`](#setforceflush).

Same failover guarantees as with the [Postgres-based store](/en/sdk/squid-sdk/resources/persisting-data/typeorm) are provided: the processor will roll back to the last successful state after a restart.

#### Example

Save ERC20 `Transfer` events retrieved by [EVM processor](/en/sdk/squid-sdk) in `transfers.csv` files:

```typescript theme={"system"}
import {EvmBatchProcessor} from '@subsquid/evm-processor'
import * as erc20abi from './abi/erc20'
import {Database, LocalDest} from '@subsquid/file-store'
import {Column, Table, Types} from '@subsquid/file-store-csv'

const processor = /* processor definition */

const dbOptions = {
  tables: {
    TransfersTable: new Table('transfers.csv', {
      from: Column(Types.String()),
      to: Column(Types.String()),
      value: Column(Types.Numeric())
    })
  },
  dest: new LocalDest('./data'),
  chunkSizeMb: 10
}

processor.run(new Database(dbOptions), async (ctx) => {
  for (let c of ctx.blocks) {
    for (let log of c.logs) {
      if (/* the log item is a Transfer we're interested in */) {
        let { from, to, value } =
          erc20abi.events.Transfer.decode(log)
        ctx.store.TransfersTable.write({ from, to, value })
      }
    }
  }
})
```

The resulting `./data` folder may look like this:

```bash theme={"system"}
./data/
├── 0000000000-0007688959
│   └── transfers.csv
├── 0007688960-0007861589
│   └── transfers.csv
...
├── 0016753040-0016762029
│   └── transfers.csv
└── status.txt
```

Each of the folders here contains a little over 10 MBytes of data. `status.txt` contains the height of the last indexed block and its hash.

## Packages

`@subsquid/file-store` is the core package that contains the implementation of `Database` for filesystems. At least one file format add-on must be installed alongside it:

* **CSV**: Supported via [`@subsquid/file-store-csv`](/en/sdk/squid-sdk/reference/store/file/csv).
* **Parquet**: An [advanced format](https://parquet.apache.org) that works well for larger data sets. Supported via [`@subsquid/file-store-parquet`](/en/sdk/squid-sdk/reference/store/file/parquet).

Data in either of these formats can be written to

* **A local filesystem**: Supported by `@subsquid/file-store` out of the box.
* **A bucket in an Amazon S3-compatible cloud**: Supported via [`@subsquid/file-store-s3`](/en/sdk/squid-sdk/reference/store/file/s3-dest).

## `Database` Options

Constructor of the `Database` class from `file-store` accepts a configuration object as its only argument. Its format is as follows:

```typescript theme={"system"}
DatabaseOptions {
  tables: Record<string, Table>
  dest: Dest
  chunkSizeMb?: number
  hooks?: DatabaseHooks<Dest>
}
```

Here,

* `Table` is an interface for classes that make [table writers](#table-writer-interface), objects that convert in-memory tabular data into format-specific file contents. An implementation of `Table` is available for every file format supported by `file-store`. Consult [pages about specific output formats](/en/sdk/squid-sdk/reference/store/file) to find out how to define `Table`s.
* **`tables`** is a mapping from developer-defined string handles to `Table` instances. A table writer will be created for each `Table` in this mapping. It will be exposed at `ctx.store.<tableHandle>`.
* **`dest`** is an instance of `Dest`, an interface for objects that take the properly formatted file contents and write them onto a particular filesystem. An implementation of `Dest` is available for every filesystem supported by `file-store`. For local filesystems use the `LocalDest` class from the `@subsquid/file-store` package and supply `new LocalDest(outputDirectoryName)` here. For other targets consult [documentation pages specific to your filesystem choice](/en/sdk/squid-sdk/reference/store/file).
* **`chunkSizeMb`** governs the size of the internal data buffer. A dataset partition will be written as soon as this buffer fills up, or at the end of the batch if [setForceFlush()](#setforceflush) was called.
* **`hooks`** are useful for [persisting data between batches](#hooks).

## Table Writer Interface

For each `Table` supplied via the `tables` field of the constructor argument, `Database` adds a table writer property to [`ctx.store`](/en/sdk/squid-sdk/resources/persisting-data/overview). The writer is exposed at `ctx.store.<tableHandle>`, where `<tableHandle>` is the key of the `Table` instance in the `tables` mapping. It has the following methods:

**`ctx.store.<tableHandle>.write(record: T)`**\
**`ctx.store.<tableHandle>.writeMany(records: T[])`**

Here, `T` is a `Table` implementation-specific data row type. See the [documentation pages on specific file formats](/en/sdk/squid-sdk/reference/store/file) for details.

These synchronous methods add rows of data to an in-memory buffer and perform no actual filesystem writes. Instead, the write happens automatically when the internal buffer reaches [`chunkSizeMb`](#overview) or at the end of the batch during which [`setForceFlush()`](#setforceflush) was called. The methods return the table writer instance and can be chained.

For example, with a `Database` defined like this:

```ts theme={"system"}
const db = new Database({
  tables: {
    TransfersTable: new Table(/* table options */)
  },
  // ...dest and dataset partitioning options
})
```

the following calls become available in the [batch handler](/en/sdk/squid-sdk/reference/processors/architecture#processorrun):

```ts theme={"system"}
processor.run(db, async ctx => {
  let record = // row in a format specific to Table implementation
  ctx.store.TransfersTable.write(record)
  ctx.store.TransfersTable.writeMany([record])
})
```

## `setForceFlush()`

Both of the following calls

**`ctx.setForceFlush()`**

**`ctx.setForceFlush(true)`**

set the force flush flag within the store. If the flag is still set at the end of a [batch](/en/sdk/squid-sdk/reference/processors/architecture#processorrun), a dataset partition will be written regardless of how full the data buffer currently is. This is useful e.g. in ensuring that the partition size is not much greater than a constant value when writing data at a low rate:

```ts theme={"system"}
let blockCount = 0

processor.run(db, async ctx => {
  // ...data is transformed and queued for writing
  blockCount += ctx.blocks.length
  if (blockCount >= 500_000) {
    ctx.store.setForceFlush()
  }
})
```

Unset the flag with

**`ctx.setForceFlush(false)`**

## Hooks

By default, `Database` maintains a record of the syncing progress in the `status.txt` file. When the processor with a `Database` instance starts, it calls the `onStateRead()` function that reads the highest reached block from `status.txt` on the target filesystem and returns its hash and height. If the file does not exist, the function returns -1 for height and the syncing resumes starting at the next (zeroth/genesis) block.

Syncing status record is updated every time a new partition is written to the dataset: the processor calls `onFlush()` which overwrites `status.txt` with the new highest reached block.

As a result, the integrity of the data set is guaranteed given the blockchain history up to the point recorded in `status.txt`.

The functions `onStateRead()` and `onStateUpdate()` can be overridden using the **`hooks`** constructor argument field. To do that, set that field to

```typescript theme={"system"}
DatabaseHooks<Dest> {
  onStateRead(dest: Dest): Promise<HashAndHeight | undefined>
  onStateUpdate(dest: Dest, info: HashAndHeight): Promise<void>
}
```

Parameters:

* **dest**: the `Dest` object used by `Database`. Use it to access the filesystem.
* **info**: a `{height: number, hash: string}` object.

Overriding these functions can be useful for transferring some processor state between batches reliably. A basic example of using **`hooks`** can be found [here](https://github.com/subsquid/squid-file-store/blob/fcc8a63dc2deb8c3f843aa525f806ff9cdf778fb/test/src/main.ts#L36).


# Store interface
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/persisting-data/overview

Squid SDK Store interface — the persistence API used inside batch handlers to write indexed entities into Postgres, BigQuery, or files.

`Store` is a generic interface exposed by [`DataHandlerContext`](/en/sdk/squid-sdk/reference/processors/architecture#batch-context) at `.store`. It is used in the [batch handler](/en/sdk/squid-sdk/reference/processors/architecture#processorrun) to persist data. Its concrete type is inferred from the `Database` argument of the [`run()` processor method](/en/sdk/squid-sdk/reference/processors/architecture#processorrun):

```typescript theme={"system"}
run<Store>(
  db: Database<Store>,
  handler: (ctx: DataHandlerContext<Store, F extends FieldSelection>)
    => Promise<void>
): void
```

The `Database` implementation supplied here defines the store and the way the processor [persists its status](/en/sdk/squid-sdk/faq#how-do-squids-keep-track-of-their-sync-progress). Squid SDK supports `Database` implementations for:

* [`TypeORM`-compatible databases](/en/sdk/squid-sdk/resources/persisting-data/typeorm)
* [file-based datasets](/en/sdk/squid-sdk/resources/persisting-data/file)
* [Google BigQuery](/en/sdk/squid-sdk/resources/persisting-data/bigquery)

Support for more databases and analytics storage will be added in the future.

## Custom `Database`

A custom implementation of the `Database` interface is the recommended solution for squids with multiple data sinks and/or for non-transactional or non-relational databases. In such a case, the inferred `Store` facade exposed to the handlers may provide multiple targets for persisting the data. `Database.transact` should handle potential edge-cases when writes to either of the data sinks fail.

In order to implement a custom data sink adapter, it suffices to implement the `Database` interface:

```ts theme={"system"}
export type Database<S> = FinalDatabase<S> | HotDatabase<S>

export interface FinalDatabase<S> {
    supportsHotBlocks?: false
    connect(): Promise<HashAndHeight>
    transact(info: FinalTxInfo, cb: (store: S) => Promise<void>): Promise<void>
}

export interface HotDatabase<S> {
    supportsHotBlocks: true
    connect(): Promise<HotDatabaseState>
    transact(info: FinalTxInfo, cb: (store: S) => Promise<void>): Promise<void>
    transactHot(info: HotTxInfo, cb: (store: S, block: HashAndHeight) => Promise<void>): Promise<void>
}
```

Consult [this file](https://github.com/subsquid/squid-sdk/blob/master/util/util-internal-processor-tools/src/database.ts) for details.

The interface only defines how the processor advances with the indexing and connects to the data sink. The following is left as implementation details:

* persisting the indexing status
* opening and rolling back the transaction (if applicable)

Consult these implementations for details:

* (Transactional) Postgres-based [TypeormDatabase](https://github.com/subsquid/squid-sdk/blob/master/typeorm/typeorm-store/src/database.ts)
* (Non-transactional) [File-based database](https://github.com/subsquid/squid-csv-store/blob/main/src/database.ts)


# Saving to PostgreSQL
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/persisting-data/typeorm

Save Squid SDK indexed data to PostgreSQL via the typeorm-store package — auto-generated entity models, batched upserts, and migration tooling.

`TypeormDatabase` context store provides a wrapper over the [TypeORM `EntityManager`](https://typeorm.io/entity-manager-api) optimized for batch saving. It currently supports only Postgres-compatible databases and seamlessly integrates with entity classes generated from the [schema file](/en/sdk/squid-sdk/reference/schema-file).

Check out [this section of the reference page](/en/sdk/squid-sdk/reference/store/typeorm#database-connection-parameters) to learn how to specify the database connection parameters.

## Usage

```ts theme={"system"}
import { Store, TypeormDatabase } from '@subsquid/typeorm-store'
import { FooEntity } from './model'

const dbOptions = {/* ... constructor options ... */}

processor.run(new TypeormDatabase(dbOptions), async ctx => {
  // ...  
  await ctx.store.upsert([new FooEntity({ id: '1'}), new FooEntity({ id: '2'})])
})
```

Here,

* `FooEntity` represents a TypeORM entity class. In squids, these are typically generated from [schema files](/en/sdk/squid-sdk/reference/schema-file) with [squid-typeorm-codegen](/en/sdk/squid-sdk/reference/schema-file/intro#typeorm-codegen).
* `TypeormDatabase` constructor options govern the behavior of the class at the highest level. They are described in [this section of the reference](/en/sdk/squid-sdk/reference/store/typeorm#typeormdatabase-constructor-arguments).
* `ctx.store` is of type `Store`. See the [`Store` interface](/en/sdk/squid-sdk/reference/store/typeorm#store-interface) section of the reference page for more details.

## Database migrations

The Squid SDK manages the database schema with [TypeORM-based database migrations](https://typeorm.io/migrations) by means of the `squid-typeorm-migration(1)` tool.
The tool auto-generates the schema migrations from the TypeORM entities created by [codegen](/en/sdk/squid-sdk/reference/schema-file/intro#typeorm-codegen), so that custom migration scripts are rarely needed.

Here are some useful commands:

```bash theme={"system"}
npx squid-typeorm-migration apply # Apply the DB migrations
```

```bash theme={"system"}
npx squid-typeorm-migration generate # Generate a DB migration matching the TypeORM entities
```

```bash theme={"system"}
rm -r db/migrations # Clean the migrations folder
```

Inspect the full list of available options with

```bash theme={"system"}
npx squid-typeorm-migration --help
```

To generate or update the migrations after a schema change, follow the steps below.

### Updating after schema changes

In most cases the simplest way to update the schema is to drop the database and regenerate the migrations from scratch.

**1. Update schema.graphql**

**2. Regenerate the TypeORM entity classes**

```bash theme={"system"}
# generate entites code from the schema file
npx squid-typeorm-codegen
```

**3. Recreate the database**

```bash theme={"system"}
# drop the database
docker compose down
```

```bash theme={"system"}
# start a blank database
docker compose up -d
```

Note that without dropping the database the next step will generate a migration only for the schema difference.

**4. Build the squid code**

```bash theme={"system"}
npm run build
```

**5. Recreate the database migration**

```bash theme={"system"}
rm -r db/migrations
```

```bash theme={"system"}
npx squid-typeorm-migration generate
```

**6. Apply the database migration**

```bash theme={"system"}
npx squid-typeorm-migration apply
```

### Updating a deployed squid schema

In some rare cases it is possible to update the schema without dropping the database and restarting the squid from a blank state. The most important case is adding an index to an entity field. More complex changes are usually not feasible.

Updating a running squid requires that you add an incremental migration.

**1. Ensure that your local database is running and has the same schema as the database of your Cloud squid**

In most situations re-creating the database container and applying existing migrations should be enough:

```bash theme={"system"}
docker compose down
docker compose up -d
npx squid-typeorm-migration apply
```

**2. Update schema.graphql**

For example, [add an index](/en/sdk/squid-sdk/reference/schema-file/indexes-and-constraints)

**3. Regenerate the model classes and build the code**

```bash theme={"system"}
npx squid-typeorm-codegen
npm run build
```

**4. Add a new database migration**

```bash theme={"system"}
npx squid-typeorm-migration generate
```

This will create a new file in `db/migrations`. You may want to examine it before proceeding to the next step.

**5. Update the squid in Cloud**

If the squid is deployed to SQD Cloud, [update the deployed version](/en/sdk/squid-sdk/squid-cli/deploy).

If you're self-hosting it, update your remote codebase and run

```bash theme={"system"}
npx squid-typeorm-migration apply
```

### SQD Cloud deployment

By default, the TypeORM migrations are automatically applied by Cloud with the command `npx squid-typeorm-migration apply` before the squid services are started. For custom behavior, one can override the migration script using the optional `migrate:` section of [squid.yaml](/en/cloud/reference/manifest#deploy).

<Info>
  To force Cloud to reset the database and start with a blank state after a schema change, use the `--hard-reset` flag of [sqd deploy](/en/sdk/squid-sdk/squid-cli/deploy).
</Info>


# Self-hosting
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/self-hosting

Self-host Squid SDK indexers locally or on-premises with Docker — single-image setup for processor and GraphQL server with Postgres backing storage.

<Tip>
  Many tips from our [best practices guide](/en/cloud/resources/best-practices) apply to self-hosted squids, too. Check it out if you intend to use your squid in production.
</Tip>

To deploy a squid locally or on-premises, use the following Dockerfile template to build a single image for both `api` and `processor` services:

```dockerfile title="Dockerfile" theme={"system"}
FROM node:20-alpine AS node
FROM node AS node-with-gyp
FROM node-with-gyp AS builder
WORKDIR /squid
ADD package.json .
ADD package-lock.json .
# remove if needed
ADD assets assets
# remove if needed
ADD db db
# remove if needed
ADD schema.graphql .
RUN npm ci
ADD tsconfig.json .
ADD src src
RUN npm run build
FROM node-with-gyp AS deps
WORKDIR /squid
ADD package.json .
ADD package-lock.json .
RUN npm ci --production
FROM node AS squid
WORKDIR /squid
COPY --from=deps /squid/package.json .
COPY --from=deps /squid/package-lock.json .
COPY --from=deps /squid/node_modules node_modules
COPY --from=builder /squid/lib lib
# remove if no assets folder
COPY --from=builder /squid/assets assets
# remove if no db folder
COPY --from=builder /squid/db db
# remove if no schema.graphql is in the root
COPY --from=builder /squid/schema.graphql schema.graphql
# remove if no commands.json is in the root
ADD commands.json .
RUN echo -e "loglevel=silent\\nupdate-notifier=false" > /squid/.npmrc
RUN npm i -g @subsquid/commands && mv $(which squid-commands) /usr/local/bin/sqd
ENV PROCESSOR_PROMETHEUS_PORT 3000
```

Then build an image with

```bash theme={"system"}
docker buildx build . -t my-squid
```

or with

```bash theme={"system"}
docker build . -t my-squid
```

if you're using an older Docker version.

## Sample compose file

Once can the run the squid services with the freshly built image. Here is a sample `docker-compose` file:

```yaml theme={"system"}
services:
  db:
    image: postgres:15
    environment:
      POSTGRES_DB: squid
      POSTGRES_PASSWORD: postgres
    healthcheck:
      test: ["CMD-SHELL", "pg_isready", "-d", "squid"]
      interval: 5s
      timeout: 5s
      retries: 5
    # Uncomment for logging all SQL statements
    # command: ["postgres", "-c", "log_statement=all"]
  api:
    image: my-squid
    environment:
      - DB_NAME=squid
      - DB_PORT=5432
      - DB_HOST=db
      - DB_PASS=postgres
      - GQL_PORT=4350
    ports:
      # GraphQL endpoint at port 4350
      - "4350:4350"
    command: ["sqd", "serve:prod"]
    depends_on:
      db:
        condition: service_healthy
  processor:
    image: my-squid
    environment:
      - DB_NAME=squid
      - DB_PORT=5432
      - DB_HOST=db
      - DB_PASS=postgres
      # any other variables that your squid processor may be using
    ports:
      # prometheus metrics exposed at port 3000
      - "3000:3000"
    command: ["sqd", "process:prod"]
    depends_on:
      db:
        condition: service_healthy
```

Note that `sqd serve:prod` and `sqd process:prod` commands are defined in the [`commands.json` file](/en/sdk/squid-sdk/squid-cli/commands-json).


# Serving GraphQL
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/serving-graphql

Choose a GraphQL server for your Squid SDK indexer — OpenReader, Hasura, and PostGraphile compared, with deployment notes for SQD Cloud and self-host.

It is common (although not required) for squids to serve GraphQL APIs. Historically, the most common way to do that was to [persist the squid data to PostgreSQL](/en/sdk/squid-sdk/resources/persisting-data/typeorm), then attach [OpenReader](#openreader) to it. Although this is still supported, we encourage using [PostGraphile](#postgraphile) or [Hasura](#hasura) in new PostgreSQD-based projects. See [OpenReader's limitations](/en/sdk/squid-sdk/reference/openreader-server/overview#limitations) if you're curious about our motivation.

## PostGraphile

[PostGraphile](https://www.graphile.org/postgraphile/) is an open-source tool that builds powerful, extensible and performant GraphQL APIs from PostgreSQL schemas. Its pros include:

* aggregations;
* reliable support for subscriptions;
* capability for deep API customization;
* organization of API customization code into plugins.

The recommended way of integrating PostGraphile into squid projects is by making a dedicated entry point at `src/api.ts`. A complete example squid implementing this approach is available in [this repository](https://github.com/subsquid-labs/squid-postgraphile-example/).

With this entry point in place, we [create a `sqd` command](https://github.com/subsquid-labs/squid-postgraphile-example/blob/f1fd1691eb59da2c9d57c475a71d0ed44cfed891/commands.json#L58) for running PostGraphile with [`commands.json`](/en/sdk/squid-sdk/squid-cli/commands-json), then use it in the [`deploy.api` entry](https://github.com/subsquid-labs/squid-postgraphile-example/blob/f1fd1691eb59da2c9d57c475a71d0ed44cfed891/squid.yaml#L15) of [Squid manifest](/en/cloud/reference/manifest). Although none of this is required, this makes it easier to run the squid both locally (with [`sqd run`](/en/sdk/squid-sdk/squid-cli/run)) and in the [Cloud](/en/cloud).

As per usual with PostGraphile installations, you can freely extend it with plugins, including your own. Here is an [example plugin for serving the `_squidStatus` queries](https://github.com/subsquid-labs/squid-postgraphile-example/blob/f1fd1691eb59da2c9d57c475a71d0ed44cfed891/src/api.ts#L11) from the standard Squid SDK GraphQL server schema. A plugin for making PostGraphile API fully compatible with [old APIs](/en/sdk/squid-sdk/reference/openreader-server/api) served by OpenReader will be made available soon.

## Hasura

[Hasura](https://hasura.io) is a powerful open-source GraphQL engine. You can use it to:

* expose multiple data sources of different kinds (various databases, APIs etc) via a single API;
* reliably serve subscriptions;
* perform aggregations;
* deeply customize your CRUD API.

You can integrate Hasura with your squid in two ways:

1. **Use Hasura to gather data from multiple sources, including your squid.**

   For this scenario we recommend separating your Hasura instance from your squid, which should consist of just one service, [the processor](/en/sdk/squid-sdk/reference/processors/architecture), plus the database. Supply your database credentials to Hasura, then configure it to produce the desired API.

   If you run your squid in our [Cloud](/en/cloud) you can find database credentials in [the app](https://app.subsquid.io/squids):2. **Run a dedicated Hasura instance for serving the data just from your squid.**

   A complete example implementing this approach is available in [this repository](https://github.com/subsquid-labs/squid-hasura-example). Here's how it works:

   * Locally, Hasura runs in a [Docker container](https://github.com/subsquid-labs/squid-hasura-example/blob/70bb6d703dc90c1bb00b47f3fef7f388ab54e565/docker-compose.yml#L14C1-L28C20). In the Cloud it is managed via the [Hasura addon](/en/cloud/reference/hasura).
   * Hasura metadata is shared among all squid instances by means of the [Hasura configuration tool](/en/sdk/squid-sdk/resources/tools/hasura-configuration). The tool can automatically create an initial configuration based on your [TypeORM models](/en/sdk/squid-sdk/reference/schema-file/intro#typeorm-codegen), then persist any changes you might make with the web GUI and metadata exports.
   * Admin authentication secret is set via the `HASURA_GRAPHQL_ADMIN_SECRET`. The variable is set in `.env` locally and from a [secret](/en/cloud/resources/env-variables#secrets) in Cloud deployments.

   See the [configuration tool page](/en/sdk/squid-sdk/resources/tools/hasura-configuration) and the [repo readme](https://github.com/subsquid-labs/squid-hasura-example#readme) for more details.

## OpenReader

[OpenReader](/en/sdk/squid-sdk/reference/openreader-server) is a GraphQL server developed by the SQD team. Although still supported, it's not recommeded for new PostgreSQL-powered projects due to its [limitations](/en/sdk/squid-sdk/reference/openreader-server/overview#limitations), especially for APIs implementing GraphQL subscriptions.

The server uses the [schema file](/en/sdk/squid-sdk/reference/schema-file) to produce its [core API](/en/sdk/squid-sdk/reference/openreader-server/api) that can be extended with [custom resolvers](/en/sdk/squid-sdk/reference/openreader-server/configuration/custom-resolvers). Extra features include [DoS protection](/en/sdk/squid-sdk/reference/openreader-server/configuration/dos-protection) and [caching](/en/sdk/squid-sdk/reference/openreader-server/configuration/caching).


# Substrate data sourcing
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/substrate/data-sourcing-miniguide

Pick the right data sources for Substrate Squid SDK indexers — events vs calls, finality, runtime upgrades, and types bundles for clean indexing.

## How do I know which events and calls I need on Substrate?

This part depends on the runtime business logic of the chain. The primary and the most reliable source of information is thus the Rust sources for the pallets used by the chain.

For a quick lookup of the documentation and the data format, it is often useful to check Runtime section of Subscan (e.g. [Statemine](https://assethub-kusama.subscan.io/runtime)). One can see the deployed pallets and drill down to events and extrinsics from there. One can also choose the spec version on the drop down.


# Frontier EVM support
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/substrate/frontier-evm

Index Frontier-EVM contracts running on Substrate with Squid SDK — use the @subsquid/frontier package to decode EVM events on Astar, Moonbeam, etc.

<Tip>
  Method documentation is also available inline and can be accessed via suggestions in most IDEs.
</Tip>

<Info>
  Check out the [Caveats](#caveats) section of this page to avoid common issues.
</Info>

This section describes additional options available for Substrate chains with the Frontier EVM pallet like Astar. On ArrowSquid, use the [frontier-evm template](https://github.com/subsquid-labs/squid-frontier-evm-template) as the starting point.

The overall approach is to use the `SubstrateBatchProcessor` methods [`addEvmLog()`](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#addevmlog) and [`addEthereumTransaction()`](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#addethereumtransaction) to request the required EVM data. The data can then be converted into a standard EVM format using the utils from [the frontier package](/en/sdk/squid-sdk/reference/frontier) and decoded with utility classes [generated by `squid-evm-typegen`](/en/sdk/squid-sdk/resources/tools/typegen/generation).

Contract state can also be accessed via wrapped RPC queries using the [classes provided by the `squid-evm-typegen` tool](/en/sdk/squid-sdk/resources/tools/typegen/state-queries).

## Examples

#### Subscribe to event logs

```typescript theme={"system"}
const processor = new SubstrateBatchProcessor()
  .setGateway('https://v2.archive.subsquid.io/network/astar-substrate')
  .setRpcEndpoint('https://astar-rpc.dwellir.com')
  .addEvmLog({
    address: [
      '0xb654611f84a8dc429ba3cb4fda9fad236c505a1a',
      '0x6a2d262d56735dba19dd70682b39f6be9a931d98'
    ],
    topic0: [erc721.events.Transfer.topic],
    extrinsic: true
  })
  .setFields({
    event: {
      phase: true
    },
    extrinsics: {
      hash: true
    }
  })
```

#### Request all EVM calls to two contracts

```ts theme={"system"}
processor.addEthereumTransaction({
  to: [
    '0x6a2d262d56735dba19dd70682b39f6be9a931d98'
    '0x3795c36e7d12a8c252a20c5a7b455f7c57b60283'
  ]
})
```

#### Request all `transfer(address,uint256)` EVM calls on the network:

```ts theme={"system"}
processor.addEthereumTransaction({sighash: ['0xa9059cbb']})
```

#### Parse events and transactions

```typescript theme={"system"}
const processor = new SubstrateBatchProcessor()
  .setGateway('https://v2.archive.subsquid.io/network/astar-substrate')
  .setRpcEndpoint('https://astar-rpc.dwellir.com')
  .addEthereumTransaction({})
  .addEvmLog({})

processor.run(new TypeormDatabase(), async ctx => {
  for (const block of ctx.blocks) {
    for (const event of block.events) {
      if (event.name === 'EVM.Log') {
        // no need to supply any extra data to determine
        // the runtime version: event has all necessary references
        const {address, data, topics} = getEvmLog(event)

        // process evm log data
      }
    }
    for (const call of block.calls) {
      if (call.name==='Ethereum.transact') {
        const txn = getTransaction(call)
        // process evm txn data
      }   
    }
  }
})
```

#### Access contract state

```ts theme={"system"}
// ...
const CONTRACT_ADDRESS= "0xb654611f84a8dc429ba3cb4fda9fad236c505a1a"

processor.run(new TypeormDatabase(), async ctx => {
  for (const block of ctx.blocks) { 
    for (const event of block.events) {
      if (event.name === "EVM.Log") {
        const contract = new erc721.Contract(ctx, block, CONTRACT_ADDRESS);
        // query the contract state
        const uri = await contract.tokenURI(1137)
      }
    }
  }
})
```

## Factory contracts

It some cases the set of contracts to be indexed by the squid is not known in advance. For example, a DEX contract typically creates a new contract for each trading pair added, and each such trading contract is of interest.

While the set of handler subscriptions is static and defined at the processor creation, one can leverage the wildcard subscriptions and filter the contracts of interest in runtime. This pattern is [described extensively](/en/sdk/squid-sdk/resources/evm/factory-contracts) in EVM documentation, but it can be used with EVM methods of `SubstrateBatchProcessor` as well. A (somewhat outdated) example is available in [this archive repo](https://github.com/subsquid/beamswap-squid/blob/master/src/processor.ts).

## Caveats

* If your use case does not require any Substrate-specific data (e.g. extrinsic hashes), use [`EvmBatchProcessor`](/en/sdk/squid-sdk/reference/processors/evm-batch) instead. EVM-only SQD Network gateways are [available](/en/data/evm) for all major EVM-on-Substrate chains.

* Passing `[]` as a set of parameter values *selects no data*. Pass `undefined` for a wildcard selection:
  ```typescript theme={"system"}
  .addEvmLog({address: []}) // selects no events
  .addEvmLog({}) // selects all events
  ```

* If contract address(es) supplied to the processor configuration methods are stored in any wide-scope variables, it is recommended to convert them to flat lower case. This precaution is necessary because same variable(s) are often reused in the [batch handler](/en/sdk/squid-sdk/reference/processors/architecture#processorrun) for item filtration, and all contract addresses in the items are **always** in flat lower case.


# Gear support
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/substrate/gear

Index Gear programs running on Substrate with Squid SDK — additional pallets, message routing, and program-specific data sources for the Gear runtime.

<Info>
  SQD Network has gateways for two networks that use Gear Protocol: **Vara** and **Vara Testnet**. Here are their endopint URLs:

  ```
  https://v2.archive.subsquid.io/network/vara
  ```

  ```
  https://v2.archive.subsquid.io/network/vara-testnet
  ```
</Info>

Indexing [Gear](https://gear-tech.io/) programs is supported with [`addGearMessageQueued()`](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#addgearmessagequeued) and [`addGearUserMessageSent()`](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#addgearusermessagesent) specialized data requests. These subscribe to the events [`Gear.MessageQueued`](https://wiki.gear-tech.io/docs/api/events/#messagequeued) and [`Gear.UserMessageSent`](https://wiki.gear-tech.io/docs/api/events/#usermessagesent) emitted by a specified Gear program.

The processor can also subscribe to any other event with [`addEvent()`](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#events) and filter by program ID in the batch handler, if so necessary.

An example of a squid indexing a Gear program (an NFT contract) can be found [here](https://github.com/subsquid/squid-sdk/tree/master/test/gear-nft).


# ink! contracts support
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/substrate/ink

Index ink! WASM smart contracts on Substrate with Squid SDK — typegen, contract event decoding, and storage state queries for ink-based dApps.

This section describes additional options available for indexing [ink!-based WASM contracts](https://use.ink), supported by chains with a `Contracts` pallet. At the moment of writing, AlephZero, Shibuya (Astar testnet), Shiden (Kusama parachain) and Astar (Polkadot parachain) are the most popular chains for deploying ink! contracts.

[Generate an ink! indexing squid automatically](/en/sdk/squid-sdk/resources/tools/squid-gen), follow the [WASM squid tutorial](/en/sdk/squid-sdk/tutorials/ink) for a step-by-step instruction or check out the [squid-wasm-template](https://github.com/subsquid-labs/squid-wasm-template) reference project.

## Processor configuration

Request events by the contract address as described on the [`SubstrateBatchProcessor` reference page](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests#addcontractscontractemitted), e.g.

```ts theme={"system"}
import * as ss58 from '@subsquid/ss58'
import {toHex} from '@subsquid/util-internal-hex'

const ADDRESS = toHex(ss58.decode('XnrLUQucQvzp5kaaWLG9Q3LbZw5DPwpGn69B5YcywSWVr5w').bytes)

const processor = new SubstrateBatchProcessor()
  .setGateway('https://v2.archive.subsquid.io/network/shibuya-substrate')
  .setRpcEndpoint('https://shibuya.public.blastapi.io')
  .addContractsContractEmitted({
    contractAddress: [ADDRESS],
    extrinsic: true
  })
  .setFields({
    event: {
      phase: true
    }
  })
```

Generate and use the facade classes for decoding ink! smart contract data as described in the [typegens reference](/en/sdk/squid-sdk/resources/tools/typegen). You can also make [direct contract state queries](/en/sdk/squid-sdk/resources/tools/typegen/state-queries) using the `Contract` class generated by `squid-ink-typegen`.


# Substrate types bundles
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/substrate/types-bundle-miniguide

Source Substrate runtime metadata via types bundles in Squid SDK — handle runtime upgrades, custom pallet types, and deprecated metadata format gracefully.

### Where do I get a types bundle for my chain?

Types bundle is only needed for pre-Metadata v14 blocks and only if SQD does not offer a built-in support for the chain in question.

Most chains publish their type bundles as an npm package (for example: [Edgeware](https://www.npmjs.com/package/@edgeware/node-types)). One of the best places to check for the latest version is the [polkadot-js/app](https://github.com/polkadot-js/apps/tree/master/packages/apps-config/src/api/spec) and [polkadot-js/api](https://github.com/polkadot-js/api/tree/master/packages/types-known/src/spec) repositories.

<Note>
  The type bundle format for typegen is slightly different from `OverrideBundleDefinition` of `polkadot.js`. The structure is as follows, all the fields are optional.
</Note>

```javascript theme={"system"}
{
  types: {}, // top-level type definitions, as `.types` option of `ApiPromise`
  typesAlias: {}, // top-level type aliases, as `.typesAlias` option of `ApiPromise`
  versions: [ // spec version specific overrides, same as `OverrideBundleDefinition.types` of `polkadot.js`
    {
       minmax: [0, 1010] // spec range
       types: {}, // type overrides for the spec range
       typesAlias: {}, // type alias overrides for the spec range
    }
  ]
}
```


# Hasura configuration tool
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/tools/hasura-configuration

Configure Hasura for a Squid SDK indexer with the hasura-configuration tool — auto-generate metadata from your schema and apply it to a Hasura instance.

<Info>
  For info on migrating to `@subsquid/hasura-configuration>=2.0.0` see [this page](/en/sdk/squid-sdk/resources/migrate/migrate-to-hasura-configuration-tool-v2).
</Info>

[`@subsquid/hasura-configuration`](https://www.npmjs.com/package/@subsquid/hasura-configuration) is a tool for managing Hasura configuration in [PostgreSQL-powered squids](/en/sdk/squid-sdk/resources/persisting-data/typeorm). Install it with

```bash theme={"system"}
npm i @subsquid/hasura-configuration
```

Make sure that the following environment variables are set:

* `HASURA_GRAPHQL_ENDPOINT` for Hasura URL (defaults to `http://localhost:8080`).
* `HASURA_GRAPHQL_ADMIN_SECRET` for admin access (only required to use `squid-hasura-configuration apply`).
* If your Hasura instance(s) use a role other than `public` to serve the anonymous part of your API, also set `HASURA_GRAPHQL_UNAUTHORIZED_ROLE`.

## Generating the initial configuration

The tool uses your squid's [TypeORM models](/en/sdk/squid-sdk/reference/schema-file/intro#typeorm-codegen) as input when generating the initial configuration. Make sure they are up to date.

<Accordion title="Here's how">
  1. Finalize any edits to [`schema.graphql`](/en/sdk/squid-sdk/reference/schema-file)

  2. Update the TypeScript code of your models with
     ```bash theme={"system"}
     npx squid-typeorm-codegen
     ```

  3. Compile your models with
     ```bash theme={"system"}
     npm run build
     ```

  4. Regenerate your [DB migrations](/en/sdk/squid-sdk/resources/persisting-data/typeorm#database-migrations) with a clean database to make sure they match your updated schema.
     ```bash theme={"system"}
     docker compose down
     docker compose up -d
     rm -r db
     npx squid-typeorm-migration generate
     ```

  You can turn off your database after doing that - Hasura configuration tool does not use it to make its initial configuration
</Accordion>

When done, run

```bash theme={"system"}
npx squid-hasura-configuration regenerate
```

The generated configuration will be available at `hasura_metadata.json`. It enables:

* tracking all tables that correspond to [schema entities](/en/sdk/squid-sdk/reference/schema-file/entities);
* `SELECT` permissions for the `public` (or `$HASURA_GRAPHQL_UNAUTHORIZED_ROLE` if it is defined) role for all columns in these tables;
* tracking all [entity relationships](/en/sdk/squid-sdk/reference/schema-file/entity-relations).

## Applying the configuration

Make sure your database is up, your Hasura instance is connected to it and the schema is up to date. If necessary, apply the migrations:

```bash theme={"system"}
npx squid-typeorm-migration apply
```

When done, you can apply the generated config with

```bash theme={"system"}
npx squid-hasura-configuration apply
```

or import it using the *Settings > Metadata Actions > Import metadata* function of the web GUI.

## Persisting configuration changes

<Warning>
  Regenerating `hasura_metadata.json` removes any modifications you might have made via metadata exporting. So, it is advisable that you finalize your schema before you begin any manual API fine-tuning.
</Warning>

When running a squid with a dedicated Hasura instance you will notice that squid resetting operations (`docker compose down; docker compose up -d` and `sqd deploy -r`) restore your Hasura API to its non-configured state. As you develop your API further you may want to persist your changes. `squid-hasura-configuration` helps with that by being compatible with the *Settings > Metadata Actions > Import/Export metadata* functions of the web GUI.

Any extra configuration you may make via the web GUI or [Hasura metadata API](https://hasura.io/docs/2.0/api-reference/metadata-api/index) can be persisted by exporting the metadata to `hasura_metadata.json` via the *Export metadata* function, then applying it to blank Hasura instances with

```bash theme={"system"}
npx squid-hasura-configuration apply
```

## Example

See [this repo](https://github.com/subsquid-labs/squid-hasura-example).


# TypeORM migration generation
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/tools/migrations-gen

Generate Postgres migrations from your Squid SDK schema with sqd codegen — apply schema changes incrementally without dropping your indexed data.

The [`typeorm-migration`](https://github.com/subsquid/squid-sdk/tree/master/typeorm/typeorm-migration) tool is used to generate, apply and revert database migrations. It follows the conventions below.

* The migrations are generated in the `db/migrations` folder.
* The database connection is inferred from the `DB_XXX` environment variables.
* All entities should be exported from `lib/model` commonjs module, i.e. the entity classes must be compiled from TypeScript.

Here are some useful commands:

```bash theme={"system"}
npx squid-typeorm-migration apply # apply pending migrations
```

```bash theme={"system"}
npx squid-typeorm-migration generate # generate the migration for the schema defined in schema.graphql
```

```bash theme={"system"}
rm -r db/migrations # clean the db/migrations folder
```


# TypeORM model generation
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/tools/model-gen

Generate TypeORM model classes from a Squid SDK schema.graphql file with the model-gen tool — used by the typeorm-store and consumed by the processor.

TypeORM entities can be automatically generated from the [schema file](/en/sdk/squid-sdk/reference/schema-file) defined in `schema.graphql`.
The tool is called [`squid-typeorm-codegen(1)`](https://github.com/subsquid/squid-sdk/tree/master/typeorm/typeorm-codegen) and has no additonal options.

Install with

```sh theme={"system"}
npm i @subsquid/typeorm-codegen --save-dev
```

Invoke with

```sh theme={"system"}
npx squid-typeorm-codegen
```


# Squid generation tools
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/tools/squid-gen

Squid SDK code generation tools — squid-gen and abi2squid generate complete indexer projects from contract ABIs for any EVM chain.

<Warning>
  As of 2025-01-01 we no longer maintain the old `squid-gen` tool.
</Warning>

SQD provides [tools](https://github.com/subsquid/squid-gen) for generating ready-to-use squids that index events and function calls of smart contracts. EVM/Solidity and WASM/ink! smart contracts are supported. The tools can be configured to make squids that save data to a [PostgreSQL database](/en/sdk/squid-sdk/resources/persisting-data/typeorm) or to a [file-based dataset](/en/sdk/squid-sdk/resources/persisting-data/file). All that is required is NodeJS, [Squid CLI](/en/sdk/squid-sdk/squid-cli/installation) and, if your squid will be using a database, Docker.

Squid generation procedure is very similar for both contract types. Here are the steps:

1. Create a new blank squid with `sqd init` using a suitable template:
   ```bash theme={"system"}
   # for EVM/Solidity contracts
   sqd init my-squid -t abi
   # OR
   # for WASM/ink! contracts
   sqd init my-squid -t https://github.com/subsquid-labs/squid-ink-abi-template
   ```
   Enter the squid folder and install the dependencies:
   ```bash theme={"system"}
   cd my-squid
   npm ci
   ```

2. Write the [configuration](#configuration) of the future squid to `squidgen.yaml`. Retrieve any necessary contract ABIs and store them at `./abi`.

   Alternatively, skip to the next step and specify the configuration via CLI. **Note:** some features will not be available.

3. Generate and build the squid code:
   ```bash theme={"system"}
   npx squid-gen config squidgen.yaml
   ```
   ```bash theme={"system"}
   npm run build
   ```
   If you chose to configure the tool via CLI instead, do so now. Here's an example:
   ```bash theme={"system"}
   npx squid-gen-abi \
     --address 0x2E645469f354BB4F5c8a05B3b30A929361cf77eC \
     --archive https://v2.archive.subsquid.io/network/ethereum-mainnet \
     --event NewGravatar \
     --event UpdatedGravatar \
     --function '*' \
     --from 6000000
   ```
   See `npx squid-gen-abi --help` for all options.

4. Prepare your squid for launching. If it is using a database, start a PostgreSQL container, then regenerate and apply migrations:
   ```bash theme={"system"}
   docker compose up -d
   ```
   ```bash theme={"system"}
   npx squid-typeorm-migration generate
   ```
   ```bash theme={"system"}
   npx squid-typeorm-migration apply
   ```
   If it is storing its data to a dataset, [strip the project folder of database-related facilities](#strip-the-squid-folder-for-file-store) that are no longer needed.

5. Test the complete squid by running it locally. Start a [processor](/en/sdk/squid-sdk/overview) with
   ```bash theme={"system"}
   node -r dotenv/config lib/main.js
   ```
   If your squid will be serving GraphQL also run `npx squid-graphql-server` in a separate terminal. Make sure that the squid saves the requested data to its target:
   * if it is serving GraphQL, visit the local [GraphiQL playground](http://localhost:4350/graphql);
   * for PostgreSQL-based squids you can also connect to the database with `PGPASSWORD=postgres psql -U postgres -p 23798 -h localhost squid` and take a look at the contents;
   * if it is storing data to a file-based dataset, wait for the first filesystem sync then verify that all the expected files are present and contain the expected data. If your squid produces data at a low rate, you may have to tweak the [`chunkSizeMb` setting](/en/sdk/squid-sdk/resources/persisting-data/file#overview) and/or add a [`ctx.store.setForceFlush()`](/en/sdk/squid-sdk/resources/persisting-data/file#setforceflush) call to manually write dataset chunks at appropriate intervals.

At this point your squid is ready. You can run it on your own infrastructure or [deploy it to SQD Cloud](/en/cloud).

## Configuration

A valid config for the `squid-gen config` is a YAML file with the following sections:

* **archive** is an endpoint URL of a [SQD Network](/en/network/overview) gateway. Find an appropriate gateway at the [Supported networks](/en/data/evm) page or with [`sqd gateways`](/en/sdk/squid-sdk/squid-cli/gateways).

* **target** section describes how the scraped data should be stored. Set
  ```yaml theme={"system"}
  target:
    type: postgres
  ```
  to use a PostgreSQL database that can be presented to users as a GraphQL API or used as-is. Another option is to [store the data to a file-based dataset](#file-store-targets).

* **contracts** is a list of contracts to be indexed. Define the following fields for each contract:
  * **name**
  * **address**
  * **range** (optional): block range to be indexed. An object with `from` and `to` properties, each of which can be omitted. Defaults to indexing the whole chain.
  * **abi** (optional on EVM): path to the contract JSON ABI. If omitted for an EVM contract, the tool will attempt to fetch the ABI by address from the Etherscan API or a compatible alternative set by the `etherscanApi` root option.
  * **proxy** (EVM-only): when indexing a [proxy contract](https://eips.ethereum.org/EIPS/eip-1967) for events or calls defined in the implementation, set this option to its address and the `address` option to the address of the implementation contract. That way the tool will retrieve the ABI of the implementation and use it to index the output of the proxy.
  * **events** (optional): a list of events to be indexed or a boolean value. Set to `true` to index all events defined in the ABI. Defaults to `false`, meaning that no events are to be indexed.
  * **functions** (EVM-only, optional): a list of functions the calls of which are to be indexed or a boolean value. Set to `true` to index calls of all functions defined in the ABI. Defaults to `false`, meaning that no function calls are to be indexed.

* **etherscanApi** (EVM-only, optional): Etherscan API-compatible endpoint to fetch contract ABI by a known address. Default: [https://api.etherscan.io/](https://api.etherscan.io/).

## `file-store` targets

Currently the only [file-based data target type](/en/sdk/squid-sdk/resources/persisting-data/file) supported by `squid-gen` packages is [`parquet`](/en/sdk/squid-sdk/reference/store/file/parquet). When used, it requires that the `path` field is also set alongside `type`. A `path` can be a local path or an URL pointing to a folder within a bucket on an S3-compatible cloud service.

Support for `file-store` is in alpha stage. Known caveats:

* If a S3 URL is used, then the S3 region, endpoint and user credentials will be [taken from the default environment variables](/en/sdk/squid-sdk/reference/store/file/s3-dest). Fill your `.env` file and/or set your [SQD Cloud secrets](/en/cloud/resources/env-variables) accordingly.

* Unlike their PostgreSQL-powered equivalents, the squids that use `file-store` may not write their data often. You may have to configure the `chunkSizeMb` parameter of the `Database` class and/or call [`ctx.store.setForceFlush()`](/en/sdk/squid-sdk/resources/persisting-data/file#setforceflush) when appropriate to strike an acceptable balance between the lag of the indexed data and the number of files in the resulting dataset. See [Filesystem store overview](/en/sdk/squid-sdk/resources/persisting-data/file) for details.

* For `parquet` targets, the [`Decimal(38)`](/en/sdk/squid-sdk/reference/store/file/parquet#columns) column type is used by the code generator to represent `uint256`. This is done for compatibility reasons: very few tools seem to support reading wider decimals from Parquet files. If you're getting a lot of errors containing `value ... does not fit into Decimal(38, 0)`, consider replacing the `Decimal(38)` column type with `Decimal(78)` or `String()` at `src/table.ts`.

* At the moment, squids generated with file-based data targets contain a lot of facilities for managing the database and have to be [stripped](#strip-the-squid-folder-for-file-store) of them before use.

## Strip the squid folder for `file-store`

Steps to convert a squid made with a database-enabled template for use with `file-store`:

1. Remove unneeded files and packages.

```bash theme={"system"}
rm docker-compose.yml
npm uninstall @subsquid/graphql-server @subsquid/typeorm-migration @subsquid/typeorm-store @subsquid/util-internal-json pg typeorm @subsquid/typeorm-codegen
```

2. Replace `commands.json` with the one from the [file-store-parquet-example repo](https://github.com/subsquid-labs/file-store-parquet-example).

```bash theme={"system"}
curl -o commands.json https://raw.githubusercontent.com/subsquid-labs/file-store-parquet-example/main/commands.json
```

3. In `squid.yaml`, remove the `deploy.addons` section and replace the `deploy.api` section with

```bash theme={"system"}
  api:
    cmd: [ "sleep", "3600" ]
```

4. Install any required `file-store` packages.

```bash theme={"system"}
# if target.type was `parquet`
npm install @subsquid/file-store-parquet
# if target.path was an S3 URL
npm install @subsquid/file-store-s3
```

## Configuration examples

### EVM/Solidity

* Index `LiquidationCall` events of the [AAVE V2 Lending Pool contract](https://etherscan.io/address/0x7d2768dE32b0b80b7a3454c06BdAc94A69DDc7A9), starting from block 11362579 when it was deployed. Save the results to PostgreSQL. Use the ABI located at `./abi/aave-v2-pool.json`.
  ```yaml theme={"system"}
  archive: eth-mainnet
  target:
    type: postgres
  contracts:
    - name: aave-v2-pool
      address: "0x7d2768dE32b0b80b7a3454c06BdAc94A69DDc7A9"
      abi: ./abi/aave-v2-pool.json
      range:
        from: 11362579
      events:
        - LiquidationCall
  ```

* Index events and function calls by the [DPX contract](https://arbiscan.io/address/0x6c2c06790b3e3e3c38e12ee22f8183b37a13ee55) (a proxy) on Arbitrum, based on the ABI of the [implementation contract](https://arbiscan.io/address/0x3f770ac673856f105b586bb393d122721265ad46) retrieved from [Arbiscan](https://arbiscan.io) API. Save the results to Parquet files at './data'.
  ```yaml theme={"system"}
  archive: arbitrum
  target:
    type: parquet
    path: ./data
  contracts:
    - name: dpx
      address: "0x3f770Ac673856F105b586bb393d122721265aD46"
      proxy: "0x6C2C06790b3E3E3c38e12Ee22F8183b37a13EE55"
      events: true
      functions: true
  etherscanApi: https://api.arbiscan.io/
  ```
  **Note:** this example is known to run into the integer length issue described in the [`file-store` targets](#file-store-targets) section. One way to make it work is to widen all `Decimal` column types from 38 to 78 symbols:
  ```bash theme={"system"}
  sed -i -e 's/38/78/g' src/table.ts
  ```

* Index all events and function calls of the [Positions NFT](https://etherscan.io/address/0xc36442b4a4522e871399cd717abdd847ab11fe88) and [Factory](https://etherscan.io/address/0x1f98431c8ad98523631ae4a59f267346ea31f984) contracts of Uniswap, send the results to the `uniswap-data` folder of the `subsquid-testing-data` bucket.
  ```yaml theme={"system"}
  archive: eth-mainnet
  target:
    type: parquet
    path: s3://subsquid-testing-bucket/uniswap-data
  contracts:
    - name: positions
      address: "0xC36442b4a4522E871399CD717aBDD847Ab11FE88"
      events: true
      functions: true
    - name: factory
      address: "0x1F98431c8aD98523631AE4a59f267346ea31F984"
      events: true
      functions: true
  ```
  **Notes:**
  * This example is also susceptible to the integer length issue and will drop at least two events if used as-is, without widening the column types.
  * The generated squid requires some variables to be set to connect to S3. Here's an example of what `.env` may look like:
    ```bash theme={"system"}
    S3_REGION=us-east-1
    S3_ENDPOINT=https://s3.filebase.com
    S3_ACCESS_KEY_ID=myAccessKeyId
    S3_SECRET_ACCESS_KEY=mySecretAccessKey
    ```

### WASM/ink!

* Index `Transfer` events emitted by an ERC20 contract on Shibuya, save results to PostgreSQL. Do not forget to use the [ink-abi template](https://github.com/subsquid-labs/squid-ink-abi-template)!
  ```yaml theme={"system"}
  archive: shibuya
  target:
    type: postgres
  contracts:
    - name: testToken
      abi: "./abi/erc20.json"
      address: "0x5207202c27b646ceeb294ce516d4334edafbd771f869215cb070ba51dd7e2c72"
      events:
        - Transfer
  ```
  **Note:** you can get the ABI from the `squid-gen` repository:
  ```bash theme={"system"}
  curl -o abi/erc20.json https://raw.githubusercontent.com/subsquid/squid-gen/master/tests/ink-erc20/abi/erc20.json
  ```


# Data decoding
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/tools/typegen/decoding

Use generated typegen modules to decode EVM and Substrate data in Squid SDK — type-safe accessors for events, calls, and storage queries.

<Tabs>
  <Tab title="squid-evm-typegen">
    Decoding events:

    ```ts theme={"system"}
    import * as depositsAbi from './abi/deposits'

    processor.run(new TypeormDatabase(), async (ctx) => {
      for (let c of ctx.blocks) {
        for (let log of c.logs) {
          if (log.address === CONTRACT_ADDRESS &&
              log.topics[0] == depositsAbi.events.Deposit.topic) {

            // type-safe decoding of the Deposit event data
            const amt = depositsAbi.events.Deposit.decode(log).wad
          }
        }
      }
    })
    ```

    Similar to `events`, transaction access is provided by the `functions` object for each contract method defined in the ABI.
  </Tab>

  <Tab title="squid-substrate-typegen">
    To decode events and calls, first determine the appropriate runtime version with `.is()`, then decode them with `.decode()`:

    ```typescript theme={"system"}
    import {events, calls} from './types'

    processor.run(new TypeormDatabase(), async ctx => {
      for (let block of ctx.blocks) {
         for (let event of block.events) {
          if (event.name == events.balances.transfer.name) {
            let rec: {from: Bytes, to: Bytes, amount: bigint}
            if (events.balances.transfer.v1020.is(event)) {
              let [from, to, amount, fee] =
                events.balances.transfer.v1020.decode(event)
              rec = {from, to, amount}
            }
            // ... decode all runtime versions similarly
            // with events.balances.transfer.${ver}.is/.decode
          }
        }
        for (let call of block.calls) {
          if (call.name == calls.balances.forceTransfer.name) {
            let rec: {source: Bytes, dest: Bytes, value: bigint} | undefined
            if (calls.balances.forceTransfer.v1020.is(call)) {
              let res =
                calls.balances.forceTransfer.v1020.decode(call)
              assert(res.source.__kind === 'AccountId')
              assert(res.dest.__kind === 'AccountId')
              rec = {
                source: res.source.value,
                dest: res.dest.value,
                value: res.value
              }
            }
            // ... decode all runtime versions similarly
            // with calls.balances.forceTransfer.${ver}.is/.decode
        }
      }
    })
    ```
  </Tab>

  <Tab title="squid-ink-typegen">
    Here is an example of a utility module generated by the ink! typegen:

    ```typescript title="src/abi/erc20.ts" theme={"system"}
    const _abi = new Abi(metadata)

    export function decodeEvent(hex: string): Event {
      return _abi.decodeEvent(hex)
    }

    export function decodeMessage(hex: string): Message {
      return _abi.decodeMessage(hex)
    }

    export function decodeConstructor(hex: string): Constructor {
      return _abi.decodeConstructor(hex)
    }
    ```

    The usage in a batch handler is straightforward:

    ```ts theme={"system"}
    processor.run(new TypeormDatabase(), async ctx => {
      for (let block of ctx.blocks) {
        for (let event of block.events) {
          if (event.name==='Contracts.ContractEmitted' &&
              event.args.contract===CONTRACT_ADDRESS) {

            let event = erc20.decodeEvent(event.args.data)
            if (event.__kind==='Transfer') {
              // event is of type `Event_Transfer`
            }
          }
        }
      }
    })
    ```
  </Tab>
</Tabs>


# Generating utility modules
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/tools/typegen/generation

Generate task-specific utility modules with Squid SDK typegen — produce typed clients for EVM ABIs and Substrate metadata for type-safe indexing.

<Tabs>
  <Tab title="squid-evm-typegen">
    The `squid-evm-typegen(1)` tool generates TypeScript facades for EVM transactions, logs and `eth_call` queries.

    The generated facades are assumed to be used by squids indexing EVM data.

    The tool takes a JSON ABIs as an input. Those can be specified in three ways:

    1. as a plain JSON file(s):

       ```bash theme={"system"}
       npx squid-evm-typegen src/abi abi/erc20.json
       ```

       To include all files in `./abi` and add an interface for the Multicall contract, run

       ```bash theme={"system"}
       npx squid-evm-typegen ./src/abi ./abi/*.json --multicall
       ```

       You can get JSON ABIs for explorer (Etherscan, Bscscan etc) verified contract by visiting the contract page, going to the "Contract" tab and scrolling down to the "Contract ABI" section. *Do not use the "Export ABI" function!* Copy the contents to the clipboard and paste them to a new JSON file.

    2. **(requires an Etherscan API key)** as a contract address. One can pass multiple addresses at once.

       ```bash theme={"system"}
       npx squid-evm-typegen --etherscan-api-key <your_key> src/abi 0xBB9bc244D798123fDe783fCc1C72d3Bb8C189413
       ```

    <Info>
      Please check if your contract is a proxy when using this method. If it is,
      consult [this page](/en/sdk/squid-sdk/resources/evm/proxy-contracts) for guidance.
    </Info>

    3. as an arbitrary URL:

       ```bash theme={"system"}
       npx squid-evm-typegen src/abi https://example.com/erc721.json
       ```

    In all cases typegen will use `basename` of the ABI as the root name for the generated files. You can change the basename of generated files using the fragment `(#)` suffix.

    ```bash theme={"system"}
    squid-evm-typegen src/abi 0xBB9bc244D798123fDe783fCc1C72d3Bb8C189413#my-contract-name
    ```

    **Arguments:**

    |              |                                                                                         |
    | ------------ | --------------------------------------------------------------------------------------- |
    | `output-dir` | output directory for generated definitions                                              |
    | `abi`        | A contract address, an URL or a local path to the ABI file. Accepts multiple contracts. |

    **Options:**

    |                         |                                                                                                                                                                                                                                  |
    | ----------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
    | `--multicall`           | generate a facade for the MakerDAO multicall contract. May significantly improve the performance of contract state calls by [batching RPC requests](/en/sdk/squid-sdk/resources/tools/typegen/state-queries#batch-state-queries) |
    | `--etherscan-api <url>` | etherscan API to fetch contract ABI by a known address. By default, `https://api.etherscan.io/`                                                                                                                                  |
    | `--clean`               | delete output directory before the run                                                                                                                                                                                           |
    | `-h, --help`            | display help for command                                                                                                                                                                                                         |

    <Warning>
      The generated modules depend on `@subsquid/evm-abi`. Please add it to your
      `package.json` as a peer dependency if it's not there already: `bash npm i
              @subsquid/evm-abi `
    </Warning>

    ## Usage

    The generated utility modules have three intended uses:

    1. Constants: EVM log topic0 values and sighashes for transactions. Example:

       ```ts theme={"system"}
       // generated by evm-typegen
       import * as weth from "./abi/weth";

       const CONTRACT_ADDRESS =
         "0xC02aaA39b223FE8D0A0e5C4F27eAD9083C756Cc2".toLowerCase();

       const processor = new EvmBatchProcessor()
         .setGateway("https://v2.archive.subsquid.io/network/ethereum-mainnet")
         .addLog({
           address: [CONTRACT_ADDRESS],
           topic0: [weth.events.Deposit.topic, weth.events.Withdrawal.topic],
         });
       ```

    2. [Decoding of EVM logs and transactions](/en/sdk/squid-sdk/resources/tools/typegen/decoding)

    3. [Direct chain state queries](/en/sdk/squid-sdk/resources/tools/typegen/state-queries), including [queries to multicall](/en/sdk/squid-sdk/resources/tools/typegen/state-queries#batch-state-queries).
  </Tab>

  <Tab title="squid-substrate-typegen">
    The substrate typegen tool is a part of Squid SDK. It generates TypeScript wrappers for interfacing Substrate events and calls.

    Usage:

    ```bash theme={"system"}
    npx squid-substrate-typegen typegen.json
    ```

    If necessary, multiple config files can be supplied:

    ```bash theme={"system"}
    npx squid-substrate-typegen typegen0.json typegen1.json ...
    ```

    The structure of the `typegen.json` config file is best illustrated with an example:

    ```json theme={"system"}
    {
      "outDir": "src/types",
      "specVersions": "https://v2.archive.subsquid.io/metadata/kusama",
      "pallets": {
        // add one such section for each pallet
        "Balances": {
          "events": [
            // list of events to generate wrappers for
            "Transfer"
          ],
          "calls": [
            // list of calls to generate wrappers for
            "transfer_allow_death"
          ],
          "storage": ["Account"],
          "constants": ["ExistentialDeposit"]
        }
      }
    }
    ```

    The `specVersions` field is either

    * a metadata service endpoint URL, like
      ```
      https://v2.archive.subsquid.io/metadata/{network}
      ```
      or
    * a path to a [`jsonl`](https://jsonlines.org) file generated by [`substrate-metadata-explorer(1)`](https://github.com/subsquid/squid-sdk/tree/master/substrate/substrate-metadata-explorer).

    To generate all items defined by a given pallet, set any of the `events`, `calls`, `storage` or `constants` fields to `true`, e.g.

    ```json theme={"system"}
    {
      "outDir": "src/types",
      "specVersions": "kusamaVersions.jsonl",
      "pallets": {
        "Balances": {
          // generate wrappers for all Balances pallet constants
          "constants": true
        }
      }
    }
    ```

    <Info>
      In the rare cases when the typegen needs a types bundle, supply it alongside metadata:

      ```json theme={"system"}
      {
        "outDir": "src/types",
        "specVersions": "westendVersions.jsonl",
        "typesBundle": "westendTypes.json",
        ...
      }
      ```
    </Info>

    ## TypeScript wrappers

    Wrappers generated by the typegen command can be found in the specified `outDir` (`src/types` by convention). Assuming that this folder is imported as `types` (e.g. with `import * as types from './types'`), you'll be able to find the wrappers at:

    * for events: `types.events.${palletName}.${eventName}`
    * for calls: `types.calls.${palletName}.${callName}`
    * for storage items: `types.storage.${palletName}.${storageItemName}`
    * for constants: `types.constants.${palletName}.${constantName}`

    All identifiers (pallet name, call name etc) are lowerCamelCased. E.g. the constant `Balances.ExistentialDeposit` becomes `types.events.balances.existentialDeposit` and the call `Balances.set_balance` becomes `types.calls.setBalance`.

    ## Usage

    1. Item name constants (e.g. `events.balances.transfer.name`).

    2. Runtime versioning-aware [decoding](/en/sdk/squid-sdk/resources/tools/typegen/decoding).

    3. [Chain storage queries](/en/sdk/squid-sdk/resources/tools/typegen/state-queries)

    4. Runtime constants:
       ```typescript theme={"system"}
       import {constants} from './types'
       // ...
       processor.run(new TypeormDatabase(), async ctx => {
         for (let block of ctx.blocks) {
           if (constants.balances.existentialDeposit.v1020.is(block.header)) {
             let c = new constants.balances.existentialDeposit.v1020.get(block.header)
             ctx.log.info(`Balances.ExistentialDeposit (runtime version V1020): ${c}`)
           }
         }
       })
       ```
  </Tab>

  <Tab title="squid-ink-typegen">
    Use [`squid-ink-typegen`](https://github.com/subsquid/squid-sdk/tree/master/substrate/ink-typegen) to generate facade classes for decoding ink! smart contract data from JSON ABI metadata.

    Usage:

    ```bash theme={"system"}
    npx squid-ink-typegen --abi abi/erc20.json --output src/abi/erc20.ts
    ```

    The generated source code exposes the [decoding methods](/en/sdk/squid-sdk/resources/tools/typegen/decoding), some useful types and a class for performing [wrapped RPC queries](/en/sdk/squid-sdk/resources/tools/typegen/state-queries), using `@subsquid/ink-abi` under the hood
  </Tab>
</Tabs>


# Direct RPC queries
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/tools/typegen/state-queries

Query chain state directly via wrapped RPC calls in Squid SDK — typegen-generated state-call wrappers for fetching contract storage and balances.

<Tabs>
  <Tab title="squid-evm-typegen">
    In order to make on-chain calls, one must set up a JSON-RPC endpoint using [setRpcEndpoint()](/en/sdk/squid-sdk/reference/processors/evm-batch/general#set-rpc-endpoint). We recommend using a private endpoint from e.g. [BlastAPI](https://blastapi.io/) or the SQD Cloud's [RPC addon](/en/cloud/resources/rpc-proxy), and set it via an environment variable:

    ```ts theme={"system"}
    //...
      .setRpcEndpoint(process.env.RPC_ETH_HTTP)
    //...
    ```

    You can define the `RPC_ETH_HTTP` in three ways:

    * for local runs, simply update the local `.env` file;
    * for squids deployed to Cloud define it as a [secret](/en/cloud/resources/env-variables) on your Cloud account;
    * if you are using the [RPC addon](/en/cloud/resources/rpc-proxy), leave it to the Cloud to define it for you.

    ## `Contract` class

    The EVM contract state is accessed using the `Contract` class generated by `squid-evm-typegen`. It takes a handler context and a contract address as constructor arguments. The state is always accessed at the context block height unless explicitly defined in the constructor.

    For example, assume that we index an ERC721 contract. Typescript ABI module generated with `squid-evm-typegen` will contain the following class:

    ```typescript theme={"system"}
    export class Contract extends ContractBase {
      //...
      balanceOf(owner: string): Promise<ethers.BigNumber> {
        return this.call('balanceOf', [owner])
      }
      //...
    }
    ```

    Now suppose we want to query our contract from the batch handler. To create a `Contract` pass the context object and the current `Block` to its constructor, then query the contract state at that block:

    ```ts theme={"system"}
    // ...
    const CONTRACT_ADDRESS = '0xb654611f84a8dc429ba3cb4fda9fad236c505a1a'

    processor.run(new TypeormDatabase(), async ctx => {
      for (const block of ctx.blocks) { 
        const contract = new abi.Contract(ctx, block.header, CONTRACT_ADDRESS)
        // query the contract state at the current block
        const uri = await contract.balanceOf('0xd8da6bf26964af9d7eed9e03e53415d37aa96045')
        // ...
      }
    })
    ```

    ## Batch state queries

    The [MakerDAO Multicall contract](https://github.com/makerdao/multicall) was designed to batch multiple state queries into a single contract call. In the context of indexing, it normally significantly improves the indexing speed since JSON RPC calls are typically the bottleneck.

    Multicall contracts are deployed in many EVM chains, see the [contract repo](https://github.com/makerdao/multicall) for addresses. You can use any of them with a `multicall` Typescript module that is generated when running `squid-evm-typegen` with `--multicall` option. The module exports a `Multicall` class with this method:

    ```ts theme={"system"}
    tryAggregate<Args extends any[], R>(
      func: Func<Args, {}, R>,
      calls: [address: string, args: Args][],
      paging?: number
    ): Promise<MulticallResult<R>[]>
    ```

    The arguments are as follows:

    * `func`: the contract function to be called
    * `calls`: an array of tuples `[contractAddress: string, args]`. Each specified contract will be called with the specified arguments.
    * `paging` an (optional) argument for the maximal number of calls to be batched into a single JSON PRC request. Note that large page sizes may cause timeouts.

    A typical usage is as follows:

    ```ts title="src/main.ts" theme={"system"}
    // generated by evm-typegen
    import { functions } from './abi/mycontract'
    import { Multicall } from './abi/multicall'

    const MY_CONTRACT = '0xac5c7493036de60e63eb81c5e9a440b42f47ebf5'
    const MULTICALL_CONTRACT = '0x5ba1e12693dc8f9c48aad8770482f4739beed696'

    processor.run(new TypeormDatabase(), async (ctx) => {
      for (let c of ctx.blocks) {
        // some logic
      }
      const lastBlock = ctx.blocks[ctx.blocks.length - 1]
      const multicall = new Multicall(
        ctx,
        lastBlock,
        MULTICALL_CONTRACT
      )
      // call MY_CONTRACT.myContractMethod('foo') and MY_CONTRACT.myContractMethod('bar')
      const args = ['foo', 'bar']
      const results = await multicall.tryAggregate(
        functions.myContractMethod,
        args.map(a => [MY_CONTRACT, a]) as [string, any[]],
        100
      )

      results.forEach((res, i) => {
        if (res.success) {
          ctx.log.info(`Result for argument ${args[i]} is ${res.value}`)
        }
      })
    })
    ```
  </Tab>

  <Tab title="squid-substrate-typegen">
    It is sometimes impossible to extract the required data with only event and call data without querying the runtime state.
    The context exposes a lightweight gRPC client to the chain node accessible via `ctx._chain`.
    It exposes low-level methods for accessing the storage. However, the recommended way to query the storage is with type-safe wrappers generated with `squid-substrate-typegen`.

    Substrate typegen tool exposes storage access wrappers at `src/types/storage.ts`. The wrappers follow the [general naming pattern](/en/sdk/squid-sdk/resources/tools/typegen/generation#typescript-wrappers) used by Substrate typegen:

    ```
    storage.${palletName}.${storageName}
    ```

    with all identifiers lowerCamelCased. Each wrapper exposes a generated `get()` query method and, if available, methods for multi-key queries, listing keys, key-value pairs and retrieving the default value.

    Note that the generated getters **always query historical blockchain state at the height derived from their `block` argument**.

    Here is an example of one such wrapper:

    ```typescript title="src/types/balances/storage.ts" theme={"system"}
    import {sts, Block, Bytes, Option, Result, StorageType} from '../support'
    import * as v1050 from '../v1050'
    import * as v9420 from '../v9420'

    export const account = {
      v1050: new StorageType('Balances.Account', 'Default', [v1050.AccountId], v1050.AccountData) as AccountV1050,
      v9420: new StorageType('Balances.Account', 'Default', [v9420.AccountId32], v9420.AccountData) as AccountV9420,
    }

    export interface AccountV1050  {
      getDefault(block: Block): v1050.AccountData
      get(block: Block, key: v1050.AccountId): Promise<v1050.AccountData | undefined>
      getMany(block: Block, keys: v1050.AccountId[]): Promise<v1050.AccountData | undefined[]>
    }

    export interface AccountV9420  {
      getDefault(block: Block): v9420.AccountData
      get(block: Block, key: v9420.AccountId32): Promise<v9420.AccountData | undefined>
      getMany(block: Block, keys: v9420.AccountId32[]): Promise<v9420.AccountData | undefined[]>
      getKeys(block: Block): Promise<v9420.AccountId32[]>
      getKeys(block: Block, key: v9420.AccountId32): Promise<v9420.AccountId32[]>
      getKeysPaged(pageSize: number, block: Block): AsyncIterable<v9420.AccountId32[]>
      getKeysPaged(pageSize: number, block: Block, key: v9420.AccountId32): AsyncIterable<v9420.AccountId32[]>
      getPairs(block: Block): Promise<[k: v9420.AccountId32, v: v9420.AccountData | undefined][]>
      getPairs(block: Block, key: v9420.AccountId32): Promise<[k: v9420.AccountId32, v: v9420.AccountData | undefined][]>
      getPairsPaged(pageSize: number, block: Block): AsyncIterable<[k: v9420.AccountId32, v: v9420.AccountData | undefined][]>
      getPairsPaged(pageSize: number, block: Block, key: v9420.AccountId32): AsyncIterable<[k: v9420.AccountId32, v: v9420.AccountData | undefined][]>
    }
    ```

    The generated access interface provides methods for accessing:

    * the default storage value with `getDefault(block)`
    * a single storage value with `get(block, key)`
    * multiple values in a batch call with `getMany(block, keys[])`
    * all storage keys with `getKeys(block)`
    * all keys with a given prefix with `getKeys(block, keyPrefix)` (only if the storage keys are decodable)
    * paginated keys via `getKeysPaged(pageSize, block)` and `getKeysPaged(pageSize, block, keyPrefix)`
    * key-value pairs via `getPairs(block)` and `getPairs(block, keyPrefix)`
    * paginated key-value pairs via `getPairsPaged(pageSize, block)` and `getPairsPaged(pageSize, block, keyPrefix)`

    ### Example

    ```typescript title="src/main.ts" theme={"system"}
    import {storage} from './types'

    processor.run(new TypeormDatabase(), async ctx => {
      let aliceAddress = ss58.decode('5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY').bytes
      for (const blockData of ctx.blocks) {
        if (storage.balances.account.v1050.is(blockData.header)) {
          let aliceBalance = (await storage.balances.account.v1050.get(blockData.header, aliceAddress))?.free
          ctx.log.info(`Alice free account balance at block ${blockData.header.height}: ${aliceBalance}`)
        }
      }
    })
    ```
  </Tab>

  <Tab title="squid-ink-typegen">
    The generated `Contract` class provides facades for all state calls that don't mutate the contract state. The info about the state mutability is taken from the contract metadata.

    ```ts theme={"system"}
    // Generated code:
    export class Contract {
        // optional blockHash indicates the block at which the state is queried
        constructor(private ctx: ChainContext, private address: string, private blockHash?: string) { }

        total_supply(): Promise<Balance> {
            return this.stateCall('0xdb6375a8', [])
        }

        balance_of(owner: Uint8Array): Promise<bigint> {
            return this.stateCall('0x6568382f', [owner])
        }
    }
    ```

    ### Example

    ```ts theme={"system"}
    processor.run(new TypeormDatabase(), async ctx => {
      for (let block of ctx.blocks) {
        for (let event of block.events) {
           // query the balance at the current block
           const contract = new Contract(ctx, CONTRACT_ADDRESS, block.header.hash)
           let aliceAddress = ss58.decode('5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY').bytes
           let aliceBalance = await contract.balance_of(aliceAddress)
        }
      }
    })
    ```
  </Tab>
</Tabs>


# RPC ingestion and reorgs
Source: https://docs.sqd.dev/en/sdk/squid-sdk/resources/unfinalized-blocks

How Squid SDK handles unfinalized blocks and reorgs — RPC ingestion, finality assumptions, and the rollback model for chain consensus changes.

Starting with the ArrowSquid release, the processor can ingest data either from a [SQD Network](/en/network) gateway or directly from an RPC endpoint. If both a network gateway and a RPC endpoint are provided, the processor will use network data until it reaches the highest block available there, then index the few remaining blocks using the RPC endpoint. This allows squids to combine low sync times with near real-time chain data access. It is, however, possible to use just the RPC endpoint (e.g. for [local development](/en/sdk/squid-sdk/tutorials/evm-local)). On [EVM](/en/sdk/squid-sdk/reference/processors/evm-batch) it is also possible to use just the gateway (e.g. for non-realtime analytics).

RPC ingestion can create a heavy load on node endpoints. The load is typically short in duration when a SQD Network gateway is used, and the total number of requests is low. However, the request rate may be sufficient to trigger HTTP 429 responses. Use private endpoints and rate limit your requests with the `rateLimit` option ([EVM](/en/sdk/squid-sdk/reference/processors/evm-batch/general#set-rpc-endpoint)/[Substrate](/en/sdk/squid-sdk/reference/processors/substrate-batch/general#set-rpc-endpoint)).

## Indexing unfinalized blocks

When ingesting from RPC, Squid SDK can index blocks before they are [finalized](https://info.etherscan.com/epoch-in-ethereum/), enabling real-time use cases. If a blockhain reorganization happens, [processor](/en/sdk/squid-sdk/reference/processors/architecture) will roll back any changes to the database made due to orphaned blocks, then re-run its batch handler on consensus blocks.

To take advantage of this feature, a squid processor must

1. have a [node RPC endpoint](/en/sdk/squid-sdk/reference/processors/evm-batch/general#set-rpc-endpoint) as one of its data sources and
2. use a [`Database`](/en/sdk/squid-sdk/resources/persisting-data/overview) with unfinalized blocks support (e.g. [`TypeormDatabase`](/en/sdk/squid-sdk/resources/persisting-data/typeorm)) in its [`processor.run()`](/en/sdk/squid-sdk/reference/processors/architecture#processorrun) call, and
3. not disable RPC ingestion explicitly with [`setRpcDataIngestionSettings({ disabled: true })`](/en/sdk/squid-sdk/reference/processors/evm-batch/general#set-rpc-data-ingestion-settings).

With this, the processor will consider some blocks to be "hot":

<Tabs>
  <Tab title="EVM">
    All blocks with fewer confirmations than the number set by the [`setFinalityConfirmations()`](/en/sdk/squid-sdk/reference/processors/evm-batch/general#set-finality-confirmation) setting
  </Tab>

  <Tab title="Substrate">
    All blocks above the latest finalized block provided by the [`chain_getFinalizedHead()`](https://polkadot.js.org/docs/substrate/rpc/#getfinalizedhead-blockhash) RPC method
  </Tab>
</Tabs>

The processor will periodically (interval setting for [EVM](/en/sdk/squid-sdk/reference/processors/evm-batch/general#set-rpc-data-ingestion-settings), [Substrate](/en/sdk/squid-sdk/reference/processors/substrate-batch/general#set-rpc-data-ingestion-settings)) poll the RPC endpoint for changes in consensus. When the consensus changes, it will re-run the [batch handler](/en/sdk/squid-sdk/reference/processors/architecture#processorrun) with the new consensus data and ask the `Database` to adjust its state. The `Database` then must roll back the changes made due to orphaned blocks and apply the new changes. With this, the state of the `Database` reflects the current blockchain consensus at all times.


# sqd auth
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/auth

Log in to SQD Cloud with the sqd auth CLI command — exchange your account credentials for an API token to authenticate sqd deploy and other commands.

Log in to the Cloud

* [`sqd auth`](#sqd-auth-1)

## `sqd auth`

Log in to the Cloud

```
USAGE
  $ sqd auth -k <value> [--interactive]

FLAGS
  -k, --key=<value>       (required) Cloud auth key. Log in to https://app.subsquid.io to create or update your key.
      --[no-]interactive  Disable interactive mode
```

*See code: [src/commands/auth.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/auth.ts)*


# sqd autocomplete
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/autocomplete

Display sqd CLI shell autocomplete installation instructions — supported shells (bash, zsh, fish, PowerShell) and how to enable command completion.

Display autocomplete installation instructions.


# commands.json
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/commands-json

Define custom sqd subcommands and aliases via commands.json in your Squid SDK project — extend the workflow with reusable scripts for build and deploy.

The `sqd` tool automatically discovers and loads any extra commands defined in the `commands.json` file. Here is a sample file demonstrating the available features:

```json theme={"system"}
{ // comments are ok
  "$schema": "https://subsquid.io/schemas/commands.json",
  "commands": {
    "clean": {
      "description": "delete all build artifacts",
      "cmd": ["rm", "-rf", "lib"]
    },
    "build": {
      "description": "build the project",
      "deps": ["clean"], // commands to execute before
      "cmd": ["tsc"]
    },
    "typegen": {
      "hidden": true, // Don't show in the overview listing
      "workdir": "abi", // change working dir
      "command": [
        "squid-evm-typegen", // node_modules/.bin is in the PATH
        "../src/abi",
        {"glob": "*.json"} // cross-platform glob expansion
      ],
      "env": { // additional environment variables
        "DEBUG": "*"
      }
    }
  }
}
```

This functionality is managed by the [`@subsquid/commands`](https://github.com/subsquid/squid-sdk/tree/master/util/commands) package.

All [squid templates](/en/sdk/squid-sdk/how-to-start/squid-development#templates) include such a file with a predefined set of useful shortcuts. See [Cheatsheet](/en/sdk/squid-sdk/how-to-start/cli-cheatsheet).


# sqd deploy
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/deploy

Deploy new or update an existing squid deployment in the Cloud. Squid name and also optionally slot and/or tag are taken from the provided deployment manifest.

Deploy new or update an existing squid deployment in the Cloud. Squid name and also optionally slot and/or tag are taken from the provided deployment manifest.

* [`sqd deploy SOURCE`](#sqd-deploy-source)

## `sqd deploy SOURCE`

Deploy new or update an existing squid in the Cloud

```
USAGE
  $ sqd deploy SOURCE [--interactive]
    [-r [<org>/]<name>(@<slot>|:<tag>) | -o <code> | -n <name> | [-s <slot>] | [-t <tag>]]
    [-m <manifest_path>] [--hard-reset] [--stream-logs] [--add-tag <value>]
    [--allow-update] [--allow-tag-reassign] [--allow-manifest-override]

ARGUMENTS
  SOURCE  [default: .] Squid source. Could be:
          - a relative or absolute path to a local folder (e.g. ".")
          - a URL to a .tar.gz archive
          - a github URL to a git repo with a branch or commit tag

FLAGS
  -m, --manifest=<manifest_path>  [default: squid.yaml] Specify the relative local path
                                  to a squid manifest file in the squid working directory
      --add-tag=<value>           Add a tag to the deployed squid
      --allow-manifest-override   Allow overriding the manifest during deployment
      --allow-tag-reassign        Allow reassigning an existing tag
      --allow-update              Allow updating an existing squid
      --hard-reset                Perform a hard reset before deploying. This will drop
                                  and re-create all squid resources, including the
                                  database, causing a short API downtime
      --[no-]interactive          Disable interactive mode
      --[no-]stream-logs          Attach and stream squid logs after the deployment

SQUID FLAGS
  -n, --name=<name>                               Name of the squid
  -r, --reference=[<org>/]<name>(@<slot>|:<tag>)  Fully qualified reference of the squid.
                                                  It can include the organization, name,
                                                  slot, or tag
  -s, --slot=<slot>                               Slot of the squid
  -t, --tag=<tag>                                 Tag of the squid

ORG FLAGS
  -o, --org=<code>  Code of the organization

DESCRIPTION
  Deploy new or update an existing squid in the Cloud

EXAMPLES

  // Create a new squid with name provided in the manifest file
  $ sqd deploy .

  // Create a new squid deployment and override it's name to "my-squid-override"
  $ sqd deploy . -n my-squid-override

  // Update the "my-squid" squid with slot "asmzf5"
  $ sqd deploy . -n my-squid -s asmzf5

  // Use a manifest file located in ./path-to-the-squid/squid.prod.yaml
  $ sqd deploy ./path-to-the-squid -m squid.prod.yaml

  // Full paths are also fine
  $ sqd deploy /Users/dev/path-to-the-squid -m /Users/dev/path-to-the-squid/squid.prod.yaml
```

*See code: [src/commands/deploy.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/deploy.ts)*


# sqd gateways
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/gateways

Explore SQD Network gateways and data sources for a squid with the sqd gateways CLI command — list endpoints, statuses, and chains supported.

Explore data sources for a squid

* [`sqd gateways list`](#sqd-gateways-list)

## `sqd gateways list`

List available gateways

```
USAGE
  $ sqd gateways list [--interactive] [-t <evm|substrate>] [-n <regex>] [-c <number>]

FLAGS
  -c, --chain=<number>        Filter by chain ID or SS58 prefix
  -n, --name=<regex>          Filter by network name
  -t, --type=<evm|substrate>  Filter by network type
      --[no-]interactive      Disable interactive mode

ALIASES
  $ sqd gateways ls
```

*See code: [src/commands/gateways/ls.ts](https://github.com/subsquid/squid-cli/blob/master/src/commands/gateways/ls.ts)*


# sqd init
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/init

Initialize a new Squid SDK project from a template or GitHub repo with sqd init — choose EVM, Substrate, Solana, Tron, or Fuel starters and scaffold files.

Setup a new squid project from a template or github repo

* [`sqd init NAME`](#sqd-init-name)

## `sqd init NAME`

Setup a new squid project from a template or github repo

```
USAGE
  $ sqd init NAME [--interactive] [-t <value>] [-d <value>] [-r]

ARGUMENTS
  NAME  The squid name. It must contain only alphanumeric or dash ("-") symbols and must not start with "-".

FLAGS
  -d, --dir=<value>
      The target location for the squid. If omitted, a new folder NAME is created.

  -r, --remove
      Clean up the target directory if it exists

  -t, --template=<value>
      A template for the squid. Accepts:
      - a github repository URL containing a valid squid.yaml manifest in the root folder
      or one of the pre-defined aliases:
      - evm  A minimal squid template for indexing EVM data.
      - abi  A template to auto-generate a squid indexing events and txs from a contract ABI
      - multichain  A template for indexing data from multiple chains
      - gravatar  A sample EVM squid indexing the Gravatar smart contract on Ethereum.
      - substrate  A template squid for indexing Substrate-based chains.
      - ink  A template for indexing Ink! smart contracts
      - ink-abi  A template to auto-generate a squid from an ink! contract ABI
      - frontier-evm  A template for indexing Frontier EVM chains, like Moonbeam and Astar.

  --[no-]interactive
      Disable interactive mode
```

*See code: [src/commands/init.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/init.ts)*


# Installation
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/installation

Install the sqd CLI in your local environment — npm install -g, standalone binaries for macOS/Linux/Windows, and shell autocomplete configuration.

Squid CLI is a command line tool for

* scaffolding new squids from templates
* running SDK tools and scripts defined in `commands.json` in a cross-platform way
* managing squid deployments in [SQD Cloud](/en/cloud) (former Aquarium)

The CLI is distributed as a [`npm` package](https://www.npmjs.com/package/@subsquid/cli).

To install Squid CLI, follow the steps below.

<Steps>
  <Step title="0. Install and setup Squid CLI">
    First, install the latest version of Squid CLI as a global `npm` package:

    ```bash theme={"system"}
    npm i -g @subsquid/cli@latest
    ```

    Check the version:

    ```bash theme={"system"}
    sqd --version
    ```

    Make sure the output looks like `@subsquid/cli@<version>`.

    <Info>
      The next steps are **optional** for building and running squids. A key is required to enable the CLI commands managing the [SQD Cloud](/en/cloud) deployments.
    </Info>
  </Step>

  <Step title="1. Obtain a SQD Cloud deployment key">
    Sign in to [Cloud](https://app.subsquid.io/), and obtain (or refresh) the deployment key page by clicking at the profile picture > "Deployment key":

    <Frame>
      <img alt="SQD Cloud deployment key page" />
    </Frame>
  </Step>

  <Step title="2. Authenticate Squid CLI">
    Open a terminal window and run

    ```bash theme={"system"}
    sqd auth -k <DEPLOYMENT_KEY>
    ```
  </Step>

  <Step title="3. Explore with `--help`">
    Use `sqd --help` to get a list of the available command and `sqd <command> --help` to get help on the available options for a specific command, e.g.

    ```bash theme={"system"}
    sqd deploy --help
    ```
  </Step>
</Steps>


# sqd list
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/list

List squids deployed to SQD Cloud with the sqd ls command — running, stopped, and archived deployments per organization with their slots and tags.

List squids deployed to the Cloud

* [`sqd list`](#sqd-list-1)

## `sqd list`

List squids deployed to the Cloud

```
USAGE
  $ sqd list [--interactive] [--truncate]
    [-r [<org>/]<name>(@<slot>|:<tag>) | -o <code> | -n <name> | [-s <slot>] | [-t <tag>]]

FLAGS
  --[no-]interactive  Disable interactive mode
  --[no-]truncate     Truncate data in columns: false by default

SQUID FLAGS
  -n, --name=<name>                               Name of the squid
  -r, --reference=[<org>/]<name>(@<slot>|:<tag>)  Fully qualified reference
                                                  of the squid. It can include
                                                  the organization, name, slot,
                                                  or tag
  -s, --slot=<slot>                               Slot of the squid
  -t, --tag=<tag>                                 Tag of the squid

ORG FLAGS
  -o, --org=<code>  Code of the organization

ALIASES
  $ sqd ls
```

*See code: [src/commands/ls.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/ls.ts)*


# sqd logs
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/logs

Fetch live and historical logs from a squid deployed to SQD Cloud with the sqd logs command — filter by container, time range, slot, and tag.

Fetch logs from a squid deployed to the Cloud

* [`sqd logs`](#sqd-logs-1)

## `sqd logs`

Fetch logs from a squid deployed to the Cloud

```
USAGE
  $ sqd logs [--interactive] [--since <value>] [--search <value>] [-f | -p <value>]
    [-r [<org>/]<name>(@<slot>|:<tag>) | -o <code> | [-s <slot> -n <name>] | [-t <tag> ]]
    [-c processor|query-node|api|db-migrate|db...]
    [-l error|debug|info|warning...] [--since <value>]

FLAGS
  -c, --container=<option>...  Container name
                               <options: processor|query-node|api|db-migrate|db>
  -f, --follow                 Follow
  -l, --level=<option>...      Log level
                               <options: error|debug|info|warning>
  -p, --pageSize=<value>       [default: 100] Logs page size
      --[no-]interactive       Disable interactive mode
      --search=<value>         Filter by content
      --since=<value>          [default: 1d] Filter by date/interval

SQUID FLAGS
  -n, --name=<name>                               Name of the squid
  -r, --reference=[<org>/]<name>(@<slot>|:<tag>)  Fully qualified reference of the squid.
                                                  It can include the organization, name,
                                                  slot, or tag
  -s, --slot=<slot>                               Slot of the squid
  -t, --tag=<tag>                                 Tag of the squid

ORG FLAGS
  -o, --org=<code>  Code of the organization
```

Notes:

* `--since` accepts the notation of the [`ms` library](https://www.npmjs.com/package/ms): 1d, 10h, 1m.
* With `--level=error` `sqd` will fetch logs emitted by three different calls to the [SDK Logger](/en/sdk/squid-sdk/reference/logger), namely
  * `ctx.log.error`
  * `ctx.log.fatal`
  * `ctx.log.trace`

*See code: [src/commands/logs.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/logs.ts)*


# sqd remove
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/remove

Remove a squid deployment from SQD Cloud with the sqd rm command — delete a specific slot or tag and free up resources from your organization quota.

Remove a squid deployed to the Cloud

* [`sqd remove`](#sqd-remove-1)

## `sqd remove`

Remove a squid deployed to the Cloud

```
USAGE
  $ sqd remove [--interactive] [-f]
    [-r [<org>/]<name>(@<slot>|:<tag>) | -o <code> | [-s <slot> -n <name>] | [-t <tag> ]]

FLAGS
  -f, --force             Does not prompt before removing a squid or its version
      --[no-]interactive  Disable interactive mode

SQUID FLAGS
  -n, --name=<name>                               Name of the squid
  -r, --reference=[<org>/]<name>(@<slot>|:<tag>)  Fully qualified reference of the squid. It can include the organization, name, slot, or tag
  -s, --slot=<slot>                               Slot of the squid
  -t, --tag=<tag>                                 Tag of the squid

ORG FLAGS
  -o, --org=<code>  Code of the organization

ALIASES
  $ sqd rm
```

*See code: [src/commands/rm.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/rm.ts)*


# sqd restart
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/restart

Restart a squid deployment on SQD Cloud with the sqd restart command — recycle the processor container without redeploying source code or migrations.

Restart a squid deployed to the Cloud

* [`sqd restart`](#sqd-restart-1)

## `sqd restart`

Restart a squid deployed to the Cloud

```
USAGE
  $ sqd restart [--interactive]
    [-r [<org>/]<name>(@<slot>|:<tag>) | -o <code> | [-s <slot> -n <name>] | [-t <tag> ]]

FLAGS
  --[no-]interactive  Disable interactive mode

SQUID FLAGS
  -n, --name=<name>                               Name of the squid
  -r, --reference=[<org>/]<name>(@<slot>|:<tag>)  Fully qualified reference of the squid.
                                                  It can include the organization, name,
                                                  slot, or tag
  -s, --slot=<slot>                               Slot of the squid
  -t, --tag=<tag>                                 Tag of the squid

ORG FLAGS
  -o, --org=<code>  Code of the organization
```

*See code: [src/commands/restart.ts](https://github.com/subsquid/squid-cli/blob/master/src/commands/restart.ts)*


# sqd run
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/run

Run a squid locally with the sqd run command — load environment variables and execute the processor and GraphQL server using the deployment manifest.

Run a squid locally according to the [deployment manifest](/en/cloud/reference/manifest).

* [sqd run PATH](#sqd-run-path)

Notes:

* The command is especially useful for running [multichain squids](/en/sdk/squid-sdk/resources/multichain), as it runs all services in the same terminal and handles failures gracefully.

## `sqd run PATH`

Run a squid project locally

```
USAGE
  $ sqd run PATH [--interactive] [-m <value>] [-f <value>] [-i <value>... | -e <value>...] [-r <value>]

FLAGS
  -e, --exclude=<value>...  Do not run specified services
  -f, --envFile=<value>     [default: .env] Relative path to an additional environment file
  -i, --include=<value>...  Run only specified services
  -m, --manifest=<value>    [default: squid.yaml] Relative path to a squid manifest file
  -r, --retries=<value>     [default: 5] Attempts to restart failed or stopped services
      --[no-]interactive    Disable interactive mode
```

*See code: [src/commands/run.ts](https://github.com/subsquid/squid-cli/blob/master/src/commands/run.ts)*


# sqd secrets
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/secrets

Manage SQD Cloud organization secrets with the sqd secrets CLI — set, list, and delete API keys, RPC endpoints, and database credentials.

Manage account secrets

The secrets are [exposed as a context](/en/cloud/resources/env-variables#secrets), and are accessible to all the squids deployed by the current SQD Cloud [organization](/en/cloud/resources/organizations).

* [`sqd secrets list`](#sqd-secrets-list)
* [`sqd secrets remove NAME`](#sqd-secrets-remove-name)
* [`sqd secrets set NAME VALUE`](#sqd-secrets-set-name-value)

## `sqd secrets list`

List organization secrets in the Cloud

```
USAGE
  $ sqd secrets list [--interactive] [-o <value>]

FLAGS
  -o, --org=<value>       Organization
      --[no-]interactive  Disable interactive mode

ALIASES
  $ sqd secrets ls
```

*See code: [src/commands/secrets/ls.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/secrets/ls.ts)*

## `sqd secrets remove NAME`

Delete an organization secret in the Cloud

```
USAGE
  $ sqd secrets remove NAME [--interactive] [-o <value>]

ARGUMENTS
  NAME  The secret name

FLAGS
  -o, --org=<value>       Organization
      --[no-]interactive  Disable interactive mode

ALIASES
  $ sqd secrets rm
```

*See code: [src/commands/secrets/rm.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/secrets/rm.ts)*

## `sqd secrets set NAME VALUE`

Add or update an organization secret in the Cloud. If value is not specified, it is read from standard input. The secret will be exposed as an environment variable with the given name to all the squids in the organization. NOTE: The changes take affect only after a squid is restarted or updated.

```
USAGE
  $ sqd secrets set NAME VALUE [--interactive] [-o <value>]

ARGUMENTS
  NAME   The secret name
  VALUE  The secret value

FLAGS
  -o, --org=<value>       Organization
      --[no-]interactive  Disable interactive mode
```

*See code: [src/commands/secrets/set.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/secrets/set.ts)*


# sqd tags
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/tags

Manage version tags for squids deployed to SQD Cloud with the sqd tags CLI — promote, alias, and roll back releases following the slots-and-tags model.

Manage squid deployements' tags. See the [slots and tags guide](/en/cloud/resources/slots-and-tags).

* [`sqd tags add`](#sqd-tags-add)
* [`sqd tags remove`](#sqd-tags-remove)

## `sqd tags add`

Add a tag to a squid deployment / slot

```
USAGE
  $ sqd tags add TAG [--interactive] [--allow-tag-reassign]
    [-r [<org>/]<name>(@<slot>|:<tag>) | -o <code> | [-s <slot> -n <name>] | [-t <tag> ]]

ARGUMENTS
  TAG  New tag to assign

FLAGS
  --allow-tag-reassign  Allow reassigning an existing tag
  --[no-]interactive    Disable interactive mode

SQUID FLAGS
  -n, --name=<name>                               Name of the squid
  -r, --reference=[<org>/]<name>(@<slot>|:<tag>)  Fully qualified reference of the squid.
                                                  It can include the organization, name,
                                                  slot, or tag
  -s, --slot=<slot>                               Slot of the squid
  -t, --tag=<tag>                                 Tag of the squid

ORG FLAGS
  -o, --org=<code>  Code of the organization
```

*See code: [src/commands/tags/add.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/tags/add.ts)*

## `sqd tags remove`

Remove a tag from a squid deployment / slot

```
USAGE
  $ sqd tags remove TAG [--interactive]
    [-r [<org>/]<name>(@<slot>|:<tag>) | -o <code> | [-s <slot> -n <name>] | [-t <tag> ]]

ARGUMENTS
  TAG  New tag to assign

FLAGS
  --[no-]interactive  Disable interactive mode

SQUID FLAGS
  -n, --name=<name>                               Name of the squid
  -r, --reference=[<org>/]<name>(@<slot>|:<tag>)  Fully qualified reference of the squid.
                                                  It can include the organization, name,
                                                  slot, or tag
  -s, --slot=<slot>                               Slot of the squid
  -t, --tag=<tag>                                 Tag of the squid

ORG FLAGS
  -o, --org=<code>  Code of the organization
```

*See code: [src/commands/tags/remove.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/tags/remove.ts)*


# sqd whoami
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/whoami

Show the SQD Cloud user account currently authenticated with the sqd CLI — display username, organization, and roles for the active session.

Show the user details for the current Cloud account

* [`sqd whoami`](#sqd-whoami-1)

## `sqd whoami`

Show the user details for the current Cloud account

```
USAGE
  $ sqd whoami [--interactive]

FLAGS
  --[no-]interactive  Disable interactive mode
```

*See code: [src/commands/whoami.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/whoami.ts)*


# SQD Firehose
Source: https://docs.sqd.dev/en/sdk/squid-sdk/subgraphs-support

Run subgraphs on SQD Network via Firehose — a high-throughput SQD-native gRPC sync layer that replaces RPC-based ingestion for The Graph subgraphs.

# Run subgraphs without full nodes

<Warning>
  This tutorial uses alpha-quality software. If you encouter any issues while using it please let us know at the [SquidDevs Telegram chat](https://t.me/HydraDevs).
</Warning>

<Info>
  **Dependencies**: Docker, Git, NodeJS, Yarn.
</Info>

Developing and running [subgraphs](https://thegraph.com/docs/en/glossary/) is hard, as one has to run a full archival node, and for many networks it is not feasible to run a full archival node to at all.

**SQD Firehose** is an open-source lightweight adapter run as a side-car to a graph indexer node, ingesting and filtering the data directly from SQD Network instead of an RPC endpoint. However, since the network does not provide the real-time blocks, the most recent and unfinalized blocks are (optionally) ingested from a complementary RPC endpoint in a seamless way.

Currently it is only possible to run subgraphs against a production-ready [permissioned SQD Network instance](/en/network/overview#open-private-network) (formerly known as Subsquid Archives). Running against the decentralized SQD Network is scheduled to be supported in Q2 2024 (see [SQD Network Overview](/en/network/overview) to learn more about the difference).

The easiest way to run a subgraph with SQD Firehose to use our [graph-node-setup](https://github.com/subsquid-labs/graph-node-setup) repo. Here's how:

1. Clone the repo and install the dependencies:

   ```bash theme={"system"}
   git clone https://github.com/subsquid-labs/graph-node-setup
   cd graph-node-setup
   npm ci
   ```

2. Interactively configure the environment with

   ```bash theme={"system"}
   npm run configure
   ```

   <Frame>
     <img alt="Configuration interface" />
   </Frame>

   You will be asked to select a network. You can pick any network from our [supported EVM networks](/en/data/evm); networks that are not currently [supported by TheGraph](https://thegraph.com/docs/en/developing/supported-networks/) will be available their under SQD names.

   Optionally you can also provide an RPC endpoint. If you do, it will be used to sync a few thousands of blocks at the chain end, so it does not have to be a paid one. However, `firehose-grpc` does not limit its request rate yet, so using a public RPC might result in a cooldown.

   If you do not provide an RPC endpoint, your subgraph deployments will be a few thousands of blocks behind the chain head.

   <Warning>
     Running against the decentralized testnet is not currently supported. This feature will be re-enabled in Q2 2024.
   </Warning>

3. Download and deploy your subgraph of choice! For example, if you configured the environment to use Ethereum mainnet (`eth-mainnet`), you can deploy the well known Gravatar subgraph:

   ```bash theme={"system"}
   git clone https://github.com/graphprotocol/example-subgraph
   cd example-subgraph

   # the repo is a bit outdated, giving it a deps update
   rm yarn.lock
   npx --yes npm-check-updates --upgrade
   yarn install

   # generate classes for the smart contract
   # and events used in the subgraph
   npm run codegen

   # create and deploy the subgraph
   npm run create-local
   npm run deploy-local
   ```

   GraphiQL playground will be available at [http://127.0.0.1:8000/subgraphs/name/example/graphql](http://127.0.0.1:8000/subgraphs/name/example/graphql).

## Troubleshooting

Do not hesitate to let us know about any issues (whether listed here or not) at the [SquidDevs Telegram chat](https://t.me/HydraDevs).

* If your subgraph is not syncing and you're getting
  ```
  thread 'tokio-runtime-worker' panicked at 'called `Option::unwrap()` on a `None` value', src/ds_rpc.rs:556:80
  ```
  errors in the `graph-node-setup-firehose` container logs, that likely means that the chain RPC is not fully Ethereum-compatible and a workaround is not yet implemented in `firehose-grpc`. You can still sync your subgraph with RPC ingestion disabled.


# SQD vs The Graph
Source: https://docs.sqd.dev/en/sdk/squid-sdk/subsquid-vs-thegraph

Compare SQD and The Graph — feature parity, indexing throughput, query performance, decentralization model, and pricing for blockchain indexers.

[The Graph](https://thegraph.com) is an indexing protocol and framework for EVM chains. In The Graph, indexing projects are called subgraphs. A Graph indexing node is a black-box that executes subgraphs compiled into WASM. The data is sourced directly from an archival blockchain node, and the processed data is stored in a built-in Postgres database.

On the contrary, SQD employs a radically open modular architecture with:

* a separated layer for efficient data extraction and batch queries ([SQD Network](/en/network))
* a client-side libraries for data transformation and presentation.

The data transformation and presentation is enabled by a growing list of tools and SDKs that consume the raw data from SQD Network:

* Typescript based [Squid SDK](/en/sdk/squid-sdk) offers pluggable data targets for both online and offline use-cases
* Python-based [DipDup SDK](https://dipdup.io/docs/quickstart-evm?ref=blog.subsquid.io)
* Subgraphs (via [SQD Firehose adapter](/en/sdk/squid-sdk/subgraphs-support))
* ApeWorx (via [ape-subsquid plugin](/en/sdk/squid-sdk/apeworx))

## Feature matrix

|                                         |                                                                     SQD Network + Squid SDK                                                                     |             The Graph             |
| :-------------------------------------: | :-------------------------------------------------------------------------------------------------------------------------------------------------------------: | :-------------------------------: |
|           Programming language          |                                                                            Typescript                                                                           | AssemblyScript (compiled to WASM) |
|              Indexing speed             |                                                                           \~1k-50k bps                                                                          |           \~100-150 bps           |
|           ABI-based generator           |                                                                               Yes                                                                               |                Yes                |
| Real-time indexing (unfinalized blocks) |                                                                               Yes                                                                               |                 No                |
|              Off-chain data             |                                                                               Yes                                                                               |                 No                |
|               Data targets              |                                                                           Customizable                                                                          |           Postgres-only           |
|        Customizable DB migrations       |                                                                               Yes                                                                               |                 No                |
|        Factory contract indexing        |                                                                        Yes, via wildcards                                                                       |                Yes                |
|         Multi-contract indexing         |                                                                               Yes                                                                               |              Limited              |
|          Analytic data targets          |                                                                      BigQuery, Parquet, CSV                                                                     |                 No                |
|               Local setup               |                                                                               Easy                                                                              |     Requires an archival node     |
|               GraphQL API               |                                                                 Generated from `schema.graphql`                                                                 |  Generated from `schema.graphql`  |
|      Custom resolvers and mutations     |                                                                               Yes                                                                               |                 No                |
|              Subscriptions              |                                                                               Yes                                                                               |           Via middleware          |
|              Hosted service             |                                                                               Yes                                                                               |         Yes (to be sunset)        |
|           Secret env variables          |                                                                               Yes                                                                               |                 No                |
|                 Payment                 |                                                          [Fiat, subscription-based](/en/cloud/pricing)                                                          |         GRT, pay-per-query        |
|             Decentralization            | Decentralized data sourcing via [SQD Network](/en/network), with opt-in decentralized data targets (Kwil DB, Ceramic) and processing (via Lava, in development) |         The Graph network         |

## Architecture difference

By design, The Graph indexing node is a black-box that executes subgraphs compiled into WASM. The data is sourced directly from the archival node and local IPFS, and the processed data is stored in a built-in Postgres database. The data stored in the database is considered to be "sealed", so that no external process can modify or read the data except through the GraphQL interface.


# Squid SDK Troubleshooting
Source: https://docs.sqd.dev/en/sdk/squid-sdk/troubleshooting

Common issues, errors, and fixes for Squid SDK developers — schema migration failures, processor crashes, sync delays, and TypeScript build errors.

Many issues can be resolved by following the [best practices guide](/en/cloud/resources/best-practices) of SQD Cloud.

## Processor

<AccordionGroup>
  <Accordion title="`Error: data out-of-bounds` `ethers` errors on EVM">
    * Usually this means that you're using the decoder on the wrong data. Make sure that the decoder receives *only* the data you intend it to.

      Example: suppose you want to add the processing of a `Mint` event to a squid that is currently processing only `Transfer` events. You change the processor configuration to get the `Mint` events for you, but you forget to sort the events in the batch handler and a data item with a `Mint` event finds its way into a decoder of `Transfer`s.

    * Another common source of this error is faulty RPC endpoints. If your EVM processor crashes during RPC ingestion on a log with `'0x'` in its data field, try switching to another RPC provider or, if you are developing locally, to another Ethereum network emulator.
  </Accordion>

  <Accordion title="`BAD_DATA` when using a Multicall contract">
    This error can occur for a variety of reasons, but one common cause is choosing a Multicall deployment that is newer than the oldest blocks that have to be indexed. When [batch state queries](/en/sdk/squid-sdk/resources/tools/typegen/state-queries#batch-state-queries) are performed on historical chain state older than the Multicall deployment, EVM detects that and refuses to run.

    Solutions:

    1. Use an older Multicall deployment.
    2. Delay your chain state queries until a later block.

    These issues are explored in [Part 4 of the BAYC tutorial](/en/sdk/squid-sdk/tutorials/bayc/step-four-optimizations).
  </Accordion>
</AccordionGroup>

## Data sinks

<AccordionGroup>
  <Accordion title="`QueryFailedError: relation &#x22;...&#x22; does not exist`">
    It is likely that the generated migrations in the `db/migrations` folder are outdated and do not match the schema file.
    Recreate the migrations from scratch as detailed on [this page](/en/sdk/squid-sdk/resources/persisting-data/typeorm#updating-after-schema-changes).
  </Accordion>

  <Accordion title="`Query runner already released. Cannot run queries anymore`, or `too late to perform db updates, make sure you haven't forgot to await on db query`">
    If your squid [saves its data to a database](/en/sdk/squid-sdk/resources/persisting-data/typeorm), all operations with `ctx.store` are asynchronous. Make sure you `await` on all `store` operations like `upsert`, `update`, `find`, `save` etc.

    You may find the [require-await](https://eslint.org/docs/latest/rules/require-await) eslint rule to be helpful.
  </Accordion>

  <Accordion title="`QueryFailedError: invalid byte sequence for encoding &#x22;UTF8&#x22;: 0x00`">
    PostgreSQL doesn't support storing `NULL (\0x00)` characters in text fields. Usually the error occurs when a raw bytes string (like `UIntArray` or `Bytes`) is inserted into a `String` field. If this is the case, use hex encoding, e.g. using [`util-internal-hex`](https://github.com/subsquid/squid/tree/master/util/util-internal-hex) library. For addresses, use the [`ss58` encoding library](https://github.com/subsquid/squid/tree/master/ss58).
  </Accordion>
</AccordionGroup>

## GraphQL

<AccordionGroup>
  <Accordion title="API queries are too slow">
    * Make sure all the necessary fields are [indexed](/en/sdk/squid-sdk/reference/schema-file/indexes-and-constraints). The best way to do that is to perform the [query optimization procedure](/en/cloud/resources/query-optimization).
    * Annotate the schema and [set reasonable limits](/en/sdk/squid-sdk/reference/openreader-server/configuration/dos-protection) for the incoming queries to protect against DoS attacks
  </Accordion>

  <Accordion title="`response might exceed the size limit`">
    Make sure the input query has limits set or the entities are decorated with `@cardinality`. We recommend using `XXXConnection` queries for pagination. For configuring limits and max response sizes, see [DoS protection](/en/sdk/squid-sdk/reference/openreader-server/configuration/dos-protection).
  </Accordion>
</AccordionGroup>


# Processor in action
Source: https://docs.sqd.dev/en/sdk/squid-sdk/tutorials/batch-processor-in-action

Walkthrough of a Squid SDK batch processor in action — see how blocks are batched, fed through handlers, and persisted in a real indexer example.

<Tabs>
  <Tab title="EVM">
    An end-to-end idiomatic usage of `EvmBatchProcessor` can be inspected in the [gravatar template repository](https://github.com/subsquid-labs/gravatar-squid) and also learned from more elaborate [examples](/en/sdk/squid-sdk/examples).

    In order to illustrate the concepts covered in the [development guide](/en/sdk/squid-sdk/how-to-start/squid-development), here we highlight the key steps, put together a processor configuration and a data handling definition.

    <Info>
      **Pre-requisites:** NodeJS, Git, Docker, [Squid CLI](/en/sdk/squid-sdk/squid-cli/installation), any of the [EVM templates](/en/sdk/squid-sdk/how-to-start/squid-development#templates).
    </Info>

    ## 1. Model the target schema and generate entity classes

    Create or edit `schema.graphql` to define the target entities and relations. Consult [the schema reference](/en/sdk/squid-sdk/reference/schema-file).

    Update the entity classes, start a fresh database and regenerate migrations:

    ```bash theme={"system"}
    npx squid-typeorm-codegen
    ```

    ```bash theme={"system"}
    docker compose down
    ```

    ```bash theme={"system"}
    docker compose up -d
    ```

    ```bash theme={"system"}
    rm -r db/migrations
    ```

    ```bash theme={"system"}
    npx squid-typeorm-migration generate
    ```

    Apply the migrations with

    ```bash theme={"system"}
    npx squid-typeorm-migration apply
    ```

    ## 2. Generate Typescript ABI modules

    Use [`evm-typegen`](/en/sdk/squid-sdk/resources/tools/typegen) to generate the facade classes, for example like this:

    ```bash theme={"system"}
    npx squid-evm-typegen src/abi 0x2E645469f354BB4F5c8a05B3b30A929361cf77eC#Gravity --clean
    ```

    ## 3. Configuration

    See [Configuration section](/en/sdk/squid-sdk/reference/processors/evm-batch) for more details.

    ```ts theme={"system"}
    const processor = new EvmBatchProcessor()
      .setGateway('https://v2.archive.subsquid.io/network/ethereum-mainnet')
      .setRpcEndpoint('<my_eth_rpc_url>')
      .setFinalityConfirmation(75)
      .setBlockRange({ from: 6175243 })
      // fetch logs emitted by '0x2E645469f354BB4F5c8a05B3b30A929361cf77eC'
      // matching either `NewGravatar` or `UpdatedGravatar`
      .addLog({
        address: ['0x2E645469f354BB4F5c8a05B3b30A929361cf77eC'],
        topic0: [
          events.NewGravatar.topic,
          events.UpdatedGravatar.topic,
        ]
      })
    ```

    ## 4. Iterate over the batch items and group events

    The following code snippet illustrates a typical data transformation in a batch. The strategy is to

    * Iterate over `ctx.blocks` and `block.logs`
    * Decode each log using a suitable facade class
    * Enrich and transform the data
    * Upsert arrays of entities in batches using `ctx.save()`

    The `processor.run()` method the looks as follows:

    ```ts theme={"system"}
    processor.run(new TypeormDatabase(), async (ctx) => {
      // storing the new/updated entities in
      // an in-memory identity map
      const gravatars: Map<string, Gravatar> = new Map()
      // iterate over the data batch stored in ctx.blocks
      for (const c of ctx.blocks) {
        for (const log of c.logs) {
          // decode the log data
          const { id, owner, displayName, imageUrl } = extractData(log)
          // transform and normalize to match the target entity (Gravatar)
          gravatars.set(id.toHexString(), new Gravatar({
            id: id.toHexString(),
            owner: decodeHex(owner),
            displayName,
            imageUrl
          }))
        }
      }
      // Upsert the entities that were updated.
      // Note that store.save() automatically updates
      // the existing entities and creates new ones.
      // It splits the data into suitable chunks to
      // guarantee an adequate performance.
      await ctx.store.save([...gravatars.values()])
    });
    ```

    In the snippet above, we decode both `NewGravatar` and `UpdatedGravatar` with a single helper function that uses the generated `events` facade class:

    ```ts theme={"system"}
    function extractData(evmLog: any): { id: ethers.BigNumber, owner: string, displayName: string, imageUrl: string} {
      if (evmLog.topics[0] === events.NewGravatar.topic) {
        return events.NewGravatar.decode(evmLog)
      }
      if (evmLog.topics[0] === events.UpdatedGravatar.topic) {
        return events.UpdatedGravatar.decode(evmLog)
      }
      throw new Error('Unsupported topic')
    }
    ```

    ## 5. Run the processor and store the transformed data into the target database

    Run the processor with

    ```bash theme={"system"}
    node -r dotenv/config lib/main.js
    ```

    The script will build the code automatically before running.

    In a separate terminal window, run

    ```bash theme={"system"}
    npx squid-graphql-server
    ```

    Inspect the GraphQL API at [`http://localhost:4350/graphql`](http://localhost:4350/graphql).
  </Tab>

  <Tab title="Substrate">
    An end-to-end idiomatic usage of `SubstrateBatchProcessor` can be inspected in the [squid-substrate-template](https://github.com/subsquid-labs/squid-substrate-template) and also learned from more elaborate [examples](/en/sdk/squid-sdk/examples).

    Here we highlight the key steps and put together a configuration and a data handling definition to illustrate the concepts covered in the [squid development guide](/en/sdk/squid-sdk/how-to-start/squid-development).

    <Info>
      **Pre-requisites:** NodeJS, Git, Docker, [Squid CLI](/en/sdk/squid-sdk/squid-cli/installation), any of the [Substrate templates](/en/sdk/squid-sdk/how-to-start/squid-development#templates).
    </Info>

    ## 1. Set up the processor

    See [Setup section](/en/sdk/squid-sdk/reference/processors/substrate-batch) for more details.

    ```ts title="src/processor.ts" theme={"system"}
    import {
      SubstrateBatchProcessor
    } from '@subsquid/substrate-processor'

    export const processor = new SubstrateBatchProcessor()
      // set the data source
      .setGateway('https://v2.archive.subsquid.io/network/kusama')
      .setRpcEndpoint('https://kusama-rpc.dwellir.com')
      // add data requests
      .addEvent({
        name: ['Balances.Transfer'],
        extrinsic: true
      })
      .addCall({
        name: ['Balances.transfer_keep_alive'],
        extrinsic: true
      })
      // define the data
      // to be fetched for each data item
      .setFields({
        extrinsic: {
          hash: true,
          fee: true
        },
        call: {
          success: true
        },
        block: {
          timestamp: true
        }
      })
    ```

    ## 2. Define a custom data facade and extract normalized data from `BatchContext`

    The following code snippet illustrates how the data is extracted and normalized into some user-specific facade interface `TransferData`. Note how the data items in each of the block data iterables (`events`, `calls`, `extrinsics`) are filtered in the batch handler to make sure they match the data requests.

    Type-safe access and decoding of the call and event data is done with the help of the classes generated by a suitable [typegen tool](/en/sdk/squid-sdk/glossary#typegen), in this case [squid-substrate-typegen](/en/sdk/squid-sdk/resources/tools/typegen).

    ```ts title="src/main.ts" theme={"system"}
    import {Store} from '@subsquid/typeorm-store'
    import * as ss58 from '@subsquid/ss58'

    import {events} from './types'
    import {ProcessorContext} from './processor'

    // some normalized user-define data
    interface TransferData {
      id: string
      timestamp: Date | undefined
      extrinsicHash: string | undefined
      from: string
      to: string
      amount: bigint
      fee?: bigint
    }

    // extract and normalize
    function getTransfers(ctx: ProcessorContext<Store>): TransferData[] {
      let transfers: TransferData[] = []
      for (let block of ctx.blocks) {
        for (let event of block.events) {
          if (event.name==='Balances.Transfer') {
            // decode the event data with runtime-versioned
            // classes generated by typegen
            let rec: {from: string, to: string, amount: bigint}
            if (events.balances.transfer.v1020.is(event)) {
              let [from, to, amount,] =
                events.balances.transfer.v1020.decode(event)
              rec = {from, to, amount}
            } else if (events.balances.transfer.v1050.is(event)) {
              let [from, to, amount] =
                events.balances.transfer.v1050.decode(event)
              rec = {from, to, amount}
            } else {
              rec = events.balances.transfer.v9130.decode(event)
            }
            // extract and normalize the `item.event` data
            let timestamp =
              block.header.timestamp == null ?
              undefined :
              new Date(block.header.timestamp)
            transfers.push({
              id: event.id,
              timestamp,
              extrinsicHash: event.extrinsic?.hash,
              from: ss58.codec('kusama').encode(rec.from),
              to: ss58.codec('kusama').encode(rec.to),
              amount: rec.amount,
              fee: event.extrinsic?.fee || 0n
            })
          }
        }
        for (let call of block.calls) {
          if (call.name==='Balances.transfer_keep_alive') {
            // extranct and normalize the call data
            transfer.push({
              /* some data manipulation on the call data */
            })
          }
        }
      }
      return transfers
    }
    ```

    ## 3. Run the processor and store the transformed data into the target database

    The snippet below assumes that we are using `TypeormDatabase` and that the database schema together with the model entity types has already been prepared. Consult the [Schema file](/en/sdk/squid-sdk/reference/schema-file) section for more details.

    ```ts title="src/main.ts" theme={"system"}
    import {TypeormDatabase} from '@subsquid/typeorm-store'
    // an entity type generated from the schema file
    import {Transfer} from './model'
    import {processor} from './processor'

    // ... `getTransfers(Ctx)` definition

    processor.run(new TypeormDatabase(), async ctx => {
      // get the normalized data from the context
      let transfersData = getTransfers(ctx)

      // an array of entity class instances
      let transfers: Transfer[] = []

      for (let t of transfersData) {
        let {id, blockNumber, timestamp, extrinsicHash, amount, fee} = t

        // join some extra data
        let from = getAccount(accounts, t.from)
        let to = getAccount(accounts, t.to)

        // populate entity class instances
        // with the extracted data
        transfers.push(new Transfer({
          id,
          blockNumber,
          timestamp,
          extrinsicHash,
          from,
          to,
          amount,
          fee
        }))
      }

      // persist an array of entities
      // in a single statement
      await ctx.store.insert(transfers)
    })
    ```
  </Tab>
</Tabs>


# Step 4: Optimization
Source: https://docs.sqd.dev/en/sdk/squid-sdk/tutorials/bayc/step-four-optimizations

BAYC indexer tutorial step 4: optimization — sync faster while tracking dynamic NFT metadata using Squid SDK batch handler tuning techniques.

This is the fourth part of the tutorial where we build a squid that indexes [Bored Ape Yacht Club](https://boredapeyachtclub.com) NFTs, their transfers, and owners from the [Ethereum blockchain](https://ethereum.org), fetches the metadata from [IPFS](https://ipfs.tech/) and regular HTTP URLs, stores all the data in a database, and serves it over a GraphQL API. In the first three parts ([1](/en/sdk/squid-sdk/tutorials/bayc/step-one-indexing-transfers), [2](/en/sdk/squid-sdk/tutorials/bayc/step-two-deriving-owners-and-tokens), [3](/en/sdk/squid-sdk/tutorials/bayc/step-three-adding-external-data)), we created a squid that does all the above but performs many IO operations sequentially, resulting in a long sync time. In this part, we discuss strategies for mitigating that shortcoming. We also discuss an alternative metadata fetching strategy that reduces redundant fetches and handles the changes in metadata of "cold" (i.e., not involved in any transfers) tokens more effectively.

Pre-requisites: Node.js, [Squid CLI](/en/sdk/squid-sdk/squid-cli/installation), Docker, a project folder with the code from the third part ([this commit](https://github.com/subsquid-labs/bayc-squid-1/tree/ab5f094ae34e8822dfb912f6e6116df2cfa800b5)).

## Using Multicall for aggregating state queries

We begin by introducing [batch processing](/en/sdk/squid-sdk/resources/batch-processing) wherever possible, and our first step is to replace individual contract state queries with [batch calls](/en/sdk/squid-sdk/resources/tools/typegen/state-queries#batch-state-queries) to a [MakerDAO multicall contract](https://github.com/mds1/multicall). Retrieve the multicall contract ABI by re-running `squid-evm-typegen` with `--multicall` option:

```bash theme={"system"}
npx squid-evm-typegen --multicall src/abi 0xbc4ca0eda7647a8ab7c2061c2e118a18a936f13d#bayc
```

This adds a Typescript ABI interface at `src/abi/multicall.ts`. Let us use it in a rewrite of `completeTokens()`:

```typescript title="src/main.ts" theme={"system"}
import {Multicall} from './abi/multicall'

const MULTICALL_ADDRESS = '0xeefba1e63905ef1d7acba5a8513c70307c1ce441'
const MULTICALL_BATCH_SIZE = 100

// ...

async function completeTokens(
    ctx: Context,
    partialTokensMap: Map<string, PartialToken>
): Promise<Map<string, Token>> {

    let partialTokens: PartialToken[] = [...partialTokensMap.values()]

    let tokens: Map<string, Token> = new Map()
    if (partialTokens.length === 0) return tokens

    let lastBatchBlockHeader = ctx.blocks[ctx.blocks.length-1].header
    let contract = new Multicall(ctx, lastBatchBlockHeader, MULTICALL_ADDRESS)

    let tokenURIs = await contract.aggregate(
        bayc.functions.tokenURI,
        CONTRACT_ADDRESS,
        partialTokens.map(t => [t.tokenId]),
        MULTICALL_BATCH_SIZE // paginating to avoid RPC timeouts
    )

    for (let [i, ptoken] of partialTokens.entries()) {
        let uri = tokenURIs[i]
        let metadata: TokenMetadata | undefined = await fetchTokenMetadata(ctx, uri)
        tokens.set(ptoken.id, new Token({
            ...ptoken,
            uri,
            ...metadata
        }))
    }

    return tokens
}

// ...
```

Here we replaced the direct calls to `tokenURI()` of the BAYC token contract with aggreaged batches of 100 state calls and called `aggregate()` of the multicall contract for each batch. `Multicall.aggregate()` takes care of splitting the set of state calls into chunks and merging the results together. The number 100 for the batch size was chosen to be as large as possible while not triggering any response size limits on the public RPC endpoint we use. With a private RPC endpoint one would try to further increase the batch size.

Note that we used the old Multicall V1 contract that was deployed before the BAYC contract. This is required because we're making queries to the historical state, and some of them will be made at block heights just a few hundred blocks above the BAYC contract deployment. Those queries would fail we used a multicall contract that did not yet exist back then.

You can find the full code after this optimization at [this commit](https://github.com/subsquid-labs/bayc-squid-1/tree/fca1918bf9f307071fe867916f989627bfc73893). In our test this optimization reduced the time required for retrieving the contract state for all 10000 tokens once from 114 minutes to 94 seconds (figures out of date).

## Retrieving metadata from HTTPS concurrently

Next, we make our metadata retrieval requests concurrent.

Many metadata URIs point to the same HTTPS server or to IPFS that we are accessing through a single gateway. When retrieving data from just a few servers, it is a common courtesy to not send all the requests at once; often, this is enforced by rate limits. Our code takes that into account, limiting the rate of the outgoing requests.

We implement HTTPS batching at `src/metadata.ts`:

```typescript theme={"system"}
import { asyncSleep, splitIntoBatches } from './util'

const MAX_REQ_SEC = 10

export async function fetchTokenMetadatasConcurrently(
    ctx: Context,
    uris: string[]
): Promise<(TokenMetadata | undefined)[]> {

    let metadatas: (TokenMetadata | undefined)[] = []
    for (let batch of splitIntoBatches(uris, MAX_REQ_SEC)) {
        let m = await Promise.all(batch.map((uri, index) => {
            // spread out the requests evenly within a second interval
            let sleepMs = Math.ceil(1000*(index+1)/MAX_REQ_SEC)
            return asyncSleep(sleepMs).then(() => fetchTokenMetadata(ctx, uri))
        }))
        metadatas.push(...m)
    }
    return metadatas
}
```

then we call the function from [`completeTokens()`](https://github.com/subsquid-labs/bayc-squid-1/blob/9e9c93859632f193f30aca0ff1362f408f493e62/src/main.ts#L110) and use its output to populate metadata fields of `Token` entity instances. Utility functions `asyncSleep()` and `splitIntoBathches()` are implemented [here](https://github.com/subsquid-labs/bayc-squid-1/blob/9e9c93859632f193f30aca0ff1362f408f493e62/src/util.ts). Full code is available at [this commit](https://github.com/subsquid-labs/bayc-squid-1/tree/9e9c93859632f193f30aca0ff1362f408f493e62).

On the first processor batch, when we retrieve metadata from `us-central1-bayc-metadata.cloudfunctions.net` for all 10000 tokens, this optimization reduced the required time from 45 to 21 minutes (figures out of date).

## Retrieving immutable metadata once

Another glaring inefficiency in our current code is the fact that we often retrieve metadata from each URI more than once. This is understandable when we work with HTTPS links that point to mutable data; however, the data IPFS links point to is immutable, so there is no need to retrieve it more than once.

To avoid these repeated retrievals we find the already known tokens in the database. If a token metadata URI is an immutable IPFS link (that is, does not [point to MFS](https://gateway.ipfs.io/ipfs/QmTkzDwWqPbnAh5YiV5VwcTLnGdwSNsNTn2aDxdXBFca7D/example#/ipfs/QmThrNbvLj7afQZhxH72m5Nn1qiVn3eMKWFYV49Zp2mv9B/ipns/readme.md)) and metadata is already available, we skip the retrieval. The implementation is at `src/metadata.ts`:

```typescript theme={"system"}
import { Token } from './model'
import { In } from 'typeorm'

export async function selectivelyUpdateMetadata(
    ctx: Context,
    tokens: Map<string, Token>
): Promise<Map<string, Token>> {

    let knownTokens: Map<string, Token> = await ctx.store.findBy(
            Token,
            {id: In([...tokens.keys()])}
        )
        .then(ts => new Map(ts.map(t => [t.id, t])))

    let updatedTokens: Map<string, Token> = new Map()
    let tokensToBeUpdated: Token[] = []
    for (let [id, t] of tokens) {
        let ktoken: Token | undefined = knownTokens.get(id)
        if (ktoken != null &&
            ktoken.image != null && ktoken.attributes != null &&
            ktoken.uri === t.uri && uriPointsToImmutable(t.uri)) {

            ctx.log.info(`Repeated retrieval from ${t.uri} skipped`)
            updatedTokens.set(id, ktoken)
        }
        else {
            ctx.log.info(`Re-retrieving from ${t.uri}`)
            tokensToBeUpdated.push(t)
        }
    }

    let metadatas: (TokenMetadata | undefined)[] = await fetchTokenMetadatasConcurrently(
        ctx,
        tokensToBeUpdated.map(t => t.uri)
    )

    for (let [i, t] of tokensToBeUpdated.entries()) {
        let m = metadatas[i]
        if (m != null) {
            t.image = m.image
            t.attributes = m.attributes
        }
        updatedTokens.set(t.id, t)
    }

    return updatedTokens
}
```

`uriPointsToImmutable()` used to classify metadata URIs rejects any non-IPFS links and IPFS links that may point to MFS:

```typescript theme={"system"}
export function uriPointsToImmutable(uri: string): boolean {
    return uri.startsWith('ipfs://') && !uri.includes('ipns')
}
```

Full code is available at [this commit](https://github.com/subsquid-labs/bayc-squid-1/tree/714aee44d9eb9f43902ae55bf41cedad0384000e). In our test this optimization reduced the total sync time from about 4.1 to 1.5 hours (figures out of date).

## Alternative: Post-sync retrieval of metadata

Despite all the optimizations, our squid still takes 1.5 hours to sync instead of 11 minutes it needed [before we introduced metadata retrieval](https://github.com/subsquid-labs/bayc-squid-1/tree/3568f70b7ac1802fbbf80079e435a3348b0d3a62) (figures out of date). This is the cost of maintaining a fully populated database at all stages of the sync, which is rarely a requirement. An alternative is to begin retrieving metadata only after the rest of the data has been fully synced, and the squid has caught up with the blockchain. We do that by reading `Token` entities from the database after the initial sync, retrieving their metadata and persisting them.

This approach has another advantage: metadata URIs can change without notice, and our old retrieval strategy would only pick up the changes when the token has been transferred. If our goal is to keep token metadata as up-to-date as possible, we have to constantly renew it even for tokens that are not involved in any recent transfers. This is easy to implement with our new metadata retrieval strategy: simply add a field to the `Token` entity that tracks the block height of the most recent metadata update and select `Token`s that were updated some fixed number of blocks ago for metadata updates.

Here is how the new metadata retrieval strategy reflects in the batch handler code:

```diff theme={"system"}
processor.run(new TypeormDatabase(), async (ctx) => {
     let rawTransfers: RawTransfer[] = getRawTransfers(ctx)
 
     let owners: Map<string, Owner> = createOwners(rawTransfers)
     let tokens: Map<string, Token> = await createTokens(ctx, rawTransfers, owners)
     let transfers: Transfer[] = createTransfers(rawTransfers, owners, tokens)
 
     await ctx.store.upsert([...owners.values()])
     await ctx.store.upsert([...tokens.values()])
     await ctx.store.insert(transfers)
+
+    if (ctx.isHead) {
+        let updatedTokens = await updateTokensWithOutdatedMetadata(ctx)
+        await ctx.store.upsert(updatedTokens)
+        ctx.log.info(`Updated metadata for ${updatedTokens.length} tokens`)
+    }
 })
```

Full implementation requires changes to the schema, replacement of `completeTokens()` with `updateTokensWithOutdatedMetadata()` and rewrites of `createTokens()` and `selectivelyUpdateMetadata()`. It is available in [this branch](https://github.com/subsquid-labs/bayc-squid-1/tree/lazy-metadata). The resulting squid took 10 minutes for the initial sync, then 50 minutes more to retrieve metadata at least once for every token (figures out of date).

## Extra: Using Multicall for metadata exploration

In part 3 of this tutorial we [explored metadata URIs](/en/sdk/squid-sdk/tutorials/bayc/step-three-adding-external-data#exploring-token-metadata) by running `tokenURI()` directly on the BAYC token contract. This process took several hours. Replacing the exploration code with its \[multicall]-based equivalent, we can reduce that time to about 17 minutes (figure out of date). Starting with the code [as it was at the end of part two](https://github.com/subsquid-labs/bayc-squid-1/tree/3568f70b7ac1802fbbf80079e435a3348b0d3a62), get `src/abi/multicall.ts` by running

```bash theme={"system"}
npx squid-evm-typegen --multicall src/abi 0xbc4ca0eda7647a8ab7c2061c2e118a18a936f13d#bayc
```

then add the code for batch URI retrieval to the batch handler:

```diff title="src/main.ts" theme={"system"}
+import {Multicall} from './abi/multicall'

+const MULTICALL_ADDRESS = '0xeefba1e63905ef1d7acba5a8513c70307c1ce441'
+const MUTLTICALL_BATCH_SIZE = 100
 
 processor.run(new TypeormDatabase(), async (ctx) => {
     let tokens: Map<string, Token> = createTokens(rawTransfers, owners)
     let transfers: Transfer[] = createTransfers(rawTransfers, owners, tokens)
 
+    let lastBatchBlockHeader = ctx.blocks[ctx.blocks.length-1].header
+    let contract = new Multicall(ctx, lastBatchBlockHeader, MULTICALL_ADDRESS)
+    let tokenURIs = await contract.aggregate(
+        bayc.functions.tokenURI,
+        CONTRACT_ADDRESS,
+        [...tokens.values()].map(t => [t.tokenId]),
+        MUTLTICALL_BATCH_SIZE
+    )
+    for (let uri of tokenURIs) {
+        ctx.log.info(`Retrieved a metadata URI: ${uri}`)
+    }
+
     await ctx.store.upsert([...owners.values()])
     await ctx.store.upsert([...tokens.values()])
     await ctx.store.insert(transfers)
})
```


# Step 1: Transfer events
Source: https://docs.sqd.dev/en/sdk/squid-sdk/tutorials/bayc/step-one-indexing-transfers

BAYC indexer tutorial step 1: index Transfer events from the BAYC ERC-721 contract on Ethereum using Squid SDK and the EvmBatchProcessor.

# Step 1: Indexing Transfer events

In this step-by-step tutorial we will build a squid that gets data about [Bored Ape Yacht Club](https://boredapeyachtclub.com) NFTs, their transfers and owners from the [Ethereum blockchain](https://ethereum.org), indexes the NFT metadata from [IPFS](https://ipfs.tech/) and regular HTTP URLs, stores all the data in a database and serves it over a GraphQL API. Here we do the first step: build a squid that indexes only the `Transfer` events emitted by the [BAYC token contract](https://etherscan.io/address/0xbc4ca0eda7647a8ab7c2061c2e118a18a936f13d).

Pre-requisites: Node.js, [Squid CLI](/en/sdk/squid-sdk/squid-cli/installation), Docker.

## Starting with a template

Begin by retrieving the `evm` template and installing the dependencies:

```bash theme={"system"}
sqd init my-bayc-squid -t evm
cd my-bayc-squid
npm i
```

The resulting code can be found at [this commit](https://github.com/subsquid-labs/bayc-squid-1/tree/4ad7bdfed0771aa6b6e9cf0368ac7f15f09cd629).

## Interfacing with the contract ABI

First, we inspect which data is available for indexing. For EVM contracts, the metadata descrbing the shape of the smart contract logs, transactions and contract state methods is distributed as an [Application Binary Interface](https://docs.soliditylang.org/en/latest/abi-spec.html) (ABI) JSON file. For many popular contracts ABI files are published on Etherscan (as in the case of the BAYC NFT contract). SQD provides a [tool](/en/sdk/squid-sdk/resources/tools/typegen/state-queries) for retrieving contract ABIs from Etherscan-like APIs and generating the boilerplate for retrieving and decoding the data. For the contract of interest, this can be done with

```bash theme={"system"}
npx squid-evm-typegen src/abi 0xbc4ca0eda7647a8ab7c2061c2e118a18a936f13d#bayc
```

Here, `src/abi` is the destination folder and the `bayc` suffix sets the base name for the generated file.

Checking out the generated `src/abi/bayc.ts` file we see all events and contract functions listed in the ABI. Among them there is the `Transfer` event:

```typescript theme={"system"}
export const events = {
    ...
    Transfer: new LogEvent<([from: string, to: string, tokenId: bigint] & {from: string, to: string, tokenId: bigint})>(
        abi, '0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef'
    ),
}
```

Reading about [elsewhere](https://eips.ethereum.org/EIPS/eip-721) we learn that it is emitted every time an NFT changes hand and that its [logs](https://medium.com/mycrypto/understanding-event-logs-on-the-ethereum-blockchain-f4ae7ba50378) contain the addresses of both involved parties, as well as the unique ID of the token. This is the data we need, so we proceed to configure our squid to retrieve it.

## Configuring the data filters

A "squid processor" is the Node.js process and the [object that powers it](/en/sdk/squid-sdk/overview). Together they are responsible for retrieving filtered blockchain data from a specialized data lake ([SQD Network](/en/network/overview)), transforming it and saving the result to a destination of choice. To configure the processor (object) to retrieve the `Transfer` events of the BAYC token contract, we initialize it like this:

```typescript title="src/processor.ts" theme={"system"}
// ...
import * as bayc from './abi/bayc'

export const CONTRACT_ADDRESS = '0xbc4ca0eda7647a8ab7c2061c2e118a18a936f13d'

export const processor = new EvmBatchProcessor()
    .setGateway('https://v2.archive.subsquid.io/network/ethereum-mainnet')
    .setRpcEndpoint({
        url: '<my_eth_rpc_url>',
        rateLimit: 10
    })
    .setFinalityConfirmation(75)
    .setBlockRange({
        from: 12_287_507,
    })
    .addLog({
        address: [CONTRACT_ADDRESS],
        topic0: [bayc.events.Transfer.topic]
    })
    .setFields({
        log: {
            transactionHash: true
        }
    })
// ...
```

Here,

* `'https://v2.archive.subsquid.io/network/ethereum-mainnet'` is the URL the public SQD Network gateway for Ethereum mainnet. Check out the exhaustive [SQD Network gateways list](/en/data/evm).
* `'<eth_rpc_endpoint_url>'` is a public RPC endpoint we chose to use in this example. When an endpoint is available, the processor will begin ingesting data from it once it reaches the highest block available within SQD Network. Please use a private endpoint or SQD Cloud's [RPC addon](/en/cloud/resources/rpc-proxy) in production.
* `setFinalityConfirmation(75)` call instructs the processor to consider blocks final after 75 confirmations when ingesting data from an RPC endpoint.
* `12_287_507` is the block at which the BAYC token contract was deployed. Can be found on the [contract's Etherscan page](https://etherscan.io/address/0xbc4ca0eda7647a8ab7c2061c2e118a18a936f13d).
* The argument of [`addLog()`](/en/sdk/squid-sdk/reference/processors/evm-batch/logs) is a set of filters that tells the processor to retrieve all event logs emitted by the BAYC contract with topic0 matching the hash of the full signature of the `Transfer` event. The hash is taken from the previously generated Typescript ABI.
* The argument of [`setFields()`](/en/sdk/squid-sdk/reference/processors/evm-batch/field-selection) specifies the exact data we need on every event to be retrieved. In addition to the data that is [provided by default](/en/sdk/squid-sdk/reference/processors/evm-batch/field-selection#logs) we are requesting hashes of parent transactions for all event logs.

See [configuration](/en/sdk/squid-sdk/reference/processors/evm-batch) for more options.

## Decoding the event data

The other part of processor configuration is the callback function used to process batches of the filtered data, the [batch handler](/en/sdk/squid-sdk/reference/processors/architecture#processorrun). It is typically defined at the `processor.run()` call at `src/main.ts`, like this:

```typescript theme={"system"}
processor.run(db, async (ctx) => {
    // data transformation and persistence code here
})
```

Here,

* `db` is a [`Database` implementation](/en/sdk/squid-sdk/resources/persisting-data/overview) specific to the target data sink. We want to store the data in a PostgreSQL database and present with a GraphQL API, so we provide a [`TypeormDatabase`](/en/sdk/squid-sdk/resources/persisting-data/typeorm) object here.
* `ctx` is a [batch context](/en/sdk/squid-sdk/reference/processors/architecture#batch-context) object that exposes a batch of data retrieved from SQD Network or a RPC endpoint (at `ctx.blocks`) and any data persistence facilities derived from `db` (at `ctx.store`).

Batch handler is where the raw on-chain data is decoded, transformed and persisted. This is the part we'll be concerned with for the rest of the tutorial.

We begin by defining a batch handler decoding the `Transfer` event:

```typescript title="src/main.ts" theme={"system"}
import {TypeormDatabase} from '@subsquid/typeorm-store'
import {processor, CONTRACT_ADDRESS} from './processor'
import * as bayc from './abi/bayc'

processor.run(new TypeormDatabase(), async (ctx) => {
    for (let block of ctx.blocks) {
        for (let log of block.logs) {
            if (log.address === CONTRACT_ADDRESS && log.topics[0] === bayc.events.Transfer.topic) {
                let {from, to, tokenId} = bayc.events.Transfer.decode(log)
                ctx.log.info(`Parsed a Transfer of token ${tokenId} from ${from} to ${to}`)
            }
        }
    }
})
```

This goes through all the log items in the batch, verifies that they indeed are `Transfer` events emitted by the BAYC contract and decodes the data of each log item, then logs the results to the terminal. The verification step is required because the processor does not guarantee that it won't supply any extra data, only that it *will* supply the data matching the filters. The decoding is done with the `bayc.events.Transfer.decode()` function from the Typescript ABI we previously generated.

At this point the squid is ready for its first test run. Execute

```bash theme={"system"}
docker compose up -d
npm run build
npx squid-typeorm-migration apply
node -r dotenv/config lib/main.js
```

and you should see lots of lines like these in the output:

```
03:56:02 INFO  sqd:processor Parsed a Transfer of token 6325 from 0x0000000000000000000000000000000000000000 to 0xb136c6A1Eb83d0b4B8e4574F28e622A57F8EF01A
03:56:02 INFO  sqd:processor Parsed a Transfer of token 6326 from 0x0000000000000000000000000000000000000000 to 0xb136c6A1Eb83d0b4B8e4574F28e622A57F8EF01A
03:56:02 INFO  sqd:processor Parsed a Transfer of token 4407 from 0x082C99d47E020a00C95460D50a83338d509A0e3a to 0x5cf0E6da6Ec2bd7165edcD52D3d31f2528dCf007
```

The full code can be found at [this commit](https://github.com/subsquid-labs/bayc-squid-1/tree/b7ccfc06b283a885878cddcb18483c4bf571e97c).

## Extending and persisting the data

`TypeormDatabase` requires us to define a TypeORM data model to actually send the data to the database. In SQD, the same data model is also used by the GraphQL server to generate the API schema. To avoid any potential discrepancies, processor and GraphQL server rely on a shared data model description defined at `schema.graphql` in a GraphQL schema dialect fully documented [here](/en/sdk/squid-sdk/reference/schema-file).

TypeORM code is generated from `schema.graphql` with the [`squid-typeorm-codegen`](/en/sdk/squid-sdk/resources/persisting-data/typeorm) tool and must be regenerated every time the schema is changed. This is usually accompanied by regenerating the [database migrations](/en/sdk/squid-sdk/resources/persisting-data/typeorm) and recreating the database itself. The migrations are applied before every run of the processor, ensuring that whenever any TypeORM code within the processor attempts to access the database, the database is in a state that allows it to succeed.

The main unit of data in `schema.graphql` is [entity](/en/sdk/squid-sdk/reference/schema-file/entities). These map onto [TypeORM entites](https://typeorm.io/entities) that in turn map onto database tables. We define one for `Transfer` events by replacing the file contents with

```graphql title="schema.graphql" theme={"system"}
type Transfer @entity {
    id: ID!
    tokenId: BigInt! @index
    from: String! @index
    to: String! @index
    timestamp: DateTime!
    blockNumber: Int!
    txHash: String! @index
}
```

Here,

* The `id` is a required field and the `ID` type is an alias for `String`. Entity IDs must be unique.
* `@index` decorators tell the codegen tool that the corresponding database columns should be indexed.
* Extra fields `timestamp` and `blockNumber` are added to make the resulting GraphQL API more convenient. We will fill them using the block metadata available in `ctx`.

Once we're done editing the schema, we regenerate the TypeORM code, recreate the database and regenerate the migrations:

```bash theme={"system"}
npx squid-typeorm-codegen
npm run build
docker compose down
docker compose up -d
rm -r db/migrations
npx squid-typeorm-migration generate
```

The generated code is in `src/model`. We can now import a `Transfer` entity class from there and use it to perform [various operations](/en/sdk/squid-sdk/resources/persisting-data/typeorm) on the corresponding database table. Let us rewrite our batch handler to save the parsed `Transfer` events data to the database:

```typescript title="src/main.ts" theme={"system"}
import {TypeormDatabase} from '@subsquid/typeorm-store'
import {processor, CONTRACT_ADDRESS} from './processor'
import * as bayc from './abi/bayc'
import {Transfer} from './model'

processor.run(new TypeormDatabase(), async (ctx) => {
    let transfers: Transfer[] = []

    for (let block of ctx.blocks) {
        for (let log of block.logs) {
            if (log.address === CONTRACT_ADDRESS && log.topics[0] === bayc.events.Transfer.topic) {
                let {from, to, tokenId} = bayc.events.Transfer.decode(log)
                transfers.push(new Transfer({
                    id: log.id,
                    tokenId,
                    from,
                    to,
                    timestamp: new Date(block.header.timestamp),
                    blockNumber: block.header.height,
                    txHash: log.transactionHash,
                }))
            }
        }
    }

    await ctx.store.insert(transfers)
})
```

Note a few things here:

* A unique event log ID is available at `log.id` - no need to generate your own!
* `tokenId` returned from the decoder is an `ethers.BigNumber`, so it has to be explicitly converted to `number`. The conversion is valid only because we know that BAYC NFT IDs run from 0 to 9999; in most cases we would use `BigInt` for the entity field type and convert with `tokenId.toBigInt()`.
* `block.header` contains block metadata that we use to fill the extra fields.
* Accumulating the `Transfer` entity instances before using `ctx.store.insert()` on the whole array of them in the end allows us to get away with just one database transaction per batch. This is [crucial for achieving a good syncing performance](/en/sdk/squid-sdk/resources/batch-processing).

At this point we have a squid that indexes the data on BAYC token transfers and is capable of serving it over a GraphQL API. Full code is available at [this commit](https://github.com/subsquid-labs/bayc-squid-1/tree/aeb6268168385cc605ce04fe09d0159f708efe47). Test it by running

```bash theme={"system"}
npm run build
npx squid-typeorm-migration apply
node -r dotenv/config lib/main.js
```

then

```bash theme={"system"}
npx squid-graphql-server
```

in a separate terminal. If all is well, a GraphiQL playground should become available at [localhost:4350/graphql](http://localhost:4350/graphql):


# Step 3: External data
Source: https://docs.sqd.dev/en/sdk/squid-sdk/tutorials/bayc/step-three-adding-external-data

BAYC indexer tutorial step 3: enrich on-chain data with state calls, IPFS metadata, and HTTP requests using Squid SDK and the @subsquid packages.

# Step 3: Adding external data

This is the third part of the tutorial in which we build a squid that indexes [Bored Ape Yacht Club](https://boredapeyachtclub.com) NFTs, their transfers and owners from the [Ethereum blockchain](https://ethereum.org), fetches the metadata from [IPFS](https://ipfs.tech/) and regular HTTP URLs, stores all the data in a database and serves it over a GraphQL API. In the first two parts of the tutorial ([1](/en/sdk/squid-sdk/tutorials/bayc/step-one-indexing-transfers), [2](/en/sdk/squid-sdk/tutorials/bayc/step-two-deriving-owners-and-tokens)) we created a squid that scraped `Transfer` events emitted by the [BAYC token contract](https://etherscan.io/address/0xbc4ca0eda7647a8ab7c2061c2e118a18a936f13d) and derived some information on tokens and their owners from that data. In this part we enrich token data with information obtained from contract state calls, IPFS and regular HTTP URLs.

Prerequisites: Node.js, [Squid CLI](/en/sdk/squid-sdk/squid-cli/installation), Docker, a project folder with the code from the second part ([this commit](https://github.com/subsquid-labs/bayc-squid-1/tree/3568f70b7ac1802fbbf80079e435a3348b0d3a62)).

## Exploring token metadata

Now that we have a record for each BAYC NFT, let's explore how we can retrieve more data for each token.

[EIP-721](https://eips.ethereum.org/EIPS/eip-721) suggests that token metadata contracts may make token data available in a JSON referred to by the output of the `tokenURI()` contract function. Upon examining `src/abi/bayc.ts`, we find that the BAYC token contract implements this function. Also, the public ABI has no obvious contract methods that may set token URI or events that may be emitted on its change. In other words, it appears that the only way to retrieve this data is by [querying the contract state](/en/sdk/squid-sdk/resources/tools/typegen/state-queries).

This requires a RPC endpoint of an archival Ethereum node, but we do not need to add one here: processor will reuse the endpoint we [supplied in part one](/en/sdk/squid-sdk/tutorials/bayc/step-one-indexing-transfers#configuring-the-data-filters) of the tutorial for use in [RPC ingestion](/en/sdk/squid-sdk/resources/unfinalized-blocks).

The next step is to prepare for retrieving and parsing the metadata proper. For this, we need to understand the protocols used in the URIs and the structure of metadata JSONs. To learn that, we retrieve and inspect some URIs ahead of the main squid sync. The most straightforward way to achieve this is by adding the following to the batch handler:

```diff title="src/main.ts" theme={"system"}
processor.run(new TypeormDatabase(), async (ctx) => {
     let tokens: Map<string, Token> = createTokens(rawTransfers, owners)
     let transfers: Transfer[] = createTransfers(rawTransfers, owners, tokens)
+
+    let lastBatchBlockHeader = ctx.blocks[ctx.blocks.length-1].header
+    let contract = new bayc.Contract(ctx, lastBatchBlockHeader, CONTRACT_ADDRESS)
+    for (let t of tokens.values()) {
+        const uri = await contract.tokenURI(t.tokenId)
+        ctx.log.info(`Token ${t.id} has metadata at "${uri}"`)
+    }

     await ctx.store.upsert([...owners.values()])
     await ctx.store.upsert([...tokens.values()])
     await ctx.store.insert(transfers)
})
```

Here, we utilize an instance of the `Contract` class provided by the `src/abi/bayc.ts` module. It uses an RPC endpoint supplied by the processor via `ctx` to call methods of contract `CONTRACT_ADDRESS` at the height corresponding to the last block of each batch. Once we have the `Contract` instance, we call `tokenURI()` for each token mentioned in the batch and print the retrieved URI.

This simple approach is rather slow: the modified squid needs about three to eight hours to get a reasonably sized sample of URIs (figure out of date). A faster yet more complex [alternative](/en/sdk/squid-sdk/tutorials/bayc/step-four-optimizations#extra-using-multicall-for-metadata-exploration) will be discussed in the next part of the tutorial.

Running the modified squid reveals that some metadata URIs point to HTTPS and some point to IPFS. Here is [one](<https://us-central1-bayc-metadata.cloudfunctions.net/api/tokens/ 0>) of the metadata JSONs:

```json theme={"system"}
{
    "image": "https://ipfs.io/ipfs/QmRRPWG96cmgTn2qSzjwr2qvfNEuhunv6FNeMFGa9bx6mQ",
    "attributes": [
        {
            "trait_type": "Fur",
            "value": "Robot"
        },
        {
            "trait_type": "Eyes",
            "value": "X Eyes"
        },
        {
            "trait_type": "Background",
            "value": "Orange"
        },
        {
            "trait_type": "Earring",
            "value": "Silver Hoop"
        },
        {
            "trait_type": "Mouth",
            "value": "Discomfort"
        },
        {
            "trait_type": "Clothes",
            "value": "Striped Tee"
        }
    ]
}
```

Note how it does not conform to the [ERC721 Metadata JSON Schema](https://eips.ethereum.org/EIPS/eip-721).

Summary of our findings:

* BAYC metadata URIs can point to HTTPS or IPFS -- we need to be able to retrieve both.
* Metadata JSONs have two fields: `"image"`, a string, and `"attributes"`, an array of pairs `{"trait_type": string, "value": string}`.

Once finished, roll back the exploratory code:

```diff theme={"system"}
processor.run(new TypeormDatabase(), async (ctx) => {
     let tokens: Map<string, Token> = createTokens(rawTransfers, owners)
     let transfers: Transfer[] = createTransfers(rawTransfers, owners, tokens)
-
-    let lastBatchBlockHeader = ctx.blocks[ctx.blocks.length-1].header
-    let contract = new bayc.Contract(ctx, lastBatchBlockHeader, CONTRACT_ADDRESS)
-    for (let t of tokens.values()) {
-        const uri = await contract.tokenURI(t.tokenId)
-        ctx.log.info(`Token ${t.id} has metadata at "${uri}"`)
-    }
```

## Extending the `Token` entity

We will save both `image` and `attributes` metadata fields and the metadata URI to the database. To do this, we need to add some new fields to the existing `Token` entity:

```diff theme={"system"}
 type Token @entity {
     id: ID! # string form of tokenId
     tokenId: Int!
     owner: Owner!
+    uri: String!
+    image: String
+    attributes: [Attribute!]
     transfers: [Transfer!]! @derivedFrom(field: "token")
 }
+
+type Attribute {
+    traitType: String!
+    value: String!
+}
```

Here, `Attribute` is a [non-entity type](/en/sdk/squid-sdk/reference/schema-file/unions-and-typed-json#typed-json) that we use to type the `attributes` field.

Once `schema.graphql` is updated, we regenerate the TypeORM data model code::

```bash theme={"system"}
npx squid-typeorm-codegen
```

To populate the new fields, let us add an extra step at the end of `createTokens()`:

```typescript theme={"system"}
async function createTokens(
    ctx: Context,
    rawTransfers: RawTransfer[],
    owners: Map<string, Owner>
): Promise<Map<string, Token>> {

    let tokens: Map<string, PartialToken> = new Map()
    for (let t of rawTransfers) {
        let tokenIdString = `${t.tokenId}`
        let ptoken: PartialToken = {
            id: tokenIdString,
            tokenId: t.tokenId,
            owner: owners.get(t.to)!
        }
        tokens.set(tokenIdString, ptoken)
    }
    return await completeTokens(ctx, tokens)
}

interface PartialToken {
    id: string
    tokenId: bigint
    owner: Owner
}
```

Here, `PartialToken` stores the incomplete `Token` information obtained purely from blockchain events and function calls, before any [state queries](/en/sdk/squid-sdk/resources/tools/typegen/state-queries) or enrichment with [external data](/en/sdk/squid-sdk/resources/external-api).
The function `completeTokens()` is responsible for filling `Token` fields that are missing in `PartialToken`s. This involves IO operations, so both the function and its caller `createTokens()` have to be asynchronous. The functions also require a batch context for state queries and logging. We modify the `createTokens()` call in the batch handler to accommodate these changes:

```diff theme={"system"}
 processor.run(new TypeormDatabase(), async (ctx) => {
     let rawTransfers: RawTransfer[] = getRawTransfers(ctx)
 
     let owners: Map<string, Owner> = createOwners(rawTransfers)
-    let tokens: Map<string, Token> = createTokens(rawTransfers, owners)
+    let tokens: Map<string, Token> = await createTokens(ctx, rawTransfers, owners)
     let transfers: Transfer[] = createTransfers(rawTransfers, owners, tokens)
```

Next, we implement `completeTokens()`:

```typescript theme={"system"}
async function completeTokens(
    ctx: Context,
    partialTokens: Map<string, PartialToken>
): Promise<Map<string, Token>> {

    let tokens: Map<string, Token> = new Map()
    if (partialTokens.size === 0) return tokens

    let lastBatchBlockHeader = ctx.blocks[ctx.blocks.length-1].header
    let contract = new bayc.Contract(ctx, lastBatchBlockHeader, CONTRACT_ADDRESS)

    for (let [id, ptoken] of partialTokens) {
        let uri = await contract.tokenURI(ptoken.tokenId)
        ctx.log.info(`Retrieved metadata URI ${uri}`)
        let metadata: TokenMetadata | undefined = await fetchTokenMetadata(ctx, uri)
        tokens.set(id, new Token({
            ...ptoken,
            uri,
            ...metadata
        }))
    }

    return tokens
}
```

URI retrieval here is similar to what we did in the exploration step: we create a `Contract` object and use it to call the `tokenURI()` method of the BAYC token contract. The retrieved URIs are then used by the `fetchTokenMetadata()` function, which is responsible for HTTPS/IPFS metadata retrieval and parsing. Once we have its output, we can create and return the final `Token` entity instances.

## Retrieving external resources

In the `fetchTokenMetadata()` implementation we first classify the URIs depending on the protocol. For IPFS links we replace `'ipfs://'` with an address of an IPFS gateway, then retrieve the metadata from all links using a regular HTTPS client. Here for the demonstration purposes we use the public [ipfs.io](https://ipfs.io) gateway, which is slow and prone to dropping requests due to rate-limiting. For production squids we recommend using a dedicated gateway, e.g. from [Filebase](https://filebase.com/dedicated-gateways/).

```typescript theme={"system"}
export async function fetchTokenMetadata(
  ctx: Context,
  uri: string
): Promise<TokenMetadata | undefined> {

    try {
        if (uri.startsWith('ipfs://')) {
            const gatewayURL = path.posix.join(IPFS_GATEWAY, ipfsRegExp.exec(uri)![1])
            let res = await client.get(gatewayURL)
            ctx.log.info(`Successfully fetched metadata from ${gatewayURL}`)
            return res.data
        } else if (uri.startsWith('http://') || uri.startsWith('https://')) {
            let res = await client.get(uri)
            ctx.log.info(`Successfully fetched metadata from ${uri}`)
            return res.data
        } else {
            ctx.log.warn(`Unexpected metadata URL protocol: ${uri}`)
            return undefined
        }
    } catch (e) {
        throw new Error(`Failed to fetch metadata at ${uri}. Error: ${e}`)
    }
}

const ipfsRegExp = /^ipfs:\/\/(.+)$/
```

We use [Axios](https://axios-http.com) for HTTPS retrieval. Install it with

```bash theme={"system"}
npm i axios
```

To avoid reinitializing the HTTPS client every time we call the function we bind it to a module-scope constant:

```typescript theme={"system"}
const client = axios.create({
    headers: {'Content-Type': 'application/json'},
    httpsAgent: new https.Agent({keepAlive: true}),
    transformResponse(res: string): TokenMetadata {
        let data: {image: string; attributes: {trait_type: string; value: string}[]} = JSON.parse(res)
        return {
            image: data.image,
            attributes: data.attributes.map((a) => new Attribute({traitType: a.trait_type, value: a.value})),
        }
    },
})
```

We move all the code related to metadata retrieval to a separate module `src/metadata.ts`. Examine its full contents [here](https://github.com/subsquid-labs/bayc-squid-1/blob/ab5f094ae34e8822dfb912f6e6116df2cfa800b5/src/metadata.ts).

Then all that is left is to import the relevant parts in `src/main.ts`:

```diff title="src/main.ts" theme={"system"}
+import {TokenMetadata, fetchTokenMetadata} from './metadata'
```

and we are done with the processor code for this part of the tutorial. Full squid code at this point is available at [this commit](https://github.com/subsquid-labs/bayc-squid-1/tree/ab5f094ae34e8822dfb912f6e6116df2cfa800b5).

Recreate the database, rebuild the code and refresh the migrations with

```bash theme={"system"}
docker compose down
docker compose up -d
npm run build
rm -r db/migrations
npx squid-typeorm-migration generate
npx squid-typeorm-migration apply
```

and test the processor by running

```bash theme={"system"}
node -r dotenv/config lib/main.js
```

It runs much slower than before, requiring about three hours to get through the first batch and more than a day to sync (figures out of date). This is something we will address in the next part of the tutorial.

Nevertheless, the squid is already fully capable of scraping token metadata and serving it over GraphQL. Verify that by running `npx squid-graphql-server` and visiting the [local GraphiQL playground](http://localhost:4350/graphql). It is now possible to retrieve image URLs and attributes for each token:


# Step 2: Owners & tokens
Source: https://docs.sqd.dev/en/sdk/squid-sdk/tutorials/bayc/step-two-deriving-owners-and-tokens

BAYC indexer tutorial step 2: derive Owner and Token entities from BAYC Transfer events with Squid SDK and persist them in Postgres via TypeORM.

# Step 2: Deriving owners and tokens

This is the second part of the tutorial in which we build a squid that indexes [Bored Ape Yacht Club](https://boredapeyachtclub.com) NFTs, their transfers, and owners from the [Ethereum blockchain](https://ethereum.org), fetches the metadata from [IPFS](https://ipfs.tech/) and regular HTTP URLs, stores all the data in a database, and serves it over a GraphQL API. In the [first part](/en/sdk/squid-sdk/tutorials/bayc/step-one-indexing-transfers) we created a simple squid that scraped Transfer events emitted by the [BAYC token contract](https://etherscan.io/address/0xbc4ca0eda7647a8ab7c2061c2e118a18a936f13d). Here, we go a step further and derive separate entities for the NFTs and their owners from the transfers. The new entities will reference the corresponding `Transfer` entities. It will be automatically translated into primary key-foreign key references in the new database schema, and enable efficient cross-entity GraphQL queries.

Prerequisites: Node.js, [Squid CLI](/en/sdk/squid-sdk/squid-cli/installation), Docker, a project folder with the code from the first part ([this commit](https://github.com/subsquid-labs/bayc-squid-1/tree/aeb6268168385cc605ce04fe09d0159f708efe47)).

## Writing `schema.graphql`

Start the process by adding new [entities](/en/sdk/squid-sdk/reference/schema-file/entities) to the `schema.graphql` file:

```graphql theme={"system"}
# any unique string can be used as id
type Owner @entity {
    id: ID! # owner address
}

type Token @entity {
    id: ID! # string form of tokenId
    tokenId: BigInt!
}
```

Next, add [entity relations](/en/sdk/squid-sdk/reference/schema-file/entity-relations). Let us begin with adding a simple relation linking tokens to their owners:

```diff theme={"system"}
 type Token @entity {
     id: ID! # string form of tokenId
     tokenId: BigInt!
+    owner: Owner!
 }
```

Now, `Token` is considered an *owning entity* in relation to `Owner`. As a result,

* On the database side: the `token` table that maps to the `Token` entity gains a foreign key column `owner_id` holding primary keys of the `owner` table. The column is automatically indexed - no need to add `@index`.
* On the Typeorm side: the `Token` entity gains an `owner` field decorated with [`@ManyToOne`](https://github.com/typeorm/typeorm/blob/master/docs/many-to-one-one-to-many-relations.md). To create a well-formed `Token` entity instance in processor code, we now will have to first get a hold of an appropriate `Owner` entity instance and populate the `owner` field of a new `Token` with a reference to it:
  ```typescript theme={"system"}
  let newOwner: Owner = new Owner({
      id: '0xd8dA6BF26964aF9D7eEd9e03E53415D37aA96045'
  })
  let newToken: Token = new Token({
      id: '1',
      tokenId: 1,
      owner: newOwner // the whole entity instance
  })
  ```
* On the GraphQL side: queries to `token` can now select `owner` and any of its subfields (`id` is the only one available now).

Introduce more entity relations by replacing the `from`, `to` and `tokenId` fields of the `Transfer` entity with fields from the new entity types:

```diff theme={"system"}
 type Transfer @entity {
     id: ID!
-    tokenId: BigInt! @index
-    from: String! @index
-    to: String! @index
+    token: Token!
+    from: Owner!
+    to: Owner!
     timestamp: DateTime!
     blockNumber: Int!
     txHash: String! @index
 }
```

Lastly, include the virtual (i.e., not mapped to a column in the database schema) [reverse lookup fields](/en/sdk/squid-sdk/reference/schema-file/entity-relations):

```diff theme={"system"}
 type Owner @entity {
     id: ID! # owner address
+    ownedTokens: [Token!]! @derivedFrom(field: "owner")
 }

 type Token @entity {
     id: ID! # string form of tokenId
     tokenId: BigInt!
+    transfers: [Transfer!]! @derivedFrom(field: "token")
 }
```

This addition doesn't create any new database columns, but it makes the `ownedTokens` and `transfers` fields accessible through GraphQL and Typeorm.

You can find the final version of schema.graphql [here](https://github.com/subsquid-labs/bayc-squid-1/blob/6a148b1345f6c26acc2678e10b60aadcd4bbbcee/schema.graphql). Once you're finished, regenerate the Typeorm entity code with the following command:

```bash theme={"system"}
npx squid-typeorm-codegen
```

We also need to regenerate the database migrations to match the new schema. However, we'll postpone this step for now: it requires recompiling the squid code and it is not possible until we fix the entity creation code.

## Creating the entities

Note how the entities we define form an acyclic dependency graph:

* `Owner` entity instances can be made straight from the raw events data;
* `Token`s require the raw data plus the `Owner`s;
* `Transfer` entities require all the above.

As a consequence, the creation of entity instances must proceed in a [particular order](https://en.wikipedia.org/wiki/Topological_sorting). Squids usually use small graphs like this one, and in these the order can be easily found manually (e.g. `Owner`s then `Token`s then `Transfer`s in this case). We will assume that it can be hardcoded by the programmer.

Further, at each step we will [process the data for the whole batch](/en/sdk/squid-sdk/resources/batch-processing) instead of handling the items individually. This is crucial for achieving a good syncing performance.

With all that in mind, let's create a batch processor that generates and persists all of our entities:

```typescript title="src/main.ts" theme={"system"}
import {Owner, Token} from './model'

processor.run(new TypeormDatabase(), async (ctx) => {
    let rawTransfers: RawTransfer[] = getRawTransfers(ctx)

    let owners: Map<string, Owner> = createOwners(rawTransfers)
    let tokens: Map<string, Token> = createTokens(rawTransfers, owners)
    let transfers: Transfer[] = createTransfers(rawTransfers, owners, tokens)

    await ctx.store.upsert([...owners.values()])
    await ctx.store.upsert([...tokens.values()])
    await ctx.store.insert(transfers)
})
```

where

```typescript theme={"system"}
interface RawTransfer {
    id: string
    tokenId: bigint
    from: string
    to: string
    timestamp: Date
    blockNumber: number
    txHash: string
}
```

is an interface very similar to that of the `Transfer` entity as it was at the beginning of this part of the tutorial. This allows us to reuse most of the code of the old batch handler in `getRawTransfers()`:

```typescript title="src/main.ts" theme={"system"}
import {Context} from './processor'

function getRawTransfers(ctx: Context): RawTransfer[] {
    let transfers: RawTransfer[] = []

    for (let block of ctx.blocks) {
        for (let log of block.logs) {
            if (log.address === CONTRACT_ADDRESS && log.topics[0] === bayc.events.Transfer.topic) {
                let {from, to, tokenId} = bayc.events.Transfer.decode(log)
                transfers.push({
                    id: log.id,
                    tokenId,
                    from,
                    to,
                    timestamp: new Date(block.header.timestamp),
                    blockNumber: block.header.height,
                    txHash: log.transactionHash,
                })
            }
        }
    }

    return transfers
}
```

The next step involves creating `Owner` entity instances. We will need these to create both `Tokens` and `Transfers`. In both scenarios, we'll have the IDs of the owners (i.e., their addresses) prepared. To simplify future lookups, we choose to return the `Owner` instances as a `Map<string, Owner>`:

```typescript title="src/main.ts" theme={"system"}
function createOwners(rawTransfers: RawTransfer[]): Map<string, Owner> {
    let owners: Map<string, Owner> = new Map()
    for (let t of rawTransfers) {
        owners.set(t.from, new Owner({id: t.from}))
        owners.set(t.to, new Owner({id: t.to}))
    }
    return owners
}
```

Similarly, `Token` instances will also need to be looked up later, so we return them as a `Map<string, Token>`. To identify the most recent owner of each token, we traverse all the transfers in the order they appear on the blockchain and assign the owner of any involved tokens to their recipient:

```typescript title="src/main.ts" theme={"system"}
function createTokens(
    rawTransfers: RawTransfer[],
    owners: Map<string, Owner>
): Map<string, Token> {

    let tokens: Map<string, Token> = new Map()
    for (let t of rawTransfers) {
        let tokenIdString = `${t.tokenId}`
        tokens.set(tokenIdString, new Token({
            id: tokenIdString,
            tokenId: t.tokenId,
            owner: owners.get(t.to)
        }))
    }
    return tokens
}
```

Some `Token` and `Owner` instances might have been created in previous batches, so we use `ctx.store.upsert()` to store these instances while updating any existing ones.

<Info>
  In some circumstances we might have had to retrieve the old entity instances from the database before updating, but here we have all the required fields populated, so we simply overwrite the whole entity with `ctx.store.upsert()`.
</Info>

Finally, we create an array of `Transfer` entity instances through a simple mapping:

```typescript title="src/main.ts" theme={"system"}
function createTransfers(
    rawTransfers: RawTransfer[],
    owners: Map<string, Owner>,
    tokens: Map<string, Token>
): Transfer[] {

    return rawTransfers.map(t => new Transfer({
        id: t.id,
        token: tokens.get(`${t.tokenId}`),
        from: owners.get(t.from),
        to: owners.get(t.to),
        timestamp: t.timestamp,
        blockNumber: t.blockNumber,
        txHash: t.txHash
    }))
}
```

Since `Transfer`s are unique, we can safely use `ctx.store.insert()` to persist them.

At this point, the squid has accomplished everything planned for this part of the tutorial. The only remaining task is to drop and recreate the database (if it's running), then regenerate and apply the migrations:

```bash theme={"system"}
npm run build
docker compose down
docker compose up -d
rm -r db/migrations
npx squid-typeorm-migration generate
npx squid-typeorm-migration apply
```

Full code can be found at [this commit](https://github.com/subsquid-labs/bayc-squid-1/tree/3568f70b7ac1802fbbf80079e435a3348b0d3a62).

To test it, start the processor and the GraphQL server by running

```bash theme={"system"}
node -r dotenv/config lib/main.js
```

and

```bash theme={"system"}
npx squid-graphql-server
```

in separate terminals. Then, visit the [GraphiQL playground](http://localhost:4350/graphql):

The new entities should be displayed in the query schema.

Thanks to the added entity relations, we can now execute more complex nested queries. For example, the one displayed in the screenshot selects a transfer, retrieves its token, looks up its owner, and finds out which tokens are currently owned by them.


# Squid SDK Case Studies
Source: https://docs.sqd.dev/en/sdk/squid-sdk/tutorials/case-studies

Deep dives into larger Squid SDK projects — multichain indexers, NFT analytics, governance dashboards, and DeFi data platforms built on SQD.

Follow the links from the [SQD Medium blog](https://medium.com/subsquid) for deep dives into the development process of larger projects built from scratch. They use the FireSquid version of the framework, so much of the involved code is outdated. Still, the general approaches they illustrate endure.

* [DeFi Dashboard](https://medium.com/subsquid/build-your-first-defi-dashboard-ad3ce1e9fc73). A React app for a real-time dashboard showing key statistics of the Moonwell lending protocol
* [Rave name service indexing on Fantom](https://medium.com/subsquid/building-a-fast-and-scalable-web3-api-on-fantom-blockchain-94c79933b55).
* [Indexing Uniswap data into parquets](https://medium.com/subsquid/how-to-scale-blockchain-data-science-for-large-datasets-b49d078c15eb). A step-by-step guide on how to extract, decode and analyze Uniswap trading data with SQD, Pandas and a Python notebook.
* [Analyzing Lens Protocol](https://medium.com/subsquid/how-to-analyze-lens-protocol-activity-data-with-subsquid-e1ee3b7b43fa). An end-to-end tutorial on how to extract and analyze the social data of the Lens protocol. Includes an extra step on how to build a UI for the dashboards.


# Use with Ganache or Hardhat
Source: https://docs.sqd.dev/en/sdk/squid-sdk/tutorials/evm-local

Use Squid SDK with Ganache or Hardhat — index a local Ethereum dev node from your own RPC endpoint without involving the SQD Network or Portal.

# Run squids with Ganache or Hardhat

## Objective

Here we show how it is possible to index an Ethereum development node running locally. This is useful for those who want to start developing a squid ETL or API in a local environment without waiting for the contract to be deployed to a testnet.

## Pre-requisites

* Either one of [Ganache](https://trufflesuite.com/ganache/) or [Hardhat](https://hardhat.org/)
* [Squid CLI](/en/sdk/squid-sdk/squid-cli/installation)
* Docker

## Setup

In this tutorial we will be using a layout in which the squid and the Hardhat/Ganache setup share the same folder. Create a new squid by running:

```bash theme={"system"}
sqd init ethereum-local-indexing -t evm
```

Here, `ethereum-local-indexing` is the project name and `evm` refers to the [`evm` template](https://github.com/subsquid-labs/squid-evm-template).

Install the dependencies and update `evm-processor` and `typeorm-store` to their ArrowSquid versions:

```bash theme={"system"}
cd ethereum-local-indexing
npm i
npm i @subsquid/evm-processor@next @subsquid/typeorm-store@next
```

Next, we set up a local EVM development environment. We consider two common options: [Hardhat](#hardhat) and [Ganache](#ganache).

## If you chose Hardhat

[Install the package](https://hardhat.org/hardhat-runner/docs/getting-started#installation).

#### 1. Create project

In a terminal, navigate to the squid project root folder. Prepare it by running

```bash theme={"system"}
rm README.md
mv tsconfig.json tsconfig.json.save
```

This will prevent collisions between SQD and Hardhat files.

Next, run

```bash theme={"system"}
npx hardhat
```

to initialize the Hardhat project. When prompted, choose **TypeScript project** and keep all other options at defaults. Finally, overwrite `tsconfig.json` provided by Hardhat:

```bash theme={"system"}
mv tsconfig.json.save tsconfig.json
```

#### 2. Configure Hardhat automining

Then, open the `hardhat.config.ts` and replace the `HardhatUserConfig` object with this one:

```typescript theme={"system"}
const config: HardhatUserConfig = {
  defaultNetwork: "hardhat",
  networks: {
    hardhat: {
      chainId: 1337,
      mining: {
        auto: false,
        interval: [4800, 5200]
      }
    },
  },
  solidity: "0.8.17",
};
```

The `mining` configuration will continuously mine blocks, even if no events or transactions are executed. This can be useful for debugging. You can read more about it on [hardhat official documentation](https://hardhat.org/hardhat-network/docs/reference#mining-modes).

#### 3. Sample contract

There should be a `contracts` subfolder in the project folder now, with a sample contract named `Lock.sol`. To compile this contract and verify its correctness, run

```bash theme={"system"}
npx hardhat compile
```

You should find the contract's ABI at `artifacts/contracts/Lock.sol/Lock.json`. **We will use this file in squid development.**

#### 4. Launch hardhat node

From the project root folder, run

```bash theme={"system"}
npx hardhat node
```

#### 5. Deploy the contract

The node will keep the console window busy. In a different terminal, run this command:

```bash theme={"system"}
npx hardhat run scripts/deploy.ts --network localhost
```

If the contract deployed successfully, the output should look like

```
Lock with 1 ETH and unlock timestamp 1704810454 deployed to 0x5FbDB2315678afecb367f032d93F642f64180aa3
```

Take note of the contract address, **you'll need it later**.

## If you chose Ganache

Install [Truffle](https://trufflesuite.com/docs/truffle/how-to/install/) and [Ganache](https://trufflesuite.com/docs/ganache/quickstart/#1-install-ganache) packages.

#### 1. Truffle project, sample contract

Let's create a new truffle project with a sample contract. In the project main folder run

```bash theme={"system"}
truffle unbox metacoin
```

Compile the contracts with

```bash theme={"system"}
truffle compile
```

You should find the contract's ABI at `build/contracts/MetaCoin.json`. **It will be useful for indexing**.

#### 2. Create a workspace

Launch the Ganache tool and select the *New Workspace (Ethereum)* option.

Next, provide a name for the workspace and link the Truffle project we just created to it. To do that, click **Add project** and select the `truffle-config.js` file in the project root folder. Finally, select the `Server` tab at the top.

In this window, change the server configuration to the exact values reported in this image.

The [`AUTOMINE` option](https://trufflesuite.com/docs/ganache/reference/cli-options/#miner) is disabled to ease the debugging.

Finally, click "Start" to launch the blockchain emulator.

#### 3. Deploy a smart contract

Configure Truffle by uncommenting the `development` section in `truffle-config.js` and setting its properties as follows:

```javascript theme={"system"}
    development: {
      host: "127.0.0.1",     // Localhost (default: none)
      port: 8545,            // Standard Ethereum port (default: none)
      network_id: "1337",    // Any network (default: none)
    },
```

Deploy the sample smart contract by running

```bash theme={"system"}
truffle migrate
```

Log information similar to this should be displayed:

```bash theme={"system"}
   Deploying 'MetaCoin'
   --------------------
   > transaction hash:    0xcdb820827306ebad7c6905d750d07536c3db93f4ef76fd777180bdac16eaa2ca
   > Blocks: 1            Seconds: 4
   > contract address:    0xd095211a90268241D75919f12397b389b1062f6F
   > block number:        329
   > block timestamp:     1673277461
   > account:             0x131D37F433BAf649111278c1d6E59843fFB26D28
   > balance:             99.98855876
   > gas used:            414494 (0x6531e)
   > gas price:           20 gwei
   > value sent:          0 ETH
   > total cost:          0.00828988 ETH

   > Saving artifacts
   -------------------------------------
   > Total cost:          0.01144124 ETH
```

Take note of the contract address, **you'll need it later**.

## Squid development

The indexing setup is ready to use. To test it, replace the contents of `src/processor.ts` with the following.

```ts theme={"system"}
import {TypeormDatabase} from '@subsquid/typeorm-store'
import {EvmBatchProcessor} from '@subsquid/evm-processor'

const processor = new EvmBatchProcessor()
  .setRpcEndpoint('http://localhost:8545')
  .setFinalityConfirmation(5)
  .addTransaction({})
  .setFields({
    transaction: {
      contractAddress: true
    }
  })

processor.run(new TypeormDatabase(), async (ctx) => {
  for (let blk of ctx.blocks) {
    for (let txn of blk.transactions) {
      console.log(txn)
    }
  }
})
```

This defines a squid that retrieves all chain transactions from the local node RPC endpoint without filtering, making sure to retrieve the addresses of the deployed contracts for deployment transactions. Run the squid with

```bash theme={"system"}
docker compose up -d
npm run build
npx squid-typeorm-migration apply
node -r dotenv/config lib/main.js
```

You should see the data of one (for Hardhat) or two (for Truffle+Ganache) contract deployment transactions printed to your terminal.

Now you can develop a SQD-based indexer alongside your contracts. Head over to the [dedicated tutorial](/en/sdk/squid-sdk/tutorials/bayc) for guidance on squid development. Use the **contract's ABI** ([here](#3-sample-contract) or [here](#1-truffle-project-sample-contract)) and contract **address** ([here](#5-deploy-the-contract) and [here](#3-deploy-a-smart-contract)) from previous steps and be mindful that the data source of the processor class needs to be set to the local node RPC endpoint, as in the example above:

```typescript theme={"system"}
// ...
  .setRpcEndpoint('http://localhost:8545')
// ...
```

You can also set the data source through an environment variable like it is done in [this complete end-to-end project example](https://medium.com/subsquid/how-to-build-a-performant-and-scalable-full-stack-nft-marketplace-63c12466b959) (outdated, but still useful). This will make sure the project code stays the same and only the environment variables change depending on where the project is deployed.


# Index to local CSV files
Source: https://docs.sqd.dev/en/sdk/squid-sdk/tutorials/file-csv

Tutorial: index blockchain data into local CSV files for analytics with Squid SDK — schema definitions, batch flushing, and post-processing examples.

# Save indexed data to local CSV files

## Objective

This tutorial describes how to use the SQD indexing framework for saving processed blockchain data to local CSV files. The intent is to show how Squid SDK can be used for data analytics prototyping.

File-based data formats like CSV are convenient for data analysis, especially in the early prototyping stages. This convenience motivated SQD Team to develop some extensions to allow saving processed data to file-based storage.

We chose MATIC transactions on Ethereum mainnet for this example. This selection provides enough data to highlight the performance of the SQD framework and is interesting from a data analysis standpoint.

An article about this demo project [has been published on Medium](https://link.medium.com/1NPC1S2czxb). The project source code can be found in the [`local-csv-indexing` repository](https://github.com/subsquid-labs/local-csv-indexing).

<Warning>
  As of 2023-12-17, the `local-csv-indexing` repo is mostly still sound, but already somewhat outdated. You can take a look at the less sophisticated, yet regularly updated example [here](https://github.com/subsquid-labs/file-store-csv-example).
</Warning>

## Pre-requisites

* [Squid CLI](/en/sdk/squid-sdk/squid-cli/installation)
* (optional) Python

## Setup

Let's start by creating a new blockchain indexer, or "squid" in SQD terminology. In a terminal, launch this command:

```bash theme={"system"}
sqd init local-csv-indexing -t evm
```

Here, `local-csv-indexing` is the name of the project, and can be changed to anything else. The `-t evm` option specifies that the [`evm` template](https://github.com/subsquid-labs/squid-evm-template) should be used as a starting point.

<Note>
  The template actually has more than what we need for this project. Unnecessary packages have been removed in the tutorial repository. You can grab [`package.json`](https://github.com/subsquid-labs/local-csv-indexing/blob/main/package.json) from there to do the same.

  Files-wise, `docker-compose.yml`, `schema.graphql` and `squid.yaml` were removed. [`commands.json`](/en/sdk/squid-sdk/squid-cli/commands-json), the list of local `sqd` scripts, has been significantly shortened ([here is the updated version](https://github.com/subsquid-labs/local-csv-indexing/blob/main/commands.json)).
</Note>

### ERC-20 token ABI

To be able to index transfers of a token, it's necessary to know the address and the ABI ([Application Binary Interface](https://docs.ethers.org/v5/api/utils/abi/)) of the token contract. The ABI defines the contract's functions and events, including their typed inputs and outputs.

Luckily, both of these can be found on block explorers like [Etherscan](https://medium.com/r/?url=https%3A%2F%2Fetherscan.io%2Ftoken%2F0x7d1afa7b718fb893db30a3abc0cfc608aacfebb0).

The indexer needs the ABI for locating the contract events or functions in the EVM execution trace and decoding their inputs. Squid SDK has a handy command to generate some boilerplate TypeScript code to achieve this:

```bash theme={"system"}
npx squid-evm-typegen src/abi 0x7D1AfA7B718fb893dB30A3aBc0Cfc608AaCfeBB0#matic
```

<Info>
  The typegen tool uses Etherscan API to fetch the contract ABI. Other compatible APIs are supported via the `--etherscan-api` flag. For example, if the contract was deployed to Polygon and its ABI was available from Polygonscan, it could still be fetched with the same command extended with `--etherscan-api https://api.polygonscan.io`.

  Alternatilvely, the same command can be used with a path (local or URL) to a JSON ABI in place of the contract address.
</Info>

This will generate some files under the `src/abi` folder, the most interesting of which is `matic.ts`.

### CSV, Tables and Databases

For writing local CSVs we will need the `file-store-csv` package:

```bash theme={"system"}
npm i @subsquid/file-store-csv
```

<Info>
  Packages are also available for [writing to parquet files](/en/sdk/squid-sdk/reference/store/file/parquet) and for [uploading to S3-compatible cloud services](/en/sdk/squid-sdk/reference/store/file/s3-dest).
</Info>

Next, let's create a new file at `src/tables.ts`. This is where it's possible to provide filenames for the CSV files, as well as configure their data structure, in much the same way as if they were a database table (the class name is no coincidence):

```typescript theme={"system"}
import {Table, Column, Types} from '@subsquid/file-store-csv'

export const Transfers = new Table(
  'transfers.csv',
  {
    blockNumber: Column(Types.Numeric()),
    timestamp: Column(Types.DateTime()),
    contractAddress: Column(Types.String()),
    from: Column(Types.String()),
    to: Column(Types.String()),
    amount: Column(Types.Numeric()),
  },
  {
    header: false,
  }
)
```

Let's create another file next, this time named `src/db.ts`, to configure the data abstraction layer. Here we export an instance of the [`Database` class](/en/sdk/squid-sdk/resources/persisting-data/overview) implementation from the `file-store` package (a dependency of `file-store-csv`). We will use this instance in much the same way as we would use a [`TypeormDatabase`](/en/sdk/squid-sdk/resources/persisting-data/typeorm) instance in a PostgreSQL-based squid.

```typescript theme={"system"}
import {Database, LocalDest, Store} from '@subsquid/file-store'
import { Transfers } from './tables'

export const db = new Database({
  tables: {
    Transfers,
  },
  dest: new LocalDest('./data'),
  chunkSizeMb: 100
})
```

<Info>
  Note the `chunkSizeMb` option. A new chunk (that is, a new folder with a new CSV file in it) will be written when either the amount of data stored in the processor buffer exceeds `chunkSizeMb`, or at the end of the batch during which [`ctx.store.setForceFlush()`](/en/sdk/squid-sdk/resources/persisting-data/file#setforceflush) was called.
</Info>

### Data indexing

All the indexing logic is defined in `src/processor.ts`, so let's open it and edit the `EvmBatchProcessor` class configuration. We should request data for the right smart contract and the right EVM event log:

```typescript theme={"system"}
export const contractAddress =
  '0x7D1AfA7B718fb893dB30A3aBc0Cfc608AaCfeBB0'.toLowerCase();

const processor = new EvmBatchProcessor()
  .setGateway('https://v2.archive.subsquid.io/network/ethereum-mainnet')
  .setRpcEndpoint({
    url: process.env.RPC_ENDPOINT,
    rateLimit: 10
  })
  .setFinalityConfirmation(75)
  .addLog({
    address: [contractAddress],
    topic0: [events.Transfer.topic]
  });
```

<Warning>
  Note: the RPC\_ENDPOINT environment variable is used, so make sure to edit the `.env` file and use a valid URL of an Ethereum RPC node, e.g.:

  ```bash theme={"system"}
  DB_NAME=squid
  DB_PORT=23798
  GQL_PORT=4350
  # JSON-RPC node endpoint, both wss and https endpoints are accepted
  RPC_ENDPOINT="<eth_rpc_endpoint_url>"
  ```
</Warning>

Let's define the logic to process a batch of EVM log data, and save it to CSV files.

A double loop is necessary to explore all the blocks in each batch and the logs in each block. The processor provides no guarantees that it won't provide any data not matching the filters, so it's necessary to check that the logs have been generated by the right address and that they have the right topic signature.

It's then possible to decode the events and prepare objects with the right data structure, which are then written to the `Transfers` CSV file.
Here is a short summary of the logic:

```typescript theme={"system"}
processor.run(db, async (ctx) => {
  let decimals = 18
  for (let block of ctx.blocks) {
    for (let log of block.logs) {
      if (log.address !== contractAddress) continue;
      if (log.topic0 !== events.Transfer.topic) continue;

      const { from, to, value } = events.Transfer.decode(log);

      ctx.store.Transfers.write(({
        blockNumber: block.header.height,
        timestamp: new Date(block.header.timestamp),
        contractAddress: log.address,
        from: from.toLowerCase(),
        to: to.toLowerCase(),
        amount: BigDecimal(value.toBigInt(), decimals).toNumber(),
      });
    }
  }
});
```

<Info>
  The file in the GitHub repository is slightly different, as there's some added logic to obtain the number of decimals for the token. For that, the processor [interacts with the smart contract deployed on chain](/en/sdk/squid-sdk/resources/tools/typegen/state-queries).
</Info>

### Launch the project

Build, then launch the project with

```bash theme={"system"}
npm run build
```

```bash theme={"system"}
node -r dotenv/config lib/main.js
```

And in a few minutes, a few sub-folders (whose names are the block ranges where the data is coming from) should be created under the `data` directory, each containing a `transfer.csv` file.

### Data analysis with Python

If you want to learn how to analyze this data using Python and Pandas, refer to [the Medium article dedicated to this demo project](https://link.medium.com/1NPC1S2czxb), or consult the [project's repository on GitHub](https://github.com/subsquid-labs/local-csv-indexing/local-csv-indexing).

## Conclusions

The purpose of this project was to demonstrate how to use SQD's indexing framework for data analytics prototyping: the indexer was able to ingest all this data, process it, and dump it to local CSV files in roughly 20 minutes (figure out of date). The simple Python script in the project's repository shows how to read multiple CSV files, and perform some data analysis with Pandas.

SQD Team seeks feedback on this new tool. If you want to share any thoughts or have any suggestions, feel free to reach out to us at [the SquidDevs Telegram channel](https://t.me/HydraDevs).


# Index to Parquet files
Source: https://docs.sqd.dev/en/sdk/squid-sdk/tutorials/file-parquet

Tutorial: index blockchain data into Apache Parquet files for analytics with Squid SDK — columnar schema, compression, and batch flushing examples.

# Save indexed data in Parquet files

## Objective

This tutorial describes how to use the SQD indexing framework to save the processed blockchain data to [Parquet files](https://parquet.apache.org/) instead of a database. The intent is to show how Squid SDK can be used for data analytics, this time with focus on tools suitable for larger datasets.

File-based data formats like [CSV](/en/sdk/squid-sdk/tutorials/file-csv) are convenient for data analysis, especially in the early prototyping stages. However, when working with large datasets the ability to read data files partially is a common requirement. This is rarely possible with CSV.

In contrast, the Parquet format is designed for efficient read/write operations without processing the entire file. To better serve data analysts' needs, the SQD Team developed a library for storing indexer data in this format.

The subject of this tutorial is the Uniswap DApp, namely the data from its pool contracts and positions held by investors. Uniswap was chosen because it generates a very large amount of information, and ultimately this helps to better show how to leverage a more performance-oriented format.

An article about this demo project [has been published on Medium](https://link.medium.com/7gU0BrLbDxb). The project source code can be found in the [`squid-parquet-storage` repo](https://github.com/subsquid-labs/squid-parquet-storage).

<Warning>
  As of 2023-12-17, the `squid-parquet-storage` repo is mostly still sound, but already somewhat outdated. You can take a look at the less sophisticated, yet regularly updated example [here](https://github.com/subsquid-labs/file-store-parquet-example/).
</Warning>

## Pre-requisites

* [Squid CLI](/en/sdk/squid-sdk/squid-cli/installation)
* (optional) Python

## Setup

Let's start by creating a new blockchain indexer, or a "squid" in SQD terminology. In a terminal, launch this command:

```bash theme={"system"}
sqd init local-parquet-indexing -t evm
```

Here, `local-parquet-indexing` is the name of the project, and can be changed to anything else. The `-t evm` option specifies that the [`evm` template](https://github.com/subsquid-labs/squid-evm-template) should be used as a starting point.

<Note>
  The template actually has more than what we need for this project. Unnecessary packages have been removed in the tutorial repository. You can grab [`package.json`](https://github.com/subsquid-labs/squid-parquet-storage/blob/main/package.json) from there to do the same.

  Files-wise, `docker-compose.yml`, `schema.graphql` and `squid.yaml` were removed. [`commands.json`](/en/sdk/squid-sdk/squid-cli/commands-json), the list of local `sqd` scripts, has been significantly shortened ([here is the updated version](https://github.com/subsquid-labs/squid-parquet-storage/blob/main/commands.json)).
</Note>

Finally, make sure to install the dependencies:

```bash theme={"system"}
npm i
```

### ERC-20 token ABI

Due to the size of this project most of the code will be explained rather than directly listed. The code of this section is no exception since this project indexes data from three kinds of Uniswap contracts, one of which is a factory that deploys new contracts. Additionally, it uses a Multicall contract.

For this project, you will need:

* The [Uniswap V3 Factory](https://etherscan.io/address/0x1f98431c8ad98523631ae4a59f267346ea31f984#code) smart contract address and ABI
* The ABI of [any one of the Uniswap V3 Pool](https://etherscan.io/address/0x390a4d096ba2cc450e73b3113f562be949127ceb#code) smart contracts deployed by the Factory
* The [Uniswap V3 Positions](https://etherscan.io/address/0xc36442b4a4522e871399cd717abdd847ab11fe88) smart contract address, and ABI
* The address of any deployed Maker DAO's Multicall smart contract. Luckily, [Uniswap have their own](https://etherscan.io/address/0x5ba1e12693dc8f9c48aad8770482f4739beed696)
* The ABI of an ERC-20 token (can be compiled from [OpenZeppelin repository](https://github.com/OpenZeppelin/openzeppelin-contracts/blob/master/contracts/token/ERC20/ERC20.sol), or downloaded, for example from [here](https://gist.github.com/veox/8800debbf56e24718f9f483e1e40c35c), or from the [repository of this article's project](https://github.com/subsquid-labs/squid-parquet-storage/tree/main/abi)). Save it as `abi/ERC20.json`.

<Note>
  The project also uses [`ERC20NameBytes`](https://github.com/subsquid-labs/squid-parquet-storage/blob/main/abi/ERC20NameBytes.json) and [`ERC20SymbolBytes`](https://github.com/subsquid-labs/squid-parquet-storage/blob/main/abi/ERC20SymbolBytes.json) ABIs, which OpenZeppelin defines in [`IERC20Metadata.sol`](https://github.com/OpenZeppelin/openzeppelin-contracts/blob/master/contracts/token/ERC20/extensions/IERC20Metadata.sol) and includes in [`ERC20.sol`](https://github.com/OpenZeppelin/openzeppelin-contracts/blob/master/contracts/token/ERC20/ERC20.sol). Save these files to `./abi`.
</Note>

<Note>
  You can find a pool and its ABI by inspecting the internal calls of one of the [Create Pool transactions of the Factory contract](https://etherscan.io/tx/0xd9e0fc1d737479e940d9807a535c75a31a2c8458792e2c66e2cc33f19f88e546#internal).
</Note>

Generate TypeScript code for them:

```bash theme={"system"}
npx squid-evm-typegen ./src/abi ./abi/*.json
npx squid-evm-typegen ./src/abi 0x1f98431c8ad98523631ae4a59f267346ea31f984#factory
npx squid-evm-typegen ./src/abi 0xc36442b4a4522e871399cd717abdd847ab11fe88#NonfungiblePositionManager
npx squid-evm-typegen ./src/abi 0x390a4d096ba2cc450e73b3113f562be949127ceb#pool --multicall
```

This should create a few files in the `src/abi` folder for you. The last command also automatically generates a Typescript ABI for the Multicall contract due to the `--multicall` flag.

### Tables and Databases

The `@subsquid/file-store` library defines the `Table` and `Database` classes.

The `Database` class gets its name from the interface that was originally developed to [access an actual database](/en/sdk/squid-sdk/resources/persisting-data/typeorm). Here, the interface is [used without modification](/en/sdk/squid-sdk/resources/persisting-data/overview) in a class designed to access a filesystem. `Table`s play a similar role to that of tables of an actual database: they represent collections of rows, all of which share same set of fields/columns. Each such data structure requires one or more data files to store it in both CSV and Parquet, hence the mapping of `Table`s to files.

To summarize, `Table` instances are used to define data files along with their schemas and hold file-specific settings. `Database` facilitates the interactions with the processor, coordinates writing to the files and maintains any state that facilitates that process (configuration, cloud connections and so on).

There are two main differences from the [CSV tutorial](/en/sdk/squid-sdk/tutorials/file-csv) and the first one is that for this project we will be using a `Table` implementation from `@subsquid/file-store-parquet`. Let's install it:

```bash theme={"system"}
npm i @subsquid/file-store-parquet
```

The other one is that this project is more involved and is, in fact, using ten different tables instead of one.

It's advisable to define these tables in a separate file. The original project has them under `src/tables.ts`. The syntax is pretty much the same as for the CSV tables, except now the `Table`, `Column`, and `Types` classes are imported from the `@subsquid/file-store-parquet` library and the set of available types is different. We will also configure the parquet file compression using the new `Compression` class.

Here's a snippet:

```typescript theme={"system"}
import {Table, Column, Compression, Types} from '@subsquid/file-store-parquet'

export const Tokens = new Table(
    'tokens.parquet',
    {
        blockNumber: Column(Types.Uint32()),
        timestamp: Column(Types.Timestamp()),
        contractAddress: Column(Types.String()),
        symbol: Column(Types.String()),
        name: Column(Types.String()),
        totalSupply: Column(Types.Uint64()),
        decimals: Column(Types.Uint16()),
    },
    {
        compression: Compression.ZSTD,
    }
)
```

The rest of the tables definitions can be found [here](https://github.com/subsquid-labs/squid-parquet-storage/blob/main/src/tables.ts).

Similarly, a `src/db.ts` file should be created to configure the `Database` class. Here we specify the tables used, as well as the destination and the size of the chunks in which the data is going to be split.

Here are the contents of this file in full:

```typescript theme={"system"}
import assert from 'assert'
import {Database, LocalDest, Store} from '@subsquid/file-store'
import {
    FactoryPairCreated,
    PoolBurn,
    PoolInitialize,
    PoolMint,
    PoolSwap,
    PositionCollect,
    PositionDecreaseLiquidity,
    PositionIncreaseLiquidity,
    PositionTransfer,
    Tokens,
} from './tables'
import {PoolsRegistry} from './utils'
import {S3Dest} from '@subsquid/file-store-s3'

type Metadata = {
    height: number
    pools: string[]
}

export const db = new Database({
    tables: {
        Tokens,
        FactoryPairCreated,
        PoolInitialize,
        PoolMint,
        PoolBurn,
        PoolSwap,
        PositionCollect,
        PositionDecreaseLiquidity,
        PositionIncreaseLiquidity,
        PositionTransfer,
    },
    dest: process.env.DEST === 'S3' ? new S3Dest('./uniswap', 'csv-store') : new LocalDest('./data'),
    hooks: {
        async onConnect(dest) {
            if (await dest.exists('status.json')) {
                let {height, pools}: Metadata = await dest.readFile('status.json').then(JSON.parse)
                assert(Number.isSafeInteger(height))

                let registry = PoolsRegistry.getRegistry()
                for (let pool of pools) {
                    registry.add(pool)
                }

                return height
            } else {
                return -1
            }
        },
        async onFlush(dest, range) {
            let metadata: Metadata = {
                height: range.to,
                pools: PoolsRegistry.getRegistry().values(),
            }
            await dest.writeFile('status.json', JSON.stringify(metadata))
        },
    },
    chunkSizeMb: 50,
})

export type Store_ = typeof db extends Database<infer R, any> ? Store<R> : never
```

<Note>
  The `chunkSizeMb` option defines the size (in MB) of a parquet file before it's saved on disk, and a new one is created.
</Note>

### Data indexing

The orchestration of the indexing logic is defined in the file named `src/processor.ts`:

```typescript theme={"system"}
import {EvmBatchProcessor} from '@subsquid/evm-processor'
import * as positionsAbi from './abi/NonfungiblePositionManager'
import * as factoryAbi from './abi/factory'
import * as poolAbi from './abi/pool'
import {db} from './db'
import {processFactory} from './mappings/factory'
import {processPools} from './mappings/pools'
import {FACTORY_ADDRESS, POSITIONS_ADDRESS} from './utils/constants'
import {processPositions} from './mappings/positions'

let processor = new EvmBatchProcessor()
    .setBlockRange({from: 12369621})
    .setGateway('https://v2.archive.subsquid.io/network/ethereum-mainnet')
    .setRpcEndpoint({
        url: process.env.ETH_CHAIN_NODE,
        rateLimit: 10
    })
    .setFinalityConfirmation(75)
    .addLog({
        address: [FACTORY_ADDRESS],
        topic0: [factoryAbi.events.PoolCreated.topic]
    })
    .addLog({
        topic0: [
            poolAbi.events.Burn.topic,
            poolAbi.events.Mint.topic,
            poolAbi.events.Initialize.topic,
            poolAbi.events.Swap.topic,
        ]
    })
    .addLog({
        address: [POSITIONS_ADDRESS],
        topic0: [
            positionsAbi.events.IncreaseLiquidity.topic,
            positionsAbi.events.DecreaseLiquidity.topic,
            positionsAbi.events.Collect.topic,
            positionsAbi.events.Transfer.topic,
        ]
    })

processor.run(db, async (ctx) => {
    await processFactory(ctx)
    await processPools(ctx)
    await processPositions(ctx)
})
```

Here's a brief explanation of the code above:

* The `EvmBatchProcessor` class is instantiated and set to connect to the Ethereum gateway of SQD Network, as well as a blockchain node, requesting data after a certain block (**make sure to add a node URL to `ETH_CHAIN_NODE` variable in the `.env` file**). Real-time consensus data will be considered final after 75 block confirmations / 15 minutes.

* It is also configured to request data for EVM logs generated by the Factory and Positions smart contracts, filtering for certain events (`PoolCreated` and `IncreaseLiquidity`, `DecreaseLiquidity`, `Collect`, `Transfer`, respectively).

* Processor is configured to also request EVM logs from **any address** with topic0 matching one of the signatures of the following Pool smart contract events: `Burn`, `Mint`, `Initialize`, `Swap`. This will guarantee that events generated by all Pool contracts are captured regardless of when the contracts were deployed.

* Finally, the processor is launched and data is processed in batches, by functions defined in `src/mappings`.

For a brief explanation of what [`processFactory`](https://github.com/subsquid-labs/squid-parquet-storage/blob/main/src/mappings/factory.ts), [`processPools`](https://github.com/subsquid-labs/squid-parquet-storage/blob/main/src/mappings/pools.ts) and [`processPositions`](https://github.com/subsquid-labs/squid-parquet-storage/blob/main/src/mappings/positions.ts) do, let's take the `processPositions` functions as an example:

* it needs to "unbundle" the batch of logs received

* for each EVM log found it checks that it belongs to one of the pool addresses generated by the factory

* compares the log's topic0 against the topics of the Events of the position NFT contract

* uses the corresponding Event TypeScript class to decode the log

* writes the decoded information to the corresponding table (parquet file)

To better understand how data is transformed, and how the other functions are defined as well, it's advised to [browse the repository](https://github.com/subsquid-labs/squid-parquet-storage/tree/main/src/mappings) and inspect the code. Be sure to check the [`utils`](https://github.com/subsquid-labs/squid-parquet-storage/tree/main/src/utils) folder as well, as there are some auxiliary files and functions used in the mapping logic.

### Launch the project

When the logic is fully implemented, to launch the project and start indexing, open a terminal and run these three commands:

```bash theme={"system"}
npm run build
```

```bash theme={"system"}
node -r dotenv/config lib/main.js
```

The indexer should be able to catch up with the Ethereum blockchain, and **reach the chain's head in a very short time**.

<Info>
  Bear in mind that this may vary a lot, depending on the Ethereum node used and on your hardware, as well as the connection, or physical distance from the Ethereum node. It took \~45 minutes while testing for this article. A test on a connection with a much higher latency and the same configuration finished indexing in 5 hours (figures out of date).
</Info>

The process will generate a series of sub-folders in the `data` folder, labelled after the block ranges where the data is coming from, and in each one of these folders there should be one `*.parquet` file for each of the *tables* we defined.

### Data analysis with Python

If you want to learn how to analyze this data using Python and Pandas, refer to [the Medium article dedicated to this demo project](https://link.medium.com/7gU0BrLbDxb).

## Conclusions

The purpose of this tutorial was to demonstrate how to use the SQD indexing framework for data analytics on larger datasets. The parquet data format is one of the most successful tools used in this setting, supported by the most common data analysis libraries such as Pandas and Pyarrow. We tested the performance of our parquet format tools on Uniswap, indexing the data from its pools and positions held by investors.

The project described here was able to index the entirety of Uniswap Pool events, across all the pools created by the Factory contract, as well as the Positions held by investors, in less than an hour (\~45 minutes) (figure out of date).

<Note>
  Indexing time may vary, depending on factors, such as the Ethereum node used, on the hardware, and quality of the connection.
</Note>

The simple Python script in the project's repository shows how to read multiple Parquet files, and perform some data analysis with Pandas.

SQD Team seeks feedback on this new tool. If you want to share any thoughts or have any suggestions, feel free to reach out to us at [the SquidDevs Telegram channel](https://t.me/HydraDevs).


# Frontier EVM-indexing squid
Source: https://docs.sqd.dev/en/sdk/squid-sdk/tutorials/frontier-evm

Build a Frontier-EVM indexing squid on Astar — Squid SDK tutorial for tracking NFTs on a Substrate chain with the Frontier EVM compatibility layer.

# Frontier EVM-indexing Squid

## Objective

The goal of this tutorial is to guide you through creating a simple blockchain indexer ("squid") using Squid SDK. The squid will be indexing the data from two contracts ([AstarDegens](https://blockscout.com/astar/address/0xd59fC6Bfd9732AB19b03664a45dC29B8421BDA9a) and [AstarCats](https://blockscout.com/astar/address/0x8b5d62f396Ca3C6cF19803234685e693733f9779)) deployed on the [Astar network](https://astar.network/). The objective will be to track ownership and transfers of all NFTs issued by these contracts.

A somewhat outdated version of the final result can be browsed [here](https://github.com/subsquid/squid-astar-example/tree/astar-degens).

## Pre-requisites

* Familiarity with Git
* A properly set up [development environment](/en/sdk/squid-sdk/how-to-start/development-environment-set-up) consisting of Node.js, Git and Docker
* [Squid CLI](/en/sdk/squid-sdk/squid-cli/installation)

## Scaffold using `sqd init`

We will start with the [`frontier-evm` squid template](https://github.com/subsquid-labs/squid-frontier-evm-template/) available through [`sqd init`](/en/sdk/squid-sdk/squid-cli/init). It is built to index EVM smart contracts deployed on Astar/Shiden, but it is also capable of indexing Substrate events. To retrieve the template and install the dependencies, run

```bash theme={"system"}
sqd init astar-evm-tutorial --template frontier-evm
cd astar-tutorial
npm i
```

## Define Entity Schema

Next, we ensure that the data [schema](/en/sdk/squid-sdk/reference/schema-file) of the squid defines [entities](/en/sdk/squid-sdk/reference/schema-file/entities) that we would like to track. We are interested in:

* Token transfers
* Ownership of tokens
* Contracts and their minted tokens

Here is a schema that defines the exact entities we need:

```graphql title="schema.graphql" theme={"system"}
type Token @entity {
  id: ID!
  owner: Owner
  uri: String
  transfers: [Transfer!]! @derivedFrom(field: "token")
  contract: Contract
}

type Owner @entity {
  id: ID!
  ownedTokens: [Token!]! @derivedFrom(field: "owner")
  balance: BigInt
}

type Contract @entity {
  id: ID!
  name: String
  symbol: String
  totalSupply: BigInt
  mintedTokens: [Token!]! @derivedFrom(field: "contract")
}

type Transfer @entity {
  id: ID!
  token: Token!
  from: Owner
  to: Owner
  timestamp: Int!
  block: Int!
}
```

It's worth noting a couple of things in this [schema definition](/en/sdk/squid-sdk/reference/schema-file):

* **`@entity`**: Signals that this type will be translated into an ORM model that is going to be persisted in the database.
* **`@derivedFrom`**: Signals that the field will not be persisted in the database. Instead, it will be [derived from](/en/sdk/squid-sdk/reference/schema-file/entity-relations) the entity relations.
* **type references** (e.g. `from: Owner`): When used on entity types, they establish a relation between two entities.

TypeScript entity classes have to be regenerated whenever the schema is changed, and to do that we use the `squid-typeorm-codegen` tool:

```bash theme={"system"}
npx squid-typeorm-codegen
```

The (re)generated entity classes can then be browsed at `src/model/generated`.

## ABI Definition and Wrapper

SQD maintains [tools](/en/sdk/squid-sdk/resources/tools/typegen/state-queries) for automated generation of TypeScript classes for handling EVM logs and transactions based on a [JSON ABI](https://docs.ethers.io/v5/api/utils/abi/) of the contract.

For our squid we will need such a module for the [ERC-721](https://eips.ethereum.org/EIPS/eip-721)-compliant part of the contracts' interfaces. Once again, the template repository already includes it, but it is still important to explain what needs to be done in case one wants to index a different type of contract.

Place any ABIs you requre for interfacing your contracts at `./abi` and run

```bash theme={"system"}
npx squid-evm-typegen ./src/abi ./abi/*.json --multicall
```

The results will be stored at `src/abi`. One module will be generated for each ABI file, and it will include constants useful for filtering and functions for decoding EVM events and functions defined in the ABI.

## Processor object and the batch handler

Squid SDK provides users with the [`SubstrateBatchProcessor` class](/en/sdk/squid-sdk). Its instances connect to [SQD Network](/en/network/overview) gateways at chain-specific URLs, to get chain data and apply custom transformations. The indexing begins at the starting block and keeps up with new blocks after reaching the tip.

`SubstrateBatchProcessor`s [expose methods](/en/sdk/squid-sdk/reference/processors/substrate-batch) that "subscribe" them to specific data such as Substrate events and calls. There are also [specialized methods](/en/sdk/squid-sdk/resources/substrate/frontier-evm) for subscribing to EVM logs and transactions by address. The actual data processing is then started by calling the `.run()` function. This will start generating requests to the SQD Network gateway for [*batches*](/en/sdk/squid-sdk/resources/batch-processing) of data specified in the configuration, and will trigger the callback function, or *batch handler* (passed to `.run()` as second argument) every time a batch is returned by the gateway.

It is in this callback function that all the mapping logic is expressed. This is where chain data decoding should be implemented, and where the code to save processed data on the database should be defined.

### Managing the EVM contract

Before we begin defining the mapping logic of the squid, we are going to write a `src/contracts.ts` utility module for managing the involved EVM contracts. It will export:

* Addresses of [astarDegens](https://blockscout.com/astar/address/0xd59fC6Bfd9732AB19b03664a45dC29B8421BDA9a) and [astarCats](https://blockscout.com/astar/address/0x8b5d62f396Ca3C6cF19803234685e693733f9779) contracts.
* A `Map` from the contract addresses to hardcoded `Contract` [entity](/en/sdk/squid-sdk/reference/schema-file/entities) instances.

Here are the full file contents:

```ts title="src/contracts.ts" theme={"system"}
import { Store } from '@subsquid/typeorm-store'
import { Contract } from './model'

export const astarDegensAddress = '0xd59fC6Bfd9732AB19b03664a45dC29B8421BDA9a'.toLowerCase();
export const astarCatsAddress = '0x8b5d62f396Ca3C6cF19803234685e693733f9779'.toLowerCase();

export const contractMapping: Map<string, Contract> = new Map()

contractMapping.set(astarDegensAddress, new Contract({
  id: astarDegensAddress,
  name: 'AstarDegens',
  symbol: 'DEGEN',
  totalSupply: 10000n,
  mintedTokens: []
}))

contractMapping.set(astarCatsAddress, new Contract({
  id: astarCatsAddress,
  name: 'AstarCats',
  symbol: 'CAT',
  totalSupply: 7777n,
  mintedTokens: []
})
```

## Create the processor object

The `src/processor.ts` file is where squids instantiate and configure their processor objects. We will use an instance of [`SubstrateBatchProcessor`](/en/sdk/squid-sdk).

We adapt the template code to handle two contracts instead of one and point the processor data source setting to the `astar` [SQD Network gateway URL](/en/data/evm). Here is the end result:

```ts title="src/processor.ts" theme={"system"}
import {assertNotNull} from '@subsquid/util-internal'
import {
    BlockHeader,
    DataHandlerContext,
    SubstrateBatchProcessor,
    SubstrateBatchProcessorFields,
    Event as _Event,
    Call as _Call,
    Extrinsic as _Extrinsic
} from '@subsquid/substrate-processor'
import * as erc721 from './abi/erc721'

import {astarDegensAddress, astarCatsAddress} from './contracts'

const processor = new SubstrateBatchProcessor()
  .setBlockRange({ from: 442693 })
  .setGateway('https://v2.archive.subsquid.io/network/astar-substrate')
  .setRpcEndpoint({
    url: assertNotNull(process.env.RPC_ENDPOINT),
    rateLimit: 10,
  })
  .addEvmLog({
    address: [astarDegensAddress],
    range: { from: 442693 },
    topic0: [erc721.events.Transfer.topic]
  })
  .addEvmLog({
    address: [astarCatsAddress],
    range: { from: 800854 },
    topic0: [erc721.events.Transfer.topic]
  })

export type Fields = SubstrateBatchProcessorFields<typeof processor>
export type Block = BlockHeader<Fields>
export type Event = _Event<Fields>
export type Call = _Call<Fields>
export type Extrinsic = _Extrinsic<Fields>
export type ProcessorContext<Store> = DataHandlerContext<Store, Fields>
```

<Warning>
  This code expects to find an URL of a working Astar RPC endpoint in the `RPC_ENDPOINT` environment variable. Set it in the `.env` file and in [SQD Cloud secrets](/en/cloud/resources/env-variables) if and when you deploy your squid there. We tested the code using a public endpoint available at `wss://astar.public.blastapi.io`; for production, we recommend using private endpoints or our [RPC addon](/en/cloud/resources/rpc-proxy).
</Warning>

## Define the batch handler

We change the batch handler logic taking care to avoid token ID clashing:

```ts title="src/main.ts" theme={"system"}
import { Store, TypeormDatabase } from '@subsquid/typeorm-store'
import { In } from 'typeorm'
import {
  astarDegensAddress,
  astarCatsAddress,
  contractMapping
} from './contracts'
import { Owner, Token, Transfer } from './model'
import * as erc721 from './abi/erc721'
import {
  processor,
  ProcessorContext,
  Event,
  Block
} from './processor'

var contractsSaved = false

processor.run(new TypeormDatabase(), async (ctx) => {
  const transfersData: TransferData[] = [];

  for (const block of ctx.blocks) {
    for (const event of block.events) {
      if (event.name === 'EVM.Log') {
        const transfer = handleTransfer(block.header, event)
        transfersData.push(transfer)
      }
    }
  }

  if (!contractsSaved) {
    await ctx.store.upsert([...contractMapping.values()])
    contractsSaved = true
  }
  await saveTransfers(ctx, transfersData)
})

type TransferData = {
  id: string
  from: string
  to: string
  token: bigint
  timestamp: number
  block: number
  contractAddress: string
}

function handleTransfer(block: Block, event: Event): TransferData {
  const { from, to, tokenId } = erc721.events.Transfer.decode(event)
  return {
    id: event.id,
    from,
    to,
    token: tokenId,
    timestamp: block.timestamp,
    block: block.height,
    contractAddress: event.args.address
  }
}

async function saveTransfers(
  ctx: ProcessorContext<Store>,
  transfersData: TransferData[]
) {
  const getTokenId = transferData => `${contractMapping.get(transferData.contractAddress)?.symbol ?? ""}-${transferData.token.toString()}`

  const tokensIds: Set<string> = new Set()
  const ownersIds: Set<string> = new Set()

  for (const transferData of transfersData) {
    tokensIds.add(getTokenId(transferData))
    ownersIds.add(transferData.from)
    ownersIds.add(transferData.to)
  }

  const tokens: Map<string, Token> = new Map(
    (await ctx.store.findBy(Token, { id: In([...tokensIds]) }))
      .map(token => [token.id, token])
  )

  const owners: Map<string, Owner> = new Map(
    (await ctx.store.findBy(Owner, { id: In([...ownersIds]) }))
      .map(owner => [owner.id, owner])
  )

  const transfers: Set<Transfer> = new Set()

  for (const transferData of transfersData) {
    const contract = new erc721.Contract(
      // temporary workaround for SDK issue 212
      // passing just the ctx as first arg may already work
      {_chain: {client: ctx._chain.rpc}},
      { height: transferData.block },
      transferData.contractAddress
    )

    let from = owners.get(transferData.from)
    if (from == null) {
      from = new Owner({ id: transferData.from, balance: 0n })
      owners.set(from.id, from)
    }

    let to = owners.get(transferData.to)
    if (to == null) {
      to = new Owner({ id: transferData.to, balance: 0n })
      owners.set(to.id, to)
    }

    const tokenId = getTokenId(transferData)
    let token = tokens.get(tokenId)
    if (token == null) {
      token = new Token({
        id: tokenId,
        uri: await contract.tokenURI(transferData.token),
        contract: contractMapping.get(transferData.contractAddress)
      })
      tokens.set(token.id, token)
    }

    token.owner = to

    const { id, block, timestamp } = transferData

    const transfer = new Transfer({
      id,
      block,
      timestamp,
      from,
      to,
      token
    })

    transfers.add(transfer)
  }

  await ctx.store.upsert([...owners.values()])
  await ctx.store.upsert([...tokens.values()])
  await ctx.store.insert([...transfers])
}
```

<Info>
  The `contract.tokenURI` call is accessing the **state** of the contract via a chain RPC endpoint. This is slowing down the indexing a little bit, but this data is only available this way. You'll find more information on accessing state in the [dedicated section of our docs](/en/sdk/squid-sdk/resources/substrate/frontier-evm#access-contract-state).
</Info>

## Database and the migration

Before giving your squid processor a local test, launch a PostgreSQL container with

```bash theme={"system"}
docker compose up -d
```

Squid projects automatically manage the database connection and schema via an [ORM abstraction](https://en.wikipedia.org/wiki/Object%E2%80%93relational_mapping). In this approach the schema is managed through migration files. Since we've made changes to the schema, we need to remove the existing migration(s) and create a new one. This involves the following steps:

1. Build the code:

   ```bash theme={"system"}
   npm run build
   ```

2. Make sure you start with a clean Postgres database. The following commands drop-create the Postgres instance in Docker:

   ```bash theme={"system"}
   docker compose down
   docker compose up -d
   ```

   Skip this step if you haven't used your database since the last `docker compose up -d`.

3. Regenerate the DB migration:

   ```bash theme={"system"}
   rm -r db/migrations
   ```

   ```bash theme={"system"}
   npx squid-typeorm-migration generate
   ```

4. Apply the migration:

   ```bash theme={"system"}
   npx squid-typeorm-migration apply
   ```

## Launch the Project

To launch the processor run the following command (this will block the current terminal):

```bash theme={"system"}
node -r dotenv/config lib/main.js
```

Finally, in a separate terminal window, launch the GraphQL server:

```bash theme={"system"}
npx squid-graphql-server
```

Visit [`localhost:4350/graphql`](http://localhost:4350/graphql) to access the [GraphiQL](https://github.com/graphql/graphiql) console. From this window, you can perform queries such as this one, to find out the account owners with the biggest balances:

```graphql theme={"system"}
query MyQuery {
  owners(limit: 10, where: {}, orderBy: balance_DESC) {
    balance
    id
  }
}
```

Or this other one, looking up the tokens owned by a given owner:

```graphql theme={"system"}
query MyQuery {
  tokens(where: {owner: {id_eq: "0x1210f3ea18ef463c162fff9084cee5b6e5ccab37"}}) {
    uri
    contract {
      id
      name
      symbol
      totalSupply
    }
  }
}
```

Have fun playing around with queries, after all, it's a *playground*!


# ink! contract indexing
Source: https://docs.sqd.dev/en/sdk/squid-sdk/tutorials/ink

Build a Squid SDK indexer for an ink! WASM smart contract — set up typegen, decode contract events, and persist them to Postgres for GraphQL queries.

## Objective

This tutorial starts with the [`substrate` squid template](https://github.com/subsquid-labs/squid-substrate-template) and goes through all the necessary changes to index the events of a WASM contract developed with [ink!](https://www.parity.io/blog/ink-3-0-paritys-rust-based-language-gets-a-major-update). This approach is taken to illustrate the development process. If you want to start indexing ASAP, consider starting with the [`ink` template](https://github.com/subsquid/squid-wasm-template) that contains the final code of this tutorial:

```bash theme={"system"}
sqd init <your squid name here> --template ink
```

Here we will use a simple test ERC20-type token contract deployed to [Shibuya](https://shibuya.subscan.io/) at `XnrLUQucQvzp5kaaWLG9Q3LbZw5DPwpGn69B5YcywSWVr5w`. Our squid will track all the token holders and account balances, together with the historical token transfers.

<Info>
  Squid SDK only supports WASM contracts executed by the [Contracts pallet](https://crates.parity.io/pallet_contracts/index.html) natively. The pallet is enabled by the following network runtimes:

  * `Astar` (a `Polkadot` parachain)
  * `Shibuya` (`Astar` testnet)
  * `Shiden` (`Kusama`-cousin of `Astar`)
  * `AlephZero` (a standalone Substrate-based chain)
</Info>

## Pre-requisites

* Familiarity with Git
* A properly set up [development environment](/en/sdk/squid-sdk/how-to-start/development-environment-set-up) consisting of Node.js, Git and Docker
* [Squid CLI](/en/sdk/squid-sdk/squid-cli/installation)

## Run the template

Retrieve the `substrate` template with `sqd init`:

```bash theme={"system"}
sqd init ink-tutorial --template substrate
cd ink-tutorial
```

and run it:

```bash theme={"system"}
npm i
npm run build
docker compose up -d
npx squid-typeorm-migration apply

node -r dotenv/config lib/main.js # should begin to ingest blocks

# open a separate terminal for this next command
npx squid-graphql-server # should begin listening on port 4350
```

After this test, shut down both processes with Ctrl-C and proceed.

## Define the data schema

The next step is to define the target data entities and their relations at `schema.graphql`, then use that file to autogenerate TypeORM entity classes.

We track:

* Wallet balances
* Token transfers

Our [schema definition](/en/sdk/squid-sdk/reference/schema-file) for modelling this data is straightforward:

```graphql theme={"system"}
# schema.graphql

type Owner @entity {
  id: ID!
  balance: BigInt! @index
}
 
type Transfer @entity {
  id: ID!
  from: Owner
  to: Owner
  amount: BigInt! @index
  timestamp: DateTime! @index
  block: Int!
}
 
```

Note:

* a one-to-many [relation](/en/sdk/squid-sdk/reference/schema-file/entity-relations) between `Owner` and `Transfer`;
* `@index` decorators for properties that we want to be able to filter the data by.

Next, we generate `TypeORM` entity classes from the schema with the `squid-typeorm-codegen` tool:

```bash theme={"system"}
npx squid-typeorm-codegen
```

The generated entity classes can be found under `src/model/generated`.

Finally, we create [database migrations](/en/sdk/squid-sdk/resources/persisting-data/typeorm) to match the changed schema. We restore the database to a clean state, then replace any existing migrations with the new one:

```bash theme={"system"}
docker compose down
docker compose up -d
rm -r db/migrations
npx squid-typeorm-migration generate
```

Apply the migration with

```bash theme={"system"}
npx squid-typeorm-migration apply
```

## WASM ABI Tools

The `Contracts` pallet stores the contract execution logs (calls and events) in a binary format. The decoding of this data is contract-specific and is done with the help of an ABI file typically published by the contract developer. ABI for ERC20 contracts like the one we're indexing can be found [here](https://raw.githubusercontent.com/subsquid-labs/squid-wasm-template/master/abi/erc20.json).

Download that file to the `abi` folder and install the following two tools from Squid SDK:

* `@subsquid/ink-abi` -- A performant library for decoding binary ink! contract data.
* `@subsquid/ink-typegen` -- A tool for making TypeScript modules for handling contract event and call data based on ABIs of contracts.

```bash theme={"system"}
npm i @subsquid/ink-abi && npm i @subsquid/ink-typegen --save-dev
```

Since `@subsquid/ink-typegen` is only used to generate source files, we install it as a dev dependency.

Generate the contract data handling module by running

```bash theme={"system"}
npx squid-ink-typegen --abi abi/erc20.json --output src/abi/erc20.ts
```

The generated `src/abi/erc20.ts` module defines interfaces to represent WASM data defined in the ABI, as well as functions necessary to decode this data (e.g. the `decodeEvent` function).

## Define the processor object

Squid SDK provides users with the [`SubstrateBatchProcessor` class](/en/sdk/squid-sdk). Its instances connect to [SQD Network](/en/network/overview) gateways at chain-specific URLs to get chain data and apply custom transformations. The indexing begins at the starting block and keeps up with new blocks after reaching the tip.

`SubstrateBatchProcessor`s [exposes methods](/en/sdk/squid-sdk/reference/processors/substrate-batch) to "subscribe" them to specific data such as Substrate events, extrinsics, storage items etc. The `Contracts` pallet emits `ContractEmitted` events wrapping the logs emitted by the WASM contracts. Processor [allows one](/en/sdk/squid-sdk/resources/substrate/ink) to subscribe to such events emitted by a specific contract.

The processor is instantiated and configured at the `src/processor.ts`. Here are the changes we need to make there:

* Change the gateway used to `https://v2.archive.subsquid.io/network/shibuya-substrate`.
* Remove the `addEvent` function call, and add `addContractsContractEmitted` instead, specifying the address of the contract we are interested in. The address should be represented as a hex string, so we need to decode our ss58 address of interest, `XnrLUQucQvzp5kaaWLG9Q3LbZw5DPwpGn69B5YcywSWVr5w`.

Here is the end result:

```ts title="src/processor.ts" theme={"system"}
import {assertNotNull} from '@subsquid/util-internal'
import {toHex} from '@subsquid/util-internal-hex'
import * as ss58 from '@subsquid/ss58'
import {
    BlockHeader,
    DataHandlerContext,
    SubstrateBatchProcessor,
    SubstrateBatchProcessorFields,
    Event as _Event,
    Call as _Call,
    Extrinsic as _Extrinsic
} from '@subsquid/substrate-processor'

export const SS58_NETWORK = 'astar' // used for the ss58 prefix, astar shares it with shibuya

const CONTRACT_ADDRESS_SS58 = 'XnrLUQucQvzp5kaaWLG9Q3LbZw5DPwpGn69B5YcywSWVr5w'
export const CONTRACT_ADDRESS = ss58.codec(SS58_NETWORK).decode(CONTRACT_ADDRESS_SS58)

export const processor = new SubstrateBatchProcessor()
    .setGateway('https://v2.archive.subsquid.io/network/shibuya-substrate')
    .setRpcEndpoint({
        url: assertNotNull(process.env.RPC_ENDPOINT),
        rateLimit: 10
    })
    .addContractsContractEmitted({
        contractAddress: [CONTRACT_ADDRESS],
        extrinsic: true
    })
    .setFields({
        block: {
            timestamp: true
        },
        extrinsic: {
            hash: true
        }
    })
    .setBlockRange({
        // genesis block happens to not have a timestamp, so it's easier
        // to start from 1 in cases when the deployment height is unknown
        from: 1
    })

export type Fields = SubstrateBatchProcessorFields<typeof processor>
export type Block = BlockHeader<Fields>
export type Event = _Event<Fields>
export type Call = _Call<Fields>
export type Extrinsic = _Extrinsic<Fields>
export type ProcessorContext<Store> = DataHandlerContext<Store, Fields>
```

## Define the batch handler

Once requested, the events can be processed by calling the `.run()` function that starts generating requests to SQD Network for [*batches*](/en/sdk/squid-sdk/resources/batch-processing) of data.

Every time a batch is returned by the Network, it will trigger the callback function, or *batch handler* (passed to `.run()` as second argument). It is in this callback function that all the mapping logic is expressed. This is where chain data decoding should be implemented, and where the code to save processed data on the database should be defined.

Batch handler is typically defined at the squid processor entry point, `src/main.ts`. Here is one that works for our task:

```ts title="src/main.ts" theme={"system"}
import {In} from 'typeorm'
import assert from 'assert'

import * as ss58 from '@subsquid/ss58'
import {Store, TypeormDatabase} from '@subsquid/typeorm-store'

import * as erc20 from './abi/erc20'
import {Owner, Transfer} from "./model"
import {
    processor,
    SS58_NETWORK,
    CONTRACT_ADDRESS,
    ProcessorContext
} from './processor'

processor.run(new TypeormDatabase({supportHotBlocks: true}), async ctx => {
    const txs: TransferRecord[] = getTransferRecords(ctx)

    const owners: Map<string, Owner> = await createOwners(ctx, txs)
    const transfers: Transfer[] = createTransfers(txs, owners)

    await ctx.store.upsert([...owners.values()])
    await ctx.store.insert(transfers)
})

interface TransferRecord {
    id: string
    from?: string
    to?: string
    amount: bigint
    block: number
    timestamp: Date
    extrinsicHash: string
}

function getTransferRecords(ctx: ProcessorContext<Store>): TransferRecord[] {
    const records: TransferRecord[] = []
    for (const block of ctx.blocks) {
        assert(block.header.timestamp, `Block ${block.header.height} had no timestamp`)
        for (const event of block.events) {
            if (event.name === 'Contracts.ContractEmitted' && event.args.contract === CONTRACT_ADDRESS) {
                assert(event.extrinsic, `Event ${event} arrived without a parent extrinsic`)
                const decodedEvent = erc20.decodeEvent(event.args.data)
                if (decodedEvent.__kind === 'Transfer') {
                    records.push({
                        id: event.id,
                        from: decodedEvent.from && ss58.codec(SS58_NETWORK).encode(decodedEvent.from),
                        to: decodedEvent.to && ss58.codec(SS58_NETWORK).encode(decodedEvent.to),
                        amount: decodedEvent.value,
                        block: block.header.height,
                        timestamp: new Date(block.header.timestamp),
                        extrinsicHash: event.extrinsic.hash
                    })
                }
            }
        }
    }
    return records
}

async function createOwners(ctx: ProcessorContext<Store>, txs: TransferRecord[]): Promise<Map<string, Owner>> {
    const ownerIds = new Set<string>()
    txs.forEach(tx => {
        if (tx.from) {
            ownerIds.add(tx.from)
        }
        if (tx.to) {
            ownerIds.add(tx.to)
        }
    })

    const ownersMap = await ctx.store.findBy(Owner, {
        id: In([...ownerIds])
    }).then(owners => {
        return new Map(owners.map(owner => [owner.id, owner]))
    })

    return ownersMap
}


function createTransfers(txs: TransferRecord[], owners: Map<string, Owner>): Transfer[] {
    return txs.map(tx => {
        const transfer = new Transfer({
            id: tx.id,
            amount: tx.amount,
            block: tx.block,
            timestamp: tx.timestamp,
            extrinsicHash: tx.extrinsicHash
        })

        if (tx.from) {
            transfer.from = owners.get(tx.from)
            if (transfer.from == null) {
                transfer.from = new Owner({id: tx.from, balance: 0n})
                owners.set(tx.from, transfer.from)
            }
            transfer.from.balance -= tx.amount
        }

        if (tx.to) {
            transfer.to = owners.get(tx.to)
            if (transfer.to == null) {
                transfer.to = new Owner({id: tx.to, balance: 0n})
                owners.set(tx.to, transfer.to)
            }
            transfer.to.balance += tx.amount
        }

        return transfer
    })
}
```

The `getTransferRecords` function generates a list of `TransferRecord` objects that contain the data we need to fill the models we have defined with our schema. This data is extracted from the events found in the batch context, `ctx`. We use the in the main body of the batch handler, the arrow function used as the second argument of the `.run()` function call, to fetch or create `Owner` instance and create a `Transfer` instance for every event found in the context.

Finally, these [TypeORM entity](/en/sdk/squid-sdk/reference/schema-file/entities) instances are saved to the database, all in one go. This is done to reduce the number of database queries.

<Info>
  In the `getTransferRecords` function we loop over the blocks and over the events contained in them, then filter the events with an `if`. The filtering is redundant when there's only one event type to process but will be needed when the processor is subscribed to multiple ones.
</Info>

## Launch the Project

Build, then launch the processor with

```bash theme={"system"}
npm run build
```

```bash theme={"system"}
node -r dotenv/config lib/main.js
```

This will block the current terminal. In a separate terminal window, launch the GraphQL server:

```bash theme={"system"}
npx squid-graphql-server
```

Visit [`localhost:4350/graphql`](http://localhost:4350/graphql) to access the [GraphiQl](https://github.com/graphql/graphiql) console. There you can perform queries such as this one, to find out the account owners with the biggest balances:

```graphql theme={"system"}
query MyQuery {
  owners(limit: 10, where: {}, orderBy: balance_DESC) {
    balance
    id
  }
}
```

Or this other one, looking up the largest transfers:

```graphql theme={"system"}
query MyQuery {
  transfers(limit: 10, orderBy: amount_DESC) {
    amount
    block
    id
    timestamp
    to {
      balance
      id
    }
    from {
      balance
      id
    }
  }
}

```

Have fun playing around with queries, after all, it's a *playground*!


# Simple Substrate squid
Source: https://docs.sqd.dev/en/sdk/squid-sdk/tutorials/substrate

Build a starter Substrate Squid SDK indexer — set up the processor, subscribe to balance transfer events, and persist to Postgres in 30 minutes.

## Objective

The goal of this tutorial is to guide you through creating a simple blockchain indexer ("squid") using Squid SDK. In this example we will query the [Crust storage network](https://crust.network). Our objective will be to observe which files have been added and deleted from the network. Additionally, our squid will be able to tell us the groups joined and the storage orders placed by a given account.

We will start with the `substrate` squid template, then go on to run the project, define a schema, and generate TypeScript interfaces. From there, we will be able to interact directly with SQD Network, using the and the metadata service to get a Crust types bundle.

We expect that experienced software developers should be able to complete this tutorial in around **10-15 minutes**.

## Pre-requisites

* Familiarity with Git
* A properly set up [development environment](/en/sdk/squid-sdk/how-to-start/development-environment-set-up) consisting of Node.js, Git and Docker
* [Squid CLI](/en/sdk/squid-sdk/squid-cli/installation)

## Scaffold with `sqd init`

Use [`sqd init`](/en/sdk/squid-sdk/squid-cli/init) and come up with some unique name for your squid. This tutorial will index data on Crust, a Substrate-based network, so use the `substrate` template:

```sh theme={"system"}
sqd init substrate-crust-tutorial --template substrate
cd substrate-crust-tutorial
```

### Run the project

Now you can follow the [quickstart](/en/sdk/squid-sdk/how-to-start/squid-development) guide to get the project up and running. Here is a summary:

```bash theme={"system"}
npm i
npm run build
docker compose up -d
npx squid-typeorm-migration apply
```

```bash theme={"system"}
node -r dotenv/config lib/main.js # will begin ingesting blocks
```

```bash theme={"system"}
# open a separate terminal for this next command
npx squid-graphql-server # should begin listening on port 4350
```

After this test, shut down both processes with Ctrl-C and proceed.

## Define the schema and generate entity classes

Next, we make changes to the data [schema](/en/sdk/squid-sdk/reference/schema-file) of the squid and define [entities](/en/sdk/squid-sdk/reference/schema-file/entities) that we would like to track. We are interested in:

* Files added to and deleted from the chain;
* Active accounts;
* Groups joined by accounts;
* Storage orders placed by accounts.

For this, we use the following `schema.graphql`:

```graphql title="schema.graphql" theme={"system"}
type Account @entity {
  id: ID! #Account address
  workReports: [WorkReport] @derivedFrom(field: "account")
  joinGroups: [JoinGroup] @derivedFrom(field: "member")
  storageOrders: [StorageOrder] @derivedFrom (field: "account")
}

type WorkReport @entity {
  id: ID! #event id
  account: Account!
  addedFiles: [[String]]
  deletedFiles: [[String]]
  extrinsicHash: String
  createdAt: DateTime
  blockNum: Int!
}

type JoinGroup @entity {
  id: ID!
  member: Account!
  owner: String!
  extrinsicHash: String
  createdAt: DateTime
  blockNum: Int!
}

type StorageOrder @entity {
  id: ID!
  account: Account!
  fileCid: String!
  extrinsicHash: String
  createdAt: DateTime
  blockNum: Int!
}
```

Notice that the `Account` entity is almost completely [derived](/en/sdk/squid-sdk/reference/schema-file/entity-relations). It is there to tie the other three entities together.

To finalize this step, run the `codegen` tool:

```bash theme={"system"}
npx squid-typeorm-codegen
```

This will automatically generate TypeScript entity classes for our schema. They can be found in the `src/model/generated` folder of the project.

## Generate TypeScript wrappers for events

We generate these using the [squid-substrate-typegen](/en/sdk/squid-sdk/tutorials/batch-processor-in-action) tool. Its configuration file is `typegen.json`; there, we need to

1. Set the `"specVersions"` field to a valid source of Crust chain runtime metadata. We'll use an URL of SQD-maintained metadata service:
   ```json theme={"system"}
   "specVersions": "https://v2.archive.subsquid.io/metadata/crust",
   ```
2. List all Substrate pallets we will need the data from. For each pallet we list all events, calls, storage items and constants needed.

<Info>
  Refer to [this note](/en/sdk/squid-sdk/resources/substrate/data-sourcing-miniguide) if you are unsure what Substrate data to use in your project.
</Info>

Our final `typegen.json` looks like this:

```json title="typegen.json" theme={"system"}
{
  "outDir": "src/types",
  "specVersions": "https://v2.archive.subsquid.io/metadata/crust",
  "pallets": {
    "Swork": {
      "events": [
        "WorksReportSuccess",
        "JoinGroupSuccess"
      ]
    },
    "Market": {
      "events": [
        "FileSuccess"
      ]
    }
  }
}
```

Once done with the configuration, we run the tool with

```bash theme={"system"}
npx squid-substrate-typegen ./typegen.json
```

The generated Typescript wrappers are at `src/types`.

## Set up the processor object

The next step is to create a [`SubstrateBatchProcessor`](/en/sdk/squid-sdk/reference/processors/substrate-batch) object which [subscribes](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests) to all the events we need. We do it at `src/processor.ts`:

```ts title="src/processor.ts" theme={"system"}
import {
  SubstrateBatchProcessor,
  SubstrateBatchProcessorFields,
  DataHandlerContext
} from '@subsquid/substrate-processor'
import {events} from './types' // the wrappers generated in previous section

const processor = new SubstrateBatchProcessor()
  .setGateway('https://v2.archive.subsquid.io/network/crust')
  .setRpcEndpoint({
    url: 'https://crust.api.onfinality.io/public',
    rateLimit: 10
  })
  .setBlockRange({ from: 583000 })
  .addEvent({
    name: [
      events.market.fileSuccess.name,
      events.swork.joinGroupSuccess.name,
      events.swork.worksReportSuccess.name
    ],
    call: true,
    extrinsic: true
  })
  .setFields({
    extrinsic: {
      hash: true
    },
    block: {
      timestamp: true
    }
  })

type Fields = SubstrateBatchProcessorFields<typeof processor>
export type ProcessorContext<Store> = DataHandlerContext<Store, Fields>
```

This creates a processor that

* Uses SQD Network as its main data source and a chain RPC for [real-time updates](/en/sdk/squid-sdk/resources/unfinalized-blocks). URLs of the SQD Network gateways are available on [this page](/en/data/substrate) and via [`sqd gateways`](/en/sdk/squid-sdk/squid-cli/gateways). See [this page](/en/sdk/squid-sdk/reference/processors/substrate-batch/general) for the reference on data sources configuration;
* [Subscribes](/en/sdk/squid-sdk/reference/processors/substrate-batch/data-requests) to `Market.FileSuccess`, `Swork.JoinGroupSuccess` and `Swork.WorksReportSuccess` events emitted at heights starting at 583000;
* Additionally subscribes to calls that emitted the events and the corresponding extrinsics;
* [Requests](/en/sdk/squid-sdk/reference/processors/substrate-batch/field-selection) the `hash` data field for all retrieved extrinsics and the `timestamp` field for all block headers.

We also export the `ProcessorContext` type to be able to pass the sole argument of the batch handler function around safely.

## Define the batch handler

Squids [batch process](/en/sdk/squid-sdk/resources/batch-processing) chain data from multiple blocks. Compared to the [handlers](/en/sdk/squid-sdk/resources/batch-processing#migrate-from-handlers) approach this results in a much lower database load. Batch processing is fully defined by processor's [batch handler](/en/sdk/squid-sdk/reference/processors/architecture#processorrun), the callback supplied to the `processor.run()` call at the entry point of each processor (`src/main.ts` by convention).

We begin defining our batch handler by importing the entity model classes and Crust event types that we generated in previous sections. We also import the processor and its types:

```ts title="src/main.ts" theme={"system"}
import {Account, WorkReport, JoinGroup, StorageOrder} from './model'
import {processor, ProcessorContext} from './processor'
```

Let's skip for now the `process.run()` call - we are going to come back to it in a second - and scroll down to the `getTransferEvents` function. In the template repository this function loops through the items contained in the context, extracts the events data and stores it in a list of objects.

For this project we are still going to extract events data from the context, but this time we have more than one event type so we have to sort them. We also need to handle the account information. Let's start with deleting the `TransferEvent` interface and defining this instead:

```typescript theme={"system"}
type Tuple<T,K> = [T,K]
interface EventsInfo {
  joinGroups: Tuple<JoinGroup, string>[]
  marketFiles: Tuple<StorageOrder, string>[]
  workReports: Tuple<WorkReport, string>[]
  accountIds: Set<string>
}
```

Now, let's replace the `getTransferEvents` function with the below snippet that

* extracts event information in a manner specific to its name (known from `e.name`);
* stores event information in an object (we are going to use entity classes for that) and extracts `accountId`s from it;
* store all `accountId`s in a set.

```typescript theme={"system"}
import {toHex} from '@subsquid/substrate-processor'
import * as ss58 from '@subsquid/ss58'

function getEventsInfo(ctx: ProcessorContext<Store>): EventInfo {
  let eventsInfo: EventsInfo = {
    joinGroups: [],
    marketFiles: [],
    workReports: [],
    accountIds: new Set<string>()
  }
  for (let block of ctx.blocks) {
    const blockTimestamp = block.header.timestamp ? new Date(block.header.timestamp) : undefined
    for (let e of block.events) {
      if (e.name === events.swork.joinGroupSuccess.name) {
        const decoded = events.swork.joinGroupSuccess.v1.decode(e)
        const memberId = ss58.codec('crust').encode(decoded[0])
        eventsInfo.joinGroups.push([
          new JoinGroup({
            id: e.id,
            owner: ss58.codec('crust').encode(decoded[1]),
            blockNum: block.header.height,
            createdAt: blockTimestamp,
            extrinsicHash: e.extrinsic?.hash
          }),
          memberId
        ])
        // add encountered account ID to the set of unique accountIDs
        eventsInfo.accountIds.add(memberId)
      }
      if (e.name === events.market.fileSuccess.name) {
        const decoded = events.market.fileSuccess.v1.decode(e)
        const accountId = ss58.codec('crust').encode(decoded[0])
        eventsInfo.marketFiles.push([
          new StorageOrder({
            id: e.id,
            fileCid: toHex(decoded[1]),
            blockNum: block.header.height,
            createdAt: blockTimestamp,
            extrinsicHash: e.extrinsic?.hash
          }),
          accountId
        ])
        eventsInfo.accountIds.add(accountId)
      }
      if (e.name === events.swork.worksReportSuccess.name) {
        const decoded = events.swork.worksReportSucces.v1.decode(e)
        const accountId = ss58.codec('crust').encode(decoded[0])

        const addedExtr = e.call?.args.addedFiles
        const deletedExtr = e.call?.args.deletedFiles

        const addedFiles = addedExtr.map(v => v.map(ve => String(ve)))
        const deletedFiles = deletedExtr.map(v => v.map(ve => String(ve)))

        if (addedFiles.length > 0 || deletedFiles.length > 0) {
          eventsInfo.workReports.push([
            new WorkReport({
              id: e.id,
              addedFiles: addedFiles,
              deletedFiles: deletedFiles,
              blockNum: block.header.height,
              createdAt: blockTimestamp,
              extrinsicHash: e.extrinsic?.hash,
            }),
            accountId
          ])
          eventsInfo.accountIds.add(accountId)
        }
      }
    }
  }
  return eventsInfo
}
```

Next, we want to create an entity (`Account`) object for every `accountId` in the set, then add the `Account` information to every event entity object. Finally, we save all the created and modified entity models into the database.

Take the code inside `processor.run()` and change it so that it looks like this:

```typescript theme={"system"}
processor.run(new TypeormDatabase(), async (ctx) => {
  const eventsInfo = getEventsInfo(ctx)

  let accounts = await ctx.store
    .findBy(Account, { id: In([...eventsInfo.accountIds]) })
    .then(accounts => new Map(accounts.map(a => [a.id, a]))
  for (let aid of eventsInfo.accountIds) {
    if (!accounts.has(aid)) {
      accounts.set(aid, new Account({ id: aid }))
    }
  }

  for (const jg of eventsInfo.joinGroups) {
    // necessary to add this field to the previously created model
    // because now we have the Account created.
    jg[0].member = accounts.get(jg[1])
  }
  for (const mf of eventsInfo.marketFiles) {
    mf[0].account = accounts.get(mf[1])
  }
  for (const wr of eventsInfo.workReports) {
    wr[0].account = accounts.get(wr[1])
  }

  await ctx.store.save([...accounts.values()]);
  await ctx.store.insert(eventsInfo.joinGroups.map(el => el[0]));
  await ctx.store.insert(eventsInfo.marketFiles.map(el => el[0]));
  await ctx.store.insert(eventsInfo.workReports.map(el => el[0]));
})
```

## Apply changes to the database

Squid projects automatically manage the database connection and schema via an [ORM abstraction](https://en.wikipedia.org/wiki/Object%E2%80%93relational_mapping) provided by [TypeORM](https://typeorm.io). Previously we changed the data schema at `schema.graphql` and reflected these changes in our Typescript code using `npx squid-typeorm-codegen`. Here, we [apply the corresponding changes to the database itself](/en/sdk/squid-sdk/resources/persisting-data/typeorm).

First, we'll need to compile our updated project code. Do this with:

```bash theme={"system"}
npm run build
```

Next we ensure that the database is at blank state:

```bash theme={"system"}
docker compose down
docker compose up -d
```

Then we replace any old migrations with the new one with

```bash theme={"system"}
rm -r db/migrations
```

```bash theme={"system"}
npx squid-typeorm-migration generate
```

The new migration will be generated from the TypeORM entity classes we previously made out of `schema.graphql`. Apply it with:

```bash theme={"system"}
npx squid-typeorm-migration apply
```

## Launch the project

It's finally time to run the project! Run

```bash theme={"system"}
node -r dotenv/config lib/main.js
```

in one terminal, then open another one and run

```bash theme={"system"}
npx squid-graphql-server
```

Now you can see the results of our hard work by visiting [`localhost:4350/graphql`](http://localhost:4350/graphql) in a browser and accessing the [GraphiQL](https://github.com/graphql/graphiql) console.

From this window we can perform queries. This one displays info on ten latest work reports, including all involved files and the account id:

```graphql theme={"system"}
query MyQuery {
  workReports(limit: 10, orderBy: blockNum_DESC) {
    account {
      id
    }
    addedFiles
    deletedFiles
  }
}
```

## Credits

This sample project is adapted from a real integration, developed by our very own [Mikhail Shulgin](https://github.com/ma-shulgin). Credit for building it and helping with the guide goes to him.


# Deployment Guide
Source: https://docs.sqd.dev/en/cloud/deployment-guide

Step-by-step guide to deploy a squid indexer to SQD Cloud — set up the manifest, push to GitHub, configure addons, and run your first deployment.

## Deployment Guide

This guide shows you how to deploy a production-ready indexer ("squid") to SQD Cloud.

## Prerequisites

Before you begin, ensure you have:

* **Docker** - [Download here](https://www.docker.com/) (for local development)
* **Squid CLI** - [Installation guide](/en/cloud/reference/cli/installation)
* **Working squid** - A locally tested indexer ready for deployment

<Steps>
  <Step title="Prepare the squid for deployment" icon="list-check">
    Ensure your squid is ready for production deployment:

    * **Test locally** - Verify the squid works as expected in your local environment
    * **Performance check** - Ensure no major performance issues. See [Best practices](/en/cloud/resources/best-practices) for guidance
    * **Update configuration** - Update the `squid.yaml` file with correct values for your use case
    * **Set up secrets** - Configure SQD Cloud secrets if your squid requires sensitive data

    <Warning>
      Production deployments should be thoroughly tested locally to avoid issues in
      the cloud environment.
    </Warning>

    <Check>Your squid is tested and ready for cloud deployment.</Check>
  </Step>

  <Step title="Register a SQD account" icon="user-plus">
    Create your SQD Cloud account:

    1. Visit the [SQD Cloud console](https://app.subsquid.io)
    2. Click **Create an account** and fill in the required information
    3. Or sign in with your GitHub or Google account

    <Check>You'll receive a confirmation email to verify your account.</Check>
  </Step>

  <Step title="Create a professional organization" icon="building">
    Set up your organization for production use:

    1. Create a new [organization](/en/cloud/resources/organizations) in SQD Cloud
    2. Upgrade to [professional](/en/cloud/resources/organizations/#professional-organizations) by attaching a valid credit card in the "Billing" tab

    <Note>
      You can skip this step, but you'll only be able to deploy to your account's
      [playground](/en/cloud/resources/organizations/#playgrounds). Playground
      squids are trial/development solutions with limited functionality and are not
      suitable for production use.
    </Note>

    <Check>
      Your professional organization is ready for production deployments.
    </Check>
  </Step>

  <Step title="Edit the squid.yaml file" icon="file-code">
    Configure your deployment manifest for SQD Cloud:

    ### Basic Configuration

    First, set squid name, description, and other metadata in the [header section](/en/cloud/reference/manifest/#header) of the `squid.yaml` file.

    If necessary, configure your [`build` options](/en/cloud/reference/manifest/#build) next. If the defaults (NodeJS v20 and `npm` for package manager) work for you, just create an empty `build` section.

    Finally, edit the [`deploy` section](/en/cloud/reference/manifest/#deploy) to specify the deployment options.

    **Example configuration:**

    ```yaml squid.yaml theme={"system"}
    manifest_version: subsquid.io/v0.1
    name: sample-squid

    build:

    deploy:
      processor:
        cmd: ["sqd", "process:prod"]
      api:
        cmd: ["sqd", "serve:prod"]
    ```

    <Note>
      The `processor` and `api` commands are
      [commands.json](/en/cloud/reference/cli/commands-json)-based. This is optional but
      recommended for complex deployments.
    </Note>

    ### Configure Addons

    SQD Cloud addons enable extra services for your squid. Enable them in the `deploy.addons` section based on your requirements.

    <AccordionGroup>
      <Accordion title="RPC Addon" icon="network-wired">
        Use the [`rpc` addon](/en/cloud/resources/rpc-proxy) for real-time data access:

        **Steps to configure:**

        1. Open the **RPC endpoints** tab in the SQD Cloud sidebar
        2. Copy the URL of your chosen endpoint
        3. Add it to your `.env` file:

        ```bash .env theme={"system"}
        RPC_ARBITRUM_ONE_HTTP=<endpoint-url>
        ```

        4. Enable the addon in your `squid.yaml`:

        ```yaml squid.yaml theme={"system"}
        deploy:
          addons:
            rpc:
              - arbitrum-one.http
        ```

        5. Configure your processor to use the RPC endpoint:

        ```typescript src/main.ts theme={"system"}
        import { assertNotNull } from "@subsquid/util-internal";

        export const processor = new EvmBatchProcessor().setRpcEndpoint(
          assertNotNull(
            process.env.RPC_ARBITRUM_ONE_HTTP,
            "Required env variable RPC_ARBITRUM_ONE_HTTP is missing"
          )
        );
        // ...the rest of the processor configuration
        ```

        <Info>
          The RPC addon provides reliable access to blockchain nodes for real-time data
          queries and state access.
        </Info>
      </Accordion>

      <Accordion title="PostgreSQL Addon" icon="database">
        For squids that [write their data to PostgreSQL](/en/sdk/squid-sdk/resources/persisting-data/typeorm), use the [`postgres` addon](/en/cloud/reference/pg):

        **Basic configuration:**

        ```yaml squid.yaml theme={"system"}
        deploy:
          addons:
            postgres:
        ```

        **Production scaling:**

        <Warning>
          Always configure the [`scale` section for the
          addon](/en/cloud/reference/pg/#scaling) when deploying to production.
        </Warning>

        ```yaml squid.yaml theme={"system"}
        scale:
          addons:
            postgres:
              storage: 100G
              profile: medium
        ```

        <Check>
          PostgreSQL addon automatically provides database connection credentials
          through environment variables.
        </Check>
      </Accordion>

      <Accordion title="Hasura Addon" icon="server">
        If your squid [uses a dedicated Hasura instance](/en/sdk/squid-sdk/resources/serving-graphql#hasura), configure the [`hasura` addon](/en/cloud/reference/hasura) to provision it.

        ```yaml squid.yaml theme={"system"}
        deploy:
          addons:
            hasura:
        ```

        <Tip>
          Use Hasura when you need advanced GraphQL features like subscriptions,
          permissions, or complex query capabilities beyond the default OpenReader.
        </Tip>
      </Accordion>
    </AccordionGroup>

    ### Scale Settings

    Configure resource allocation for your squid components to ensure optimal performance.

    #### API Service Configuration

    Squids include a GraphQL service by default. You can enable or disable it by adding or removing the `deploy.api` section. Use the [`scale.api` section](/en/cloud/reference/scale/#api) to configure resources and replicas.

    ```yaml squid.yaml theme={"system"}
    deploy:
      api:
        cmd: ["sqd", "serve:prod"]
    scale:
      api:
        profile: large
        # load-balance across three replicas
        replicas: 3
    ```

    <Note>
      This configuration works with
      [OpenReader](/en/sdk/squid-sdk/resources/serving-graphql#openreader) (default) and
      [PostGraphile](/en/sdk/squid-sdk/resources/serving-graphql#postgraphile). For
      [Hasura](/en/sdk/squid-sdk/resources/serving-graphql#hasura), see the [`hasura` addon
      documentation](/en/cloud/reference/hasura).
    </Note>

    #### Processor Configuration

    Configure the indexer processor resources based on your data processing requirements. See the [processor scale reference](/en/cloud/reference/scale/#processor) for all available options.

    ```yaml squid.yaml theme={"system"}
    scale:
      processor:
        profile: medium
    ```

    <Tip>
      Choose a processor profile based on your indexing workload. Start with `small`
      for testing and scale up to `medium` or `large` for production workloads.
    </Tip>

    #### Dedicated Deployment

    [Dedicated deployment](/en/cloud/reference/scale/#dedicated) ensures your squid runs on dedicated resources for optimal performance and reliability. This is the default for professional organizations.

    <Warning>
      If your `squid.yaml` contains `dedicated: false` (from previous playground
      deployments), you must remove it or set it to `true` for production use.
      **Collocated squids are not recommended for production environments.**
    </Warning>

    To explicitly enable dedicated deployment:

    ```yaml squid.yaml theme={"system"}
    scale:
      dedicated: true
    ```

    <Check>
      Dedicated deployment is enabled by default for all professional organization
      deployments.
    </Check>

    ### Complete Configuration Example

    Here's a complete `squid.yaml` file with all deployment options configured:

    ```yaml squid.yaml theme={"system"}
    manifest_version: subsquid.io/v0.1
    name: sample-squid

    build:

    deploy:
      addons:
        postgres:
        rpc:
          - arbitrum-one.http
      processor:
        cmd: ["sqd", "process:prod"]
      api:
        cmd: ["sqd", "serve:prod"]

    scale:
      addons:
        postgres:
          storage: 100G
          profile: medium
      processor:
        profile: medium
      api:
        profile: large
        # load-balance across three replicas
        replicas: 3
      dedicated: true
    ```

    <Check>
      Your `squid.yaml` is now configured for production deployment on SQD Cloud.
    </Check>

    <Tip>
      For detailed information on all available deployment options, see the
      [deployment manifest reference](/en/cloud/reference/manifest).
    </Tip>
  </Step>

  <Step title="Set any required secrets" icon="key">
    If your squid uses any sensitive data such as a private URL or an access key, store it securely in a [SQD Cloud secret](/en/cloud/resources/env-variables/#secrets).

    You can add secrets using either method:

    * **Via SQD Cloud Console**: Navigate to the `Secrets` tab in the sidebar and add your values
    * **Via CLI**: Use the [`sqd secrets`](/en/cloud/reference/cli/secrets) command

    <Warning>
      Never commit secrets or API keys directly to your code repository. Always use
      SQD Cloud secrets for sensitive data.
    </Warning>
  </Step>

  <Step title="Deploy the squid" icon="rocket">
    Deploy your squid to SQD Cloud using the Squid CLI:

    1. Open the **Squids** tab in the SQD Cloud sidebar
    2. Click the **Deploy a squid** button
    3. Follow the prompts to install the Squid CLI if needed
    4. Set up your authentication key as shown in the console
    5. Ensure your squid name matches the one in `squid.yaml`

    Run the deployment command from your project root:

    ```bash theme={"system"}
    sqd deploy .
    ```

    <Info>
      The deployment process will build your squid, upload it to SQD Cloud, and
      start the processor and API services according to your `squid.yaml`
      configuration.
    </Info>

    <Check>
      Once deployed, your squid will appear in the Squids list with status
      information.
    </Check>
  </Step>

  <Step title="Monitor the squid" icon="chart-mixed">
    After deployment, monitor your squid's performance and health through the SQD Cloud dashboard:

    **What you can monitor:**

    * **Logs**: View real-time logs from your processor and API services
    * **Metrics**: Track memory usage, CPU usage, and processing speed
    * **Status**: Check the health status of your squid components
    * **Sync progress**: Monitor blockchain indexing progress

    **To access monitoring:**

    1. Navigate to the **Squids** tab in the SQD Cloud sidebar
    2. Select your deployed squid from the list
    3. View the monitoring dashboard with metrics and logs

    <Tip>
      Visit the [logging documentation](/en/cloud/resources/logging) for tips on
      emitting structured logs and using log filters effectively.
    </Tip>

    <Check>
      Your squid is now running and processing blockchain data on SQD Cloud.
    </Check>
  </Step>

  <Step title="Use the squid" icon="plug">
    Your deployed squid is now accessible and ready to use:

    **Access your data:**

    * **Database access**: If your squid uses PostgreSQL, access credentials are available in the **DB access** tab of your squid's card
    * **GraphQL API**: Access your squid's GraphQL endpoint from the deployment details
    * **Direct queries**: Connect to your database using the provided credentials

    **Production workflows:**

    Squid deployments use [slots and tags](/en/cloud/resources/slots-and-tags) for flexible deployment management. This enables:

    * **Zero downtime updates**: Deploy new versions without interrupting service
    * **Version management**: Maintain multiple versions simultaneously
    * **Traffic routing**: Control which deployment serves production traffic

    <Tip>
      We strongly recommend using the [zero downtime update
      workflow](/en/cloud/resources/slots-and-tags/#zero-downtime-updates) for all
      production squids.
    </Tip>
  </Step>
</Steps>

## What's Next?

Continue optimizing and managing your deployment:

<CardGroup>
  <Card title="Pricing" icon="dollar-sign" href="/en/cloud/pricing/overview">
    Understand SQD Cloud costs and resource allocation
  </Card>

  <Card title="Organizations" icon="users" href="/en/cloud/resources/organizations">
    Manage teams and billing settings
  </Card>

  <Card title="Slots and Tags" icon="tags" href="/en/cloud/resources/slots-and-tags">
    Master zero downtime deployments
  </Card>

  <Card title="Squid CLI" icon="terminal" href="/en/cloud/reference/cli/installation">
    Learn CLI commands for deployment management
  </Card>
</CardGroup>


# SQD Cloud Indexer Hosting
Source: https://docs.sqd.dev/en/cloud/overview

Deploy and scale Squid SDK indexers on SQD Cloud — managed hosting with Postgres, ClickHouse, RPC proxy, and one-command deployments via the sqd CLI.

<Frame>
  <img alt="SQD Cloud - Deploy and scale your squids" />
</Frame>

**Enterprise-grade deployment platform for blockchain data indexers**

SQD Cloud is a managed infrastructure platform that handles the complexity of deploying, scaling, and monitoring your blockchain indexers. Focus on building your data pipelines while we handle the DevOps.

<Warning>
  SQD Cloud currently supports **Squid SDK** indexers only. Pipes SDK deployments are not supported at this time.
</Warning>

## Key Benefits

* **Zero DevOps Required** - Deploy with a single command, no infrastructure management
* **Built-in Monitoring** - Real-time logs, metrics, and health status for all components
* **Automatic Scaling** - Configure resources and replicas to match your workload
* **Production-Ready** - Dedicated deployment, managed databases, and high availability
* **Cost-Effective** - Pay only for resources you use with transparent pricing

## When to Use SQD Cloud

SQD Cloud is ideal for:

* **Production deployments** requiring reliability and uptime
* **Teams** that want to focus on building, not operations
* **Projects** that need monitoring and observability built-in
* **Applications** requiring managed PostgreSQL, RPC access, or GraphQL APIs

<Info>
  Before deploying to SQD Cloud, ensure you have a working indexer locally. See
  [Development flow](/en/sdk/squid-sdk/how-to-start/squid-development) and the [Indexing
  SDK](/en/sdk/overview) section for development guidance.
</Info>

## Getting Started

Ready to deploy your indexer to production? Our comprehensive deployment guide walks you through the entire process, from preparing your squid to monitoring it in production.

The deployment process includes:

* Setting up your SQD Cloud account and organization
* Configuring your `squid.yaml` deployment manifest
* Managing secrets and environment variables
* Deploying with a single command
* Monitoring and managing your production deployment

<Card title="Deploy Your First Squid" icon="rocket" href="/en/cloud/deployment-guide">
  Follow our step-by-step deployment guide to get your indexer running on SQD Cloud in minutes.
</Card>

## What's Next?

Continue optimizing and managing your deployment:

<CardGroup>
  <Card title="Pricing" icon="dollar-sign" href="/en/cloud/pricing/overview">
    Understand SQD Cloud costs and resource allocation
  </Card>

  <Card title="Organizations" icon="users" href="/en/cloud/resources/organizations">
    Manage teams and billing settings
  </Card>

  <Card title="Slots and Tags" icon="tags" href="/en/cloud/resources/slots-and-tags">
    Master zero downtime deployments
  </Card>

  <Card title="Squid CLI" icon="terminal" href="/en/cloud/reference/cli/installation">
    Learn CLI commands for deployment management
  </Card>
</CardGroup>


# Pricing FAQ
Source: https://docs.sqd.dev/en/cloud/pricing/faq

Frequently asked questions about SQD Cloud pricing — billing tiers, free tier limits, and how to estimate costs for your squid deployment.

<AccordionGroup>
  <Accordion title="What's changing with pricing in April 2026?">
    Effective **April 1, 2026**, we're updating our pricing structure:

    * **Cloud Hosting:** 20% increase on all compute resources (API, Processor, Database, Collocated Squids)
    * **Storage:** 20% increase ($0.50 → $0.60 per GB/month)
    * **RPC Add-on:** Overage pricing increases from $2.00 to $5.00 per 1M requests
    * **Free Tier:** Unchanged - 2M RPC requests per month remain free

    **Why the change?** Since launching SQD Cloud over two years ago, we've maintained stable pricing while continuously investing in infrastructure, reliability, and performance. This adjustment allows us to continue delivering high-quality service while investing in new capabilities.

    **When does this take effect?** April 1, 2026. All usage from that date forward will be billed at the new rates.

    For complete details, including pricing tables and FAQs, see the [full pricing announcement](/en/other/pricing-update-april-2026).
  </Accordion>

  <Accordion title="How does billing work? When am I charged?">
    SQD Cloud uses hourly billing for compute resources and monthly billing for storage. You're only charged for the actual time your squids are running.

    **Billing cycle:** Charges are calculated and billed on the first day of each month for the previous month's usage.

    **Prorated charges:** If you deploy a squid mid-month, you only pay for the hours it actually runs. For example, a squid running for 360 hours (half a month) costs exactly half of the monthly rate.

    <Note>
      You can monitor your current usage and estimated costs in the SQD Cloud
      dashboard at any time.
    </Note>
  </Accordion>

  <Accordion title="What is the difference between collocated and dedicated squids?">
    **Collocated squids** share compute resources with other squids on the same infrastructure. This makes them much cheaper (`$0.02/hr` until March 31, 2026, then `$0.024/hr` from April 1, vs separate service costs) but means performance is not guaranteed. If a neighboring squid experiences high load, your squid may become slow or unresponsive.

    **Dedicated squids** have guaranteed compute resources that are never shared. Each service (API, processor, database) runs on its own allocated resources with predictable performance.

    <Warning>
      We strongly recommend dedicated squids for production use. Collocated squids
      are perfect for development, testing, and staging environments.
    </Warning>
  </Accordion>

  <Accordion title="How do I estimate my costs before deploying?">
    Use our [interactive cost calculator](/en/cloud/pricing/overview#estimate-your-costs) to estimate costs based on your configuration. You can also review [real-world examples](/en/cloud/pricing/pricing-details#real-world-cost-examples) for common deployment scenarios.

    <Tip>
      Start with smaller profiles and monitor your actual resource usage. You can
      easily scale up if needed. It's better to start small and grow than to
      overprovision from the start.
    </Tip>
  </Accordion>

  <Accordion title="Can I change my squid's compute profiles after deployment?">
    Yes! You can modify your squid's compute profiles at any time using [manifest-based deployments](/en/cloud/reference/manifest/).

    Simply update your manifest file with the new profiles and redeploy. The new pricing takes effect immediately. See the [profile reference](/en/cloud/reference/scale/) for all available options and their specifications.

    <Info>
      Profile changes require a brief redeployment. Plan accordingly for production
      squids to minimize downtime.
    </Info>
  </Accordion>

  <Accordion title="How is database storage billed? Can I reduce storage costs?">
    Database storage is billed at `$0.50 per GB per month` (until March 31, 2026) or `$0.60 per GB per month` (from April 1, 2026) based on actual usage. This applies to both dedicated and collocated squids.

    **Cost optimization strategies:**

    * Implement data retention policies to remove old data you no longer need
    * Optimize your database schema to reduce redundancy
    * Use appropriate data types (e.g., integers instead of strings where possible)
    * Archive historical data to external storage (S3, BigQuery) if you don't need it for queries
    * Monitor your storage usage in the Cloud dashboard

    <Tip>
      For data-heavy applications, consider a processor-only squid that exports to
      external storage instead of maintaining a large database.
    </Tip>
  </Accordion>

  <Accordion title="What are the 2M free RPC requests? How do I use them?">
    Professional organizations include 2 million RPC requests per month at no additional cost through the [RPC addon](/en/cloud/resources/rpc-proxy). This allows your squids to make blockchain RPC calls without managing your own RPC infrastructure.

    **Overage pricing:** If you exceed 2M requests, additional requests cost `$2.00 per million` (until March 31, 2026) or `$5.00 per million` (from April 1, 2026).

    **How to use:** Configure the RPC proxy in your squid to route requests through SQD's infrastructure. See the [RPC addon documentation](/en/cloud/resources/rpc-proxy/) for setup instructions and rate limiting configuration.
  </Accordion>

  <Accordion title="What happens when I hibernate a squid?">
    When you hibernate a squid, all compute services (API, processor, database) are paused. You only pay for storage costs (`$0.50/GB/month` until March 31, 2026, then `$0.60/GB/month` from April 1) while the squid is hibernated.

    **Benefits:**

    * Preserve your squid's data and configuration
    * Eliminate compute costs when not needed
    * Resume quickly when you need the squid again

    **Perfect for:**

    * Seasonal applications that aren't needed year-round
    * Development squids used only during working hours
    * Backup squids kept ready but not actively running

    <Info>
      Hibernation and resumption are instant operations. You can hibernate and wake
      squids as often as needed.
    </Info>
  </Accordion>

  <Accordion title="How do I decide between collocated and dedicated for my use case?">
    Choose based on your performance requirements and budget:

    **Use collocated when:**

    * Developing or testing new features
    * Running staging environments
    * Building proof-of-concepts or MVPs
    * Budget is the primary concern
    * Occasional slowness is acceptable

    **Use dedicated when:**

    * Running production applications
    * Serving end users who expect consistent performance
    * Processing time-sensitive data
    * You need guaranteed response times
    * Reliability is critical to your application

    <Check>
      Most teams use collocated for development and dedicated for production. This
      gives you cost-effective testing while ensuring production reliability.
    </Check>
  </Accordion>

  <Accordion title="Are there any hidden fees or charges I should know about?">
    No hidden fees! SQD Cloud pricing is transparent and predictable.

    **What you pay for:**

    * Compute (hourly rates for API, processor, database)
    * Storage (monthly rate per GB)
    * RPC requests (only above 2M/month)

    **What's free:**

    * Data egress (outbound traffic)
    * GraphQL API requests (unlimited)
    * SQD Network data access
    * Deployments and redeployments
    * Monitoring and logs

    <Note>
      The only variable costs are compute time and storage. Everything else is
      included with no additional charges.
    </Note>
  </Accordion>

  <Accordion title="Do you offer discounts for high-volume usage or long-term commitments?">
    Yes! We offer enterprise plans with volume discounts, custom SLAs, and dedicated infrastructure for high-volume use cases.

    **Enterprise benefits may include:**

    * Volume-based pricing discounts
    * Custom SLA terms
    * Dedicated support channels
    * Architecture consultation
    * Priority feature requests

    [Book a demo](https://calendly.com/t-tyrie-subsquid/30min) to discuss your requirements and get a personalized quote.
  </Accordion>
</AccordionGroup>


# Pricing Overview
Source: https://docs.sqd.dev/en/cloud/pricing/overview

SQD Cloud pricing overview — subscription tiers and pay-as-you-go pricing for squid hosting, Postgres addons, RPC proxy, and storage usage.

<Warning>
  **Pricing Update:** Effective April 1, 2026, Cloud hosting prices will increase by 20% and RPC add-on pricing will update to \$5 per 1M requests. [Learn more about these changes](/en/other/pricing-update-april-2026).
</Warning>

SQD Cloud uses a transparent pay-as-you-go pricing model. You only pay for the compute and storage resources your squids consume. Data from the SQD Network is provided free of charge, with no hidden fees or data egress costs.

<Tip>
  Use our interactive cost calculator below to estimate your monthly costs based
  on your specific requirements. Estimates use pricing effective April 1, 2026.
</Tip>

## Estimate Your Costs

<CostCalculator />

## Free Tier

<Info>
  Each account includes a free playground
  [organization](/en/cloud/resources/organizations) where you can deploy a
  single squid at no cost. This is perfect for testing, learning, and small
  projects. See [playground
  organization](/en/cloud/resources/organizations/#playgrounds) for limitations
  and details.
</Info>

## Professional Organizations

To deploy production-ready squids with full [resource configuration](/en/cloud/reference/scale), upgrade to a [Professional organization](/en/cloud/resources/organizations/#professional-organizations). Professional status unlocks powerful features and generous allowances:

<CardGroup>
  <Card title="Unlimited API Requests" icon="infinity">
    No metering or throttling on GraphQL API requests. Scale your application without worrying about query limits.
  </Card>

  <Card title="Flexible Deployment Options" icon="server">
    Deploy unlimited [dedicated](/en/cloud/reference/scale/#dedicated) and
    [collocated](/en/cloud/reference/scale/#collocated) squids with full control
    over compute profiles.
  </Card>

  <Card title="Generous RPC Allowance" icon="network-wired">
    2 million [RPC addon](/en/cloud/resources/rpc-proxy) requests included
    monthly, then `$2` per additional million requests (until March 31, 2026) or `$5` per additional million requests (from April 1, 2026).
  </Card>

  <Card title="Priority Support" icon="headset">
    Get faster response times and dedicated assistance from our support team for production issues.
  </Card>
</CardGroup>

## Billing Details

Billing is based on actual resource consumption and occurs monthly, typically on the first day of each month. You're only charged for the hours your squids are running and the storage they use.

<Note>
  All compute resources are billed hourly. If you deploy a squid mid-month,
  you'll only pay for the actual hours it runs, not the full month.
</Note>

## Enterprise Plans

<Tip>
  Need custom SLAs, volume discounts, or dedicated infrastructure? We offer
  enterprise plans tailored to high-volume use cases. [Book a
  demo](https://calendly.com/t-tyrie-subsquid/30min) to discuss your
  requirements and get a personalized quote.
</Tip>

## Next Steps

Ready to dive into the details? Check out our [Pricing Details & Examples](/en/cloud/pricing/pricing-details) page for complete pricing tables, profile specifications, and real-world cost calculations.


# Pricing Details & Examples
Source: https://docs.sqd.dev/en/cloud/pricing/pricing-details

Detailed SQD Cloud pricing — billing rates, SLAs, addon costs, and example calculations for production-grade Squid SDK deployments at scale.

<Warning>
  **Pricing Update:** Effective April 1, 2026, Cloud hosting prices will increase by 20% and RPC add-on pricing will update to \$5 per 1M requests. [Learn more about these changes](/en/other/pricing-update-april-2026).
</Warning>

SQD Cloud pricing is straightforward: you pay only for the compute and storage resources your squids consume. The SQD Network provides data at no charge, making your costs predictable and transparent.

<Note>
  **Good news:** Data egress is completely free. No charges for outbound traffic
  or API responses.
</Note>

## Pay-as-you-go Pricing

Your total monthly cost is the sum of:

* **Compute costs**: Hourly rates for API, processor, and database services
* **Storage costs**: Monthly rate per GB of database storage
* **RPC costs**: Only if you exceed the 2M free monthly requests

### API and Processor Compute

Both API and processor services use the same pricing structure. They are configured and billed independently.

#### Pricing Until March 31, 2026

| Profile     | Price per Hour | Monthly Cost (24/7) | Best For                                                           |
| ----------- | -------------- | ------------------- | ------------------------------------------------------------------ |
| **small**   | `$0.04`        | `~$29`              | Development, low-traffic APIs, simple data processing              |
| **medium**  | `$0.08`        | `~$58`              | Production APIs with moderate traffic, standard indexing workloads |
| **large**   | `$0.15`        | `~$108`             | High-traffic APIs, complex data transformations                    |
| **xlarge**  | `$0.30`        | `~$216`             | Very high traffic, intensive processing requirements               |
| **2xlarge** | `$0.60`        | `~$432`             | Enterprise-scale applications, maximum performance needs           |

#### Pricing From April 1, 2026

| Profile     | Price per Hour | Monthly Cost (24/7) | Best For                                                           |
| ----------- | -------------- | ------------------- | ------------------------------------------------------------------ |
| **small**   | `$0.048`       | `~$35`              | Development, low-traffic APIs, simple data processing              |
| **medium**  | `$0.096`       | `~$70`              | Production APIs with moderate traffic, standard indexing workloads |
| **large**   | `$0.18`        | `~$130`             | High-traffic APIs, complex data transformations                    |
| **xlarge**  | `$0.36`        | `~$259`             | Very high traffic, intensive processing requirements               |
| **2xlarge** | `$0.72`        | `~$518`             | Enterprise-scale applications, maximum performance needs           |

<Info>
  **API Replicas:** If you run multiple API replicas, each replica is billed
  separately. For example, 2 replicas of medium API = `2 × $0.08 = $0.16/hr`.
</Info>

Full specifications and resource allocations:

* [API service profiles](/en/cloud/reference/scale/#api)
* [Processor service profiles](/en/cloud/reference/scale/#processor)

***

### Database Compute

Dedicated Postgres database instances with guaranteed resources.

#### Pricing Until March 31, 2026

| Profile     | Price per Hour | Monthly Cost (24/7) | vCPU | RAM  | Best For                                   |
| ----------- | -------------- | ------------------- | ---- | ---- | ------------------------------------------ |
| **small**   | `$0.08`        | `~$58`              | 1    | 2GB  | Small datasets, development                |
| **medium**  | `$0.16`        | `~$115`             | 2    | 4GB  | Medium datasets, moderate query load       |
| **large**   | `$0.33`        | `~$238`             | 4    | 8GB  | Large datasets, high query throughput      |
| **xlarge**  | `$0.66`        | `~$475`             | 8    | 16GB | Very large datasets, complex queries       |
| **2xlarge** | `$1.32`        | `~$950`             | 16   | 32GB | Enterprise-scale data, maximum performance |

#### Pricing From April 1, 2026

| Profile     | Price per Hour | Monthly Cost (24/7) | vCPU | RAM  | Best For                                   |
| ----------- | -------------- | ------------------- | ---- | ---- | ------------------------------------------ |
| **small**   | `$0.096`       | `~$69`              | 1    | 2GB  | Small datasets, development                |
| **medium**  | `$0.192`       | `~$138`             | 2    | 4GB  | Medium datasets, moderate query load       |
| **large**   | `$0.396`       | `~$286`             | 4    | 8GB  | Large datasets, high query throughput      |
| **xlarge**  | `$0.792`       | `~$570`             | 8    | 16GB | Very large datasets, complex queries       |
| **2xlarge** | `$1.584`       | `~$1,140`           | 16   | 32GB | Enterprise-scale data, maximum performance |

<Note>
  Database costs include both compute (above) and storage (billed separately at
  `$0.50/GB/month` until March 31, 2026, then `$0.60/GB/month` from April 1, 2026). See the [database scaling
  reference](/en/cloud/reference/pg/#scaling) for detailed specifications.
</Note>

***

### Database Storage

Storage is billed monthly based on actual usage.

**Price Until March 31, 2026:** `$0.50 per GB per month`

**Price From April 1, 2026:** `$0.60 per GB per month`

Applies to both dedicated and collocated squids.

<Tip>
  Monitor your storage usage in the Cloud dashboard. Optimize your schema and
  data retention policies to control costs.
</Tip>

### Collocated Squids (Development)

Collocated squids bundle API, processor, and database into a single shared-resource deployment.

**Compute Until March 31, 2026:** `$0.02/hr` (`~$15/month`)
**Compute From April 1, 2026:** `$0.024/hr` (`~$17/month`)

**Storage:** `$0.50/GB/month` until March 31, `$0.60/GB/month` from April 1 (same as dedicated)

<Warning>
  Collocated squids share compute resources and performance is **not
  guaranteed**. They're perfect for development and testing but **not
  recommended for production** use.
</Warning>

### RPC Requests

Professional organizations include 2 million RPC requests per month at no additional cost.

**Overage Pricing Until March 31, 2026:** `$2.00 per million requests`

**Overage Pricing From April 1, 2026:** `$5.00 per million requests`

See [RPC addon documentation](/en/cloud/resources/rpc-proxy) for configuration details.

### Hibernated Squids

Hibernated squids are paused and only incur storage costs. No compute charges apply.

**Cost Until March 31, 2026:** Storage only (`$0.50/GB/month`)

**Cost From April 1, 2026:** Storage only (`$0.60/GB/month`)

<Info>
  Hibernation is perfect for squids you want to keep available but don't need
  running continuously.
</Info>

### Multi-Processor Squids

Each processor in a multi-processor squid is billed independently based on its configured profile.

**Example (Until March 31, 2026):** 2 small processors + 1 medium processor = `2 × $0.04 + $0.08 = $0.16/hr`

**Example (From April 1, 2026):** 2 small processors + 1 medium processor = `2 × $0.048 + $0.096 = $0.192/hr`

## Service Level Agreement (SLA)

Our SLA applies to [dedicated](/en/cloud/reference/scale/#dedicated) squids deployed to [Professional organizations](/en/cloud/resources/organizations/#professional-organizations).

We align with the [SLAs of the Google Cloud Platform](https://cloud.google.com/compute/sla) and provide bill reductions based on uptime:

| Uptime Percentage  | Bill Reduction |
| ------------------ | -------------- |
| 95.00% - \< 99.50% | 10%            |
| 90.00% - \< 95.00% | 25%            |
| \< 90.00%          | 100%           |

<Warning>
  The SLA covers SQD Cloud infrastructure (API availability, compute
  provisioning) but **does not** cover issues caused by client code, bugs in
  your squid, or upstream problems with SQD Network/RPC providers. You're
  responsible for provisioning adequate compute resources for your expected
  traffic.
</Warning>

<Info>
  Enterprise customers can negotiate custom SLA terms. [Contact
  us](https://calendly.com/t-tyrie-subsquid/30min) to discuss your
  requirements.
</Info>

## Real-World Cost Examples

Use these examples to understand how different deployment configurations translate to monthly costs. All examples assume 24/7 operation (720 hours per month).

<Info>
  **Note:** Examples below use pricing effective April 1, 2026. For current pricing (until March 31, 2026), refer to the pricing tables above.
</Info>

<CardGroup>
  <Card title="Small Dedicated Squid" icon="seedling">
    **Use case:** Personal project or small application with moderate data requirements and low traffic.

    **Configuration:**

    * 1× API service (small)
    * 1× Processor (small)
    * 1× Database (small)
    * 50GB storage

    | Component         | Rate        | Monthly Cost  |
    | ----------------- | ----------- | ------------- |
    | API (small)       | `$0.048/hr` | `$34.56`      |
    | Processor (small) | `$0.048/hr` | `$34.56`      |
    | Database (small)  | `$0.096/hr` | `$69.12`      |
    | Storage (50GB)    | `$0.60/GB`  | `$30.00`      |
    | **Total**         | \*\*, \*\*  | **`$168.24`** |

    <Check>
      Perfect for side projects, MVPs, and applications with predictable
      low-to-moderate traffic.
    </Check>
  </Card>

  <Card title="Development Squid" icon="flask">
    **Use case:** Testing, development, or staging environment where performance guarantees aren't critical.

    **Configuration:**

    * Collocated (API + Processor + DB)
    * 50GB storage

    | Component          | Rate        | Monthly Cost |
    | ------------------ | ----------- | ------------ |
    | Collocated compute | `$0.024/hr` | `$17.28`     |
    | Storage (50GB)     | `$0.60/GB`  | `$30.00`     |
    | **Total**          | \*\*, \*\*  | **`$47.28`** |

    <Check>
      Most cost-effective option for non-production environments and
      experimentation.
    </Check>
  </Card>

  <Card title="Production Squid" icon="rocket">
    **Use case:** Production application with high traffic, requiring load balancing and robust data processing.

    **Configuration:**

    * 2× API service (medium) - for redundancy
    * 1× Processor (small)
    * 1× Database (large)
    * 500GB storage

    | Component         | Rate        | Monthly Cost  |
    | ----------------- | ----------- | ------------- |
    | 2× API (medium)   | `$0.192/hr` | `$138.24`     |
    | Processor (small) | `$0.048/hr` | `$34.56`      |
    | Database (large)  | `$0.396/hr` | `$285.12`     |
    | Storage (500GB)   | `$0.60/GB`  | `$300.00`     |
    | **Total**         | \*\*, \*\*  | **`$757.92`** |

    <Check>
      Ideal for production applications with significant traffic and data processing
      needs.
    </Check>
  </Card>

  <Card title="Data Export Pipeline" icon="database">
    **Use case:** Processor-only squid that indexes blockchain data and exports to external storage (S3, BigQuery, etc.). No API or database needed.

    **Configuration:**

    * 1× Processor (small)
    * No database or API
    * Minimal storage

    | Component         | Rate        | Monthly Cost |
    | ----------------- | ----------- | ------------ |
    | Processor (small) | `$0.048/hr` | `$34.56`     |
    | **Total**         | \*\*, \*\*  | **`$34.56`** |

    <Check>
      Most economical option for data pipelines that don't need to serve queries.
    </Check>
  </Card>
</CardGroup>

<Tip>
  **Cost optimization tips:** - Start with smaller profiles and scale up based
  on actual usage - Use collocated squids for development and testing - Consider
  hibernating squids that don't need to run 24/7 - Monitor storage usage and
  implement data retention policies - Use the [cost
  calculator](/en/cloud/pricing/overview#estimate-your-costs) to model different
  scenarios
</Tip>


# Authentication
Source: https://docs.sqd.dev/en/cloud/reference/cli/auth

Authenticate the sqd CLI with SQD Cloud — exchange your account credentials for an API token and run authenticated deploy and admin commands.

# `sqd auth`

Log in to the Cloud

* [`sqd auth`](#sqd-auth-1)

## `sqd auth`

Log in to the Cloud

```
USAGE
  $ sqd auth -k <value> [--interactive]

FLAGS
  -k, --key=<value>       (required) Cloud auth key. Log in to https://app.subsquid.io to create or update your key.
      --[no-]interactive  Disable interactive mode
```

*See code: [src/commands/auth.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/auth.ts)*


# Shell Autocomplete
Source: https://docs.sqd.dev/en/cloud/reference/cli/autocomplete

Enable shell autocomplete for the sqd CLI — install completion scripts for bash, zsh, fish, and PowerShell to speed up Squid command entry.

# `sqd autocomplete`

Display autocomplete installation instructions.


# Commands Configuration
Source: https://docs.sqd.dev/en/cloud/reference/cli/commands-json

Define custom sqd CLI commands and aliases via commands.json — extend the squid project workflow with reusable scripts for build, lint, and deploy.

The `sqd` tool automatically discovers and loads any extra commands defined in the `commands.json` file. Here is a sample file demonstrating the available features:

```json theme={"system"}
{ // comments are ok
  "$schema": "https://subsquid.io/schemas/commands.json",
  "commands": {
    "clean": {
      "description": "delete all build artifacts",
      "cmd": ["rm", "-rf", "lib"]
    },
    "build": {
      "description": "build the project",
      "deps": ["clean"], // commands to execute before
      "cmd": ["tsc"]
    },
    "typegen": {
      "hidden": true, // Don't show in the overview listing
      "workdir": "abi", // change working dir
      "command": [
        "squid-evm-typegen", // node_modules/.bin is in the PATH
        "../src/abi",
        \{"glob": "*.json"\} // cross-platform glob expansion
      ],
      "env": { // additional environment variables
        "DEBUG": "*"
      }
    }
  }
}
```

This functionality is managed by the [`@subsquid/commands`](https://github.com/subsquid/squid-sdk/tree/master/util/commands) package.

All [squid templates](/en/sdk/squid-sdk/how-to-start/squid-development#templates) include such a file with a predefined set of useful shortcuts. See [Cheatsheet](/en/sdk/squid-sdk/how-to-start/cli-cheatsheet).


# Deploy Command
Source: https://docs.sqd.dev/en/cloud/reference/cli/deploy

Use the sqd deploy CLI command to push a squid to SQD Cloud — flags for slot, tag, manifest path, and how to handle ongoing deployments.

# `sqd deploy`

Deploy new or update an existing squid deployment in the Cloud. Squid name and also optionally slot and/or tag are taken from the provided deployment manifest.

* [`sqd deploy SOURCE`](#sqd-deploy-source)

## `sqd deploy SOURCE`

Deploy new or update an existing squid in the Cloud

```
USAGE
  $ sqd deploy SOURCE [--interactive]
    [-r [<org>/]<name>(@<slot>|:<tag>) | -o <code> | -n <name> | [-s <slot>] | [-t <tag>]]
    [-m <manifest_path>] [--hard-reset] [--stream-logs] [--add-tag <value>]
    [--allow-update] [--allow-tag-reassign] [--allow-manifest-override]

ARGUMENTS
  SOURCE  [default: .] Squid source. Could be:
          - a relative or absolute path to a local folder (e.g. ".")
          - a URL to a .tar.gz archive
          - a github URL to a git repo with a branch or commit tag

FLAGS
  -m, --manifest=<manifest_path>  [default: squid.yaml] Specify the relative local path
                                  to a squid manifest file in the squid working directory
      --add-tag=<value>           Add a tag to the deployed squid
      --allow-manifest-override   Allow overriding the manifest during deployment
      --allow-tag-reassign        Allow reassigning an existing tag
      --allow-update              Allow updating an existing squid
      --hard-reset                Perform a hard reset before deploying. This will drop
                                  and re-create all squid resources, including the
                                  database, causing a short API downtime
      --[no-]interactive          Disable interactive mode
      --[no-]stream-logs          Attach and stream squid logs after the deployment

SQUID FLAGS
  -n, --name=<name>                               Name of the squid
  -r, --reference=[<org>/]<name>(@<slot>|:<tag>)  Fully qualified reference of the squid.
                                                  It can include the organization, name,
                                                  slot, or tag
  -s, --slot=<slot>                               Slot of the squid
  -t, --tag=<tag>                                 Tag of the squid

ORG FLAGS
  -o, --org=<code>  Code of the organization

DESCRIPTION
  Deploy new or update an existing squid in the Cloud

EXAMPLES

  // Create a new squid with name provided in the manifest file
  $ sqd deploy .

  // Create a new squid deployment and override it's name to "my-squid-override"
  $ sqd deploy . -n my-squid-override

  // Update the "my-squid" squid with slot "asmzf5"
  $ sqd deploy . -n my-squid -s asmzf5

  // Use a manifest file located in ./path-to-the-squid/squid.prod.yaml
  $ sqd deploy ./path-to-the-squid -m squid.prod.yaml

  // Full paths are also fine
  $ sqd deploy /Users/dev/path-to-the-squid -m /Users/dev/path-to-the-squid/squid.prod.yaml
```

*See code: [src/commands/deploy.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/deploy.ts)*


# Explorer Command
Source: https://docs.sqd.dev/en/cloud/reference/cli/explorer

Use the sqd explorer CLI command to inspect deployed squids — view logs, environment, status, and live deployment details from your terminal.

# `sqd explorer`

<Warning>
  `sqd explorer` is disabled in `@subsquid/cli>=3.0.0`. If you've been using it, please let us know in the [SquidDevs Telegram channel](https://t.me/HydraDevs).
</Warning>

Visual explorer of deployed squids

Left pane: List of deployed squids.
Right pane: details of the selected squid. Navigate by pressing the corresponding number (e.g. `1` for Summary).

1. Summary: endpoint URL, sync status, DB storage utilization

2. Logs

3. DB access details

This command requires specifying an [organization](/en/cloud/resources/organizations) with the `-o/--org` flag when invoked by accounts with more than one organization. SQD Cloud users with just one organization can omit this flag.

)

*See code: [src/commands/explorer.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/explorer.ts)*


# Gateway Management
Source: https://docs.sqd.dev/en/cloud/reference/cli/gateways

Manage SQD Network gateway endpoints with the sqd gateways CLI — list available gateways and configure which ones your local squids connect to.

# `sqd gateways`

Explore data sources for a squid

* [`sqd gateways list`](#sqd-gateways-list)

## `sqd gateways list`

List available gateways

```
USAGE
  $ sqd gateways list [--interactive] [-t <evm|substrate>] [-n <regex>] [-c <number>]

FLAGS
  -c, --chain=<number>        Filter by chain ID or SS58 prefix
  -n, --name=<regex>          Filter by network name
  -t, --type=<evm|substrate>  Filter by network type
      --[no-]interactive      Disable interactive mode

ALIASES
  $ sqd gateways ls
```

*See code: [src/commands/gateways/ls.ts](https://github.com/subsquid/squid-cli/blob/master/src/commands/gateways/ls.ts)*


# Initialize Project
Source: https://docs.sqd.dev/en/cloud/reference/cli/init

Initialize a new squid project from a template with sqd init — choose EVM, Substrate, Solana, or Tron starter, scaffold files, and install dependencies.

# `sqd init`

Setup a new squid project from a template or github repo

* [`sqd init NAME`](#sqd-init-name)

## `sqd init NAME`

Setup a new squid project from a template or github repo

```
USAGE
  $ sqd init NAME [--interactive] [-t <value>] [-d <value>] [-r]

ARGUMENTS
  NAME  The squid name. It must contain only alphanumeric or dash ("-") symbols and must not start with "-".

FLAGS
  -d, --dir=<value>
      The target location for the squid. If omitted, a new folder NAME is created.

  -r, --remove
      Clean up the target directory if it exists

  -t, --template=<value>
      A template for the squid. Accepts:
      - a github repository URL containing a valid squid.yaml manifest in the root folder
      or one of the pre-defined aliases:
      - evm  A minimal squid template for indexing EVM data.
      - abi  A template to auto-generate a squid indexing events and txs from a contract ABI
      - multichain  A template for indexing data from multiple chains
      - gravatar  A sample EVM squid indexing the Gravatar smart contract on Ethereum.
      - substrate  A template squid for indexing Substrate-based chains.
      - ink  A template for indexing Ink! smart contracts
      - ink-abi  A template to auto-generate a squid from an ink! contract ABI
      - frontier-evm  A template for indexing Frontier EVM chains, like Moonbeam and Astar.

  --[no-]interactive
      Disable interactive mode
```

*See code: [src/commands/init.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/init.ts)*


# CLI Installation
Source: https://docs.sqd.dev/en/cloud/reference/cli/installation

Install the sqd CLI on macOS, Linux, or Windows — npm and standalone install methods for the SQD Cloud command-line tool with shell autocomplete.

Squid CLI is a command line tool for

* scaffolding new squids from templates
* running SDK tools and scripts defined in `commands.json` in a cross-platform way
* managing squid deployments in [SQD Cloud](/en/cloud/overview) (former Aquarium)

The CLI is distributed as a [`npm` package](https://www.npmjs.com/package/@subsquid/cli).

To install Squid CLI, follow the steps below.

## 0. Install and setup Squid CLI

First, install the latest version of Squid CLI as a global `npm` package:

```bash theme={"system"}
npm i -g @subsquid/cli@latest
```

Check the version:

```bash theme={"system"}
sqd --version
```

Make sure the output looks like `@subsquid/cli@<version>`.

<Info>
  The next steps are **optional** for building and running squids. A key is required to enable the CLI commands managing the [SQD Cloud](/en/cloud/overview) deployments.
</Info>

## 1. Obtain a SQD Cloud deployment key

Sign in to [Cloud](https://app.subsquid.io/), and obtain (or refresh) the deployment key page by clicking at the profile picture > "Deployment key":

## 2. Authenticate Squid CLI

Open a terminal window and run

```bash theme={"system"}
sqd auth -k <DEPLOYMENT_KEY>
```

## 3. Explore with `--help`

Use `sqd --help` to get a list of the available command and `sqd <command> --help` to get help on the available options for a specific command, e.g.

```bash theme={"system"}
sqd deploy --help
```


# List Squids
Source: https://docs.sqd.dev/en/cloud/reference/cli/list

List your deployed squids on SQD Cloud with sqd ls — show running, stopped, and archived deployments per organization with their slots and tags.

# `sqd list`

List squids deployed to the Cloud

* [`sqd list`](#sqd-list-1)

## `sqd list`

List squids deployed to the Cloud

```
USAGE
  $ sqd list [--interactive] [--truncate]
    [-r [<org>/]<name>(@<slot>|:<tag>) | -o <code> | -n <name> | [-s <slot>] | [-t <tag>]]

FLAGS
  --[no-]interactive  Disable interactive mode
  --[no-]truncate     Truncate data in columns: false by default

SQUID FLAGS
  -n, --name=<name>                               Name of the squid
  -r, --reference=[<org>/]<name>(@<slot>|:<tag>)  Fully qualified reference
                                                  of the squid. It can include
                                                  the organization, name, slot,
                                                  or tag
  -s, --slot=<slot>                               Slot of the squid
  -t, --tag=<tag>                                 Tag of the squid

ORG FLAGS
  -o, --org=<code>  Code of the organization

ALIASES
  $ sqd ls
```

*See code: [src/commands/ls.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/ls.ts)*


# View Logs
Source: https://docs.sqd.dev/en/cloud/reference/cli/logs

View live and historical squid logs from SQD Cloud with the sqd logs CLI — filter by container, time range, slot, and tag for efficient debugging.

# `sqd logs`

Fetch logs from a squid deployed to the Cloud

* [`sqd logs`](#sqd-logs-1)

## `sqd logs`

Fetch logs from a squid deployed to the Cloud

```
USAGE
  $ sqd logs [--interactive] [--since <value>] [--search <value>] [-f | -p <value>]
    [-r [<org>/]<name>(@<slot>|:<tag>) | -o <code> | [-s <slot> -n <name>] | [-t <tag> ]]
    [-c processor|query-node|api|db-migrate|db...]
    [-l error|debug|info|warning...] [--since <value>]

FLAGS
  -c, --container=<option>...  Container name
                               <options: processor|query-node|api|db-migrate|db>
  -f, --follow                 Follow
  -l, --level=<option>...      Log level
                               <options: error|debug|info|warning>
  -p, --pageSize=<value>       [default: 100] Logs page size
      --[no-]interactive       Disable interactive mode
      --search=<value>         Filter by content
      --since=<value>          [default: 1d] Filter by date/interval

SQUID FLAGS
  -n, --name=<name>                               Name of the squid
  -r, --reference=[<org>/]<name>(@<slot>|:<tag>)  Fully qualified reference of the squid.
                                                  It can include the organization, name,
                                                  slot, or tag
  -s, --slot=<slot>                               Slot of the squid
  -t, --tag=<tag>                                 Tag of the squid

ORG FLAGS
  -o, --org=<code>  Code of the organization
```

Notes:

* `--since` accepts the notation of the [`ms` library](https://www.npmjs.com/package/ms): 1d, 10h, 1m.
* With `--level=error` `sqd` will fetch logs emitted by three different calls to the [SDK Logger](/en/sdk/squid-sdk/reference/logger), namely
  * `ctx.log.error`
  * `ctx.log.fatal`
  * `ctx.log.trace`

*See code: [src/commands/logs.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/logs.ts)*


# Production Deployment
Source: https://docs.sqd.dev/en/cloud/reference/cli/prod

Reference for the deprecated sqd prod command — legacy production deployment workflow now replaced by the slots-and-tags model with sqd deploy --tag prod.

`sqd prod` and the production aliases system are deprecated starting from `@subsquid/cli@3.0.0`. Zero downtime updates are now handled by the [slots and tags system](/en/cloud/resources/slots-and-tags). See also the [changelog](/en/cloud/reference/deployments-two-changelog) for a comparison between production aliases and the new system.


# Remove Squid
Source: https://docs.sqd.dev/en/cloud/reference/cli/remove

Remove a deployed squid from SQD Cloud with the sqd rm command — delete a specific slot or tag and free up resources from your organization quota.

# `sqd remove`

Remove a squid deployed to the Cloud

* [`sqd remove`](#sqd-remove-1)

## `sqd remove`

Remove a squid deployed to the Cloud

```
USAGE
  $ sqd remove [--interactive] [-f]
    [-r [<org>/]<name>(@<slot>|:<tag>) | -o <code> | [-s <slot> -n <name>] | [-t <tag> ]]

FLAGS
  -f, --force             Does not prompt before removing a squid or its version
      --[no-]interactive  Disable interactive mode

SQUID FLAGS
  -n, --name=<name>                               Name of the squid
  -r, --reference=[<org>/]<name>(@<slot>|:<tag>)  Fully qualified reference of the squid. It can include the organization, name, slot, or tag
  -s, --slot=<slot>                               Slot of the squid
  -t, --tag=<tag>                                 Tag of the squid

ORG FLAGS
  -o, --org=<code>  Code of the organization

ALIASES
  $ sqd rm
```

*See code: [src/commands/rm.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/rm.ts)*


# Restart Squid
Source: https://docs.sqd.dev/en/cloud/reference/cli/restart

Restart a deployed squid on SQD Cloud with the sqd restart CLI — recycle the processor container without redeploying source code or schema migrations.

# `sqd restart`

Restart a squid deployed to the Cloud

* [`sqd restart`](#sqd-restart-1)

## `sqd restart`

Restart a squid deployed to the Cloud

```
USAGE
  $ sqd restart [--interactive]
    [-r [<org>/]<name>(@<slot>|:<tag>) | -o <code> | [-s <slot> -n <name>] | [-t <tag> ]]

FLAGS
  --[no-]interactive  Disable interactive mode

SQUID FLAGS
  -n, --name=<name>                               Name of the squid
  -r, --reference=[<org>/]<name>(@<slot>|:<tag>)  Fully qualified reference of the squid.
                                                  It can include the organization, name,
                                                  slot, or tag
  -s, --slot=<slot>                               Slot of the squid
  -t, --tag=<tag>                                 Tag of the squid

ORG FLAGS
  -o, --org=<code>  Code of the organization
```

*See code: [src/commands/restart.ts](https://github.com/subsquid/squid-cli/blob/master/src/commands/restart.ts)*


# Run Squid Locally
Source: https://docs.sqd.dev/en/cloud/reference/cli/run

Run a squid processor and GraphQL server locally with sqd run — load env vars, apply migrations, and exercise the indexer end-to-end before deployment.

# `sqd run`

Run a squid locally according to the [deployment manifest](/en/cloud/reference/manifest).

* [sqd run PATH](#sqd-run-path)

Notes:

* The command is especially useful for running [multichain squids](/en/sdk/squid-sdk/resources/multichain), as it runs all services in the same terminal and handles failures gracefully.

## `sqd run PATH`

Run a squid project locally

```
USAGE
  $ sqd run PATH [--interactive] [-m <value>] [-f <value>] [-i <value>... | -e <value>...] [-r <value>]

FLAGS
  -e, --exclude=<value>...  Do not run specified services
  -f, --envFile=<value>     [default: .env] Relative path to an additional environment file
  -i, --include=<value>...  Run only specified services
  -m, --manifest=<value>    [default: squid.yaml] Relative path to a squid manifest file
  -r, --retries=<value>     [default: 5] Attempts to restart failed or stopped services
      --[no-]interactive    Disable interactive mode
```

*See code: [src/commands/run.ts](https://github.com/subsquid/squid-cli/blob/master/src/commands/run.ts)*


# Manage Secrets
Source: https://docs.sqd.dev/en/cloud/reference/cli/secrets

Manage squid secrets in SQD Cloud with the sqd secrets CLI — set API keys, RPC endpoints, and database credentials securely per organization.

# `sqd secrets`

Manage account secrets

The secrets are [exposed as a context](/en/cloud/resources/env-variables/#secrets), and are accessible to all the squids deployed by the current SQD Cloud [organization](/en/cloud/resources/organizations).

* [`sqd secrets list`](#sqd-secrets-list)
* [`sqd secrets remove NAME`](#sqd-secrets-remove-name)
* [`sqd secrets set NAME VALUE`](#sqd-secrets-set-name-value)

## `sqd secrets list`

List organization secrets in the Cloud

```
USAGE
  $ sqd secrets list [--interactive] [-o <value>]

FLAGS
  -o, --org=<value>       Organization
      --[no-]interactive  Disable interactive mode

ALIASES
  $ sqd secrets ls
```

*See code: [src/commands/secrets/ls.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/secrets/ls.ts)*

## `sqd secrets remove NAME`

Delete an organization secret in the Cloud

```
USAGE
  $ sqd secrets remove NAME [--interactive] [-o <value>]

ARGUMENTS
  NAME  The secret name

FLAGS
  -o, --org=<value>       Organization
      --[no-]interactive  Disable interactive mode

ALIASES
  $ sqd secrets rm
```

*See code: [src/commands/secrets/rm.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/secrets/rm.ts)*

## `sqd secrets set NAME VALUE`

Add or update an organization secret in the Cloud. If value is not specified, it is read from standard input. The secret will be exposed as an environment variable with the given name to all the squids in the organization. NOTE: The changes take affect only after a squid is restarted or updated.

```
USAGE
  $ sqd secrets set NAME VALUE [--interactive] [-o <value>]

ARGUMENTS
  NAME   The secret name
  VALUE  The secret value

FLAGS
  -o, --org=<value>       Organization
      --[no-]interactive  Disable interactive mode
```

*See code: [src/commands/secrets/set.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/secrets/set.ts)*


# Version Tags
Source: https://docs.sqd.dev/en/cloud/reference/cli/tags

Manage version tags for SQD Cloud deployments with sqd tags — promote, alias, and roll back squid releases without redeploying source code.

# `sqd tags`

Manage squid deployements' tags. See the [slots and tags guide](/en/cloud/resources/slots-and-tags).

* [`sqd tags add`](#sqd-tags-add)
* [`sqd tags remove`](#sqd-tags-remove)

## `sqd tags add`

Add a tag to a squid deployment / slot

```
USAGE
  $ sqd tags add TAG [--interactive] [--allow-tag-reassign]
    [-r [<org>/]<name>(@<slot>|:<tag>) | -o <code> | [-s <slot> -n <name>] | [-t <tag> ]]

ARGUMENTS
  TAG  New tag to assign

FLAGS
  --allow-tag-reassign  Allow reassigning an existing tag
  --[no-]interactive    Disable interactive mode

SQUID FLAGS
  -n, --name=<name>                               Name of the squid
  -r, --reference=[<org>/]<name>(@<slot>|:<tag>)  Fully qualified reference of the squid.
                                                  It can include the organization, name,
                                                  slot, or tag
  -s, --slot=<slot>                               Slot of the squid
  -t, --tag=<tag>                                 Tag of the squid

ORG FLAGS
  -o, --org=<code>  Code of the organization
```

*See code: [src/commands/tags/add.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/tags/add.ts)*

## `sqd tags remove`

Remove a tag from a squid deployment / slot

```
USAGE
  $ sqd tags remove TAG [--interactive]
    [-r [<org>/]<name>(@<slot>|:<tag>) | -o <code> | [-s <slot> -n <name>] | [-t <tag> ]]

ARGUMENTS
  TAG  New tag to assign

FLAGS
  --[no-]interactive  Disable interactive mode

SQUID FLAGS
  -n, --name=<name>                               Name of the squid
  -r, --reference=[<org>/]<name>(@<slot>|:<tag>)  Fully qualified reference of the squid.
                                                  It can include the organization, name,
                                                  slot, or tag
  -s, --slot=<slot>                               Slot of the squid
  -t, --tag=<tag>                                 Tag of the squid

ORG FLAGS
  -o, --org=<code>  Code of the organization
```

*See code: [src/commands/tags/remove.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/tags/remove.ts)*


# Check Authentication Status
Source: https://docs.sqd.dev/en/cloud/reference/cli/whoami

Display the currently authenticated SQD Cloud account with sqd whoami — useful for verifying CLI identity and switching between organizations.

# `sqd whoami`

Show the user details for the current Cloud account

* [`sqd whoami`](#sqd-whoami-1)

## `sqd whoami`

Show the user details for the current Cloud account

```
USAGE
  $ sqd whoami [--interactive]

FLAGS
  --[no-]interactive  Disable interactive mode
```

*See code: [src/commands/whoami.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/whoami.ts)*


# Changelog: slots and tags
Source: https://docs.sqd.dev/en/cloud/reference/deployments-two-changelog

Changelog for SQD Cloud slots-and-tags deployment model — new commands, breaking changes, and migration notes for users on the legacy prod workflow.

<Tip>
  Take a look at the [slots and tags guide
  page](/en/cloud/resources/slots-and-tags) for practical instructions on using
  the system.
</Tip>

## Motivation

When we initially built the [original deployment flow](/en/cloud/resources/production-alias) versions felt appropriate. A simple number you could increment on each deployment if necessary. This worked well in simple scenarios, but over time this gradually became more difficult to manage for those of you with any kind of deployment complexity.

At the same time it became clear that our solution to endpoint management, allowing you to mark a squid as production and create a permalink, was equally powerful in basic scenarios but limited as requirements grew.

It became clear that was lacking here was the ability for larger teams to collaborate effectively. Our goal with these changes is to provide the power and flexibility that more complex deployment flows need.

## Concepts

| Concepts         | Before        | After                                            |
| ---------------- | ------------- | ------------------------------------------------ |
| Version          | Existed       | Removed (replaced with slots)                    |
| Slot             | Did not exist | Introduced (unique identifier for configs)       |
| Production alias | Existed       | Deprecated (use tags)                            |
| Tag              | Did not exist | Introduced (user-defined labels for deployments) |
| Reference        | Existed       | Exists, but the format has changed               |

New concepts are explained in detail in [the guide](/en/cloud/resources/slots-and-tags).

## Format changes

| Value                                                  | Before                                                  | After                                                                     |
| ------------------------------------------------------ | ------------------------------------------------------- | ------------------------------------------------------------------------- |
| Version                                                | Must be a number                                        | Deprecated                                                                |
| Slot                                                   | Did not exist                                           | A string of up to six lower case alphanumeric characters, dashes allowed  |
| Tag                                                    | Did not exist                                           | A string of lowercase alphanumeric characters, dashes allowed             |
| Reference                                              | `<name>@v<version>`                                     | `[<org>/]<name>(@<slot>\|:<tag>)`                                         |
| Canonical URL                                          | `https://<org>.squids.live/<name>/v/v<version>/graphql` | `https://<org>.squids.live/<name>@<slot>/api/graphql`                     |
| [Production URL](/en/cloud/resources/production-alias) | `https://<org>.squids.live/<name>/graphql`              | Deprecated. See also [backwards compatibility](#backwards-compatibility). |
| Tag URL                                                | Did not exist                                           | `https://<org>.squids.live/<name>:<tag>/api/graphql`                      |

## Manifest changes

| Field   | Before  | After                          |
| ------- | ------- | ------------------------------ |
| name    | Present | Present                        |
| version | Present | Deprecated (mapped to slot)    |
| slot    | Absent  | New field (unique identifier)  |
| tag     | Absent  | New field (user-defined label) |

## CLI changes

Changes to [CLI behavior](squid-cli) are rather extensive:

* New flags `--name/-n`, `--tag/-t`, `--slot/-s`, `--reference/-r` have been added to nearly all commands.
  * For `sqd deploy` they now can override their corresponding [manifest fields](#manifest-changes). Heterogenous overrides also work: `-t` in CLI overrides `slot:` in manifest, and `-s` overrides `tag:`.
  * For other commands (`sqd logs`, `sqd restart` etc) they allow for flexible specification of deployment.
* New commands `sqd tags add` and `sqd tags remove` have been added.
* The `sqd prod` command has been removed.

Here's what the new commands look like for some common tasks:

| Action                                  | Before                                 | After                                 |
| --------------------------------------- | -------------------------------------- | ------------------------------------- |
| Deploying prod version                  | `sqd deploy . && sqd prod my-squid@v1` | `sqd deploy . --add-tag prod`         |
| Promoting to prod                       | `sqd prod my-squid@v1`                 | `sqd tags add prod -n my-squid -s v1` |
| Marking a deployment as a "dev" version | Was not possible                       | `sqd tags add dev -n my-squid -s v1`  |
| Updating the "dev" version              | Was not possible                       | `sqd deploy . -t dev`                 |
| Fetching logs                           | `sqd logs my-squid@v1`                 | `sqd logs -n my-squid -s v1`          |

### Cheatsheet

**Add tag `testtag` to the deployment of `my-squid` running in slot `v1`**

```bash theme={"system"}
sqd tags add testtag -n my-squid -s v1
```

**(Re)deploy the squid and immediately assign the tag `testtag` to the deployment**

```bash theme={"system"}
sqd deploy . --add-tag testtag
```

* The squid will be redeployed if one of the `tag:`, `slot:` or `version:` fields are defined in the manifest and the Cloud already has a deployment that matches the reference.
* If the fields are undefined or there's no matching squid in the Cloud, a new deployment will be created in a new slot.
* Tag `testtag` will be assigned to the slot of the (possibly new) deployment.

**Update the deployment tagged `dev`**

```bash theme={"system"}
sqd deploy . --tag dev
```

* If there's a deployment tagged `dev`, it will be updated.
* Otherwise, the command will exit with an error.

**Read logs of the deployment of `my-squid` tagged `prod`**

```bash theme={"system"}
sqd logs -n my-squid -t prod
```

## Backwards compatibility

Here are the measures we've taken to make the migration smoother:

* Existing deployments lose their versions; instead, they are assigned to the corresponding `v<version>` slots. New deployments that specify `version:` in the manifest will be assigned to the `v<version>` slot, too. Hence, for the time being the lines
  ```yaml theme={"system"}
  version: 42
  ```
  and
  ```yaml theme={"system"}
  slot: v42
  ```
  in the manifest file are equivalent.
* Tag `prod` has a special meaning for the duration of the transition period: deployments that have it are made available via old-style [production URLs](/en/cloud/resources/production-alias).
* Tag `prod` is assigned to all existing deployments with [production aliases](/en/cloud/resources/production-alias).
* You can, for the time being, add an old-style production URL to any deployment - just assign the `prod` tag to it.

## FAQs

#### Do I need to redeploy anything?

No. But we suggest you take a look at the [Backwards compatibility](#backwards-compatibility) section.

#### We really like versions, they work well for us, do we need to change our workflow?

No, you don't need to change your workflow at all! Your version will be migrated into the new slot field, and slots are a just an arbitrary string between 2 and 6 characters in length. Our goal was to expand functionality rather than remove it.

#### Why should I use tags? Convince me.

Tags serve two key roles. Firstly as a labelling mechanism, this allows teams to explicitly describe what each squid does and what stage it's at in development. Secondly as a way to preserve urls across slots. Tags act as an alias by creating a unique endpoint for each tag. This allows you to [migrate a consistent endpoint across slots](/en/cloud/resources/slots-and-tags/#zero-downtime-updates) with ease.

#### Will any of my endpoints change when this is released?

No. Every endpoint will remain unchanged.

#### Where should I send my feedback on the new system?

We're keen to hear from you. Come talk to us in the [SquidDevs Telegram channel](https://t.me/HydraDevs).


# addons.hasura section
Source: https://docs.sqd.dev/en/cloud/reference/hasura

Configure the Hasura GraphQL addon for a squid deployment — connect to Postgres, set authentication, and expose a managed GraphQL API on SQD Cloud.

## Running Hasura

To provision a [Hasura](https://hasura.io) instance, add an empty `deploy.addons.hasura` section to the [deployment manifest](/en/cloud/reference/manifest). Provide some basic configuration:

```yaml theme={"system"}
deploy:
  env:
    HASURA_GRAPHQL_ADMIN_SECRET: "$\{\{ secrets.HASURA_SECRET \}\}"
    HASURA_GRAPHQL_UNAUTHORIZED_ROLE: user
    HASURA_GRAPHQL_STRINGIFY_NUMERIC_TYPES: "true"
  addons:
    postgres:
    hasura:
```

Note the use of a [Cloud secret](/en/cloud/resources/env-variables/#secrets) for storing the admin password.

## Configuring a Hasura API

### For a squid

Use the [Hasura configuration tool](/en/sdk/squid-sdk/resources/tools/hasura-configuration) for squids running dedicated Hasura instances. To make Cloud initialize Hasura configuration on squid restarts, make sure that the tool runs on squid startup by adding a `deploy.init` section to the manifest, e.g. like this:

```yaml theme={"system"}
deploy:
  init:
    env:
      HASURA_GRAPHQL_ENDPOINT: 'http://hasura:8080'
    cmd:
      - npx
      - squid-hasura-configuration
      - apply
```

See also the [Hasura section of the GraphQL guide](/en/sdk/squid-sdk/resources/serving-graphql#hasura) and the [complete squid example](https://github.com/subsquid-labs/squid-hasura-example).

### For a DipDup indexer

[DipDup](https://dipdup.io) also configures Hasura automatically. See the [DipDup section](/en/sdk/alternative-clients/dipdup) for details.


# Deployment manifest
Source: https://docs.sqd.dev/en/cloud/reference/manifest

Reference for the squid deployment manifest — define processor and GraphQL services, addons, scale settings, environment variables, and secrets.

The deployment manifest is named `squid.yaml` by convention. The manifest defines how the squid should be built, customized and scaled during the deployment to SQD Cloud. It is used together with [`sqd deploy`](/en/cloud/reference/cli/deploy) to deploy or update an existing squid version.

With the introduction of the deployment manifest the add-ons (`postgres`) and the API services (`api`) become optional. This allows flexible deployment configuration for analytic use-cases (e.g. transform and write to s3 or BigQuery).

<Tip>
  Cloud [secrets](/en/cloud/resources/env-variables/#secrets) of the squid's [organization](/en/cloud/resources/organizations) are exposed as the `secrets` context. You can set any variable defined in any of the `env:` sections mentioned below:

  ```yaml theme={"system"}
  env:
    RPC_ENDPOINT: "${{ secrets.FAST_RPC_ENDPOINT_URL }}"
  ```

  See the [Environment variables](/en/cloud/resources/env-variables) page for more info.
</Tip>

## Header

The manifest header defines the squid metadata

|        Field       |                                                         Description                                                        |
| :----------------: | :------------------------------------------------------------------------------------------------------------------------: |
| `manifest_version` |                                       Only `subsquid.io/v0.1` is currently supported                                       |
|       `name`       | A globally unique squid name. Can only contain lowercase Latin letters (`a-z`) and a dash (`-`). Must be under 20 symbols. |
|      `version`     |      Squid version. **Must be an integer**. A squid deployment is canonically identified as `$\{name\}@$\{version\}`.      |
|    `description`   |                                         (Optional) A short description of the squid                                        |

## `build:`

Specifies the way the squid is built into a docker image.

|        Name       |              Type             | Default value |
| :---------------: | :---------------------------: | ------------: |
|   `node_version`  |        `18 \| 20 \| 21`       |          `20` |
| `package_manager` | `auto \| npm \| pnpm \| yarn` |        `auto` |
|   [`cmd`](#cmd)   |              list             |               |

For a successful build the following files and folders **must** be present in the root folder of the squid:

* `/src`
* `tsconfig.json`
* `package.json`
* `commands.json`

The `db` and `assets` folders are added to the build context if present in the squid folder. See [Project structure](/en/sdk/squid-sdk/how-to-start/layout) for more info.

Under the hood, Cloud builds a Docker image and runs a docker container for each service (`api`, `processor`, `migrate`) using the same image.
See [Self-hosting](/en/sdk/squid-sdk/resources/self-hosting) for instructions on how to build and run the Docker image locally.
Even though the squid services (`api`, `processor`, `migrate`) use the same single container image, the exec command is different and can is defined by the `deploy:` section as explained below.

### `cmd:`

Enables overriding the dependencies installation command, e.g.

````yaml theme={"system"}
build:
  cmd:
    - npm
    - install
```
The default is to use a shrinkwrap-based installation command appropriate for the detected package manager (e.g. `npm ci`).

## `deploy:`

The `deploy` section may define:

### `addons:`

A list of add-on services to be deployed along the squid services.

- **`postgres:`**

See [Postgres add-on](/en/cloud/reference/pg).

- **`rpc`**

See [RPC add-on](/en/cloud/resources/rpc-proxy).

### `migrate:`

**Optional** The init container ran before `processor` and `api` services are started. If it exits with an error, squid deployment fails.

| Name        | Description                                           | Type          | Default value         |         Optional   |
|:-----------:|:-----------------------------------------------------:|:-------------:|----------------------:|:------------------:|
| `cmd`       | Exec command passed to the squid container. By default runs TypeORM migrations in the `db` folder  |  `string[]`   | `['npx', 'squid-typeorm-migration', 'apply']` |   Optional   |
| `env`       | A key-value list of env variables to be set           |  `\{k:v\}[]`    |       []               | Optional                |


### `processor:`

A processor service or a list of processor services of the squid.

| Name        | Description                                           | Type          | Default value         |         Optional                            |
|:-----------:|:-----------------------------------------------------:|:-------------:|----------------------:|:-------------------------------------------:|
| `cmd`       | Processor exec command passed to the squid container  |  `string[]`   |                       |   **Required**                              |
| `name`      | Processor name                                        |  `string`     |                       |   **Required** with more than one processor |
| `env`       | A key-value list of env variables to be set           |  `\{k:v\}[]`    |   []                  |   Optional                                  |

With a single processor this section may look like this:
```yaml
  processor:
    cmd: [ "sqd", "process:prod" ]
```
For the multiprocessor case:
```yaml
  processor:
    - name: eth-processor
      cmd: [ "sqd", "process:prod:eth" ]
    - name: bsc-processor
      cmd: [ "sqd", "process:prod:bsc" ]
```
where `process:prod:bsc` and `process:prod:eth` are extra `sqd` commands defined at [`commands.json`](/en/cloud/reference/cli/commands-json):
```json title="commands.json"
...
  "process:prod:eth": {
    "deps": ["migration:apply"],
    "cmd": ["node", "lib/eth/main.js"]
   },
  "process:prod:bsc": {
     "cmd": ["node", "lib/bsc/main.js"]
   },
...
```

### `api:`

**Optional** The GraphQL API service of the squid. Automatically provisions a publicly available endpoint `https: "//\{org\}.subsquid.io/\{name\}/v/v\{version\}/graphql` and binds it to the server."

| Name        | Description                                                             | Type          | Default value         | Optional           |
|:-----------:|:-----------------------------------------------------------------------:|:-------------:|----------------------:|:------------------:|
| `cmd`       | GraphQL API server exec command passed to the squid container. The server must be listening at port `GRAPHQL_SERVER_PORT`. |  `string[]` |    |   **Required**     |
| `env`       | A key-value list of env variables to be set                             |  `\{k:v\}[]`    |  []                   |   Optional         |

### `env:`

**Optional** A key-value list of deployment-wide (i.e. visible to all services of the squid) env variables to be set

## `scale:`

See the [Scale the deployment](/en/cloud/reference/scale) section.

## Examples

A minimal example of manifest is below:

```yaml title="squid.yaml"
manifest_version: subsquid.io/v0.1
name: sample-squid
version: 1
description: |-
  My sample squid

build:

deploy:
  addons:
    postgres:
  processor:
    cmd: [ "sqd", "process:prod" ]
  api:
    cmd: [ "sqd", "serve:prod" ]
```

An extended version:

```yaml title="squid.yaml"
manifest_version: subsquid.io/v0.1
name: sample-squid
version: 1
description: |-
  My advanced squid

build:

deploy:
  addons:
    postgres:
  migrate:
    env:
      FOO: bar
    cmd: [ "echo", "skip migrations!" ]
  processor:
    # static and secret-valued env variables
    env:
      SQD_DEBUG: sqd:mapping
RPC_ENDPOINT: "$\{\{ secrets.ACALA_RPC_ENDPOINT \}\}"
COINGECKO_API_KEY: "$\{\{ secrets.COINGECKO_API_KEY \}\}"
    cmd: [ "sqd", "process:prod" ]
  api:
    env:
      SQD_DEBUG: '*'
    # custom run command for the GraphQL server
    cmd: [ "npx", "squid-graphql-server", "--subscriptions", "--max-root-fields", "10", "--sql-statement-timeout", "1000" ]

scale:
  addons:
    postgres:
      storage: 100G
      profile: medium
  processor:
    profile: medium
  api:
    profile: large
    # load-balance three replicas
    replicas: 3
````


# addons.postgres section
Source: https://docs.sqd.dev/en/cloud/reference/pg

Provision and scale a managed Postgres database for a squid via the addons.postgres manifest section — sizing, profiles, and snapshot options.

<Note>Postgres instances provisioned by SQD Cloud only have basic functionality by default. Do not use them to store data that can't be recovered via an indexer resync. Use [zero-downtime updates](/en/cloud/resources/slots-and-tags/#zero-downtime-updates) to avoid sync-related downtimes. Advanced capabilities such as snapshots, replication, high-availability setup, increased number of direct connections etc are avialable on Enterprise plans. If you're interested please [let us know](https://t.me/HydraDevs).</Note>

To provision a postgres instance, add the `addons.postgres:` section to the deployment manifest. The add-on deploys a Postgres instance and [injects variables](#variable-shadowing) with database connection params into the environment of the `api` and `processor` squid services.

## Variable shadowing

Any of the following variables set globally or for `api` or `processor` squid services in the [manifest](/en/cloud/reference/manifest) will be overwritten with Cloud-supplied values:

* `DB_SSL`
* `DB_HOST`
* `DB_PORT`
* `DB_NAME`
* `DB_USER`
* `DB_PASS`
* `DB_URL`

## Config options

The addon supports additional PG config options:

|                    Name                    |                                                       Description                                                      |   Type   | Default value | Optional |
| :----------------------------------------: | :--------------------------------------------------------------------------------------------------------------------: | :------: | :-----------: | :------: |
|         `config.statement_timeout`         |                           Max execution time after which any query is forcefully aborted, ms                           | `number` |    `60000`    | Optional |
|   `config.` `log_min_duration_statement`   |                                Log queries executing longer than the given threshold, ms                               | `number` |     `5000`    | Optional |
| `config.` `max_pred_locks_per_transaction` | See [Lock Management](https://www.postgresql.org/docs/15/runtime-config-locks.html#GUC-MAX-PRED-LOCKS-PER-TRANSACTION) | `number` |      `64`     | Optional |
|     `config.max_locks_per_transaction`     |    See [Lock Management](https://www.postgresql.org/docs/15/runtime-config-locks.html#GUC-MAX-LOCKS-PER-TRANSACTION)   | `number` |      `64`     | Optional |

## Direct access

SQD Cloud enables direct read access to the deployed PG instances. Go to the "DB access" tab of your squid deployment's card in the [Cloud web app](https://app.subsquid.io/squids) to get the PG connection string.

<Note>Outside of Enterprise plans, the number of direct connections one can open to the database is capped for optimal performance. Please avoid using more than ten (10) connections in your setups.</Note>

## Scaling

The `postgres` add-on supports storage and compute resource scaling by extending the `scale.addons.postgres` section of the deploy manifest. The following options are supported

|    Name   |               Description              |                                                            Type                                                           | Default value | Optional |
| :-------: | :------------------------------------: | :-----------------------------------------------------------------------------------------------------------------------: | :-----------: | :------: |
| `storage` | Volume size for the postgres container | [memory resource units](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#meaning-of-memory) |     `10G`     | Optional |
| `profile` |       Allocated resources profile      |                                  `small` \| `medium` \| `large` \| `xlarge` \| `2xlarge`                                  |    `small`    | Optional |

The profile specifications for a `postgres` service are as follows:

|  Profile  | colocated vCPU (max) | colocated RAM (max) | dedicated vCPU (requested) | dedicated RAM (max) |
| :-------: | :------------------: | :-----------------: | :------------------------: | :-----------------: |
|  `small`  |          0.2         |       `768Mi`       |              1             |        `2Gi`        |
|  `medium` |          0.5         |       `1.5Gi`       |              2             |        `4Gi`        |
|  `large`  |           1          |        `3Gi`        |              4             |        `4Gi`        |
|  `xlarge` |           -          |          -          |              8             |        `16Gi`       |
| `2xlarge` |           -          |          -          |             16             |        `32Gi`       |

## Examples

```yaml theme={"system"}
manifest_version: subsquid.io/v0.1
name: sample-squid
version: 1
description: |-
  My advanced squid 

build: 

deploy:
  addons:
    postgres: 
      config:
        statement_timeout: 100000
        log_min_duration_statement: 100000
  processor:
    cmd: [ "sqd", "process:prod" ]
  api:
    cmd: [ "sqd", "serve:prod" ]

scale:
  addons:
    postgres:
      storage: 100G
      profile: medium
```


# RPC service networks
Source: https://docs.sqd.dev/en/cloud/reference/rpc-proxy-networks

Request a managed RPC endpoint for an EVM or Substrate network on SQD Cloud — list of supported chains, latency, and reliability targets per region.

Enable the add-on with:

```yaml theme={"system"}
deploy:
  addons:
    rpc:
      - eth-goerli.http
```

The RPC endpoint URL will become available at the `RPC_ETH_GOERLI_HTTP` env variable.

See the [RPC addon guide](/en/cloud/resources/rpc-proxy) for details.

<details>
  <summary>Available EVM networks</summary>

  |      Network name     |           network.protocol           |
  | :-------------------: | :----------------------------------: |
  |      Arbitrum One     |          `arbitrum-one.http`         |
  |  Arbitrum One Goerli  |        `arbitrum-goerli.http`        |
  |  Arbitrum One Sepolia |        `arbitrum-sepolia.http`       |
  |     Arbitrum Nova     |         `arbitrum-nova.http`         |
  |         Astar         |             `astar.http`             |
  |          AVA          |              `ava.http`              |
  |      AVA Testnet      |          `ava-testnet.http`          |
  |          Base         |              `base.http`             |
  |      Base Goerli      |          `base-goerli.http`          |
  |      Base Sepolia     |          `base-sepolia.http`         |
  |       Berachain       |           `berachain.http`           |
  |        Blast L2       |            `blast-l2.http`           |
  |          BSC          |              `bsc.http`              |
  |      BSC Testnet      |          `bsc-testnet.http`          |
  |        Ethereum       |              `eth.http`              |
  |    Ethereum Goerli    |           `eth-goerli.http`          |
  |    Ethereum Holesky   |          `eth-holesky.http`          |
  |    Ethereum Sepolia   |          `eth-sepolia.http`          |
  |         Evmos         |             `evmos.http`             |
  |         Fantom        |             `fantom.http`            |
  |     Fantom Testnet    |         `fantom-testnet.http`        |
  |         Gnosis        |             `gnosis.http`            |
  |         Linea         |             `linea.http`             |
  |         Mantle        |             `mantle.http`            |
  |         Mantle        |         `mantle-sepolia.http`        |
  |         Metis         |             `metis.http`             |
  |        Moonbase       |         `moonbase-alpha.http`        |
  |        Moonbeam       |            `moonbeam.http`           |
  |       Moonriver       |           `moonriver.http`           |
  |          OKTC         |              `oktc.http`             |
  |         opBNB         |             `opbnb.http`             |
  |         opBNB         |         `opbnb-testnet.http`         |
  |        Optimism       |            `optimism.http`           |
  |    Optimism Goerli    |        `optimism-goerli.http`        |
  |        Polygon        |            `polygon.http`            |
  |      Polygon Amoy     |      `polygon-amoy-testnet.http`     |
  |    Polygon Testnet    |        `polygon-testnet.http`        |
  |     Polygon zkEVM     |         `polygon-zkevm.http`         |
  | Polygon zkEVM Cardona | `polygon-zkevm-cardona-testnet.http` |
  | Polygon zkEVM Testnet |     `polygon-zkevm-testnet.http`     |
  |         Scroll        |             `scroll.http`            |
  |     Scroll Sepolia    |         `scroll-sepolia.http`        |
  |        Shibuya        |            `shibuya.http`            |
  |         Shiden        |             `shiden.http`            |
  |         Sonic         |         `sonic-mainnet.http`         |
  |        Unichain       |            `unichain.http`           |
  |    Unichain Sepolia   |        `unichain-sepolia.http`       |
  |         zkSync        |             `zksync.http`            |
  |     zkSync Sepolia    |         `zksync-sepolia.http`        |

  <details>
    <summary>Available Substrate networks</summary>

    |     Network name     |       network.protocol      |
    | :------------------: | :-------------------------: |
    |         Acala        |         `acala.http`        |
    |      Aleph Zero      |      `aleph-zero.http`      |
    |  Aleph Zero Testnet  |  `aleph-zero-testnet.http`  |
    |   Asset Hub Kusama   |   `asset-hub-kusama.http`   |
    |  Asset Hub Polkadot  |  `asset-hub-polkadot.http`  |
    |   Asset Hub Rococo   |   `asset-hub-rococo.http`   |
    |   Asset Hub Westend  |   `asset-hub-westend.http`  |
    |         Astar        |    `astar-substrate.http`   |
    |       Basilisk       |       `basilisk.http`       |
    |       Bittensor      |       `bittensor.http`      |
    |   Bittensor Testnet  |   `bittensor-testnet.http`  |
    |   Bridge Hub Kusama  |   `bridge-hub-kusama.http`  |
    |  Bridge Hub Polkadot |  `bridge-hub-polkadot.http` |
    |   Bridge Hub Rococo  |   `bridge-hub-rococo.http`  |
    |  Bridge Hub Westend  |  `bridge-hub-westend.http`  |
    |      Centrifuge      |      `centrifuge.http`      |
    | Collectives Polkadot | `collectives-polkadot.http` |
    |  Collectives Westend |  `collectives-westend.http` |
    |         Crust        |         `crust.http`        |
    |       Darwinia       |       `darwinia.http`       |
    |     Darwiniacrab     |     `darwiniacrab.http`     |
    |         Eden         |         `eden.http`         |
    |       Frequency      |       `frequency.http`      |
    |        Hydradx       |        `hydradx.http`       |
    |       Interlay       |       `interlay.http`       |
    |        Karura        |        `karura.http`        |
    |         Khala        |         `khala.http`        |
    |         Kilt         |         `kilt.http`         |
    |       Kintsugi       |       `kintsugi.http`       |
    |        Kusama        |        `kusama.http`        |
    |       Litentry       |       `litentry.http`       |
    |       Moonbase       |       `moonbase.http`       |
    |       Moonbeam       |  `moonbeam-substrate.http`  |
    |       Moonriver      |  `moonriver-substrate.http` |
    |       Pendulum       |       `pendulum.http`       |
    |         Phala        |         `phala.http`        |
    |       Polkadex       |       `polkadex.http`       |
    |       Polkadot       |       `polkadot.http`       |
    |        Rococo        |        `rococo.http`        |
    |        Shibuya       |   `shibuya-substrate.http`  |
    |        Shiden        |   `shiden-substrate.http`   |
    |        Turing        |        `turing.http`        |
  </details>
</details>


# scale section
Source: https://docs.sqd.dev/en/cloud/reference/scale

Scale a squid horizontally and vertically using the manifest scale section — replicas, processor profiles, GraphQL profiles, and addon resources.

The `scale:` section of the [deployment manifest](/en/cloud/reference/manifest) allows allocating additional computing resources for the squid add-ons and services. This option is only available for paid squids deployed in [professional organizations](/en/cloud/resources/organizations/#professional-organizations).

<Info>
  Visit the [Pricing page](/en/cloud/pricing) and/or our [costs calculator](https://subsquid.io/subsquid-cloud#calculator) if you're looking for an estimate.
</Info>

The manifest supports the following scaling options:

## `dedicated:`

Default: `dedicate
d: true`.

When a dedicated profile is used, the resources that the squid requests are reserved in advance. **This is the new/current default** and is the recommended profile for production squids. Squid deployment must be dedicated to be a subject of [SQD Cloud SLA](/en/cloud/pricing).

By setting `dedicated: false` you can request for your deployment to be *collocated* - that is, share resources with other deployments. The allocation of full resources is not guaranteed for collocated squids.

All playground squids must use the collocated profile, that is, explicitly specify `dedicated: false`.

## `addons:`

### `postgres:`

See [Postgres add-on](/en/cloud/reference/pg) for details.

## `services:`

### `api:`

|    Name    |                                                    Description                                                   |                           Type                          | Default value | Optional |
| :--------: | :--------------------------------------------------------------------------------------------------------------: | :-----------------------------------------------------: | :-----------: | :------: |
|  `profile` |                                            Allocated resources profile                                           | `small` \| `medium` \| `large` \| `xlarge` \| `2xlarge` |    `small`    | Optional |
| `replicas` | The number of gateway replicas. The API requests are distributed between the replicas in the round-robin fashion |                          Number                         |      `1`      | Optional |

The profile specifications for API service replicas are as follows:

|  Profile  | colocated vCPU (max) | colocated RAM (max) | dedicated vCPU (requested) | dedicated RAM (max) |
| :-------: | :------------------: | :-----------------: | :------------------------: | :-----------------: |
|  `small`  |          0.2         |       `768Mi`       |             0.5            |       `768Mi`       |
|  `medium` |          0.5         |       `1.5Gi`       |              1             |       `1.5Gi`       |
|  `large`  |           1          |        `3Gi`        |              2             |        `3Gi`        |
|  `xlarge` |           -          |          -          |              4             |        `6Gi`        |
| `2xlarge` |           -          |          -          |              8             |        `12Gi`       |

### `processor:`

|    Name   |         Description         |                           Type                          | Default value | Optional |
| :-------: | :-------------------------: | :-----------------------------------------------------: | :-----------: | :------: |
| `profile` | Allocated resources profile | `small` \| `medium` \| `large` \| `xlarge` \| `2xlarge` |    `small`    | Optional |

The profile specifications for a processor service are as follows:

|  Profile  | colocated vCPU (max) | colocated RAM (max) | dedicated vCPU (requested) | dedicated RAM (max) |
| :-------: | :------------------: | :-----------------: | :------------------------: | :-----------------: |
|  `small`  |          0.2         |       `768Mi`       |             0.5            |       `768Mi`       |
|  `medium` |          0.5         |       `1.5Gi`       |              1             |       `1.5Gi`       |
|  `large`  |           1          |        `3Gi`        |              2             |        `3Gi`        |
|  `xlarge` |           -          |          -          |              4             |        `6Gi`        |
| `2xlarge` |           -          |          -          |              8             |        `12Gi`       |

## Example

```yaml title="squid.yaml" theme={"system"}
manifest_version: subsquid.io/v0.1
name: sample-squid

build: 

deploy:
  addons:
    postgres: 
  processor:
    cmd: [ "sqd", "process:prod" ]
  api:
    cmd: [ "sqd", "serve:prod" ]

scale:
  addons:
    postgres:
      storage: 100G
      profile: medium
  processor:
    profile: medium
  api:
    profile: large
    # load-balance three replicas
    replicas: 3
```


# .squidignore file(s)
Source: https://docs.sqd.dev/en/cloud/reference/squidignore

Exclude files from a squid Docker image with .squidignore — gitignore-style patterns to keep deployment artifacts small and free of secrets.

**Since @subsquid/cli\@2.9.0**

* If a `.squidignore` file is present in any of the squid folders (including the project root), [`sqd deploy`](/en/cloud/reference/cli/deploy) will read filename patterns from it and omit the matching files from the bundle to be sent to the Cloud.
* Filename patterns follow the [gitignore pattern format](https://git-scm.com/docs/gitignore#_pattern_format).
* Patterns read from `.squidignore` files from higher level folders (the project root being the highest) are overridden by patterns read from lower level folders.
* When no `.squidignore` files are supplied, `sqd deploy` will omit the following files and folders:
  ```
  node_modules
  builds
  lib
  Dockerfile
  .git
  .github
  .idea
  ```


# Best practices
Source: https://docs.sqd.dev/en/cloud/resources/best-practices

Production best practices for SQD Cloud deployments — manifest patterns, error handling, monitoring, secrets, and scaling tips for reliable squids.

Here is a list of items to check out before you deploy your squid for use in production:

* Make sure that you use [batch processing](/en/sdk/squid-sdk/resources/batch-processing) throughout your code. Consider using `@belopash/typeorm-store` for large projects with extensive [entity relations](/en/sdk/squid-sdk/reference/schema-file/entity-relations) and frequent [database reads](/en/sdk/squid-sdk/reference/store/typeorm#typeorm-methods).

* Filter your data in the batch handler. E.g. if you [request event logs](/en/sdk/squid-sdk/reference/processors/evm-batch/logs) from a particular contract, do check that the `address` field of the returned data items matches the contract address before processing the data. This will make sure that any future changes in your processor configuration will not cause the newly added data to be routed to your old processing code by mistake.

<Info>
  Batch handler data filtering used to be compulsory before the release of `@subsquid/evm-processor@1.13.0`. Now it is optional but highly recommended.
</Info>

* If your squid [saves its data to a database](/en/sdk/squid-sdk/resources/persisting-data/typeorm), make sure your [schema](/en/sdk/squid-sdk/reference/schema-file) has [`@index` decorators](/en/sdk/squid-sdk/reference/schema-file/indexes-and-constraints) for all entities that will be looked up frequently.
  * Follow the [queries optimization procedure](/en/cloud/resources/query-optimization) for best results.

* If your squid serves a [GraphQL API](/en/sdk/squid-sdk/resources/serving-graphql)
  1. Do not use [OpenReader](/en/sdk/squid-sdk/resources/serving-graphql#openreader) if your application uses subscriptions. Instead, use [PostGraphile](/en/sdk/squid-sdk/resources/serving-graphql#postgraphile) or [Hasura](/en/sdk/squid-sdk/resources/serving-graphql#hasura).
  2. If you do use OpenReader:
     * configure the built-in [DoS protection](/en/sdk/squid-sdk/reference/openreader-server/configuration/dos-protection) against heavy queries;
     * configure [caching](/en/sdk/squid-sdk/reference/openreader-server/configuration/caching).
  3. If you use PostGraphile or Hasura, follow their docs to harden your service in a similar way.

* If you deploy your squid to SQD Cloud:
  1. Deploy your squid to a [Professional organization](/en/cloud/resources/organizations/#professional-organizations).
  2. Do not use [`dedicated: false`](/en/cloud/reference/scale/#dedicated) in the `scale:` section of the manifest.
  3. Make sure that your [`scale:` section](/en/cloud/reference/scale) requests a sufficient but not excessive amount of resources.
  4. Set your deployment up for [zero downtime updates](/en/cloud/resources/slots-and-tags/#zero-downtime-updates). Use a tag-based URL and not slot-based URLs to access your API e.g. from your frontend app.
  5. Make sure to use [secrets](/en/cloud/resources/env-variables/#secrets) for all sensitive data you might've used in your code. The most common type of such data is API keys and URLs containing them.
  6. Follow the recommendations from the [Cloud logging page](/en/cloud/resources/logging).


# Environment variables
Source: https://docs.sqd.dev/en/cloud/resources/env-variables

Manage environment variables for squids on SQD Cloud — declare manifest variables, contexts, secrets, and Cloud-only build-time substitutions.

<Info>
  Store all your sensitive inputs (API keys, passwords etc) as
  [secrets](#secrets).
</Info>

SQD Cloud supports adding environment variables to squid deployments. The variables can be defined as key-value pairs at any of the `env:` sections of the [manifest](/en/cloud/reference/manifest).

For example, here is how to add a variable that will be visible only to the [squid processor](/en/cloud/reference/manifest/#processor):

````yaml title="squid.yaml" theme={"system"}
deploy:
  processor:
    env:
      MY_PROCESSOR_VAR: string_value
```
You can also add variables visible only to the [GraphQL server](/en/cloud/reference/manifest/#api) or to the [migration script](/en/cloud/reference/manifest/#migrate).

There is also an option to add variables [for all services](/en/cloud/reference/manifest/#env):
```yaml title="squid.yaml"
deploy:
  env:
    MY_SQUIDWIDE_VAR: string_value
```

Variables can be assigned either to strings, or to member variables of [contexts](#contexts) provided by the service. For example, to make a processor-scoped `API_KEY` variable and populate it with the value of `secrets.API_KEY`, do this:
```yaml title="squid.yaml"
deploy:
  processor:
    env:
RPC_ENDPOINT: "$\{\{ secrets.API_KEY \}\}"
```

## Variable shadowing

There is one special case in which the variables defined in the manifest will get overwritten by the Cloud: [database connection settings](/en/sdk/squid-sdk/reference/store/typeorm#database-connection-parameters) are shadowed by the system-defined values whenever the [`postgres` addon](/en/cloud/reference/pg) is enabled (see the [Variable shadowing](/en/cloud/reference/pg/#variable-shadowing) section of the addon page). For example, in the snippet below all the `DB_*` variable definitions will be ignored:
```yaml title="squid.yaml"
deploy:
  addons:
    postgres:
  env:
    DB_HOST: mypostgreshost.xyz
    DB_PORT: 5432
    DB_NAME: squid-tests
    DB_USER: me
DB_PASS: "$\{\{ secrets.DATABASE_PASSWORD \}\}"
    DB_SSL: true
```

## Contexts

The Cloud exposes some useful variables via a mechanism identical to [GitHub Actions contexts](https://docs.github.com/en/actions/learn-github-actions/contexts). Namely, any string

```
${{ secrets.EXAMPLE_VAR }}
```

```
added to the [manifest](/en/cloud/reference/manifest) at any environment variable definition gets replaced by the value supplied by the Cloud.

### Secrets

Secrets are designed to store sensitive data, such as API keys or private URLs for chain RPC endpoints. They are defined at the [organization](/en/cloud/resources/organizations) level and are exposed to all organization squids that request them in their environment variable definitions.

To add a secret:

1. Create it in the Cloud. You can do it at the [secrets page](https://app.subsquid.io/secrets) or with [`sqd secrets`](/en/cloud/reference/cli/secrets):
   ```bash
   sqd secrets set MOONRIVER_GRPC_ENDPOINT wss://moonriver.my-endpoint.com/ws/my-secret-key
````

If you do not specify the value, `sqd` will attempt to read it from standard input. This is useful when setting a value to the contents of some file:

```bash theme={"system"}
sqd secrets set MY_JSON_CREDENTIALS < creds.json
```

2. At your squid's [manifest](/en/cloud/reference/manifest), add an environment variable and assign it to the secret:
   ```yaml theme={"system"}
   deploy:
     env:
   RPC_ENDPOINT: "$\{\{ secrets.MOONRIVER_GRPC_ENDPOINT \}\}"
   ```
   Note: **a deployment requesting a secret unknown to the Cloud will fail**.
3. Access the value in the squid with `process.env`, e.g.
   ```typescript theme={"system"}
   const processor = new EvmBatchProcessor()
     .setRpcEndpoint({
       url: process.env.RPC_ENDPOINT,
       rateLimit: 1000rps
     })
   ```
   Inspect, remove and update the secrets using the [`sqd secrets`](/en/cloud/reference/cli/secrets) command.

<Info>
  Any changes to secrets will take effect only when the squid is restarted, e.g.
  with `bash sqd deploy . `
</Info>


# Inspect logs
Source: https://docs.sqd.dev/en/cloud/resources/logging

Inspect deployment logs from squids running on SQD Cloud — stream stdout/stderr from processor and GraphQL containers with filters and time ranges.

SQD Cloud automatically collects the logs emitted by the squid processor, its API server and its database. Please use the [built-in SDK logger](/en/sdk/squid-sdk/reference/logger) throughout your code when developing for SQD Cloud. You can set the severity flags for squids running in the Cloud via `SQD_DEBUG`, `SQD_TRACE` or `SQD_INFO` - see [Environment Variables](/en/cloud/resources/env-variables).

To inspect and follow the squid logs from all the squid services, use [`sqd logs`](/en/cloud/reference/cli/logs):

```bash theme={"system"}
sqd logs -n <name> -s <slot> -f
```

or

```bash theme={"system"}
sqd logs -n <name> -t <tag> -f
```

<details>
  <summary>For older version-based deployments...</summary>

  ...the slot string is `v$\{version\}`, so use

  ```bash theme={"system"}
  sqd logs -n <name> -s v<version> -f
  ```

  Check out the [Slots and tags guide](/en/cloud/resources/slots-and-tags) to learn more.
</details>

There are additional flags to filter the logs:

* `-f` to follow the logs
* `-c` allows filtering by the container (can be `processor`, `db`, `db-migrate` and `query-node`)
* `-l` allows filtering by the severity
* `-p` number of lines to fetch (default: `100`)
* `--since` cut off by the time (default: `1d`). Accepts the notation of the [`ms` library](https://www.npmjs.com/package/ms): `1d`, `10h`, `1m`.

### Example

```bash theme={"system"}
sqd logs squid-substrate-template@v1 -f -c processor -l info --since 1d
```


# Monitoring
Source: https://docs.sqd.dev/en/cloud/resources/monitoring

Monitor SQD Cloud squids with built-in Prometheus endpoints — collect metrics from processor and GraphQL services for dashboards and alerting.

Each deployed squid version exposes Prometheus metrics for external monitoring with e.g. Grafana.

## Processor metrics

The processor metrics are available at

* `https: "//$\{org\}.squids.live/$\{name\}@$\{slot\}/processors/$\{processor\}/metrics`, and at"
* `https: "//$\{org\}.squids.live/$\{name\}:$\{tag\}/processors/$\{processor\}/metrics` for each tag attached to the slot."

See the [slots and tags guide](/en/cloud/resources/slots-and-tags).

`$\{processor\}` here is the processor name; it defaults to `processor` unless specified.

The metrics are documented inline. They include some values reflecting the squid health:

* `sqd_processor_last_block`. The last processed block.
* `sqd_processor_chain_height`. Current chain height as reported by the RPC endpoint (when [RPC ingestion](/en/sdk/squid-sdk/resources/unfinalized-blocks) is enabled) or by [SQD Network](/en/network/overview) (when it is disabled).

Inspect the metrics endpoint for a full list.

## Postgres metrics

Postgres metrics will be available in the future SQD Cloud releases.

## API metrics

API metrics will be available in the future SQD Cloud releases.


# Organizations
Source: https://docs.sqd.dev/en/cloud/resources/organizations

Group squids and team members into organizations on SQD Cloud — invite users, assign roles, share Postgres addons, and manage billing per org.

All squids within SQD Cloud are grouped into *organizations*. This additional layer of hierarchy enables team access and per-team billing. It also prevents unnecessary data sharing, as [environment variables](/en/cloud/resources/env-variables) are separate between organizations.

To create an organization, click on the organizations dropdown menu in the upper left corner of the [Cloud homepage](https://app.subsquid.io/) and select "Create new organization".

## Playgrounds

A playground organization is created for each account on its first login. There, you can deploy one squid for development or prototyping, free of charge.

Playground squids cannot be used in production, as they are [collocated](/en/cloud/reference/scale/#dedicated) and run on [spot VMs](https://cloud.google.com/spot-vms). Expect 3-5 minutes of downtime once every few days. Other limitations include:

* squid manifests with the [`scale:` section](/en/cloud/reference/scale) are forbidden;
* 10 GB of database storage;
* 500k monthly requests to the [built-in RPC service](/en/cloud/resources/rpc-proxy).

Unlike other organizations, playgrounds cannot be shared or billed.

## Draft organizations

Freshly created organizations are marked as drafts until upgraded to Professional status. It is not possible to deploy squids to draft organizations, but you can invite other users into them and set [environment variables](/en/cloud/resources/env-variables).

## Professional organizations

Adding a valid payment method promotes an organization to Professional status. Visit [the billing page](https://app.subsquid.io/billing), select your organization in the dropdown menu in the top left corner and follow the upgrade instructions:

Once an organization is upgraded you can deploy as many squids as you requre. [`scale:` section](/en/cloud/reference/scale) is now unlocked: use it to request any resources suitable for your use case. Your organization will be billed according to our [pricing schedule](/en/cloud/pricing).

## Working with organizations

When your account has access to more than one organization, it is necessary to specify one when [listing](/en/cloud/reference/cli/list) or [deploying](/en/cloud/reference/cli/deploy) (with some exceptions) your squids, as well as when [setting secrets](/en/cloud/reference/cli/secrets). Do it with the `--org/-o` flag:

```bash theme={"system"}
sqd secrets ls -o my-organization
sqd secrets rm SECRET --org my-organization
sqd secrets set SECRET --org my-organization

sqd ls -o my-organization
sqd deploy . -o my-organization
```

If you omit the flag, `sqd` will ask you to choose an organization interactively.


# Production alias (deprecated)
Source: https://docs.sqd.dev/en/cloud/resources/production-alias

Reference for the deprecated production-alias system on SQD Cloud — pin a hostname to a specific squid version, now replaced by slots-and-tags.

<Warning>
  Production aliasing feature is deprecated in `@subsquid/cli>=3.0.0`. Use [Slots and tags](/en/cloud/resources/slots-and-tags) instead.
</Warning>

Version aliasing is used to switch between squid versions without a downtime and updates of the downstream clients.
Each squid has a canonical production endpoint URL of the form

```bash theme={"system"}
https://<org name>.subsquid.io/<squid name>/graphql
```

To alias a squid version to the production endpoint, use [`sqd prod`](/en/cloud/reference/cli/prod):

```bash theme={"system"}
sqd prod <squid name>@<version>
```

Note that after promoting to the production the version-specific endpoint URL of the form

```bash theme={"system"}
https://<org name>.subsquid.io/<squid name>/v/v<version>/graphql
```

remains to be available.

## Example

Assuming your organization is called `my-org`, running

```bash theme={"system"}
sqd prod my-squid@v1
```

will make the endpoint of the v1 of `my-squid` accessible at `https://my-org.subsquid.io/my-squid/graphql`.


# Query optimization
Source: https://docs.sqd.dev/en/cloud/resources/query-optimization

Optimize squid GraphQL query performance on SQD Cloud — add indexes, tune Postgres, configure caching, and identify slow queries via the explorer.

Before going to production, ensure that none of the queries that your app generates are executed suboptimally. Failure to do so can cause severe performance issues.

The optimization procedure involves executing a representative sample of your app's queries, analyzing the slow queries and iteratively adding any helpful indexes. Here is the procedure:

1. Create a development deployment of your squid.
   * Use the squid name that you intend to be using in production, a new slot and (optionally) a dev tag. See the [Slots and tags guide](/en/cloud/resources/slots-and-tags).

2. Wait for you dev deployment to sync.

3. Configure your dev deployment to log mildly slow SQL queries.
   * Set the `log_min_duration_statement` to some low value in your manifest:
     ```yaml title=squid.yaml theme={"system"}
     deploy:
       addons:
         postgres:
           config:
             log_min_duration_statement: 50
     ...
     ```
     I'm using 50 ms = 0.05 s here. All queries that take longer than that to execute will now be logged by the `db` container.
   * Update your dev deployment, e.g. with
     ```bash theme={"system"}
     sqd deploy -s <dev-slot> .
     ```

4. Send a representative set of queries to your dev deployment. For this step you can use automated or manual tests of your API consumer app.

5. View the slow queries using [`sqd logs`](/en/cloud/reference/cli/logs), e.g.
   ```bash theme={"system"}
   sqd logs -f -n <my-squid-name> -s <dev-slot> -c db
   ```

6. Connect to your dev deployment's database with `psql`. You can find the connection command by going to the [squids view of the network app](https://app.subsquid.io/squids) > your dev deployment's page > "DB access" tab.

7. Execute the slow queries with [`EXPLAIN ANALYZE`](https://www.postgresql.org/docs/current/sql-explain.html) and view their execution plans.

8. Hypothesize about which indexes that could be used to speed up your queries.
   * Aim to use as few indexes as possible: having more than necessary can hurt the performance.
   * One possible strategy is to use ChatGPT. Just give it the execution plan and ask which indexes should be added. If it struggles give it your database schema as well (e.g. in form of your `schema.graphql` file listing). It is also useful to ask it to explain its suggestion.

9. Update the indexes of your dev deployment's database:

   * Ensure that your local database container schema is the same as the schema of your dev deployment's database. Assuming that the codebases are the same,
     ```bash theme={"system"}
     docker compose up -d
     npm run build
     npx squid-typeorm-migration apply
     ```
     should do the trick. If you encounter problems, wipe the database with `docker compose down` and repeat the step.
   * Add or remove some [`@index` statements](/en/sdk/squid-sdk/reference/schema-file/indexes-and-constraints) to your squid's schema file.
   * Regenerate the TypeORM model classes.
     ```bash theme={"system"}
     npx squid-typeorm-codegen
     ```
   * Build your code.
     ```bash theme={"system"}
     npm run build
     ```
   * Generate an incremental migration file.
     ```bash theme={"system"}
     npx squid-typeorm-migration generate
     ```
   * Update your dev deployment in the Cloud without resetting its database, e.g.
     ```bash theme={"system"}
     sqd deploy -s <dev-slot> .
     ```

   Main guide is in [this section](/en/sdk/squid-sdk/resources/persisting-data/typeorm#updating-a-deployed-squid-schema).

10. Repeat steps 4-9 until there are no more slow queries.
    * If your queries are complicated, you can skip step 9 and add indexes manually in `psql`, e.g. like this
      ```sql theme={"system"}
      CREATE INDEX "my_idx" ON my_table (column);
      ```
      This can drastically reduce the time you spend per iteration, but then you'll have to manually add the combination of indexes that worked to `schema.graphql` and redo the whole optimization procedure once more to ensure consistency. Otherwise, you may run into unexpected performance issues due to lost/changed indexes as you redeploy or modify your squid.

11. Set a less strict rule for logging slow queries, e.g. as follows
    ```yaml title=squid.yaml theme={"system"}
    deploy:
      addons:
        postgres:
          config:
            log_min_duration_statement: 1000 # 1 second
    ...
    ```
    Update the deployment:
    ```bash theme={"system"}
    sqd deploy -s <dev-slot> .
    ```

12. Mark your development deployment for production use.

    * If you used a development tag, remove it with [`sqd tags remove`](/en/cloud/reference/cli/tags#sqd-tags-remove)
    * Add a production tag to the deployment with [`sqd tags add`](/en/cloud/reference/cli/tags#sqd-tags-add)

    See also the [Zero-downtime updates section](/en/cloud/resources/slots-and-tags/#zero-downtime-updates).


# RPC addon
Source: https://docs.sqd.dev/en/cloud/resources/rpc-proxy

Use SQD Cloud's built-in RPC proxy as a managed RPC source for squid processors — bundled rate limits, supported networks, and configuration options.

[List of available networks](/en/cloud/reference/rpc-proxy-networks) and their aliases is available on the reference page.

SQD Cloud provides a built-in RPC service. The service is available as an addon to all squids deployed to the Cloud, although [playground](/en/cloud/resources/organizations/#playgrounds) squids only have access to a limited number of calls.

[Professional organizations](/en/cloud/resources/organizations/#professional-organizations) do not have that limitation and [can access](#external-access) their RPC endpoints from outside of the Cloud.

Enable the RPC add-on in the [manifest](/en/cloud/reference/manifest) like this:

```yaml theme={"system"}
deploy:
  addons:
    rpc:
      - eth-goerli.http
```

## Processor configuration

With the add-on successfully enabled, your squid will get a unique proxied endpoint to the requested network. SQD Cloud will make its URL available to the deployed squid at the `RPC_$\{Upper(network)\}_$\{Upper(protocol)\}` environment variable. Assert it to avoid compilation errors. We also recommend rate limiting RPC addon requests on the processor side to the same rate as was used in the manifest:

```typescript theme={"system"}
import \{assertNotNull\} from '@subsquid/util-internal'

processor.setRpcEndpoint({
  // dash in "eth-goerli" becomes an underscore
  url: assertNotNull(process.env.RPC_ETH_GOERLI_HTTP),
  rateLimit: 10
})
```

## External access

[Professional organizations](/en/cloud/resources/organizations/#professional-organizations) can [access their RPC proxies from outside](https://app.subsquid.io/rpc) of the SQD Cloud.

Among other things, this enables a development process that uses the service for both local and Cloud runs transparently. Here an example configuration for Ethereum Mainnet:

<details>
  <summary>Here is an example configuration for Ethereum Mainnet</summary>

  1. Add the following variable to `.env`:
     ```bash theme={"system"}
     # get the url from https://app.subsquid.io/rpc/chains/eth
     RPC_ETH_HTTP=<your_rpc_url_for_external_use>
     ```

  2. Enable the RPC addon in squid.yaml:
     ```yaml theme={"system"}
     deploy:
       addons:
         rpc:
           - eth.http
     ```
     This will add and populate the `RPC_ETH_HTTP` variable for Cloud deployments.

  3. Configure the processor to use the URL from `RPC_ETH_HTTP`:
     ```typescript theme={"system"}
     import \{EvmBatchProcessor\} from '@subsquid/evm-processor'
     import \{assertNotNull\} from '@subsquid/util-internal'

     export const processor = new EvmBatchProcessor()
       .setRpcEndpoint(assertNotNull(process.env.RPC_ETH_HTTP))
       // ...the rest of the processor configuration
     ```
</details>

## Pricing

RPC addon requests are priced [at a flat rate](/en/cloud/pricing/#rpc-requests), with substantial packages included for free for all [organization types](/en/cloud/resources/organizations). Pricing does not depend on the call method.

**Current Pricing (Until March 31, 2026):**

* Free tier: 2M requests/month
* Overage: \$2.00 per 1M requests

**New Pricing (From April 1, 2026):**

* Free tier: 2M requests/month (unchanged)
* Overage: \$5.00 per 1M requests

See the [pricing announcement](/en/other/pricing-update-april-2026) for more details about upcoming changes.


# Slots and tags
Source: https://docs.sqd.dev/en/cloud/resources/slots-and-tags

Manage multiple squid deployments per project on SQD Cloud — slots-and-tags model for blue-green releases, canary versions, and rollbacks.

<Tip>
  If you used SQD Cloud before the slots and tags update, you may want to take a look at the [Changelog](/en/cloud/reference/deployments-two-changelog), including its [Backwards compatibility section](/en/cloud/reference/deployments-two-changelog/#backwards-compatibility).
</Tip>

Starting with `@subsquid/cli>=3.0.0` SQD Cloud is using a new deployment system based on the following concepts:

* **Slots:** Given a squid name and an [organization](/en/cloud/resources/organizations), slot name uniquely identifies a squid deployment.
  * Slots are identified by short strings. They directly replace versions from the old workflow, which were identified by numbers.
  * You can specify a slot on deployment or omit it. In the latter case the Cloud will create a new slot with a random identifier.
  * Deploying to an already occupied slot will result in updating the initial deployment.
* **Tags:** A way to label deployments.

  Tags rules:

  * Each tag can be assigned to only one deployment among those that have the same name + organization. Assigning an existing tag *moves* it.
  * Multiple tags can be assigned to any one deployment.

  Tags uses:

  * Each tag added to the slot adds a new custom endpoint.
  * You can address deployments by their tags to perform various operations, including redeployment.

Both tags and slots can be specified in squid manifest as well as via command line, giving rise to a variety of possible workflows. Let's take a look at a few common scenarios.

## Zero-downtime updates

<Info>
  The functionality in this section mirrors the now-deprecated [production aliases](/en/cloud/resources/production-alias).
</Info>

As you develop your squid you will sometimes need to re-sync it from scratch. With tags you can do this without a maintenance downtime.

**Here is how to prepare your squid on its initial deployment:**

1. Go to your squid project (or the [EVM template](https://github.com/subsquid-labs/squid-evm-template/) if you just want something to test this with) and open the squid manifest (`squid.yaml` by default). Remove any deployment references:

   * `version:` (old)
   * `slot:` (new)
   * `tag:` (new)

   This is needed because we'll let the Cloud auto generate a slot identifier for us. The top of your manifest file should look something like this:

   ```yaml theme={"system"}
   manifestVersion: subsquid.io/v0.1
   name: my-squid
   description: 'The very first...
   ```

2. Deploy the squid with `sqd deploy .` while in the project folder. Once complete, you’ll see a message like this:

   ```
   =================================================
   A new squid my-org/my-squid@fj0anc has been successfully created
   =================================================
   ```

   `fj0anc` is the slot name that Cloud automatically assigned to your squid deployment. Here's how this deployment looks like in [the app](https://app.subsquid.io/squids):

   At this stage you can already access your deployment's API via a slot-based URL. However, that API will go down if we reset the deployment's database. Instead, let us use a tag-based URL that can be redirected to an API of another deployment.

3. Assign a tag to your deployment:
   ```bash theme={"system"}
   sqd tags add production -n my-squid -s fj0anc
   ```
   This assigns the `production` tag to the deployment, making the API accessible via:
   ```
   https://my-org.squids.live/my-squid:production/api/graphql
   ```
   Use this URL to access the API in your frontend app.

**Now, let's say you've made some changes to your codebase and need to re-sync your squid from scratch.** Follow these steps:

1. Make one more deployment of your squid. For that you can just re-run `sqd deploy .`: since no slot, version or tag is supplied in the manifest, the Cloud will automatically create a new slot for you. Your Cloud deployments might now look like this:

2. Wait for your new deployment to sync, then test its API using its slot-based (*canonical*) URL:
   ```
   https://my-org.squids.live/my-squid@u293zi/api/graphql
   ```

3. When you're ready to switch, simply reassign the `production` tag to the new deployment:
   ```bash theme={"system"}
   sqd tags add production -n my-squid -s u293zi
   ```
   The tag-based URL will now point to the new deployment, `u293zi`, with the switch typically taking just a few seconds.

4. Finally, if you no longer need the old deployment, remove it by slot:
   ```bash theme={"system"}
   sqd rm -n my-squid -s fj0anc
   ```

## Multiple major versions

One common scenario occurs when you release a breaking change and need to maintain several major versions of your API. With tags, you can keep all these major versions under the same squid name.

Let’s say you’ve introduced a breaking change to the squid from the previous example. Deploy your new version as follows:

```bash theme={"system"}
sqd deploy .
# then, assuming that the new deployment got slot f69x40
sqd tags add production-v1 -n my-squid -s f69x40
```

Alternatively, you can use the `--add-tag` option of `sqd deploy` to do the same thing without having to handle the autogenerated slot name:

```bash theme={"system"}
sqd deploy . --add-tag production-v1
```

This will produce exactly the same result. Here's how it may look like in the Cloud app:

Now you have two independent APIs accessible by tag-based URLs and you can perform [zero-downtime updates](#zero-downtime-updates) on each.

## Development workflows

In the [previous section](#multiple-major-versions) we considered the situation when multiple tags are used to denote major versions of the API. However, the exact same approach can be used to separate, for example, development from production:

When working on your development deployment, you’ll likely need to perform a variety of operations on it, frequently and without the concern of downtime. For that, all Squid CLI commands allow addressing deployments by tag. For example, to access logs of your `development` deployment use

```bash theme={"system"}
sqd logs -n my-squid -t development
```

This command allows you to pull logs directly from the deployment tagged `development`, without needing to remember or specify the slot name.

Moreover, you can update your `development`-tagged deployment without having to know its slot name:

```bash theme={"system"}
sqd deploy . -t development
```

It's even possible to ensure that whenever you deploy from the current directory, the deployment goes to the slot tagged `development`. Simply add the following line to `squid.yaml`:

```yaml theme={"system"}
tag: development
```

This may serve as a safeguard, preventing accidental deployments to production or the creation of unintended deployments.

## Next steps

Now that you've seen how slots and tags can help you with some common tasks, you may want to use them to design new workflows uniquely suitable for the needs of your project. Here are some resources that may help you with that:

* [Squid CLI documentation](/en/cloud/reference/cli/installation), particularly the [`sqd deploy` page](/en/cloud/reference/cli/deploy). Most of it is also available via
  ```bash theme={"system"}
  sqd deploy --help
  ```
  All the flags you can use to refer to deployments and to disable protective user prompts are described there.

* The detailed [changelog](/en/cloud/reference/deployments-two-changelog) that (for now) doubles as the reference documentation page. It describes the new system by comparing it to the old one that was based on versions and [production aliases](/en/cloud/resources/production-alias).

* If you used the old system or witnessed the migration of your organization, you may want to take a look at the [Backwards compatibility](/en/cloud/reference/deployments-two-changelog/#backwards-compatibility) section of the changelog.


# SQD Cloud Troubleshooting
Source: https://docs.sqd.dev/en/cloud/troubleshooting

Common issues and solutions for SQD Cloud squid deployments — fixes for failed builds, addon errors, RPC issues, and Postgres migration problems.

### "Secrets outdated. Please restart the squid" warning

This occurs when you have a squid deployed, then create, remove or change some [secrets](/en/cloud/reference/cli/secrets) of [relevance](/en/cloud/resources/organizations). Squids must be restarted manually for such changes to have effect. Navigate to the squid version page (e.g. by clicking on the warning sign) and click restart. The restart will not touch the database, so unless your new secret values cause the squid to crash this procedure should be quick and easy.

### My squid is stuck in "Building", "Deploying" or "Starting" state

* Run with `SQD_DEBUG=*` as explained on the [Logging](/en/sdk/squid-sdk/reference/logger#overriding-the-log-level) page
* Update the squid CLI to the latest version with

```bash theme={"system"}
npm update -g @subsquid/cli
```

* Update the Squid SDK dependencies:

```bash theme={"system"}
npm run update
```

* Check that the squid adheres to the expected [structure](/en/sdk/squid-sdk/how-to-start/layout)
* Make sure you can [build and run Docker images locally](/en/sdk/squid-sdk/resources/self-hosting)

### `Validation error` when releasing a squid

Make sure the squid name contains only alphanumeric characters, underscores and hyphens. The squid version must be also alphanumeric.
Since both the squid and version name become part of the squid API endpoint URL, slashes and dots are not accepted.

### My squid ran out of disk space

Edit the [postgres addon](/en/cloud/reference/pg) section of `squid.yaml` and request more space for the database.

### My squid is behind the chain, but is shows that it is in sync

Check that your processor uses both a RPC endpoint as one of its data sources (in addition to a SQD Network gateway).


# Block data for Fuel Network
Source: https://docs.sqd.dev/en/sdk/squid-sdk/fuel-indexing/fuel-datasource/context-interfaces

Block data interfaces for Fuel Network indexing in Squid SDK — type definitions for blocks, transactions, receipts, inputs, and outputs in the context.

In Fuel Squid SDK, the data is processed by repeatedly calling the user-defined [batch handler](/en/sdk/squid-sdk/reference/processors/architecture#processorrun) function on batches of on-chain data. The sole argument of the batch handler is its context `ctx`, and `ctx.blocks` is an array of `Block` objects containing the data to be processed, aligned at the block level.

For Fuel `DataSource` the `Block` interface is defined as follows:

```ts theme={"system"}
export interface Block {
  header: BlockHeader;
  transactions: Transaction[];
  inputs: TransactionInput[];
  outputs: TransactionOutput[];
  receipts: Receipt[];
}
```

`Block.header` contains the block header data. The rest of the fields are iterables containing the four kinds of blockchain data. The items within each iterable are ordered in the same way as they are within blocks.

The exact fields available in each data item type are inferred from the `setFields()` call argument. The method is documented on the [field selection](/en/sdk/squid-sdk/fuel-indexing/fuel-datasource/field-selection) page:

* [`Input` section](/en/sdk/squid-sdk/fuel-indexing/fuel-datasource/field-selection#input);
* [`Transaction` section](/en/sdk/squid-sdk/fuel-indexing/fuel-datasource/field-selection#transaction);
* [`Output` section](/en/sdk/squid-sdk/fuel-indexing/fuel-datasource/field-selection#output);
* [`Receipt` section](/en/sdk/squid-sdk/fuel-indexing/fuel-datasource/field-selection#receipt).


# Fuel Field Selection
Source: https://docs.sqd.dev/en/sdk/squid-sdk/fuel-indexing/fuel-datasource/field-selection

Fine-tune Fuel Network data requests with setFields() in Squid SDK — choose only the block, transaction, and receipt fields you need to minimize bandwidth.

#### `setFields(options)`

Set the fields to be retrieved for data items of each supported type. The `options` object has the following structure:

```ts theme={"system"}
{
  block?:       // field selector for block headers
  transaction?: // field selector for transactions
  receipt?:     // field selector for receipts
  input?:       // field selector for inputs
  output?:      // field selector for output
}
```

Every field selector is a collection of boolean fields that map to the fields of data items within the batch context [iterables](/en/sdk/squid-sdk/fuel-indexing/fuel-datasource/context-interfaces). Defining a field of a field selector of a given type and setting it to true will cause the processor to populate the corresponding field of all data items of that type. Here is a definition of a processor that requests `hash` and `status` fields for transactions and the `contract` field for receipts:

```ts theme={"system"}
const dataSource = new DataSourceBuilder().setFields({
  transaction: {
    hash: true,
    status: true,
  },
  receipt: {
    contract: true,
  },
});
```

Same fields will be available for all data items of any given type, including the items accessed via nested references. Suppose we used the processor defined above to subscribe to some transactions as well as some receipts, and for each receipt we requested its parent transaction:

```ts theme={"system"}
dataSource
  .addTransaction({
    // some transaction data requests
  })
  .addReceipt({
    // some receipt data requests

    transaction: true
  })
  .build();
```

After populating the convenience reference fields with `augmentBlock()` from `@subsquid/fuel-objects`, the `contract` field will be available both within the data items of the `transactions` iterable of [block data](/en/sdk/squid-sdk/fuel-indexing/fuel-datasource/context-interfaces) and within the transaction items that provide parent transaction information for the receipts matching the `addReceipt()` data request:

```ts theme={"system"}
run(dataSource, database, async (ctx) => {
  let blocks = ctx.blocks.map(augmentBlock)
  for (let block of blocks) {
    for (let txn of block.transactions) {
      let contract = txn.contract; // OK
    }
    for (let rec of block.receipts) {
      if (/* rec matches the data request */) {
        let recContract = rec.transaction.contract; // also OK!
      }
    }
  }
})
```

Some data fields, like `hash` for transactions, are enabled by default but can be disabled by setting a field of a field selector to `false`. For example, this code will not compile:

```ts theme={"system"}
const dataSource = new DataSourceBuilder()
  .setFields({
    transaction: {
      hash: false,
    },
  })
  .build();

run(dataSource, database, async (ctx) => {
  for (let block of ctx.blocks) {
    for (let txn of block.transactions) {
      let hash = txn.hash; // ERROR: no such field
    }
  }
});
```

Disabling unused fields will improve sync performance, as the fields' data will not be fetched from the SQD Network gateway.

## Data item types and field selectors

<Tip>
  Most IDEs support smart suggestions to show the possible field selectors. For VS Code, press `Ctrl+Space`.
</Tip>

Here we describe the data item types as functions of the field selectors. Unless otherwise mentioned, each data item type field maps to the eponymous field of its corresponding field selector. Item fields are divided into three categories:

* Fields that are always added regardless of the `setFields()` call.
* Fields that are enabled by default and can be disabled by `setFields()`. E.g. a `hash` field will be fetched for transactions by default, but can be disabled by setting `hash: false` within the `transaction` field selector.
* Fields that can be requested by `setFields()`.

### Transaction

Fields of `Transaction` data items may be requested by the eponymous fields of the field selector. Composite fields like `inputContract` are requested in their entirety by a single selector.

Here's a detailed description of possible `Transaction` fields:

```ts theme={"system"}
Transaction {
  // independent of field selectors
  index: number

  // can be disabled with field selectors
  hash: string
  type: TransactionType
  status: Status

  // can be enabled with field selectors
  inputAssetIds?: string[]
  inputContracts?: string[]
  inputContract?: {
    utxoId: string
    balanceRoot: string
    stateRoot: string
    txPointer: string
    contractId: string
  }
  policies?: Policies
  scriptGasLimit?: bigint
  maturity?: number
  mintAmount?: bigint
  mintAssetId?: string
  mintGasPrice?: bigint
  txPointer?: string
  isScript: boolean
  isCreate: boolean
  isMint: boolean
  isUpgrade: boolean
  isUpload: boolean
  outputContract?: {
    inputIndex: number
    balanceRoot: string
    stateRoot: string
  }
  witnesses?: string[]
  receiptsRoot?: string
  script?: string
  scriptData?: string
  salt?: string
  storageSlots?: string[]
  rawPayload?: string
  bytecodeWitnessIndex?: number
  bytecodeRoot?: string
  subsectionIndex?: number
  subsectionsNumber?: number
  proofSet?: string[]
  upgradePurpose?: UpgradePurpose
}
```

* `transactionType` field has the following type:

  ```ts theme={"system"}
  type TransactionType =
    "Script" |
    "Create" |
    "Mint" |
    "Upgrade" |
    "Upload"
  ```

* `status` field has the following type:

  ```ts theme={"system"}
  type Status =
    SubmittedStatus |
    SuccessStatus |
    SqueezedOutStatus |
    FailureStatus
  ```

  `Status` can be one of the following:

  ```ts theme={"system"}
  interface SubmittedStatus {
    type: "SubmittedStatus"
    time: bigint
  }

  interface SuccessStatus {
    type: "SuccessStatus"
    transactionId: string
    time: bigint
    programState?: ProgramState
  }

  interface SqueezedOutStatus {
    type: "SqueezedOutStatus"
    reason: string
  }

  interface FailureStatus {
    type: "FailureStatus"
    transactionId: string
    time: bigint
    reason: string
    programState?: ProgramState
    totalGas: bigint
    totalFee: bigint
  }
  ```

  `ProgramState` is defined as follows:

  ```ts theme={"system"}
  interface ProgramState {
    returnType: "RETURN" | "RETURN_DATA" | "REVERT";
    data: string;
  }
  ```

* `policies` field has the following interface:

  ```ts theme={"system"}
  interface Policies {
    gasPrice?: bigint
    witnessLimit?: bigint
    maturity?: number
    maxFee?: bigint
  }
  ```

* `upgradePurpose` field is of type `ConsensusParametersPurpose | StateTransitionPurpose`, where the types are defined as:
  ```ts theme={"system"}
  interface ConsensusParametersPurpose {
    type: 'ConsensusParametersPurpose'
    witnessIndex: number
    checksum: string
  }

  interface StateTransitionPurpose {
    type: 'StateTransitionPurpose'
    root: string
  }
  ```

### Receipt

Fields of `Receipt` data items may be requested by the eponymous fields of the field selector. Here's a detailed description of possible `Receipt` fields:

```ts theme={"system"}
Receipt {
  // independent of field selectors
  index: number
  transactionIndex: number

  // can be disabled with field selectors
  receiptType: ReceiptType

  // can be enabled with field selectors
  contract?: string
  pc?: bigint
  is?: bigint
  to?: string
  toAddress?: string
  amount?: bigint
  assetId?: string
  gas?: bigint
  param1?: bigint
  param2?: bigint
  val?: bigint
  ptr?: bigint
  digest?: string
  reason?: bigint
  ra?: bigint
  rb?: bigint
  rc?: bigint
  rd?: bigint
  len?: bigint
  result?: bigint
  gasUsed?: bigint
  data?: string
  sender?: string
  recipient?: string
  nonce?: string
  contractId?: string
  subId?: string
}
```

* `receiptType` has the following type:

  ```ts theme={"system"}
  type ReceiptType =
    | "CALL"
    | "RETURN"
    | "RETURN_DATA"
    | "PANIC"
    | "REVERT"
    | "LOG"
    | "LOG_DATA"
    | "TRANSFER"
    | "TRANSFER_OUT"
    | "SCRIPT_RESULT"
    | "MESSAGE_OUT"
    | "MINT"
    | "BURN";
  ```

### Input

`Input` data items can be of types `InputCoin`, `InputContract` or `InputMessage`. Each type has its own set of fields. To access the type-specific fields, narrow down the type by asserting the value of the `type` field, e.g.

```ts theme={"system"}
if (input.type === "InputCoin") {
  // use InputCoin-specific fields here
}
```

To get a name of a field selector field, apply a type prefix to a capitalized name of the data item field, e.g.

* `amount` field of `InputCoin` input items is requested by `coinAmount: true`;
* `witnessIndex` field of `InputMessage` input items is requested by `messageWitnessIndex: true`
  etc.

All `Input*` data types have `transactionIndex`, `index` and `type` fields. These cannot be disabled. There aren't any fields that are enabled by default and can be disabled.

`InputCoin` data items may have the following fields:

```ts theme={"system"}
interface InputCoin {
  type: "InputCoin";
  index: number;
  transactionIndex: number;
  utxoId: string;
  owner: string;
  amount: bigint;
  assetId: string;
  txPointer: string;
  witnessIndex: number;
  predicateGasUsed: bigint;
  predicate: string;
  predicateData: string;
}
```

`InputContract` data items may have the following fields:

```ts theme={"system"}
interface InputContract {
  type: "InputContract";
  index: number;
  transactionIndex: number;
  utxoId: string;
  balanceRoot: string;
  stateRoot: string;
  txPointer: string;
  contractId: string;
}
```

`InputMessage` data items may have the following fields:

```ts theme={"system"}
InputMessage {
  type: "InputMessage";
  index: number;
  transactionIndex: number;
  sender: string;
  recipient: string;
  amount: bigint;
  nonce: string;
  witnessIndex: number;
  predicateGasUsed: bigint;
  data: string;
  predicate: string;
  predicateData: string;
}
```

It is possible to request more than one type of input in the same data request. For example, to request all inputs of type `InputCoin` and `InputContract`:

```ts theme={"system"}
dataSource
  .addInput({
    type: ["InputCoin", "InputContract"],
  })
  .build();
```

### Output

`Output` data items can be of types `CoinOutput`, `ContractOutput`, `ChangeOutput`, `VariableOutput` or `'ContractCreated'`. Each type has its own set of fields. To access the type-specific fields, narrow down the type by asserting the value of the `type` field, e.g.

```ts theme={"system"}
if (output.type === "CoinOutput") {
  // use CoinOutput-specific fields here
}
```

To get a name of a field selector field, apply a type prefix to a capitalized name of the data item field, e.g.

* `to` field of `CoinOutput` items is requested by `coinTo: true`;
* `inputIndex` field of `ContractOutput` items is requested by `contractInputIndex: true`;
* `amount` field of `ChangeOutput` items is requested by `changeAmount: true`;
* `assetId` field of `VariableOutput` items is requested by `variableAssetId: true`;
* `stateRoot` field of `ContractCreated` items is requested by `contractCreatedStateRoot: true`
  etc.

All output data types have `transactionIndex`, `index` and `type` fields. These cannot be disabled. There aren't any fields that are enabled by default and can be disabled.

`CoinOutput` data items may have the following fields:

```ts theme={"system"}
CoinOutput {
  type: 'CoinOutput'
  index: number
  transactionIndex: number
  to: string
  amount: bigint
  assetId: string
}
```

`ContractOutput` data items may have the following fields:

```ts theme={"system"}
ContractOutput {
  type: "ContractOutput";
  index: number;
  transactionIndex: number;
  inputIndex: number;
  balanceRoot: string;
  stateRoot: string;
}
```

`ChangeOutput` data items may have the following fields:

```ts theme={"system"}
ChangeOutput {
  type: 'ChangeOutput'
  index: number
  transactionIndex: number
  to: string
  amount: bigint
  assetId: string
}

```

`VariableOutput` data items may have the following fields:

```ts theme={"system"}
VariableOutput {
  type: 'VariableOutput'
  index: number
  transactionIndex: number
  to: string
  amount: bigint
  assetId: string
}
```

`ContractCreated` data items may have the following fields:

```ts theme={"system"}
ContractCreated {
  type: 'ContractCreated'
  index: number
  transactionIndex: number
  contract: string
  stateRoot: string
}
```

### Block header

`BlockHeader` data items may have the following fields:

```ts theme={"system"}
BlockHeader {
	// independent of field selectors
  hash: string
  height: number

  // can be disabled with field selectors
  time: bigint

  // can be enabled with field selectors
  daHeight: bigint
  transactionsRoot: string
  transactionsCount: number
  messageReceiptCount: number
  prevRoot: string
  applicationHash: string
  eventInboxRoot: string
  consensusParametersVersion: number
  stateTransitionBytecodeVersion: number
  messageOutboxRoot: string
}
```

Request the fields with eponymous field request flags.

## A complete example

```ts theme={"system"}
import { run } from "@subsquid/batch-processor";
import { augmentBlock } from "@subsquid/fuel-objects";
import { DataSourceBuilder } from "@subsquid/fuel-stream";
import { TypeormDatabase } from "@subsquid/typeorm-store";
import { Contract } from "./model";

const dataSource = new DataSourceBuilder()
  .setGateway("https://v2.archive.subsquid.io/network/fuel-mainnet")
  .setGraphql({
    url: "https://mainnet.fuel.network/v1/graphql",
  })
  .setFields({
    transaction: {
      hash: false,
      isMint: true,
    },
    receipt: {
      amount: true,
      gas: true,
    },
    input: {
      coinOwner: true,
      contractStateRoot: true,
    },
    output: {
      coinTo: true,
      contractCreatedStateRoot: true,
    },
    block: {
      daHeight: true
    }
  })
  .addTransaction({
    type: ['Mint', 'Script'],
    inputs: true,
    range: { from: 1_000_000 }
  })
  .addReceipt({
    type: ['LOG_DATA']
  })
  .addInput({
    type: ['InputCoin', 'InputContract']
  })
  .addOutput({
    type: ['CoinOutput', 'ContractCreated'],
    transaction: true
  })
  .build()
```


# Inputs
Source: https://docs.sqd.dev/en/sdk/squid-sdk/fuel-indexing/fuel-datasource/inputs

Subscribe to Fuel transaction input data with addInput() in Squid SDK — filter by asset ID, owner, and predicate for efficient indexing.

#### `addInput(options)`

Get some *or all* inputs on the network. `options` has the following structure:

```typescript theme={"system"}
{
  // data requests
  type?: InputType[]
  coinOwner?: string[]
  coinAssetId?: string[]
  contractContract?: string[]
  messageSender?: string[]
  messageRecipient?: string[]

  // related data retrieval
  transaction?: boolean

  range?: {
    from: number
    to?: number
  }
}
```

Data requests:

* `type` sets the type of the input. You can request one or more of `'InputCoin' | 'InputContract' | 'InputMessage'`. Leave it undefined to subscribe to all inputs.

Enabling the `transaction` flag will cause the processor to retrieve transactions where the selected inputs have occurred. The data will be added to the appropriate iterables within the [block data](/en/sdk/squid-sdk/fuel-indexing/fuel-datasource/context-interfaces). You can also call `augmentBlock()` from `@subsquid/fuel-objects` on the block data to populate the convenience reference fields like `input.transaction`.

Note that inputs can also be requested by the other `FuelDataSource` methods as related data.

Selection of the exact fields to be retrieved for each transaction and the optional related data items is done with the `setFields()` method documented on the [Field selection](/en/sdk/squid-sdk/fuel-indexing/fuel-datasource/field-selection) page.

## Examples

Request all inputs with `InputCoin` type and include transactions:

```ts theme={"system"}
processor
  .addInput({
    type: ["InputCoin"],
    transaction: true,
  })
  .build();
```


# Outputs
Source: https://docs.sqd.dev/en/sdk/squid-sdk/fuel-indexing/fuel-datasource/outputs

Subscribe to Fuel transaction output data with addOutput() in Squid SDK — filter by asset ID, owner, and output type for efficient indexing.

#### `addOutput(options)`

Get some *or all* outputs on the network. `options` has the following structure:

```typescript theme={"system"}
{
  // data requests
  type?: OutputType[]

  // related data retrieval
  transaction?: boolean

  range?: {
    from: number
    to?: number
  }
}
```

Data requests:

* `type` sets the type of the output. Output type has the following options: `'CoinOutput' | 'ContractOutput' | 'ChangeOutput' | 'VariableOutput' | 'ContractCreated'`. Leave it undefined to subscribe to all outputs.

Enabling the `transaction` flag will cause the processor to retrieve transactions where the selected outputs have occurred. The data will be added to the appropriate iterables within the [block data](/en/sdk/squid-sdk/fuel-indexing/fuel-datasource/context-interfaces). You can also call `augmentBlock()` from `@subsquid/fuel-objects` on the block data to populate the convenience reference fields like `output.transaction`.

Note that receipts can also be requested by the other `FuelDataSource` methods as related data.

Selection of the exact fields to be retrieved for each transaction and the optional related data items is done with the `setFields()` method documented on the [Field selection](/en/sdk/squid-sdk/fuel-indexing/fuel-datasource/field-selection) page.

## Examples

Request all outputs with `ChangeOutput` type and include transactions:

```ts theme={"system"}
processor
  .addOutput({
    type: ["ChangeOutput"],
    transaction: true,
  })
  .build();
```


# Fuel Transactions Indexing
Source: https://docs.sqd.dev/en/sdk/squid-sdk/fuel-indexing/fuel-datasource/transactions

Subscribe to Fuel transaction data with addTransaction() in Squid SDK — filter by sender, type, and contract input for efficient on-chain indexing.

#### `addTransaction(options)`

Get some *or all* transactions on the network. `options` has the following structure:

```typescript theme={"system"}
{
  // data requests
  type?: TransactionType[]

  // related data retrieval
  receipts?: boolean
  inputs?: boolean
  outputs?: boolean

  range?: {
    from: number
    to?: number
  }
}
```

Data requests:

* `type` sets the type of the transaction: `'Script' | 'Create' | 'Mint' | 'Upgrade' | 'Upload'`. Leave it undefined to subscribe to all transactions.

Enabling the `receipts` and/or `inputs` and `outputs` flags will cause the processor to retrieve receipts, inputs and outputs that occurred as a result of each selected transaction. The data will be added to the appropriate iterables within the [block data](/en/sdk/squid-sdk/fuel-indexing/fuel-datasource/context-interfaces). You can also call `augmentBlock()` from `@subsquid/fuel-objects` on the block data to populate the convenience reference fields like `transaction.receipts`.

Note that transactions can also be requested by the other `FuelDataSource` methods as related data.

Selection of the exact fields to be retrieved for each transaction and the optional related data items is done with the `setFields()` method documented on the [Field selection](/en/sdk/squid-sdk/fuel-indexing/fuel-datasource/field-selection) page.

## Examples

Request all transactions with `Create` and `Mint` types and include receipts, inputs and outputs:

```ts theme={"system"}
processor
  .addTransaction({
    type: ["Create", "Mint"],
    receipts: true,
    inputs: true,
    outputs: true,
  })
  .build();
```


# Fuel Network API
Source: https://docs.sqd.dev/en/sdk/squid-sdk/fuel-indexing/network-api/fuel-api

Fuel Network data API reference for Squid SDK — endpoint URLs, schema fields, and access notes for the Squid SDK Fuel network data source.

# Fuel SQD Network API

<Warning>
  The Fuel API of SQD Network is currently in beta. Breaking changes may be introduced in the future releases.
</Warning>

SQD Network API distributes the requests over a ([potentially decentralized](/en/other/faq)) network of *workers*. The main gateway URL points at a *router* that provides URLs of workers that do the heavy lifting. Each worker has its own range of blocks on each dataset it serves.

Suppose you want to retrieve an output of some [query](#worker-api) on a block range starting at `firstBlock` (can be the genesis block) and ending at the highest available block. Proceed as follows:

1. Retrieve the dataset height from the router with `GET /height` and make sure it's above `firstBlock`.

2. Save the value of `firstBlock` to some variable, say `currentBlock`.

3. Query the router for an URL of a worker that has the data for `currentBlock` with `GET /${currentBlock}/worker`.

4. Retrieve the data from the worker by [posting the query](#worker-api) (`POST /`), setting the `"fromBlock"` query field to `${currentBlock}`.

5. Parse the retrieved data to get a batch of query data **plus** the height of the last block available from the current worker. Take the `header.number` field of the last element of the retrieved JSON array - it is the height you want. Even if your query returns no data, you'll still get the block data for the last block in the range, so this procedure is safe.

6. Set `currentBlock` to the height from the previous step **plus one**.

7. Repeat steps 3-6 until all the required data is retrieved.

See the [Fuel section of the public gateways list](/en/data/other) for URL gateways.

Implementation examples:

<AccordionGroup>
  <Accordion title="Manually with cURL">
    Suppose we want data on Fuel receipts from block `1000000`. We begin by finding the main URL for the Fuel Mainnet dataset. Then we have to:

    1. Retrieve the dataset height from the router with

       ```bash theme={"system"}
       curl https://v2.archive.subsquid.io/network/fuel-mainnet/height
       ```

       Output

       ```
       1211611
       ```

    2. Save the value `1000000` to some variable, say `currentBlock`.

    3. Query the router for an URL of a worker that has the data for`currentBlock`

       ```bash theme={"system"}
       curl https://v2.archive.subsquid.io/network/fuel-mainnet/1000000/worker
       ```

       Output

       ```
       https://rb03.sqd-archive.net/worker/query/czM6Ly9mdWVsLXRlc3RuZXQ
       ```

    4. Retrieve the data from the worker

       ```bash theme={"system"}
       curl https://rb03.sqd-archive.net/worker/query/czM6Ly9mdWVsLXRlc3RuZXQ \
       -X 'POST' -H 'content-type: application/json' -H 'accept: application/json' \
       -d '{
           "type": "fuel",
           "fromBlock":1000000,
           "fields":{"receipt":{"contract":true, "receiptType": true}},
           "receipts":[ {"type": ["LOG_DATA"]} ]
       }' | jq
       ```

       Output:

       ```json theme={"system"}
       [
         {
           "header": {
             "number": 1000000,
             "hash": "0xdc31db7fa3c1fb4f3e0910dc5abf927e64cc985eb2eb13418a9f2e00c4b7ad23"
           },
           "receipts": []
         },
         {
           "header": {
             "number": 1002527,
             "hash": "0x649f045675405f9d4ee34bb19479d0e5706ed14615e8f97da9f34dd166e37f35"
           },
           "receipts": [
             {
               "transactionIndex": 0,
               "index": 1,
               "contract": "0xe637b4c254aa07baa9845eb9f8c7ad0965fad5c5b1194cb37193f956be0ce6f3",
               "receiptType": "LOG_DATA"
             }
           ]
         },
         ...
         {
           "header": {
             "number": 1211611,
             "hash": "0x04c9ef60f2b54d32569a410477c136f11b692c1d3eadaa5b946a0295b526223e"
           },
           "receipts": []
         }
       ]
       ```

    5. Parse the retrieved data to get a batch of query data plus the height of the last block available from the current worker. Take the `header.number` field of the last element of the retrieved JSON array - it is the height you want. Even if your query returns no data, you'll still get the block data for the last block in the range, so this procedure is safe.

    6. Set `currentBlock` to the height from the previous step plus one.

    7. Repeat steps 3-6 until all the required data is retrieved.
  </Accordion>

  <Accordion title="In Python">
    ```python theme={"system"}
    def get_text(url: str) -> str:
        res = requests.get(url)
        res.raise_for_status()
        return res.text

    def dump(
        gateway_url: str,
        query: Query,
        first_block: int,
        last_block: int
    ) -> None:
        assert 0 <= first_block <= last_block
        query = dict(query)  # copy query to mess with it later

        dataset_height = int(get_text(f'{gateway_url}/height'))
        next_block = first_block
        last_block = min(last_block, dataset_height)

        while next_block <= last_block:
            worker_url = get_text(f'{gateway_url}/{next_block}/worker')

            query['fromBlock'] = next_block
            query['toBlock'] = last_block
            res = requests.post(worker_url, json=query)
            res.raise_for_status()
            blocks = res.json()

            last_processed_block = blocks[-1]['header']['number']
            next_block = last_processed_block + 1
            for block in blocks:
                print(json.dumps(block))
    ```

    Full code [here](https://gist.github.com/eldargab/2e007a293ac9f82031d023f1af581a7d).
  </Accordion>
</AccordionGroup>

## Router API

<AccordionGroup>
  <Accordion title="GET /height (get height of the dataset)">
    **Example response:** `16576911`.
  </Accordion>

  <Accordion title="GET ${firstBlock}/worker (get a suitable worker URL)">
    The returned worker will be capable of processing `POST /` requests in which the `"fromBlock"` field is equal to `${firstBlock}`.

    **Example response:** `https://rb03.sqd-archive.net/worker/query/czM6Ly9mdWVsLXRlc3RuZXQ`.
  </Accordion>
</AccordionGroup>

## Worker API

<Accordion title="POST / (query inputs and receipts)">
  ##### Query Fields

  * **fromBlock**: Block number to start from (inclusive).
  * **toBlock**: (optional) Block number to end on (inclusive). If this is not given, the query will go on for a fixed amount of time or until it reaches the height of the dataset.
  * **includeAllBlocks**: (optional) If true, the Network will include blocks that contain no data selected by data requests into its response.
  * **fields**: (optional) A [selector](#data-fields-selector) of data fields to retrieve. Common for all data items.
  * **receipts**: (optional) A list of [receipts requests](#receipts). An empty list requests no data.
  * **inputs**: (optional) A list of [inputs requests](#inputs). An empty list requests no data.
  * **outputs**: (optional) A list of [outputs requests](#outputs). An empty list requests no data.
  * **transactions**: (optional) A list of [transactions requests](#transactions). An empty list requests no data.

  <Accordion title="Example Request">
    ```json theme={"system"}
    {
      "type": "fuel",
      "fromBlock": 1000000,
      "toBlock": 1100000,
      "fields": { "receipt": { "contract": true, "receiptType": true } },
      "receipts": [{ "type": ["LOG_DATA"] }],
      "inputs": [{ "type": ["InputCoin"] }]
    }
    ```
  </Accordion>

  <Accordion title="Example Response">
    ```json theme={"system"}
    [
      {
        "header": {
          "number": 1000000,
          "hash": "0xdc31db7fa3c1fb4f3e0910dc5abf927e64cc985eb2eb13418a9f2e00c4b7ad23"
        },
        "receipts": [],
        "inputs": []
      },
      {
        "header": {
          "number": 1002527,
          "hash": "0x649f045675405f9d4ee34bb19479d0e5706ed14615e8f97da9f34dd166e37f35"
        },
        "receipts": [
          {
            "transactionIndex": 0,
            "index": 1,
            "contract": "0xe637b4c254aa07baa9845eb9f8c7ad0965fad5c5b1194cb37193f956be0ce6f3",
            "receiptType": "LOG_DATA"
          }
        ],
        "inputs": [
          {
            "transactionIndex": 0,
            "index": 1
          }
        ]
      },
      ...
      {
        "header": {
          "number": 1100000,
          "hash": "0x4e1420d7c2cd973842ef1ce919560f15e5461376b7533c371fb895034c85dfd3"
        },
        "receipts": [],
        "inputs": []
      }
    ]
    ```
  </Accordion>
</Accordion>

## Data requests

### Receipts

```ts theme={"system"}
{
  type?: ReceiptType[]
  contract?: string[]
  transaction?: boolean
}
```

Receipts will be included in the response if it matches all the requests. An empty array matches no instructions; omit all requests to match all receipts.

See [addReceipt()](/en/sdk/squid-sdk/fuel-indexing/fuel-datasource/receipts) SDK data request method for details on this request; also see [Receipt fields](/en/sdk/squid-sdk/fuel-indexing/fuel-datasource/field-selection#receipt).

### Transactions

```ts theme={"system"}
{
  type?: TransactionType[]
  receipts?: boolean
  inputs?: boolean
  outputs?: boolean
}
```

A transaction will be included in the response if it matches all the requests. An empty array matches no transactions; omit all requests to match all transactions.

See [addTransaction()](/en/sdk/squid-sdk/fuel-indexing/fuel-datasource/transactions) SDK data request method for details on this request; also see [Transaction fields](/en/sdk/squid-sdk/fuel-indexing/fuel-datasource/field-selection#transaction).

### Inputs

```ts theme={"system"}
{
  type?: InputType[]
  coinOwner?: Bytes[]
  coinAssetId?: Bytes[]
  contractContract?: Bytes[]
  messageSender?: Bytes[]
  messageRecipient?: Bytes[]
  transaction?: boolean
}
```

An input will be included in the response if it matches all the requests. An empty array matches no inputs; omit all requests to match all inputs.

See [addInput()](/en/sdk/squid-sdk/fuel-indexing/fuel-datasource/inputs) SDK data request method for details on this request; also see [Input fields](/en/sdk/squid-sdk/fuel-indexing/fuel-datasource/field-selection#input).

### Outputs

```ts theme={"system"}
{
  type?: OutputType[]
  transaction?: boolean
}
```

An output will be included in the response if it matches all the requests. An empty array matches no outputs; omit all requests to match all outputs.

See [addOutput()](/en/sdk/squid-sdk/fuel-indexing/fuel-datasource/outputs) SDK data request method for details on this request; also see [Output fields](/en/sdk/squid-sdk/fuel-indexing/fuel-datasource/field-selection#output).

## Data fields selector

A JSON selector of fields for the returned data items. Documented in the [Field selectors](/en/sdk/squid-sdk/fuel-indexing/fuel-datasource/field-selection) section.


# sqd auth
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/auth

Log in to SQD Cloud with the sqd auth CLI command — exchange your account credentials for an API token to authenticate sqd deploy and other commands.

Log in to the Cloud

* [`sqd auth`](#sqd-auth-1)

## `sqd auth`

Log in to the Cloud

```
USAGE
  $ sqd auth -k <value> [--interactive]

FLAGS
  -k, --key=<value>       (required) Cloud auth key. Log in to https://app.subsquid.io to create or update your key.
      --[no-]interactive  Disable interactive mode
```

*See code: [src/commands/auth.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/auth.ts)*


# sqd autocomplete
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/autocomplete

Display sqd CLI shell autocomplete installation instructions — supported shells (bash, zsh, fish, PowerShell) and how to enable command completion.

Display autocomplete installation instructions.


# commands.json
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/commands-json

Define custom sqd subcommands and aliases via commands.json in your Squid SDK project — extend the workflow with reusable scripts for build and deploy.

The `sqd` tool automatically discovers and loads any extra commands defined in the `commands.json` file. Here is a sample file demonstrating the available features:

```json theme={"system"}
{ // comments are ok
  "$schema": "https://subsquid.io/schemas/commands.json",
  "commands": {
    "clean": {
      "description": "delete all build artifacts",
      "cmd": ["rm", "-rf", "lib"]
    },
    "build": {
      "description": "build the project",
      "deps": ["clean"], // commands to execute before
      "cmd": ["tsc"]
    },
    "typegen": {
      "hidden": true, // Don't show in the overview listing
      "workdir": "abi", // change working dir
      "command": [
        "squid-evm-typegen", // node_modules/.bin is in the PATH
        "../src/abi",
        {"glob": "*.json"} // cross-platform glob expansion
      ],
      "env": { // additional environment variables
        "DEBUG": "*"
      }
    }
  }
}
```

This functionality is managed by the [`@subsquid/commands`](https://github.com/subsquid/squid-sdk/tree/master/util/commands) package.

All [squid templates](/en/sdk/squid-sdk/how-to-start/squid-development#templates) include such a file with a predefined set of useful shortcuts. See [Cheatsheet](/en/sdk/squid-sdk/how-to-start/cli-cheatsheet).


# sqd deploy
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/deploy

Deploy new or update an existing squid deployment in the Cloud. Squid name and also optionally slot and/or tag are taken from the provided deployment manifest.

Deploy new or update an existing squid deployment in the Cloud. Squid name and also optionally slot and/or tag are taken from the provided deployment manifest.

* [`sqd deploy SOURCE`](#sqd-deploy-source)

## `sqd deploy SOURCE`

Deploy new or update an existing squid in the Cloud

```
USAGE
  $ sqd deploy SOURCE [--interactive]
    [-r [<org>/]<name>(@<slot>|:<tag>) | -o <code> | -n <name> | [-s <slot>] | [-t <tag>]]
    [-m <manifest_path>] [--hard-reset] [--stream-logs] [--add-tag <value>]
    [--allow-update] [--allow-tag-reassign] [--allow-manifest-override]

ARGUMENTS
  SOURCE  [default: .] Squid source. Could be:
          - a relative or absolute path to a local folder (e.g. ".")
          - a URL to a .tar.gz archive
          - a github URL to a git repo with a branch or commit tag

FLAGS
  -m, --manifest=<manifest_path>  [default: squid.yaml] Specify the relative local path
                                  to a squid manifest file in the squid working directory
      --add-tag=<value>           Add a tag to the deployed squid
      --allow-manifest-override   Allow overriding the manifest during deployment
      --allow-tag-reassign        Allow reassigning an existing tag
      --allow-update              Allow updating an existing squid
      --hard-reset                Perform a hard reset before deploying. This will drop
                                  and re-create all squid resources, including the
                                  database, causing a short API downtime
      --[no-]interactive          Disable interactive mode
      --[no-]stream-logs          Attach and stream squid logs after the deployment

SQUID FLAGS
  -n, --name=<name>                               Name of the squid
  -r, --reference=[<org>/]<name>(@<slot>|:<tag>)  Fully qualified reference of the squid.
                                                  It can include the organization, name,
                                                  slot, or tag
  -s, --slot=<slot>                               Slot of the squid
  -t, --tag=<tag>                                 Tag of the squid

ORG FLAGS
  -o, --org=<code>  Code of the organization

DESCRIPTION
  Deploy new or update an existing squid in the Cloud

EXAMPLES

  // Create a new squid with name provided in the manifest file
  $ sqd deploy .

  // Create a new squid deployment and override it's name to "my-squid-override"
  $ sqd deploy . -n my-squid-override

  // Update the "my-squid" squid with slot "asmzf5"
  $ sqd deploy . -n my-squid -s asmzf5

  // Use a manifest file located in ./path-to-the-squid/squid.prod.yaml
  $ sqd deploy ./path-to-the-squid -m squid.prod.yaml

  // Full paths are also fine
  $ sqd deploy /Users/dev/path-to-the-squid -m /Users/dev/path-to-the-squid/squid.prod.yaml
```

*See code: [src/commands/deploy.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/deploy.ts)*


# sqd gateways
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/gateways

Explore SQD Network gateways and data sources for a squid with the sqd gateways CLI command — list endpoints, statuses, and chains supported.

Explore data sources for a squid

* [`sqd gateways list`](#sqd-gateways-list)

## `sqd gateways list`

List available gateways

```
USAGE
  $ sqd gateways list [--interactive] [-t <evm|substrate>] [-n <regex>] [-c <number>]

FLAGS
  -c, --chain=<number>        Filter by chain ID or SS58 prefix
  -n, --name=<regex>          Filter by network name
  -t, --type=<evm|substrate>  Filter by network type
      --[no-]interactive      Disable interactive mode

ALIASES
  $ sqd gateways ls
```

*See code: [src/commands/gateways/ls.ts](https://github.com/subsquid/squid-cli/blob/master/src/commands/gateways/ls.ts)*


# sqd init
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/init

Initialize a new Squid SDK project from a template or GitHub repo with sqd init — choose EVM, Substrate, Solana, Tron, or Fuel starters and scaffold files.

Setup a new squid project from a template or github repo

* [`sqd init NAME`](#sqd-init-name)

## `sqd init NAME`

Setup a new squid project from a template or github repo

```
USAGE
  $ sqd init NAME [--interactive] [-t <value>] [-d <value>] [-r]

ARGUMENTS
  NAME  The squid name. It must contain only alphanumeric or dash ("-") symbols and must not start with "-".

FLAGS
  -d, --dir=<value>
      The target location for the squid. If omitted, a new folder NAME is created.

  -r, --remove
      Clean up the target directory if it exists

  -t, --template=<value>
      A template for the squid. Accepts:
      - a github repository URL containing a valid squid.yaml manifest in the root folder
      or one of the pre-defined aliases:
      - evm  A minimal squid template for indexing EVM data.
      - abi  A template to auto-generate a squid indexing events and txs from a contract ABI
      - multichain  A template for indexing data from multiple chains
      - gravatar  A sample EVM squid indexing the Gravatar smart contract on Ethereum.
      - substrate  A template squid for indexing Substrate-based chains.
      - ink  A template for indexing Ink! smart contracts
      - ink-abi  A template to auto-generate a squid from an ink! contract ABI
      - frontier-evm  A template for indexing Frontier EVM chains, like Moonbeam and Astar.

  --[no-]interactive
      Disable interactive mode
```

*See code: [src/commands/init.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/init.ts)*


# Installation
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/installation

Install the sqd CLI in your local environment — npm install -g, standalone binaries for macOS/Linux/Windows, and shell autocomplete configuration.

Squid CLI is a command line tool for

* scaffolding new squids from templates
* running SDK tools and scripts defined in `commands.json` in a cross-platform way
* managing squid deployments in [SQD Cloud](/en/cloud) (former Aquarium)

The CLI is distributed as a [`npm` package](https://www.npmjs.com/package/@subsquid/cli).

To install Squid CLI, follow the steps below.

<Steps>
  <Step title="0. Install and setup Squid CLI">
    First, install the latest version of Squid CLI as a global `npm` package:

    ```bash theme={"system"}
    npm i -g @subsquid/cli@latest
    ```

    Check the version:

    ```bash theme={"system"}
    sqd --version
    ```

    Make sure the output looks like `@subsquid/cli@<version>`.

    <Info>
      The next steps are **optional** for building and running squids. A key is required to enable the CLI commands managing the [SQD Cloud](/en/cloud) deployments.
    </Info>
  </Step>

  <Step title="1. Obtain a SQD Cloud deployment key">
    Sign in to [Cloud](https://app.subsquid.io/), and obtain (or refresh) the deployment key page by clicking at the profile picture > "Deployment key":

    <Frame>
      <img alt="SQD Cloud deployment key page" />
    </Frame>
  </Step>

  <Step title="2. Authenticate Squid CLI">
    Open a terminal window and run

    ```bash theme={"system"}
    sqd auth -k <DEPLOYMENT_KEY>
    ```
  </Step>

  <Step title="3. Explore with `--help`">
    Use `sqd --help` to get a list of the available command and `sqd <command> --help` to get help on the available options for a specific command, e.g.

    ```bash theme={"system"}
    sqd deploy --help
    ```
  </Step>
</Steps>


# sqd list
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/list

List squids deployed to SQD Cloud with the sqd ls command — running, stopped, and archived deployments per organization with their slots and tags.

List squids deployed to the Cloud

* [`sqd list`](#sqd-list-1)

## `sqd list`

List squids deployed to the Cloud

```
USAGE
  $ sqd list [--interactive] [--truncate]
    [-r [<org>/]<name>(@<slot>|:<tag>) | -o <code> | -n <name> | [-s <slot>] | [-t <tag>]]

FLAGS
  --[no-]interactive  Disable interactive mode
  --[no-]truncate     Truncate data in columns: false by default

SQUID FLAGS
  -n, --name=<name>                               Name of the squid
  -r, --reference=[<org>/]<name>(@<slot>|:<tag>)  Fully qualified reference
                                                  of the squid. It can include
                                                  the organization, name, slot,
                                                  or tag
  -s, --slot=<slot>                               Slot of the squid
  -t, --tag=<tag>                                 Tag of the squid

ORG FLAGS
  -o, --org=<code>  Code of the organization

ALIASES
  $ sqd ls
```

*See code: [src/commands/ls.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/ls.ts)*


# sqd logs
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/logs

Fetch live and historical logs from a squid deployed to SQD Cloud with the sqd logs command — filter by container, time range, slot, and tag.

Fetch logs from a squid deployed to the Cloud

* [`sqd logs`](#sqd-logs-1)

## `sqd logs`

Fetch logs from a squid deployed to the Cloud

```
USAGE
  $ sqd logs [--interactive] [--since <value>] [--search <value>] [-f | -p <value>]
    [-r [<org>/]<name>(@<slot>|:<tag>) | -o <code> | [-s <slot> -n <name>] | [-t <tag> ]]
    [-c processor|query-node|api|db-migrate|db...]
    [-l error|debug|info|warning...] [--since <value>]

FLAGS
  -c, --container=<option>...  Container name
                               <options: processor|query-node|api|db-migrate|db>
  -f, --follow                 Follow
  -l, --level=<option>...      Log level
                               <options: error|debug|info|warning>
  -p, --pageSize=<value>       [default: 100] Logs page size
      --[no-]interactive       Disable interactive mode
      --search=<value>         Filter by content
      --since=<value>          [default: 1d] Filter by date/interval

SQUID FLAGS
  -n, --name=<name>                               Name of the squid
  -r, --reference=[<org>/]<name>(@<slot>|:<tag>)  Fully qualified reference of the squid.
                                                  It can include the organization, name,
                                                  slot, or tag
  -s, --slot=<slot>                               Slot of the squid
  -t, --tag=<tag>                                 Tag of the squid

ORG FLAGS
  -o, --org=<code>  Code of the organization
```

Notes:

* `--since` accepts the notation of the [`ms` library](https://www.npmjs.com/package/ms): 1d, 10h, 1m.
* With `--level=error` `sqd` will fetch logs emitted by three different calls to the [SDK Logger](/en/sdk/squid-sdk/reference/logger), namely
  * `ctx.log.error`
  * `ctx.log.fatal`
  * `ctx.log.trace`

*See code: [src/commands/logs.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/logs.ts)*


# sqd remove
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/remove

Remove a squid deployment from SQD Cloud with the sqd rm command — delete a specific slot or tag and free up resources from your organization quota.

Remove a squid deployed to the Cloud

* [`sqd remove`](#sqd-remove-1)

## `sqd remove`

Remove a squid deployed to the Cloud

```
USAGE
  $ sqd remove [--interactive] [-f]
    [-r [<org>/]<name>(@<slot>|:<tag>) | -o <code> | [-s <slot> -n <name>] | [-t <tag> ]]

FLAGS
  -f, --force             Does not prompt before removing a squid or its version
      --[no-]interactive  Disable interactive mode

SQUID FLAGS
  -n, --name=<name>                               Name of the squid
  -r, --reference=[<org>/]<name>(@<slot>|:<tag>)  Fully qualified reference of the squid. It can include the organization, name, slot, or tag
  -s, --slot=<slot>                               Slot of the squid
  -t, --tag=<tag>                                 Tag of the squid

ORG FLAGS
  -o, --org=<code>  Code of the organization

ALIASES
  $ sqd rm
```

*See code: [src/commands/rm.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/rm.ts)*


# sqd restart
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/restart

Restart a squid deployment on SQD Cloud with the sqd restart command — recycle the processor container without redeploying source code or migrations.

Restart a squid deployed to the Cloud

* [`sqd restart`](#sqd-restart-1)

## `sqd restart`

Restart a squid deployed to the Cloud

```
USAGE
  $ sqd restart [--interactive]
    [-r [<org>/]<name>(@<slot>|:<tag>) | -o <code> | [-s <slot> -n <name>] | [-t <tag> ]]

FLAGS
  --[no-]interactive  Disable interactive mode

SQUID FLAGS
  -n, --name=<name>                               Name of the squid
  -r, --reference=[<org>/]<name>(@<slot>|:<tag>)  Fully qualified reference of the squid.
                                                  It can include the organization, name,
                                                  slot, or tag
  -s, --slot=<slot>                               Slot of the squid
  -t, --tag=<tag>                                 Tag of the squid

ORG FLAGS
  -o, --org=<code>  Code of the organization
```

*See code: [src/commands/restart.ts](https://github.com/subsquid/squid-cli/blob/master/src/commands/restart.ts)*


# sqd run
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/run

Run a squid locally with the sqd run command — load environment variables and execute the processor and GraphQL server using the deployment manifest.

Run a squid locally according to the [deployment manifest](/en/cloud/reference/manifest).

* [sqd run PATH](#sqd-run-path)

Notes:

* The command is especially useful for running [multichain squids](/en/sdk/squid-sdk/resources/multichain), as it runs all services in the same terminal and handles failures gracefully.

## `sqd run PATH`

Run a squid project locally

```
USAGE
  $ sqd run PATH [--interactive] [-m <value>] [-f <value>] [-i <value>... | -e <value>...] [-r <value>]

FLAGS
  -e, --exclude=<value>...  Do not run specified services
  -f, --envFile=<value>     [default: .env] Relative path to an additional environment file
  -i, --include=<value>...  Run only specified services
  -m, --manifest=<value>    [default: squid.yaml] Relative path to a squid manifest file
  -r, --retries=<value>     [default: 5] Attempts to restart failed or stopped services
      --[no-]interactive    Disable interactive mode
```

*See code: [src/commands/run.ts](https://github.com/subsquid/squid-cli/blob/master/src/commands/run.ts)*


# sqd secrets
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/secrets

Manage SQD Cloud organization secrets with the sqd secrets CLI — set, list, and delete API keys, RPC endpoints, and database credentials.

Manage account secrets

The secrets are [exposed as a context](/en/cloud/resources/env-variables#secrets), and are accessible to all the squids deployed by the current SQD Cloud [organization](/en/cloud/resources/organizations).

* [`sqd secrets list`](#sqd-secrets-list)
* [`sqd secrets remove NAME`](#sqd-secrets-remove-name)
* [`sqd secrets set NAME VALUE`](#sqd-secrets-set-name-value)

## `sqd secrets list`

List organization secrets in the Cloud

```
USAGE
  $ sqd secrets list [--interactive] [-o <value>]

FLAGS
  -o, --org=<value>       Organization
      --[no-]interactive  Disable interactive mode

ALIASES
  $ sqd secrets ls
```

*See code: [src/commands/secrets/ls.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/secrets/ls.ts)*

## `sqd secrets remove NAME`

Delete an organization secret in the Cloud

```
USAGE
  $ sqd secrets remove NAME [--interactive] [-o <value>]

ARGUMENTS
  NAME  The secret name

FLAGS
  -o, --org=<value>       Organization
      --[no-]interactive  Disable interactive mode

ALIASES
  $ sqd secrets rm
```

*See code: [src/commands/secrets/rm.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/secrets/rm.ts)*

## `sqd secrets set NAME VALUE`

Add or update an organization secret in the Cloud. If value is not specified, it is read from standard input. The secret will be exposed as an environment variable with the given name to all the squids in the organization. NOTE: The changes take affect only after a squid is restarted or updated.

```
USAGE
  $ sqd secrets set NAME VALUE [--interactive] [-o <value>]

ARGUMENTS
  NAME   The secret name
  VALUE  The secret value

FLAGS
  -o, --org=<value>       Organization
      --[no-]interactive  Disable interactive mode
```

*See code: [src/commands/secrets/set.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/secrets/set.ts)*


# sqd tags
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/tags

Manage version tags for squids deployed to SQD Cloud with the sqd tags CLI — promote, alias, and roll back releases following the slots-and-tags model.

Manage squid deployements' tags. See the [slots and tags guide](/en/cloud/resources/slots-and-tags).

* [`sqd tags add`](#sqd-tags-add)
* [`sqd tags remove`](#sqd-tags-remove)

## `sqd tags add`

Add a tag to a squid deployment / slot

```
USAGE
  $ sqd tags add TAG [--interactive] [--allow-tag-reassign]
    [-r [<org>/]<name>(@<slot>|:<tag>) | -o <code> | [-s <slot> -n <name>] | [-t <tag> ]]

ARGUMENTS
  TAG  New tag to assign

FLAGS
  --allow-tag-reassign  Allow reassigning an existing tag
  --[no-]interactive    Disable interactive mode

SQUID FLAGS
  -n, --name=<name>                               Name of the squid
  -r, --reference=[<org>/]<name>(@<slot>|:<tag>)  Fully qualified reference of the squid.
                                                  It can include the organization, name,
                                                  slot, or tag
  -s, --slot=<slot>                               Slot of the squid
  -t, --tag=<tag>                                 Tag of the squid

ORG FLAGS
  -o, --org=<code>  Code of the organization
```

*See code: [src/commands/tags/add.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/tags/add.ts)*

## `sqd tags remove`

Remove a tag from a squid deployment / slot

```
USAGE
  $ sqd tags remove TAG [--interactive]
    [-r [<org>/]<name>(@<slot>|:<tag>) | -o <code> | [-s <slot> -n <name>] | [-t <tag> ]]

ARGUMENTS
  TAG  New tag to assign

FLAGS
  --[no-]interactive  Disable interactive mode

SQUID FLAGS
  -n, --name=<name>                               Name of the squid
  -r, --reference=[<org>/]<name>(@<slot>|:<tag>)  Fully qualified reference of the squid.
                                                  It can include the organization, name,
                                                  slot, or tag
  -s, --slot=<slot>                               Slot of the squid
  -t, --tag=<tag>                                 Tag of the squid

ORG FLAGS
  -o, --org=<code>  Code of the organization
```

*See code: [src/commands/tags/remove.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/tags/remove.ts)*


# sqd whoami
Source: https://docs.sqd.dev/en/sdk/squid-sdk/squid-cli/whoami

Show the SQD Cloud user account currently authenticated with the sqd CLI — display username, organization, and roles for the active session.

Show the user details for the current Cloud account

* [`sqd whoami`](#sqd-whoami-1)

## `sqd whoami`

Show the user details for the current Cloud account

```
USAGE
  $ sqd whoami [--interactive]

FLAGS
  --[no-]interactive  Disable interactive mode
```

*See code: [src/commands/whoami.ts](https://github.com/subsquid/squid-cli/tree/master/src/commands/whoami.ts)*


# Starknet API
Source: https://docs.sqd.dev/en/sdk/squid-sdk/starknet-indexing/network-api/starknet-api

Access Starknet blockchain data via the Squid SDK Starknet network API — endpoint URLs, schema fields, and access notes for indexing Starknet.

# Starknet SQD Network API

<Warning>
  The Starknet API of SQD Network is currently in beta. Breaking changes may be introduced in the future releases.
</Warning>

SQD Network API distributes the requests over a ([potentially decentralized](/en/other/faq)) network of *workers*. The main gateway URL points at a *router* that provides URLs of workers that do the heavy lifting. Each worker has its own range of blocks on each dataset it serves.

Suppose you want to retrieve an output of some [query](#worker-api) on a block range starting at `firstBlock` (can be the genesis block) and ending at the highest available block. Proceed as follows:

1. Retrieve the dataset height from the router with `GET /height` and make sure it's above `firstBlock`.

2. Save the value of `firstBlock` to some variable, say `currentBlock`.

3. Query the router for an URL of a worker that has the data for `currentBlock` with `GET /${currentBlock}/worker`.

4. Retrieve the data from the worker by [posting the query](#worker-api) (`POST /`), setting the `"fromBlock"` query field to `${currentBlock}`.

5. Parse the retrieved data to get a batch of query data **plus** the height of the last block available from the current worker. Take the `header.number` field of the last element of the retrieved JSON array - it is the height you want. Even if your query returns no data, you'll still get the block data for the last block in the range, so this procedure is safe.

6. Set `currentBlock` to the height from the previous step **plus one**.

7. Repeat steps 3-6 until all the required data is retrieved.

The main URL of the Starknet gateway is

```
https://v2.archive.subsquid.io/network/starknet-mainnet
```

<Warning>
  Unlike in [the explorer](https://starkscan.co), addresses in this API do not have leading zeros (both in valid requests and in the returned data). For example, explorer's `0x00ce6c...0552` becomes `0xce6c...0552`. This is the format that Starknet RPC nodes use.
</Warning>

Implementation examples:

<AccordionGroup>
  <Accordion title="Manually with cURL">
    Suppose we want data on all txs sent by `Layerswap`/`0x19252b1deef483477c4d30cfcc3e5ed9c82fafea44669c182a45a01b4fdb97a` starting from block 600\_000. The steps are:

    1. Verify that the dataset has reached the required height:
       ```bash theme={"system"}
       curl https://v2.archive.subsquid.io/network/starknet-mainnet/height
       ```
       Output
       ```
       632494
       ```

    2. Remember that your current height is 600000.

    3. Get a worker URL for the current height:
       ```bash theme={"system"}
       curl https://v2.archive.subsquid.io/network/starknet-mainnet/600000/worker
       ```
       Output
       ```
       https://rb03.sqd-archive.net/worker/query/czM6Ly9zdGFya25ldC1tYWlubmV0
       ```

    4. Retrieve the data from the current worker

       ```bash theme={"system"}
       curl https://rb03.sqd-archive.net/worker/query/czM6Ly9zdGFya25ldC1tYWlubmV0 \
       -X 'POST' -H 'content-type: application/json' -H 'accept: application/json' \
       -d '{
           "type": "starknet",
           "fromBlock":600000,
           "toBlock":632494,
           "fields":{"transaction":{"transactionHash":true}},
           "transactions":[{"senderAddress":["0x19252b1deef483477c4d30cfcc3e5ed9c82fafea44669c182a45a01b4fdb97a"]}]
       }' | jq
       ```

       Output:

       ```json theme={"system"}
       [
         {
           "header": {
             "number": 600000,
             "hash": "0x898fe7f61f5d662199d223de496988f221d150ed054f2fe5e681b2988b9e2c"
           },
           "transactions": []
         },
         {
           "header": {
             "number": 600007,
             "hash": "0x44aa251cee1baaf3f19accefd223ce5208815686c881bf645ffb3e3348a5ddc"
           },
           "transactions": [
             {
               "transactionIndex": 24,
               "transactionHash": "0x6a88edb0713769de4ad4d450df70911c3e9a7e8253c135c9574d0b3542ced18"
             }
           ]
         },
         ...
         {
           "header": {
             "number": 617979,
             "hash": "0x7f6a8516a91eefa6a65972c47002cbe3851e3c4287f670c914850960d29ca29"
           },
           "transactions": []
         }
       ]
       ```

    5. Parse the retrieved data:
       * Grab the network data you requested from the list items with non-empty data fields (`transactions`, `events`). For the example above, this data will include the txn `0x6a88...`.
       * Observe that we received the data up to and including block 617979.

    6. To get the rest of the data, update the current height to 617980 and go to step 3.
       * Note how the worker URL you're getting while repeating step 3 points to a different host than before. This is how data storage and reads are distributed across the SQD Network.

    7. Repeat steps 3 through 6 until the dataset height of 632494 is reached.
  </Accordion>

  <Accordion title="In Python">
    ```python theme={"system"}
    def get_text(url: str) -> str:
        res = requests.get(url)
        res.raise_for_status()
        return res.text

    def dump(
        gateway_url: str,
        query: Query,
        first_block: int,
        last_block: int
    ) -> None:
        assert 0 <= first_block <= last_block
        query = dict(query)  # copy query to mess with it later

        dataset_height = int(get_text(f'{gateway_url}/height'))
        next_block = first_block
        last_block = min(last_block, dataset_height)

        while next_block <= last_block:
            worker_url = get_text(f'{gateway_url}/{next_block}/worker')

            query['fromBlock'] = next_block
            query['toBlock'] = last_block
            res = requests.post(worker_url, json=query)
            res.raise_for_status()
            blocks = res.json()

            last_processed_block = blocks[-1]['header']['number']
            next_block = last_processed_block + 1
            for block in blocks:
                print(json.dumps(block))
    ```

    Full code [here](https://gist.github.com/eldargab/2e007a293ac9f82031d023f1af581a7d).
  </Accordion>
</AccordionGroup>

## Router API

<AccordionGroup>
  <Accordion title="GET /height (get height of the dataset)">
    **Example response:** `632494`.
  </Accordion>

  <Accordion title="GET ${firstBlock}/worker (get a suitable worker URL)">
    The returned worker is capable of processing `POST /` requests in which the `"fromBlock"` field is equal to `${firstBlock}`.

    **Example response:** `https://rb06.sqd-archive.net/worker/query/czM6Ly9zdGFya25ldC1tYWlubmV0`.
  </Accordion>
</AccordionGroup>

## Worker API

<Accordion title="POST / (query transactions and events)">
  ##### Query Fields

  * **type**: `"starknet"`.
  * **fromBlock**: Block number to start from (inclusive).
  * **toBlock**: (optional) Block number to end on (inclusive). If this is not given, the query will go on for a fixed amount of time or until it reaches the height of the dataset.
  * **includeAllBlocks**: (optional) If true, the Network will include blocks that contain no data selected by data requests into its response.
  * **fields**: (optional) A [selector](#data-fields-selector) of data fields to retrieve. Common for all data items.
  * **transactions**: (optional) A list of [transaction requests](#transactions). An empty list requests no data.
  * **events**: (optional) A list of [event requests](#events). An empty list requests no data.

  The response is a JSON array of per-block data items that covers a block range starting from `fromBlock`. The last block of the range is determined by the worker. You can find it by looking at the `header.number` field of the last element in the response array.

  The first and the last block in the range are returned even if all data requests return no data for the range.

  In most cases the returned range will not contain all the range requested by the user (i.e. the last block of the range will not be `toBlock`). To continue, [retrieve a new worker URL](#router-api) for blocks starting at the end of the current range *plus one block* and repeat the query with an updated value of `fromBlock`.

  <Accordion title="Example Request">
    ```json theme={"system"}
    {
      "type": "starknet",
      "fromBlock":632000,
      "toBlock":632494,
      "fields": {
        "block": {
          "timestamp": true
        },
        "event": {
          "keys": true,
          "data": true
        },
        "transaction": {
          "transactionHash":true
        }
      },
      "events": [
        {
          "fromAddress": [
            "0x19252b1deef483477c4d30cfcc3e5ed9c82fafea44669c182a45a01b4fdb97a"
          ],
          "transaction": true
        }
      ]
    }
    ```
  </Accordion>

  <Accordion title="Example Response">
    ```json theme={"system"}
    [
      {
        "header": {
          "number": 632000,
          "hash": "0xdfebe2b6af20dfe7f27d5fe8b1b4e8ee48ad812ce9bfd9c756c9db7dbcdb22",
          "timestamp": 1712950160
        },
        "transactions": [
          {
            "transactionIndex": 110,
            "transactionHash": "0x151fa3c8633e6ed71301af4afc8f73a141ef39cca1d51d0f72d66a11911e2f3"
          },
          {
            "transactionIndex": 306,
            "transactionHash": "0x7e1c307624c5c78e311e2a08f0355dfef80e6fc6ed47c64ceda757e044f2c85"
          }
        ],
        "events": [
          {
            "transactionIndex": 110,
            "eventIndex": 1,
            "keys": [
              "0x1dcde06aabdbca2f80aa51392b345d7549d7757aa855f7e37f5d335ac8243b1",
              "0x151fa3c8633e6ed71301af4afc8f73a141ef39cca1d51d0f72d66a11911e2f3"
            ],
            "data": [
              "0x1",
              "0x1",
              "0x1"
            ]
          },
          {
            "transactionIndex": 306,
            "eventIndex": 1,
            "keys": [
              "0x1dcde06aabdbca2f80aa51392b345d7549d7757aa855f7e37f5d335ac8243b1",
              "0x7e1c307624c5c78e311e2a08f0355dfef80e6fc6ed47c64ceda757e044f2c85"
            ],
            "data": [
              "0x1",
              "0x1",
              "0x1"
            ]
          }
        ]
      },
      ...
      {
        "header": {
          "number": 632492,
          "hash": "0x440ee029f2a970b2546eb39ab23075659ec8e0246c94f62d21e21f912dfb58d",
          "timestamp": 1713049189
        },
        "transactions": [
          {
            "transactionIndex": 125,
            "transactionHash": "0x61bd3d233cfe9cb387f3b016127ffb0d66d265c2593f80a317404e2f3c334bb"
          }
        ],
        "events": [
          {
            "transactionIndex": 125,
            "eventIndex": 1,
            "keys": [
              "0x1dcde06aabdbca2f80aa51392b345d7549d7757aa855f7e37f5d335ac8243b1",
              "0x61bd3d233cfe9cb387f3b016127ffb0d66d265c2593f80a317404e2f3c334bb"
            ],
            "data": [
              "0x1",
              "0x1",
              "0x1"
            ]
          }
        ]
      },
      {
        "header": {
          "number": 632494,
          "hash": "0x2782c5ca3f1d3eb2e4c085fc17908b9b86bfe91807cd452374bcb40b2245925",
          "timestamp": 1713049569
        },
        "transactions": [],
        "events": []
      }
    ]
    ```
  </Accordion>
</Accordion>

## Data requests

### Transactions

```ts theme={"system"}
{
  contractAddress: string[],
  senderAddress: string[],
  type: string[],
  firstNonce: int,
  lastNonce: int,
  events: boolean
}
```

A transaction will be included in the response if it matches all the requests. An empty array matches no transactions; omit all requests to match all transactions.

All events emitted by the selected transactions will be included into the response if `events` is set to `true`.

See [Data fields selector](#data-fields-selector) for info on field selection.

### Events

```ts theme={"system"}
{
  fromAddress: string[],
  key0: string[],
  key1: string[],
  key2: string[],
  key3: string[],
  transaction: boolean
}
```

An event will be included in the response if it matches all the requests. An empty array matches no events; omit all requests to match all events.

If `transaction` is set to `true`, all parent transactions will be included into the response.

See [Data fields selector](#data-fields-selector) for info on field selection.

## Data fields selector

A selector of fields for the returned data items. Its structure is as follows:

```
{
  block:               // field selector for blocks
  transaction:         // field selector for transactions
  event:               // field selector for events
}
```

### Block fields

```
{
  parentHash
  status
  newRoot
  timestamp
  sequencerAddress
}
```

A valid field selector for blocks is a JSON that has a subset of these fields as keys and `true` as values, e.g. `{"status": true, "timestamp": true}`.

### Transaction fields

```
{
  transactionHash
  contractAddress
  entryPointSelector
  calldata
  maxFee
  type
  senderAddress
  version
  signature
  nonce
  classHash
  compiledClassHash
  contractAddressSalt
  constructorCalldata
}
```

A valid field selector for transactions is a JSON that has a subset of these fields as keys and `true` as values, e.g. `{"transactionHash": true, "type": true, "calldata": true}`.

### Event fields

```ts theme={"system"}
{
  fromAddress
  keys
  data
}
```

A valid field selector for logs is a JSON that has a subset of these fields as keys and `true` as values, e.g. `{"fromAddress": true, "data": true}`.


# Tron Indexing CLI Cheatsheet
Source: https://docs.sqd.dev/en/sdk/squid-sdk/tron-indexing/cli-cheatsheet

Commonly used sqd CLI commands for Tron indexers — init, deploy, run, secrets, and gateways with examples for the Squid SDK Tron data source.

# CLI cheatsheet

This guide provides a quick reference to the commands needed to launch the squid and manage the database.

### Install dependencies

```sh theme={"system"}
npm i
```

### Compile the project

```sh theme={"system"}
npx tsc
```

### Launch Postgres database to store the data

```sh theme={"system"}
docker compose up -d
```

### Apply database migrations to create the target schema

```sh theme={"system"}
npx squid-typeorm-migration apply
```

### Run indexer

```sh theme={"system"}
node -r dotenv/config lib/main.js
```

### Check out the indexed swaps

```sh theme={"system"}
docker exec "$(basename "$(pwd)")-db-1" psql -U postgres \
  -c "SELECT id, logs_count, found_at FROM contract ORDER BY logs_count desc LIMIT 10"
```

You can use the `sqd` utility to shorten these commands and manage their interrelations. [Learn more here](/en/sdk/squid-sdk/squid-cli/commands-json).


# Indexing USDT on Tron
Source: https://docs.sqd.dev/en/sdk/squid-sdk/tron-indexing/indexing-usdt-transfers

Tutorial: index TRC-20 USDT transfers on Tron with Squid SDK — set up the processor, decode events, persist to Postgres, and serve via GraphQL.

<Info>
  Tron support in SQD is now in public beta. Please report any bugs or suggestions to [Squid Devs](https://t.me/HydraDevs).
</Info>

In this tutorial we will look into a squid that indexes USDT transfers on the [Tron Network](https://tron.network).

Pre-requisites: Node.js v20 or newer, Git, Docker.

## Download the project

Begin by retrieving the template and installing the dependencies:

```bash theme={"system"}
git clone https://github.com/subsquid-labs/tron-example
cd tron-example
npm i
```

## TronBatchProcessor configuration

[Squid processor](/en/sdk/squid-sdk/reference/processors/architecture) is a term with two meanings:

1. The process responsible for retrieving and transforming the data.

2. The main object implementing that process.

On Tron, [`TronBatchProcessor`](/en/sdk/squid-sdk/tron-indexing/tron-batch-processor) is the class that should be used for processor objects. Our first step is to create the processor and configure it to fetch USDT transfers data:

```typescript title="src/main.ts" theme={"system"}
// Note that the address is in lowercase hex and has no leading `0x`.
// This is the format used throughout the SQD Tron stack.
const USDT_ADDRESS = 'a614f803b6fd780986a42c78ec9c7f77e6ded13c'
const TRANSFER_TOPIC = 'ddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef'

const processor = new TronBatchProcessor()
  // The SQD Network gateway URL - this is where the bulk of data will
  // be coming from.
  .setGateway('https://v2.archive.subsquid.io/network/tron-mainnet')
  // SQD Network is always about N blocks behind the head.
  // We must use regular HTTP API endpoint to get through the last mile
  // and stay on top of the chain.
  // This is a limitation, and we promise to lift it in the future!
  .setHttpApi({
    // Ankr public endpoint is heavily rate-limited so expect many 429 errors
    url: 'https://rpc.ankr.com/http/tron',
    strideConcurrency: 1,
    strideSize: 1,
  })
  // We request data items via `.addXxx()` methods.
  // Each `.addXxx()` method accepts item selection criteria
  // and also allows to request related items.
  .addLog({
    // select logs
    where: {
      address: [USDT_ADDRESS],
      topic0: [TRANSFER_TOPIC]
    },
    // for each log selected above load related transactions
    include: {
      transaction: true
    }
  })
  // For each data item we can specify a set of fields we want to fetch.
  // Accurate selection of only the required fields can have a notable
  // positive impact on performance when data is sourced from SQD Network.
  .setFields({
    block: {
      timestamp: true,
    },
    transaction: {
      hash: true
    },
    log: {
      address: true,
      data: true,
      topics: true
    }
  })
```

See also [`TronBatchProcessor` reference](/en/sdk/squid-sdk/tron-indexing/tron-batch-processor).

The next step is to configure data decoding and transformation.

## Processing the data

The other part of the squid processor configuration is the callback function used to process batches of filtered data, the [batch handler](/en/sdk/squid-sdk/reference/processors/architecture#processorrun). It is typically defined within a `processor.run()` call, like this:

```typescript theme={"system"}
processor.run(database, async ctx => {
  ...
}
```

Here,

* `processor` is the object described in the previous section
* `database` is a [`Database` implementation](/en/sdk/squid-sdk/resources/persisting-data/overview) specific to the target data sink. We want to store the data in a PostgreSQL database and present with a GraphQL API, so we provide a [`TypeormDatabase`](/en/sdk/squid-sdk/resources/persisting-data/typeorm) object here.
* `ctx` is the [batch context](/en/sdk/squid-sdk/reference/processors/architecture#batch-context) object that exposes a batch of data (at `ctx.blocks`) and any data persistence facilities derived from `db` (at `ctx.store`). See [Block data for Tron](/en/sdk/squid-sdk/tron-indexing/tron-batch-processor/context-interfaces) for details on how the data batches are presented.

Batch handler is where the raw on-chain data is decoded, transformed and persisted. Here is our definition:

```typescript title="src/main.ts" theme={"system"}
processor.run(new TypeormDatabase(), async ctx => {
  let transfers: Transfer[] = []

  for (let block of ctx.blocks) {
    for (let log of block.logs) {
      if (log.address == USDT_ADDRESS &&
          log.topics?.[0] === TRANSFER_TOPIC) {
        assert(log.data, 'USDT transfers always carry data')
        let tx = log.getTransaction()
        // `0x` prefixes make log data compatible with EVM codec
        let event = {
          topics: log.topics.map(t => '0x' + t),
          data: '0x' + log.data
        }
        let {from, to, value} = erc20.events.Transfer.decode(event)

        transfers.push(new Transfer({
          id: log.id,
          blockNumber: block.header.height,
          timestamp: new Date(block.header.timestamp),
          tx: tx.hash,
          from,
          to,
          amount: value
        }))
      }
    }
  }

  await ctx.store.insert(transfers)
})
```

This goes through all the logs in the block, verifies that they originate from `USDT_ADDRESS` and have a `TRANSFER_TOPIC` as the first topic, decodes the log and saves a `Transfer` record with the obtained data to the database. See [Prepare the store](/en/sdk/squid-sdk/how-to-start/squid-development#store) for more details on how the database interface works.

<Info>
  Notice that we used the SQD EVM codec and [`squid-evm-typegen`](/en/sdk/squid-sdk/resources/tools/typegen/generation)-generated module to decode events. Since Tron events are fully EVM-compatible, this is the recommended way.

  This tooling requires that the hex values are prefixed with `0x`, as shown above.
</Info>

At this point the squid is ready for its first test run. Execute

```bash theme={"system"}
npx tsc
docker compose up -d
npx squid-typeorm-migration apply
node -r dotenv/config lib/main.js
```

You can verify that the data is being stored in the database by running

```bash theme={"system"}
docker exec "$(basename "$(pwd)")-db-1" psql -U postgres -c "SELECT * FROM transfer LIMIT 100"
```

Full code can be found [here](https://github.com/subsquid-labs/tron-example).


# Tron API
Source: https://docs.sqd.dev/en/sdk/squid-sdk/tron-indexing/network-api/tron-api

Access Tron blockchain data via the Squid SDK Tron network API — endpoint URLs, schema fields, and access notes for indexing the Tron network.

# Tron SQD Network API

<Warning>
  The Tron API of SQD Network is currently in beta. Breaking changes may be introduced in the future releases.
</Warning>

[Open private SQD Network](/en/network/overview#open-private-network) offers access to Tron data. The gateway is at

```
https://v2.archive.subsquid.io/network/tron-mainnet
```

SQD Network API distributes the requests over a ([potentially decentralized](/en/other/faq)) network of *workers*. The main gateway URL points at a *router* that provides URLs of workers that do the heavy lifting. Each worker has its own range of blocks.

Suppose you want to retrieve an output of some [query](#worker-api) on a block range starting at `firstBlock` (can be the genesis block) and ending at the highest available block. Proceed as follows:

1. Retrieve the dataset height from the router with `GET /height` and make sure it's above `firstBlock`.

2. Save the value of `firstBlock` to some variable, say `currentBlock`.

3. Query the router for an URL of a worker that has the data for `currentBlock` with `GET /${currentBlock}/worker`.

4. Retrieve the data from the worker by [posting the query](#worker-api) (`POST /`), setting the `"fromBlock"` query field to `${currentBlock}`.

5. Parse the retrieved data to get a batch of query data **plus** the height of the last block available from the current worker. Take the `header.number` field of the last element of the retrieved JSON array - it is the height you want. Even if your query returns no data, you'll still get the block data for the last block in the range, so this procedure is safe.

6. Set `currentBlock` to the height from the previous step **plus one**.

7. Repeat steps 3-6 until all the required data is retrieved.

Implementation examples:

<AccordionGroup>
  <Accordion title="Manually with cURL">
    Suppose we want data on [token burns](https://developers.tron.network/docs/faq#3-what-is-the-destruction-address-of-tron) from block 58\_000\_000. We have to:

    1. Verify that the dataset has reached the required height:
       ```bash theme={"system"}
       curl https://v2.archive.subsquid.io/network/tron-mainnet/height
       ```
       Output
       ```
       61889299
       ```

    2. Remember that your current height is 58000000.

    3. Get a worker URL for the current height
       ```bash theme={"system"}
       curl https://v2.archive.subsquid.io/network/tron-mainnet/58000000/worker
       ```
       Output
       ```
       https://rb03.sqd-archive.net/worker/query/czM6Ly90cm9uLW1haW5uZXQ
       ```

    4. Retrieve the data from the worker
       ```bash theme={"system"}
       curl https://rb03.sqd-archive.net/worker/query/czM6Ly90cm9uLW1haW5uZXQ \
       -X 'POST' -H 'content-type: application/json' -H 'accept: application/json' \
       -d '{
           "fromBlock":58000000,
           "toBlock":59286799,
           "fields":{"transaction":{"hash":true}},
           "transactions":[{
             "to":[
               "0x0000000000000000000000000000000000000000"
             ]
           }]
       }' | jq
       ```
       Output:
       ```json theme={"system"}
       [
         {
           "header": {
             "number": 16000000,
             "hash": "0x3dc4ef568ae2635db1419c5fec55c4a9322c05302ae527cd40bff380c1d465dd",
             "parentHash": "0x6f377dc6bd1f3e38b9ceb8c946a88c13211fa3f084622df3ee5cfcd98cc6bb16"
           },
           "transactions": []
         },
         // ...
         {
           "header": {
             "number": 16027977,
             "hash": "0x4b332878deb33e963b68c8bbbea60cbca72a88c297b6800eafa82baab497c166",
             "parentHash": "0x2b979d67d9b03394da336938ee0bcf5aedfdf87e1b5bd574d985aee749eb8b76"
           },
           "transactions": [
             {
               "transactionIndex": 96,
               "hash": "0xbaede248ec6fce28e9d874f69ea70359bea0107ce9144d6838898674d9d10c8c"
             }
           ]
         },
         // ...
         {
           "header": {
             "number": 16031419,
             "hash": "0x9cc48c9b4ad8dddb1de86a15e30a62ffd48cf9b72930930cfa5167c4e1685d0a",
             "parentHash": "0x4ec7b4562739032f51e70d26fe5129e571e2bf0348a744c1509f8205f4381696"
           },
           "transactions": []
         }
       ]
       ```

    5. Observe that we received the transactions up to and including block 16031419. To get the rest of the data, we find a worker who has blocks from 16031420 on:
       ```bash theme={"system"}
       curl https://v2.archive.subsquid.io/network/ethereum-mainnet/16031420/worker
       ```
       Output:
       ```
       https://rb02.sqd-archive.net/worker/query/czM6Ly9ldGhlcmV1bS1tYWlubmV0
       ```
       We can see that this part of the dataset is located on another host.

    6. Retrieve the data from the new worker
       ```bash theme={"system"}
       curl https://rb02.sqd-archive.net/worker/query/czM6Ly9ldGhlcmV1bS1tYWlubmV0 \
       -X 'POST' -H 'content-type: application/json' -H 'accept: application/json' \
       -d '{
           "fromBlock":16031420,
           "toBlock":18593441,
           "fields":{"transaction":{"hash":true}},
           "transactions":[{"to":["0xd8da6bf26964af9d7eed9e03e53415d37aa96045"]}]
       }' | jq
       ```
       Output is similar to that of step 3.

    7. Repeat steps 4 and 5 until the dataset height of 18593441 reached.
  </Accordion>

  <Accordion title="In Python">
    ```python theme={"system"}
    def get_text(url: str) -> str:
        res = requests.get(url)
        res.raise_for_status()
        return res.text

    def dump(
        dataset_url: str,
        query: Query,
        first_block: int,
        last_block: int
    ) -> None:
        assert 0 <= first_block <= last_block
        query = dict(query)  # copy query to mess with it later

        dataset_height = int(get_text(f'{dataset_url}/height'))
        next_block = first_block
        last_block = min(last_block, dataset_height)

        while next_block <= last_block:
            worker_url = get_text(f'{dataset_url}/{next_block}/worker')

            query['fromBlock'] = next_block
            query['toBlock'] = last_block
            res = requests.post(worker_url, json=query)
            res.raise_for_status()
            blocks = res.json()

            last_processed_block = blocks[-1]['header']['number']
            next_block = last_processed_block + 1
            for block in blocks:
                print(json.dumps(block))
    ```

    Full code [here](https://gist.github.com/eldargab/2e007a293ac9f82031d023f1af581a7d).
  </Accordion>
</AccordionGroup>

## Router API

<AccordionGroup>
  <Accordion title="GET /height (get height of the dataset)">
    **Example response:** `65900614`.
  </Accordion>

  <Accordion title="GET ${firstBlock}/worker (get a suitable worker URL)">
    The returned worker will be capable of processing `POST /` requests in which the `"fromBlock"` field is equal to `${firstBlock}`.

    **Example response:** `https://rb06.sqd-archive.net/worker/query/czM6Ly90cm9uLW1haW5uZXQ`.
  </Accordion>
</AccordionGroup>

## Worker API

<Accordion title="POST / (query logs and transactions)">
  ##### Query Fields

  * **fromBlock**: Block number to start from (inclusive).
  * **toBlock**: (optional) Block number to end on (inclusive). If this is not given, the query will go on for a fixed amount of time or until it reaches the height of the dataset.
  * **includeAllBlocks**: (optional) If true, the Network will include blocks that contain no data selected by data requests into its response.
  * **fields**: (optional) A [selector](#data-fields-selector) of data fields to retrieve. Common for all data items.
  * **logs**: (optional) A list of [log requests](#logs). An empty list requests no data.
  * **transactions**: (optional) A list of [general transaction requests](#general-transactions). An empty list requests no data.
  * **transferTransactions**: (optional) A list of ["transfer" transaction requests](#transfer-transactions). An empty list requests no data.
  * **transferAssetTransactions**: (optional) A list of ["transfer asset" transaction requests](#transfer-asset-transactions). An empty list requests no data.
  * **triggerSmartContractTransactions**: (optional) A list of ["trigger smart contract" transaction requests](#trigger-smart-contract-transactions). An empty list requests no data.
  * **internalTransactions**: (optional) A list of [internal transaction requests](#internal-transactions). An empty list requests no data.

  <Accordion title="Example Request">
    ```json theme={"system"}
    {
      "logs": [
        {
          "address": [
            "0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48"
          ],
          "topic0": [
            "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef"
          ],
          "transaction": true
        }
      ],
      "fields": {
        "block": {
          "gasUsed": true
        },
        "log": {
          "topics": true,
          "data": true
        }
      },
      "fromBlock": 16000000,
      "toBlock": 16000000
    }
    ```
  </Accordion>

  <Accordion title="Example Response">
    ```json theme={"system"}
    [
      {
        "header": {
          "number": 16000000,
          "hash": "0x3dc4ef568ae2635db1419c5fec55c4a9322c05302ae527cd40bff380c1d465dd",
          "parentHash": "0x6f377dc6bd1f3e38b9ceb8c946a88c13211fa3f084622df3ee5cfcd98cc6bb16",
          "gasUsed": "0x121cdff"
        },
        "transactions": [
          {
            "transactionIndex": 0
          },
          {
            "transactionIndex": 124
          },
          {
            "transactionIndex": 131
          },
          {
            "transactionIndex": 140
          },
          {
            "transactionIndex": 188
          },
          {
            "transactionIndex": 205
          }
        ],
        "logs": [
          {
            "logIndex": 0,
            "transactionIndex": 0,
            "topics": [
              "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef",
              "0x000000000000000000000000ffec0067f5a79cff07527f63d83dd5462ccf8ba4",
              "0x000000000000000000000000e47872c80e3af63bd237b82c065e441fa75c4dea"
            ],
            "data": "0x0000000000000000000000000000000000000000000000000000000007270e00"
          },
          {
            "logIndex": 30,
            "transactionIndex": 124,
            "topics": [
              "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef",
              "0x000000000000000000000000f42ed7184f3bdd07b0456952f67695683afd9044",
              "0x0000000000000000000000009bbcfc016adcc21d8f86b30cda5e9f100ff9f108"
            ],
            "data": "0x0000000000000000000000000000000000000000000000000000000032430d8b"
          },
          {
            "logIndex": 34,
            "transactionIndex": 131,
            "topics": [
              "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef",
              "0x0000000000000000000000001d76271fb3d5a61184ba00052caa636e666d11ec",
              "0x00000000000000000000000074de5d4fcbf63e00296fd95d33236b9794016631"
            ],
            "data": "0x000000000000000000000000000000000000000000000000000000000fa56ea0"
          },
          {
            "logIndex": 35,
            "transactionIndex": 131,
            "topics": [
              "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef",
              "0x00000000000000000000000074de5d4fcbf63e00296fd95d33236b9794016631",
              "0x000000000000000000000000af0b0000f0210d0f421f0009c72406703b50506b"
            ],
            "data": "0x000000000000000000000000000000000000000000000000000000000fa56ea0"
          },
          {
            "logIndex": 58,
            "transactionIndex": 140,
            "topics": [
              "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef",
              "0x00000000000000000000000048c04ed5691981c42154c6167398f95e8f38a7ff",
              "0x000000000000000000000000f41d156a9bbc1fa6172a50002060cbc757035385"
            ],
            "data": "0x0000000000000000000000000000000000000000000000000000000026273075"
          },
          {
            "logIndex": 230,
            "transactionIndex": 188,
            "topics": [
              "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef",
              "0x000000000000000000000000ba12222222228d8ba445958a75a0704d566bf2c8",
              "0x00000000000000000000000053222470cdcfb8081c0e3a50fd106f0d69e63f20"
            ],
            "data": "0x00000000000000000000000000000000000000000000000000000002536916b7"
          },
          {
            "logIndex": 232,
            "transactionIndex": 188,
            "topics": [
              "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef",
              "0x00000000000000000000000053222470cdcfb8081c0e3a50fd106f0d69e63f20",
              "0x00000000000000000000000088e6a0c2ddd26feeb64f039a2c41296fcb3f5640"
            ],
            "data": "0x00000000000000000000000000000000000000000000000000000002536916b7"
          },
          {
            "logIndex": 372,
            "transactionIndex": 205,
            "topics": [
              "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef",
              "0x0000000000000000000000001116898dda4015ed8ddefb84b6e8bc24528af2d8",
              "0x0000000000000000000000002796317b0ff8538f253012862c06787adfb8ceb6"
            ],
            "data": "0x0000000000000000000000000000000000000000000000000000000018307e19"
          },
          {
            "logIndex": 374,
            "transactionIndex": 205,
            "topics": [
              "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef",
              "0x0000000000000000000000002796317b0ff8538f253012862c06787adfb8ceb6",
              "0x000000000000000000000000735b75559ebb9cd7fed7cec2372b16c3871d2031"
            ],
            "data": "0x0000000000000000000000000000000000000000000000000000000018307e19"
          }
        ]
      }
    ]
    ```
  </Accordion>
</Accordion>

## Data requests

<Warning>
  Addresses in all data requests must be in hex without `0x` and in lowercase. Example:

  ```
  a614f803b6fd780986a42c78ec9c7f77e6ded13c
  ```

  All addresses in the responses will be in this format, too.
</Warning>

### Logs

```ts theme={"system"}
{
  // data requests
  address: string[],
  topic0: string[],
  topic1: string[],
  topic2: string[],
  topic3: string[],

  // related data retrieval
  transaction: boolean
}
```

* `address`: the set of addresses of contracts emitting the logs. Omit to subscribe to events from all contracts in the network.
* `topicN`: the set of values of topicN.

A log will be included in the response if it matches all the requests. An empty array matches no logs; omitted or `null` request matches any log.

With `transaction: true` all parent transactions will be included into the response.

### Transactions

#### General

```ts theme={"system"}
{
  // data requests
  type: string[],

  // related data retrieval
  logs: boolean,
  internalTransactions: boolean
}
```

* `type`: the set of acceptable transaction types.

A transaction will be included in the response if it matches all the requests (just the `type` request in this case). An empty array matches no transactions; omitted or `null` request matches any transaction.

With `logs: true` all logs emitted by the transactions will be included into the response. With `internalTransactions: true` all the internal transactions induced by the selected transactions will be included into the response.

#### "transfer"

```ts theme={"system"}
{
  // data requests
  owner: string[],
  to: string[],

  // related data retrieval
  logs: boolean,
  internalTransactions: boolean
}
```

* `owner`: the set of owner addresses for which the transfer transactions should be retrieved.
* `to`: the set of destination addresses.

A transaction will be included in the response if it matches all the requests. An empty array matches no transactions; omitted or `null` request matches any transaction.

With `logs: true` all logs emitted by the transactions will be included into the response. With `internalTransactions: true` all the internal transactions induced by the selected transactions will be included into the response.

#### "transfer asset"

```ts theme={"system"}
{
  // data requests
  owner: string[],
  to: string[],
  asset: string[],

  // related data retrieval
  logs: boolean,
  internalTransactions: boolean
}
```

* `owner`: the set of owner addresses for which the transfer transactions should be retrieved.
* `to`: the set of destination addresses.
* `asset`: the set of asset contract addresses.

A transaction will be included in the response if it matches all the requests. An empty array matches no transactions; omitted or `null` request matches any transaction.

With `logs: true` all logs emitted by the transactions will be included into the response. With `internalTransactions: true` all the internal transactions induced by the selected transactions will be included into the response.

#### "trigger smart contract"

```ts theme={"system"}
{
  // data requests
  owner: string[],
  to: string[],

  // related data retrieval
  logs: boolean,
  internalTransactions: boolean
}
```

* `owner`: the set of owner addresses for which the transfer transactions should be retrieved.
* `to`: the set of destination addresses.

A transaction will be included in the response if it matches all the requests. An empty array matches no transactions; omitted or `null` request matches any transaction.

With `logs: true` all logs emitted by the transactions will be included into the response. With `internalTransactions: true` all the internal transactions induced by the selected transactions will be included into the response.

### Internal transactions

```ts theme={"system"}
{
  // data requests
  caller: string[],
  transferTo: string[],

  // related data retrieval
  transaction: boolean
}
```

* `caller`: the set of addresses of caller contracts.
* `transferTo`: the set of addresses of receivers of TRX or TRC10 tokens.

A transaction will be included in the response if it matches all the requests. An empty array matches no transactions; omitted or `null` request matches any transaction.

With `transaction: true` all parent transactions for the selected internal transactions will be included into the response.

## Data fields selector

A selector of fields for the returned data items. Its structure is as follows:

```
{
  block:               // field selector for blocks
  log:                 // field selector for logs
  transaction:         // field selector for transactions
  internalTransaction: // field selector for internal transactions
}
```

The `transaction` field selector is common for [all transaction types](#transactions) except for the internal transactions.

### Block fields

```
{
  number
  hash
  parentHash
  timestamp
  txTrieRoot
  version
  timestamp
  witness_address
  witness_signature
}
```

A valid field selector for blocks is a JSON that has a subset of these fields as keys and `true` as values, e.g. `{"hash": true, "timestamp": true}`.

### Log fields

```ts theme={"system"}
{
  logIndex
  transactionHash
  address
  data
  topics
}
```

A valid field selector for logs is a JSON that has a subset of these fields as keys and `true` as values, e.g. `{"address": true, "transactionHash": true}`.

### Transaction fields

```
{
  hash
  ret
  signature
  type
  parameter
  permissionId
  refBlockBytes
  refBlockHash
  feeLimit
  expiration
  timestamp
  rawDataHex
  fee
  contractResult
  contractAddress
  resMessage
  withdrawAmount
  unfreezeAmount
  withdrawExpireAmount
  cancelUnfreezeV2Amount
  result
  energyFee
  energyUsage
  energyUsageTotal
  netUsage
  netFee
  originEnergyUsage
  energyPenaltyTotal
}
```

A valid field selector for transactions is a JSON that has a subset of these fields as keys and `true` as values, e.g. `{"hash": true, "type": true, "contractResult": true}`.

## Internal transaction fields

```
{
  transactionHash
  hash
  callerAddress
  transferToAddress
  callValueInfo
  note
  rejected
  extra
}
```

A valid field selector for internal transactions is a JSON that has a subset of these fields as keys and `true` as values, e.g. `{"hash": true, "callerAddress": true, "callValueInfo": true}`.


# Block data for Tron
Source: https://docs.sqd.dev/en/sdk/squid-sdk/tron-indexing/tron-batch-processor/context-interfaces

Block data interfaces for Tron indexing with Squid SDK — type definitions for blocks, transactions, internal transactions, and event logs in the context.

In Tron Squid SDK, the data is processed by repeatedly calling the user-defined [batch handler](/en/sdk/squid-sdk/reference/processors/architecture#processorrun) function on batches of on-chain data. The sole argument of the batch handler is its context `ctx`, and `ctx.blocks` is an array of `Block` objects containing the data to be processed, aligned at the block level.

For `TronBatchProcessor` the `Block` interface is defined as follows:

```ts theme={"system"}
export interface Block<F extends FieldSelection = {}> {
  header: BlockHeader<F>
  transactions: Transaction<F>[]
  logs: Log<F>[]
  internalTransactions: InternalTransaction<F>[]
}
```

`F` here is the type of the argument of the [`setFields()`](/en/sdk/squid-sdk/tron-indexing/tron-batch-processor/field-selection) processor method.

`Block.header` contains the block header data. The rest of the fields are iterables containing the three kinds of blockchain data. The items within each iterable are ordered in the same way as they are within the block.

The exact fields available in each data item type are inferred from the `setFields()` call argument. The method is documented on the [field selection](/en/sdk/squid-sdk/tron-indexing/tron-batch-processor/field-selection) page:

* [`Transaction` section](/en/sdk/squid-sdk/tron-indexing/tron-batch-processor/field-selection#transaction);
* [`Log` section](/en/sdk/squid-sdk/tron-indexing/tron-batch-processor/field-selection#log);
* [`InternalTransaction` section](/en/sdk/squid-sdk/tron-indexing/tron-batch-processor/field-selection#internal-transaction).
* [`BlockHeader` section](/en/sdk/squid-sdk/tron-indexing/tron-batch-processor/field-selection#block-header)


# Tron Field Selection
Source: https://docs.sqd.dev/en/sdk/squid-sdk/tron-indexing/tron-batch-processor/field-selection

Fine-tune Tron data requests with setFields() in Squid SDK — choose only the block, transaction, internal transaction, and log fields you need.

#### `setFields(options)`

Set the fields to be retrieved for data items of each supported type. The `options` object has the following structure:

```ts theme={"system"}
{
  block?:               // field selector for block headers
  transaction?:         // field selector for transactions
  log?:                 // field selector for logs
  internalTransaction?: // field selector for internal transactions
}
```

Every field selector is a collection of boolean fields that map to the fields of data items within the batch context [iterables](/en/sdk/squid-sdk/tron-indexing/tron-batch-processor/context-interfaces). Defining a field of a field selector of a given type and setting it to true will cause the processor to populate the corresponding field of all data items of that type. Here is a definition of a processor that requests `hash` and `fee` fields for transactions and the `data` field for logs:

```ts theme={"system"}
const processor = new TronBatchProcessor().setFields({
  transaction: {
    hash: true,
    fee: true,
  },
  logs: {
    data: true,
  },
});
```

Same fields will be available for all data items of any given type, including the items accessed via nested references. Suppose we used the processor defined above to subscribe to some transactions as well as some logs, and for each log we requested its parent transaction:

```ts theme={"system"}
processor
  .addTransaction({
    where: {
      // some transaction data requests
    }
  })
  .addLog({
    where: {
      // some log data requests
    },
    include: {
      transaction: true
    }
  })
```

The `hash` and `fee` fields will be available both within the data items of the `transactions` iterable of [block data](/en/sdk/squid-sdk/tron-indexing/tron-batch-processor/context-interfaces) and within the transaction items populating the `.transaction` field of log data items:

```ts theme={"system"}
processor.run(database, async (ctx) => {
  for (let block of blocks) {
    for (let txn of block.transactions) {
      let fee = txn.fee; // OK
    }
    for (let log of block.logs) {
      if (/* log matches the data request */) {
        let logTxFee = log.transaction?.fee; // also OK!
      }
    }
  }
})
```

Some data fields, like `hash` for transactions, are enabled by default but can be disabled by setting a field of a field selector to `false`. For example, this code will not compile:

```ts theme={"system"}
const processor = new TronBatchProcessor()
  .setFields({
    transaction: {
      hash: false
    }
  })

processor.run(database, async (ctx) => {
  for (let block of ctx.blocks) {
    for (let txn of block.transactions) {
      let hash = txn.hash; // ERROR: no such field
    }
  }
});
```

Disabling unused fields will improve sync performance, as the fields' data will not be fetched from the SQD Network gateway.

## Data item types and field selectors

<Tip>
  Most IDEs support smart suggestions to show the possible field selectors. For VS Code, press `Ctrl+Space`.
</Tip>

Here we describe the data item types as functions of the field selectors. On Tron, each data item type field maps to the eponymous field of its corresponding field selector. Item fields are divided into three categories:

* Fields that are always added regardless of the `setFields()` call.
* Fields that are enabled by default and can be disabled by `setFields()`. E.g. a `hash` field will be fetched for transactions by default, but can be disabled by setting `hash: false` within the `transaction` field selector.
* Fields that can be requested by `setFields()`.

### Transaction

Fields of `Transaction` data items may be requested by the eponymous fields of the field selector. Here's a detailed description of possible `Transaction` fields:

```ts theme={"system"}
Transaction {
  // independent of field selectors
  transactionIndex: number

  // can be disabled with field selectors
  hash: string
  type: string

  // can be enabled with field selectors
  ret?: TransactionResult[]
  signature?: string[]
  parameter: any
  permissionId?: number
  refBlockBytes?: string
  refBlockHash?: string
  feeLimit?: bigint
  expiration?: number
  timestamp?: bigint
  rawDataHex: string
  fee?: bigint
  contractResult?: string
  contractAddress?: string
  resMessage?: string
  withdrawAmount?: bigint
  unfreezeAmount?: bigint
  withdrawExpireAmount?: bigint
  cancelUnfreezeV2Amount?: Record<string, bigint>
  result?: string
  energyFee?: bigint
  energyUsage?: bigint
  energyUsageTotal?: bigint
  netUsage?: bigint
  netFee?: bigint
  originEnergyUsage?: bigint
  energyPenaltyTotal?: bigint
}
```

Here, `TransactionResult` is defined as follows:

```ts theme={"system"}
{
  contractRet?: string
}
```

Request the fields with eponymous field request flags.

### Log

Fields of `Log` data items may be requested by the eponymous fields of the field selector. Here's a detailed description of possible `Log` fields:

```ts theme={"system"}
Log {
  // independent of field selectors
  transactionIndex: number
  logIndex: number

  // can be disabled with field selectors
  address: string
  data?: string
  topics?: string[]
}
```

Request the fields with eponymous field request flags.

### Internal transaction

Fields of `InternalTransaction` data items may be requested by the eponymous fields of the field selector. Here's a detailed description of possible `InternalTransaction` fields:

```ts theme={"system"}
InternalTransaction {
  // independent of field selectors
  transactionIndex: number
  internalTransactionIndex: number

  // can be disabled with field selectors
  callerAddress: string
  transferToAddress?: string

  // can be enabled with field selectors
  hash: string
  callValueInfo: CallValueInfo[]
  note: string
  rejected?: boolean
  extra?: string
}
```

Here, `CallValueInfo` is defined as follows:

```ts theme={"system"}
{
  callValue?: bigint
  tokenId?: string
}
```

Request the fields with eponymous field request flags.

### Block header

`BlockHeader` data items may have the following fields:

```ts theme={"system"}
BlockHeader {
	// independent of field selectors
  height: number
  hash: string
  parentHash: string

  // can be disabled with field selectors
  timestamp: number

  // can be enabled with field selectors
  txTrieRoot: string
  version?: number
  witnessAddress: string
  witnessSignature?: string
}
```

Request the fields with eponymous field request flags.

## A complete example

TBA


# Tron Batch General Settings
Source: https://docs.sqd.dev/en/sdk/squid-sdk/tron-indexing/tron-batch-processor/general

General settings for Squid SDK TronBatchProcessor — configure data source, batch size, finality, prometheus metrics, and runtime options for indexers.

<Tip>
  The method documentation is also available inline and can be accessed via suggestions in most IDEs.
</Tip>

The following setters configure the global settings of `TronBatchProcessor`. They return the modified instance and can be chained.

The only requirement is to specify at least one data souce via [`setGateway()`](#set-gateway) or [\`setHttpApi()](#set-http-api). If you need real-time data, providing an HTTP API data source is a hard requirement.

* If you add both an SQD Network gateway and an HTTP API endpoint, the processor will obtain as much data as is currently available from the gateway, then switch to ingesting recent data via HTTP API.
* If you only add an SQD Network gateway, your data will be being several thousands of blocks behind the chain head most of the time.

### `setGateway(url: string | GatewaySettings)`

Use a [SQD Network](/en/network) gateway. The argument is either a string URL of the gateway or

```ts theme={"system"}
{
  url: string // gateway URL
  requestTimeout?: number // in milliseconds
}
```

### `setHttpApi(settings?: HttpApiSettings)`

We must use a regular HTTP API endpoint to get through the last mile and stay on top of the chain. This is a limitation, and we promise to lift it in the future.

```ts theme={"system"}
interface HttpApiSettings = {
  url: string
  strideConcurrency?: // num of concurrent connections, default 2
  strideSize?: number // query size, default 10
  headPollInterval?: number // milliseconds, default 1000
}
```

### `setBlockRange({from: number, to?: number})`

Limits the range of blocks to be processed. When the upper bound is specified, processor will terminate with exit code 0 once it reaches it.

Note that block ranges can also be specified separately for each data request. This method sets global bounds for all block ranges in the configuration.

### `includeAllBlocks(range?: {from: number, to?: number})`

By default, processor will fetch only blocks which contain requested items. This method modifies such behavior to fetch all chain blocks. Optionally a range of blocks can be specified for which the setting should be effective.

### `setPrometheusPort(port: string | number)`

Sets the port for a built-in prometheus health metrics server (serving at `http://localhost:${port}/metrics`). By default, the value of PROMETHEUS\_PORT environment variable is used. When it is not set, processor will pick an ephemeral port.


# Internal transactions
Source: https://docs.sqd.dev/en/sdk/squid-sdk/tron-indexing/tron-batch-processor/internal-transactions

Subscribe to Tron internal transactions with addInternalTransaction() in Squid SDK — filter by sender, contract, and asset for efficient indexing.

#### `addInternalTransaction(options)`

Get some *or all* internal transactions on the network. `options` has the following structure:

```typescript theme={"system"}
{
  where?: {
    caller?: string[]
    transferTo?: string[]
  }
  include?: {
    transaction?: boolean
  }
  range?: {
    from: number
    to?: number
  }
}
```

**Data requests** are located in the `where` field:

* `caller` is the set of caller addresses responsible for the internal transactons. Leave it undefined to subscribe to internal txs from all callers.
* `transferTo` is the set of receiver addresses that the internal txn is addressed to.

Omit the `where` field to subscribe to all txs network-wide.

**Related data** can be requested via the `include` field:

* `transaction = true`: will retrieve parent transactions for each selected internal txn.

The data will be added to the `.transactions` iterable within [block data](/en/sdk/squid-sdk/tron-indexing/tron-batch-processor/context-interfaces) and made available via the `.transaction` field of each internal transaction item.

Note that internal transactions can also be requested by the other `TronBatchProcessor` methods as related data.

Selection of the exact fields to be retrieved for each transaction and the optional related data items is done with the `setFields()` method documented on the [Field selection](/en/sdk/squid-sdk/tron-indexing/tron-batch-processor/field-selection) page.

#### Example

TBA


# Logs
Source: https://docs.sqd.dev/en/sdk/squid-sdk/tron-indexing/tron-batch-processor/logs

Subscribe to Tron event log data with addLog() in Squid SDK — filter by contract address, topic, and signature for efficient Tron event indexing.

#### `addLog(options)`

Get event logs emitted by some *or all* contracts in the network. `options` has the following structure:

```typescript theme={"system"}
{
  where?: {
    address?: string[]
    topic0?: string[]
    topic1?: string[]
    topic2?: string[]
    topic3?: string[]
  }
  include?: {
    transaction?: boolean
  }
  range?: {
    from: number
    to?: number
  }
}
```

**Data requests** are located in the `where` field:

* `address`: the set of addresses of contracts emitting the logs. Omit to subscribe to events from all contracts in the network.
* `topicN`: the set of values of topicN.

Omit the `where` field to subscribe to all logs network-wide.

**Related data** can be requested via the `include` field:

* `transaction = true`: will retrieve the parent transacton for each selected log.

The data will be added to the `.transactions` iterable within [block data](/en/sdk/squid-sdk/tron-indexing/tron-batch-processor/context-interfaces) and made available via the `.transaction` field of each log item.

Note that logs can also be requested by the other `TronBatchProcessor` methods as related data.

Selection of the exact fields to be retrieved for each log item and the optional parent transactions is done with the `setFields()` method documented on the [Field selection](/en/sdk/squid-sdk/tron-indexing/tron-batch-processor/field-selection) page.

#### Example

Request all `Transfer(address,address,uint256)` event logs emitted by the USDT smart contract, include parent txs.

```ts theme={"system"}
const USDT_ADDRESS = 'a614f803b6fd780986a42c78ec9c7f77e6ded13c'
const TRANSFER_TOPIC = 'ddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef'

processor.addLog({
  where: {
    address: [USDT_ADDRESS],
    topic0: [TRANSFER_TOPIC]
  },
  include: {
    transaction: true
  }
})
```


# Tron Transactions Indexing
Source: https://docs.sqd.dev/en/sdk/squid-sdk/tron-indexing/tron-batch-processor/transactions

Subscribe to Tron transaction data with addTransaction() in Squid SDK — filter by from, to, and contract address for efficient on-chain indexing.

Tron Network has a diverse and dynamic set of transaction types. `TronBatchProcessor` has methods for subscribing to some common transaction types, all with fine-grained filters:

* [`addTransferTransaction()`](#add-transfer-transaction)
* [`addTransferAssetTransaction()`](#add-transfer-asset-transaction)
* [`addTriggerSmartContractTransaction()`](#add-trigger-smart-contract-transaction)

For other transaction types you can use the general method [`addTransaction()`](#add-transaction) that does not offer much filtering.

If your application would benefit from fine-grained request methods for other transaction types, please let us know in the [SquidDevs Telegram channel](https://t.me/HydraDevs).

### `addTransferTransaction(options)`

Get some *or all* [`TransferContract`](https://github.com/tronprotocol/java-tron/blob/c136a26f8140b17f2c05df06fb5efb1bb47d3baa/protocol/src/main/protos/core/Tron.proto#L340) transactions on the network. `options` has the following structure:

```typescript theme={"system"}
{
  where?: {
    owner?: string[]
    to?: string[]
  }
  include?: {
    logs?: boolean
    internalTransactions?: boolean
  }
  range?: {
    from: number
    to?: number
  }
}
```

**Data requests** are located in the `where` field:

* `owner`: the set of addresses of owners of the token being transferred.
* `to`: the set of addresses of tokens' receivers.

Omit the `where` field to subscribe to all `TransferContract` txs network-wide.

**Related data** can be requested via the `include` field:

* `logs = true`: will retrieve event logs emitted by the selected transactions.
* `internalTransactions = true`: will retrieve internal txs that occurred due to the selected transactions.

The data will be added to the appropriate iterables within [block data](/en/sdk/squid-sdk/tron-indexing/tron-batch-processor/context-interfaces) and made available via the `.logs` and `.internalTransactions` fields of each transaction item.

Note that transactions can also be requested by the other `TronBatchProcessor` methods as related data.

Selection of the exact fields to be retrieved for each transaction and the optional related data items is done with the `setFields()` method documented on the [Field selection](/en/sdk/squid-sdk/tron-indexing/tron-batch-processor/field-selection) page.

#### Example

This requests `TransferContract` txs where owner is `41da9964294c8689bfee2778606e485221625496bf`, plus their logs, on block 64000675.

```ts theme={"system"}
processor.addTransferTransaction({
  where: {
    owner: [ '41da9964294c8689bfee2778606e485221625496bf' ]
  },
  include: {
    logs: true
  },
  range: {
    from: 64000675,
    to: 64000675
  }
})
```

### `addTransferAssetTransaction(options)`

Get some *or all* [`TransferAssetContract`](https://github.com/tronprotocol/java-tron/blob/c136a26f8140b17f2c05df06fb5efb1bb47d3baa/protocol/src/main/protos/core/Tron.proto#L341) transactions on the network. `options` has the following structure:

```typescript theme={"system"}
{
  where?: {
    asset?: string[]
    owner?: string[]
    to?: string[]
  }
  include?: {
    logs?: boolean
    internalTransactions?: boolean
  }
  range?: {
    from: number
    to?: number
  }
}
```

**Data requests** are located in the `where` field:

* `asset`: the set of token names / ids.
* `owner`: the set of addresses of owners of the token being transferred.
* `to`: the set of addresses of tokens' receivers.

Omit the `where` field to subscribe to all `TransferAssetContract` txs network-wide.

**Related data** can be requested via the `include` field:

* `logs = true`: will retrieve event logs emitted by the selected transactions.
* `internalTransactions = true`: will retrieve internal txs that occurred due to the selected transactions.

The data will be added to the appropriate iterables within [block data](/en/sdk/squid-sdk/tron-indexing/tron-batch-processor/context-interfaces) and made available via the `.logs` and `.internalTransactions` fields of each transaction item.

Note that transactions can also be requested by the other `TronBatchProcessor` methods as related data.

Selection of the exact fields to be retrieved for each transaction and the optional related data items is done with the `setFields()` method documented on the [Field selection](/en/sdk/squid-sdk/tron-indexing/tron-batch-processor/field-selection) page.

#### Example

This requests `TransferAssetContract` txs where owner is `4170d66a855afef04753f06c4d6210d6b77708cdb0`, plus their internal txs, on block 64000000.

```ts theme={"system"}
processor.addTransferTransaction({
  where: {
    owner: [ '4170d66a855afef04753f06c4d6210d6b77708cdb0' ]
  },
  include: {
    internalTransactions: true
  },
  range: {
    from: 64000000,
    to: 64000000
  }
})
```

### `addTriggerSmartContractTransaction(options)`

Get some *or all* [`TriggerSmartContract`](https://github.com/tronprotocol/java-tron/blob/c136a26f8140b17f2c05df06fb5efb1bb47d3baa/protocol/src/main/protos/core/Tron.proto#L360) transactions on the network. `options` has the following structure:

```typescript theme={"system"}
{
  where?: {
    contract?: string[]
    sighash?: string[]
    owner?: string[]
  }
  include?: {
    logs?: boolean
    internalTransactions?: boolean
  }
  range?: {
    from: number
    to?: number
  }
}
```

**Data requests** are located in the `where` field:

* `contract`: the set of addresses of contracts being called.
* `sighash`: the set of 4-byte signature hashes to filter the calls by.
* `owner`: the set of addresses of call owners.

Omit the `where` field to subscribe to all `TriggerSmartContract` txs network-wide.

**Related data** can be requested via the `include` field:

* `logs = true`: will retrieve event logs emitted by the selected transactions.
* `internalTransactions = true`: will retrieve internal txs that occurred due to the selected transactions.

The data will be added to the appropriate iterables within [block data](/en/sdk/squid-sdk/tron-indexing/tron-batch-processor/context-interfaces) and made available via the `.logs` and `.internalTransactions` fields of each transaction item.

Note that transactions can also be requested by the other `TronBatchProcessor` methods as related data.

Selection of the exact fields to be retrieved for each transaction and the optional related data items is done with the `setFields()` method documented on the [Field selection](/en/sdk/squid-sdk/tron-indexing/tron-batch-processor/field-selection) page.

#### Example

This requests `TriggerSmartContract` txs made directly to the USDT contract (`41a614f803b6fd780986a42c78ec9c7f77e6ded13c`).

```ts theme={"system"}
processor.addTriggerSmartContractTransaction({
  where: {
    contract: [ '41a614f803b6fd780986a42c78ec9c7f77e6ded13c' ]
  }
})
```

### `addTransaction(options)`

Get some *or all* transactions on the network. `options` has the following structure:

```typescript theme={"system"}
{
  where?: {
    type?: string[]
  }
  include?: {
    logs?: boolean
    internalTransactions?: boolean
  }
  range?: {
    from: number
    to?: number
  }
}
```

**Data requests** are located in the `where` field:

* `type` sets the allowed transaction types. Leave it undefined to subscribe to transactions of all types.

Omit the `where` field to subscribe to all txs network-wide.

**Related data** can be requested via the `include` field:

* `logs = true`: will retrieve event logs emitted by the selected transactions.
* `internalTransactions = true`: will retrieve internal txs that occurred due to the selected transactions.

The data will be added to the appropriate iterables within [block data](/en/sdk/squid-sdk/tron-indexing/tron-batch-processor/context-interfaces) and made available via the `.logs` and `.internalTransactions` fields of each transaction item.

Note that transactions can also be requested by the other `TronBatchProcessor` methods as related data.

Selection of the exact fields to be retrieved for each transaction and the optional related data items is done with the `setFields()` method documented on the [Field selection](/en/sdk/squid-sdk/tron-indexing/tron-batch-processor/field-selection) page.

#### Example

This requests all `FreezeBalanceV2Contract` txs network-wide, along with all the logs they emitted:

```ts theme={"system"}
processor.addTransaction({
  where: {
    type: [ 'FreezeBalanceV2Contract' ]
  },
  include: {
    logs: true
  }
})
```


# Latest Finalized Block
Source: https://docs.sqd.dev/en/api/evm/finalized-head

/en/api/catalog/evm/openapi.yaml get /finalized-head
Returns the block number and hash of the highest finalized block available in the dataset, or null if no blocks are available. Useful for diagnostics.



# Stream Finalized Blocks
Source: https://docs.sqd.dev/en/api/evm/finalized-stream

/en/api/catalog/evm/openapi.yaml post /finalized-stream
Returns only finalized blocks matching the query. Same request format as /stream but never returns unfinalized blocks, so there are no chain reorganizations to handle (no 409 responses).




# Latest Block
Source: https://docs.sqd.dev/en/api/evm/head

/en/api/catalog/evm/openapi.yaml get /head
Returns the block number and hash of the highest block available in the dataset, or null if no blocks are available. Takes real-time data into account. Useful for diagnostics.



# Introduction
Source: https://docs.sqd.dev/en/api/evm/introduction



1. [POST your data request to `/stream`](./stream) or [`/finalized-stream`](./finalized-stream) and you shall receive a [JSON lines](https://jsonlines.org) stream:
   ```bash theme={"system"}
   curl https://portal.sqd.dev/datasets/ethereum-mainnet/stream \
     -X POST \
     -d '{
       "type": "evm",
       "fromBlock": 16000000,
       "toBlock": 16000001,
       "fields": {
         "block": {"number": true},
         "log": {"data": true}
       },
       "logs": [
         {"address": [
           "0x2260fac5e5542a773aa44fbcfedf7c193bc2c599"
         ]}
       ]
     }'
   ```
   outputs
   ```jsonl theme={"system"}
   {"header":{"number":16000000}}
   {"header":{"number":16000001},"logs":[{"data":"0x000000000000000000000000000000000000000000000000000000000000071e"}]}
   ```
2. By default the Portal will only include blocks with data matching the request filters. However, the stream will always include the first and the last blocks of the block range it covered, even if they have no matching data.
3. The portal may end stream before it reaches `toBlock`. You can omit `toBlock` for open-ended streaming.
4. To get the rest of the data, the client must adjust the `fromBlock` field to the number of the last received block **plus one**. When streaming unfinalized data from `/stream`, they must also include `parentBlockHash` into the request.

## Handling forks on `/stream`

5. Chain forks / reorgs sometimes make some of the data from `/stream` obsolete. The portal will detect that using the `parentBlockHash` field of the request and respond with an HTTP 409 containing a sample of new consensus blocks. When that happens, the client must roll back any changes to its own state made due to orphan blocks' data, then send another `POST /stream` that starts from an earlier block to attempt to fetch the updated chain.
6. Post-rollback request must contain `parentBlockHash`. If the Portal detects that the block is still not in the updated chain, it'll return a 409 again. Be prepared to process several of these in a row.
7. To make state management easier for the client, Portal may return `X-Sqd-Finalized-Head-Number` and `X-Sqd-Finalized-Head-Hash` headers with each response from `/stream`. The block from these headers and blocks below it won't ever reorganize.
8. That doesn't mean that `X-Sqd-Finalized-Head-Number` is a non-decreasing value: Portals can be behind load balancers and the value may decrease when the load balancer switches to a lagging instance. The client should trust the guarantees provided by either instance and treat the highest block returned with these headers as a high water mark for finalized blocks.


# Dataset Metadata
Source: https://docs.sqd.dev/en/api/evm/metadata

/en/api/catalog/evm/openapi.yaml get /metadata
Retrieves metadata describing the dataset, including its name, aliases, start block, and real-time status.



# Stream Blocks
Source: https://docs.sqd.dev/en/api/evm/stream

/en/api/catalog/evm/openapi.yaml post /stream
Returns blocks matching the query. May include unfinalized blocks from the chain tip. Response is a JSON array of blocks containing the requested fields. Set toBlock to limit the range, or omit it to stream up to the dataset height. Supports gzip compression via Accept-Encoding and Content-Encoding headers.




# EVM Networks
Source: https://docs.sqd.dev/en/data/evm

Browse the complete list of EVM and Ethereum-compatible networks supported by SQD for blockchain data indexing — chain IDs, gateways, and Portal endpoints.

<Warning>
  **Portal Migration**: We're migrating from v2 archives to Portal. V2 archives will be sunset soon. [Get Started with Portal →](/en/portal/migration)
</Warning>

## L1 fields availability

Extra information on L1 [height](/en/sdk/squid-sdk/reference/processors/evm-batch/field-selection#block-headers) and [transaction fees](/en/sdk/squid-sdk/reference/processors/evm-batch/field-selection#transactions) is available for select Layer 2 EVM networks:

* Optimism
* Optimism Sepolia
* Base
* Base Sepolia
* Zora
* Zora Sepolia
* Scroll (`l1Fee` only)
* Scroll Sepolia (`l1Fee` only)


# Configuration
Source: https://docs.sqd.dev/en/data/evm-local-setup/configuration

Configure Docker services for local EVM indexing with SQD — config.yaml dataset definition, retention strategy, and Portal endpoint setup for devnet workflows.

Set up the configuration files needed to run your local EVM indexer.

## Dataset Configuration

Create a `config.yaml` file to define your dataset:

```yaml config.yaml theme={"system"}
my-devnet:
  kind: evm
  retention_strategy:
    FromBlock:
      number: 0
    # Alternative: retain last N blocks
    # Head: 500
  data_sources:
    - http://hotblocks-service:3000
```

### Configuration Options

**`kind`**

* Set to `evm` for Ethereum-compatible chains

**`retention_strategy`**

Choose one of two strategies:

* `FromBlock`: Keep all blocks starting from the specified block number
  ```yaml theme={"system"}
  FromBlock:
    number: 0  # Start from genesis
  ```

* `Head`: Keep only the most recent N blocks (rolling window)
  ```yaml theme={"system"}
  Head: 500  # Keep last 500 blocks
  ```

**`data_sources`**

* Internal service URL (use `http://hotblocks-service:3000` as shown)
* This connects to the hotblocks-service container

## Docker Compose Configuration

Create a `docker-compose.yaml` file:

```yaml docker-compose.yaml theme={"system"}
services:
  # Loads data from RPC and serves full blocks
  hotblocks-service:
    image: subsquid/evm-hotblocks-service:4229b9b2
    environment:
      SQD_INFO: "*"
    command: >
      --http-rpc ${RPC_URL}
      --receipts
      --traces
      --diffs
      --use-debug-api-for-statediffs
      --verify-tx-sender
      --verify-tx-root
      --verify-logs-bloom
      --block-cache-size 10
    # Optional verification flags:
    # --verify-receipts-root
    # --verify-block-hash

  # Loads full blocks from hotblocks-service and provides query API
  hotblocks-db:
    image: subsquid/data-hotblocks:a8e7c369
    environment:
      RUST_LOG: "info"
    command: >
      --datasets /app/config.yaml
      --db /run/db
      --port 8000
    ports:
      - "8000:8000"
    volumes:
      - ./config.yaml:/app/config.yaml
      - db-data:/run/db
    depends_on:
      - hotblocks-service

volumes:
  db-data:
```

## Service Details

### hotblocks-service

Fetches blockchain data from your RPC endpoint.

**Key command flags:**

* `--http-rpc ${RPC_URL}` - Your RPC endpoint (set via environment variable)
* `--receipts` - Fetch transaction receipts
* `--traces` - Fetch execution traces
* `--diffs` - Fetch state diffs
* `--use-debug-api-for-statediffs` - Use debug API for state diff extraction
* `--verify-tx-sender` - Verify transaction sender addresses
* `--verify-tx-root` - Verify transaction root hashes
* `--verify-logs-bloom` - Verify logs bloom filters
* `--block-cache-size 10` - Cache 10 blocks in memory

**Optional verification flags** (commented out by default):

* `--verify-receipts-root` - Verify receipts root hash
* `--verify-block-hash` - Verify block hash

<Tip>
  Enable additional verification flags for maximum data integrity, but note they may increase processing time.
</Tip>

### hotblocks-db

Stores blockchain data and exposes the query API.

**Configuration:**

* `--datasets /app/config.yaml` - Path to dataset configuration
* `--db /run/db` - Database storage location
* `--port 8000` - API server port
* **Volumes:**
  * `./config.yaml:/app/config.yaml` - Mounts your config file
  * `db-data:/run/db` - Persistent storage for blockchain data

**Environment:**

* `RUST_LOG: "info"` - Set logging level (options: `error`, `warn`, `info`, `debug`, `trace`)

## Storage Considerations

<Warning>
  The `db-data` volume stores the entire blockchain. Disk space usage grows continuously as new blocks are added.

  **Recommendations:**

  * Monitor disk usage regularly
  * Use the `Head` retention strategy for development/testing
  * Ensure adequate disk space before syncing production chains
</Warning>

## Next Steps

Proceed to the [Usage](/en/data/evm-local-setup/usage) page to start the services and query your data.


# Local EVM Devnet Setup
Source: https://docs.sqd.dev/en/data/evm-local-setup/overview

Introduction to running a local EVM devnet indexing stack — Docker Compose setup with reth, Portal, and Postgres for end-to-end SQD development.

Index your local devnet data and query it using the SQD Portal API. This setup allows you to run a complete indexing stack on your machine using Docker.

## What This Provides

A local deployment that:

* Ingests blockchain data from your RPC endpoint
* Stores blocks with full transaction details, receipts, traces, and state diffs
* Exposes the Portal API for querying your local chain data
* Works with any EVM-compatible devnet or testnet

## Prerequisites

Before starting, ensure you have:

* **Docker** and **Docker Compose** installed on your system
* Access to an **EVM RPC endpoint** (local devnet or testnet URL)
* **Sufficient disk space** - the setup stores the entire blockchain locally

<Note>
  For production devnets with significant block history, allocate at least 50GB of disk space initially and monitor growth.
</Note>

## Architecture

The setup runs two Docker services:

### hotblocks-service

Connects to your RPC endpoint and fetches blockchain data including:

* Block headers and transactions
* Transaction receipts
* Execution traces
* State diffs

### hotblocks-db

Stores the blockchain data and provides:

* Portal API query interface on port 8000
* Efficient data retrieval and filtering
* Same API as the public SQD network

## Next Steps

Continue to the [Configuration](/en/data/evm-local-setup/configuration) page to set up your Docker services.


# Usage
Source: https://docs.sqd.dev/en/data/evm-local-setup/usage

Run and query a local EVM indexer with SQD — start the Docker services, set the RPC endpoint, and stream blockchain data via the local Portal HTTP API.

Start your local indexer and query blockchain data using the Portal API.

## Starting the Services

<Steps>
  <Step title="Set your RPC endpoint">
    Export the RPC URL as an environment variable:

    ```bash theme={"system"}
    export RPC_URL=https://your-rpc-endpoint-url
    ```

    **Examples:**

    * Local devnet: `http://localhost:8545`
    * Local Hardhat: `http://127.0.0.1:8545`
    * Remote testnet: `https://sepolia.infura.io/v3/YOUR_KEY`
  </Step>

  <Step title="Start the services">
    Launch both services in detached mode:

    ```bash theme={"system"}
    docker-compose up -d
    ```

    The services will start syncing blocks from your RPC endpoint.
  </Step>

  <Step title="Verify services are running">
    Check the logs to confirm syncing has started:

    ```bash theme={"system"}
    docker-compose logs -f
    ```

    You should see block ingestion activity. Press `Ctrl+C` to exit the logs.

    <Check>
      Look for log entries showing block numbers being processed. This confirms the indexer is working.
    </Check>
  </Step>
</Steps>

## Querying Your Data

Once running, the Portal API is available at `http://localhost:8000`.

### Basic Query Example

Fetch all blocks with their basic information:

```bash theme={"system"}
curl --compress "http://localhost:8000/datasets/my-devnet/stream" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "evm",
    "fields": {
      "block": {
        "timestamp": true,
        "number": true,
        "hash": true
      }
    },
    "includeAllBlocks": true,
    "fromBlock": 0
  }'
```

### Query Transactions

Fetch blocks with transaction details:

```bash theme={"system"}
curl --compress "http://localhost:8000/datasets/my-devnet/stream" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "evm",
    "fields": {
      "block": {
        "number": true,
        "timestamp": true
      },
      "transaction": {
        "hash": true,
        "from": true,
        "to": true,
        "value": true
      }
    },
    "fromBlock": 0
  }'
```

### Query Logs

Fetch event logs from specific contracts:

```bash theme={"system"}
curl --compress "http://localhost:8000/datasets/my-devnet/stream" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "evm",
    "fields": {
      "log": {
        "address": true,
        "topics": true,
        "data": true
      }
    },
    "logs": [
      {
        "address": ["0xYourContractAddress"]
      }
    ],
    "fromBlock": 0
  }'
```

<Note>
  Replace `my-devnet` in the URL with your dataset name from `config.yaml` if you used a different name.
</Note>

## API Reference

Your local setup exposes the same Portal API endpoints as the public network.

For complete endpoint documentation and query options, see the [Portal API Reference](/en/api/catalog/evm/openapi.yaml).

## Stopping the Services

Stop the services while preserving data:

```bash theme={"system"}
docker-compose down
```

The blockchain data remains in the `db-data` volume and will be available when you restart.

## Cleanup

To remove all data and free disk space:

```bash theme={"system"}
docker-compose down -v
```

<Warning>
  The `-v` flag deletes the blockchain data volume permanently. You'll need to re-sync from scratch if you start the services again.
</Warning>

## Troubleshooting

### Services won't start

**Check RPC connectivity:**

```bash theme={"system"}
curl --compress -X POST $RPC_URL \
  -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":1}'
```

If this fails, verify your RPC endpoint is accessible.

### No blocks appearing

**Check hotblocks-service logs:**

```bash theme={"system"}
docker-compose logs hotblocks-service
```

Look for connection errors or RPC issues.

### Out of disk space

**Check volume size:**

```bash theme={"system"}
docker system df -v
```

Consider using the `Head` retention strategy in `config.yaml` to limit storage.

### Port 8000 already in use

Modify the port mapping in `docker-compose.yaml`:

```yaml theme={"system"}
ports:
  - "8001:8000"  # Use port 8001 instead
```

Then access the API at `http://localhost:8001`.


# 0g Testnet
Source: https://docs.sqd.dev/en/data/evm/0g-testnet

Index 0g Testnet EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value        |
| ----------- | ------------ |
| Network ID  | `0g-testnet` |
| Chain ID    | `16600`      |
| Real-time   | No           |
| Traces      | ✗            |
| State Diffs | ✗            |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/0g-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/0g-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/0g-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Abstract
Source: https://docs.sqd.dev/en/data/evm/abstract-mainnet

Index Abstract EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value              |
| ----------- | ------------------ |
| Network ID  | `abstract-mainnet` |
| Chain ID    | `2741`             |
| Real-time   | No                 |
| Traces      | ✗                  |
| State Diffs | ✗                  |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/abstract-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/abstract-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/abstract-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Abstract Testnet
Source: https://docs.sqd.dev/en/data/evm/abstract-testnet

Index Abstract Testnet EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value              |
| ----------- | ------------------ |
| Network ID  | `abstract-testnet` |
| Chain ID    | `11124`            |
| Real-time   | No                 |
| Traces      | ✗                  |
| State Diffs | ✗                  |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/abstract-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/abstract-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/abstract-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# ADI Chain
Source: https://docs.sqd.dev/en/data/evm/adi-mainnet

Index ADI Chain EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value         |
| ----------- | ------------- |
| Network ID  | `adi-mainnet` |
| Chain ID    | `36900`       |
| Real-time   | Yes           |
| Traces      | ✓             |
| State Diffs | ✗             |

## Endpoint

```
https://portal.sqd.dev/datasets/adi-mainnet
```

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/adi-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Chain-specific tutorials

* [Squid SDK: real-time Portal data template](https://github.com/subsquid-labs/squid-evm-rt-template/)

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# ADI Testnet
Source: https://docs.sqd.dev/en/data/evm/adi-testnet

Index ADI Testnet EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value         |
| ----------- | ------------- |
| Network ID  | `adi-testnet` |
| Chain ID    | `undefined`   |
| Real-time   | Yes           |
| Traces      | ✓             |
| State Diffs | ✗             |

## Endpoint

```
https://portal.sqd.dev/datasets/adi-testnet
```

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/adi-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Chain-specific tutorials

* [Squid SDK: real-time Portal data template](https://github.com/subsquid-labs/squid-evm-rt-template/)

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Agung Testnet
Source: https://docs.sqd.dev/en/data/evm/agung-evm

Index Agung Testnet EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value       |
| ----------- | ----------- |
| Network ID  | `agung-evm` |
| Chain ID    | `9990`      |
| Real-time   | No          |
| Traces      | ✗           |
| State Diffs | ✗           |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/agung-evm
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/agung-evm
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/agung-evm/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Aleph Zero EVM
Source: https://docs.sqd.dev/en/data/evm/aleph-zero-evm-mainnet

Index Aleph Zero EVM EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value                    |
| ----------- | ------------------------ |
| Network ID  | `aleph-zero-evm-mainnet` |
| Chain ID    | `41455`                  |
| Real-time   | No                       |
| Traces      | ✓                        |
| State Diffs | ✓                        |

## Endpoint

```
https://portal.sqd.dev/datasets/aleph-zero-evm-mainnet
```

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/aleph-zero-evm-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Alpen Testnet
Source: https://docs.sqd.dev/en/data/evm/alpen-testnet

Index Alpen Testnet EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value           |
| ----------- | --------------- |
| Network ID  | `alpen-testnet` |
| Chain ID    | `8150`          |
| Real-time   | Yes             |
| Traces      | ✓               |
| State Diffs | ✓               |

## Endpoint

```
https://portal.sqd.dev/datasets/alpen-testnet
```

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/alpen-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Arbitrum Nova
Source: https://docs.sqd.dev/en/data/evm/arbitrum-nova

Index Arbitrum Nova EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value           |
| ----------- | --------------- |
| Network ID  | `arbitrum-nova` |
| Chain ID    | `42170`         |
| Real-time   | No              |
| Traces      | ✓               |
| State Diffs | ✗               |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/arbitrum-nova
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/arbitrum-nova
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/arbitrum-nova/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Arbitrum One
Source: https://docs.sqd.dev/en/data/evm/arbitrum-one

Index Arbitrum One EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value          |
| ----------- | -------------- |
| Network ID  | `arbitrum-one` |
| Chain ID    | `42161`        |
| Real-time   | Yes            |
| Traces      | ✓              |
| State Diffs | ✗              |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/arbitrum-one
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/arbitrum-one
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/arbitrum-one/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Arbitrum Sepolia
Source: https://docs.sqd.dev/en/data/evm/arbitrum-sepolia

Index Arbitrum Sepolia EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value              |
| ----------- | ------------------ |
| Network ID  | `arbitrum-sepolia` |
| Chain ID    | `421614`           |
| Real-time   | Yes                |
| Traces      | ✗                  |
| State Diffs | ✗                  |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/arbitrum-sepolia
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/arbitrum-sepolia
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/arbitrum-sepolia/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Arthera
Source: https://docs.sqd.dev/en/data/evm/arthera-mainnet

Index Arthera EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value             |
| ----------- | ----------------- |
| Network ID  | `arthera-mainnet` |
| Chain ID    | `10242`           |
| Real-time   | No                |
| Traces      | ✗                 |
| State Diffs | ✗                 |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/arthera-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/arthera-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/arthera-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Astar
Source: https://docs.sqd.dev/en/data/evm/astar-mainnet

Index Astar EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

<Info>Only for EVM data. For Substrate and ink! data use the corresponding Substrate gateway.</Info>

| Property    | Value           |
| ----------- | --------------- |
| Network ID  | `astar-mainnet` |
| Chain ID    | `592`           |
| Real-time   | No              |
| Traces      | ✗               |
| State Diffs | ✗               |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/astar-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/astar-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/astar-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Astar zkEVM
Source: https://docs.sqd.dev/en/data/evm/astar-zkevm-mainnet

Index Astar zkEVM EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value                 |
| ----------- | --------------------- |
| Network ID  | `astar-zkevm-mainnet` |
| Chain ID    | `3776`                |
| Real-time   | No                    |
| Traces      | ✗                     |
| State Diffs | ✗                     |

## Endpoint

```
https://portal.sqd.dev/datasets/astar-zkevm-mainnet
```

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/astar-zkevm-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Astar zKyoto
Source: https://docs.sqd.dev/en/data/evm/astar-zkyoto

Index Astar zKyoto EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value          |
| ----------- | -------------- |
| Network ID  | `astar-zkyoto` |
| Chain ID    | `6038361`      |
| Real-time   | No             |
| Traces      | ✗              |
| State Diffs | ✗              |

## Endpoint

```
https://portal.sqd.dev/datasets/astar-zkyoto
```

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/astar-zkyoto/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Avalanche
Source: https://docs.sqd.dev/en/data/evm/avalanche-mainnet

Index Avalanche EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value               |
| ----------- | ------------------- |
| Network ID  | `avalanche-mainnet` |
| Chain ID    | `43114`             |
| Real-time   | Yes                 |
| Traces      | ✓                   |
| State Diffs | ✓                   |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/avalanche-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/avalanche-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/avalanche-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Avalanche Testnet
Source: https://docs.sqd.dev/en/data/evm/avalanche-testnet

Index Avalanche Testnet EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value               |
| ----------- | ------------------- |
| Network ID  | `avalanche-testnet` |
| Chain ID    | `43113`             |
| Real-time   | No                  |
| Traces      | ✓                   |
| State Diffs | ✓                   |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/avalanche-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/avalanche-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/avalanche-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# B3
Source: https://docs.sqd.dev/en/data/evm/b3-mainnet

Index B3 EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value        |
| ----------- | ------------ |
| Network ID  | `b3-mainnet` |
| Chain ID    | `8333`       |
| Real-time   | No           |
| Traces      | ✓            |
| State Diffs | ✓            |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/b3-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/b3-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/b3-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# B3 Sepolia
Source: https://docs.sqd.dev/en/data/evm/b3-sepolia

Index B3 Sepolia EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value        |
| ----------- | ------------ |
| Network ID  | `b3-sepolia` |
| Chain ID    | `1993`       |
| Real-time   | No           |
| Traces      | ✓            |
| State Diffs | ✓            |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/b3-sepolia
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/b3-sepolia
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/b3-sepolia/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Base
Source: https://docs.sqd.dev/en/data/evm/base-mainnet

Index Base EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value          |
| ----------- | -------------- |
| Network ID  | `base-mainnet` |
| Chain ID    | `8453`         |
| Real-time   | Yes            |
| Traces      | ✓              |
| State Diffs | ✓              |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/base-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/base-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/base-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Base Sepolia
Source: https://docs.sqd.dev/en/data/evm/base-sepolia

Index Base Sepolia EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value          |
| ----------- | -------------- |
| Network ID  | `base-sepolia` |
| Chain ID    | `84532`        |
| Real-time   | No             |
| Traces      | ✓              |
| State Diffs | ✓              |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/base-sepolia
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/base-sepolia
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/base-sepolia/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Beam
Source: https://docs.sqd.dev/en/data/evm/beam-mainnet

Index Beam EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value          |
| ----------- | -------------- |
| Network ID  | `beam-mainnet` |
| Chain ID    | `4337`         |
| Real-time   | No             |
| Traces      | ✓              |
| State Diffs | ✓              |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/beam-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/beam-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/beam-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Berachain bArtio
Source: https://docs.sqd.dev/en/data/evm/berachain-bartio

Index Berachain bArtio EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value              |
| ----------- | ------------------ |
| Network ID  | `berachain-bartio` |
| Chain ID    | `80084`            |
| Real-time   | No                 |
| Traces      | ✓                  |
| State Diffs | ✓                  |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/berachain-bartio
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/berachain-bartio
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/berachain-bartio/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Berachain
Source: https://docs.sqd.dev/en/data/evm/berachain-mainnet

Index Berachain EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value               |
| ----------- | ------------------- |
| Network ID  | `berachain-mainnet` |
| Chain ID    | `80094`             |
| Real-time   | Yes                 |
| Traces      | ✓                   |
| State Diffs | ✓                   |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/berachain-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/berachain-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/berachain-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# BNB Smart Chain
Source: https://docs.sqd.dev/en/data/evm/binance-mainnet

Index BNB Smart Chain EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value             |
| ----------- | ----------------- |
| Network ID  | `binance-mainnet` |
| Chain ID    | `56`              |
| Real-time   | Yes               |
| Traces      | ✓                 |
| State Diffs | ✓                 |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/binance-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/binance-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/binance-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Binance Testnet
Source: https://docs.sqd.dev/en/data/evm/binance-testnet

Index Binance Testnet EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value             |
| ----------- | ----------------- |
| Network ID  | `binance-testnet` |
| Chain ID    | `97`              |
| Real-time   | No                |
| Traces      | ✗                 |
| State Diffs | ✗                 |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/binance-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/binance-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/binance-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Bitfinity Network
Source: https://docs.sqd.dev/en/data/evm/bitfinity-mainnet

Index Bitfinity Network EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value               |
| ----------- | ------------------- |
| Network ID  | `bitfinity-mainnet` |
| Chain ID    | `355110`            |
| Real-time   | No                  |
| Traces      | ✗                   |
| State Diffs | ✗                   |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/bitfinity-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/bitfinity-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/bitfinity-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Bitfinity Testnet
Source: https://docs.sqd.dev/en/data/evm/bitfinity-testnet

Index Bitfinity Testnet EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

<Info>Genesis block for Bitfinity Testnet is 2,760,000.</Info>

| Property    | Value               |
| ----------- | ------------------- |
| Network ID  | `bitfinity-testnet` |
| Chain ID    | `355113`            |
| Real-time   | No                  |
| Traces      | ✗                   |
| State Diffs | ✗                   |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/bitfinity-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/bitfinity-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/bitfinity-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Bitgert
Source: https://docs.sqd.dev/en/data/evm/bitgert-mainnet

Index Bitgert EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

<Info>State diffs available up to block 4298094.</Info>

| Property    | Value             |
| ----------- | ----------------- |
| Network ID  | `bitgert-mainnet` |
| Chain ID    | `32520`           |
| Real-time   | No                |
| Traces      | ✓                 |
| State Diffs | ✗                 |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/bitgert-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/bitgert-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/bitgert-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Bitgert Testnet
Source: https://docs.sqd.dev/en/data/evm/bitgert-testnet

Index Bitgert Testnet EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value             |
| ----------- | ----------------- |
| Network ID  | `bitgert-testnet` |
| Chain ID    | `64668`           |
| Real-time   | No                |
| Traces      | ✓                 |
| State Diffs | ✗                 |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/bitgert-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/bitgert-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/bitgert-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Bittensor
Source: https://docs.sqd.dev/en/data/evm/bittensor-mainnet-evm

Index Bittensor EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

<Info>First available block for Bittensor Mainnet is 4,400,000.</Info>

| Property    | Value                   |
| ----------- | ----------------------- |
| Network ID  | `bittensor-mainnet-evm` |
| Chain ID    | `964`                   |
| Real-time   | No                      |
| Traces      | ✗                       |
| State Diffs | ✗                       |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/bittensor-mainnet-evm
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/bittensor-mainnet-evm
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/bittensor-mainnet-evm/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Bittensor Testnet
Source: https://docs.sqd.dev/en/data/evm/bittensor-testnet-evm

Index Bittensor Testnet EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

<Info>First available block for Bittensor Testnet is 3,300,000.</Info>

| Property    | Value                   |
| ----------- | ----------------------- |
| Network ID  | `bittensor-testnet-evm` |
| Chain ID    | `945`                   |
| Real-time   | No                      |
| Traces      | ✗                       |
| State Diffs | ✗                       |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/bittensor-testnet-evm
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/bittensor-testnet-evm
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/bittensor-testnet-evm/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Blast
Source: https://docs.sqd.dev/en/data/evm/blast-l2-mainnet

Index Blast EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value              |
| ----------- | ------------------ |
| Network ID  | `blast-l2-mainnet` |
| Chain ID    | `81457`            |
| Real-time   | No                 |
| Traces      | ✓                  |
| State Diffs | ✓                  |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/blast-l2-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/blast-l2-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/blast-l2-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Blast Sepolia
Source: https://docs.sqd.dev/en/data/evm/blast-sepolia

Index Blast Sepolia EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value           |
| ----------- | --------------- |
| Network ID  | `blast-sepolia` |
| Chain ID    | `168587773`     |
| Real-time   | No              |
| Traces      | ✗               |
| State Diffs | ✗               |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/blast-sepolia
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/blast-sepolia
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/blast-sepolia/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# BOB
Source: https://docs.sqd.dev/en/data/evm/bob-mainnet

Index BOB EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value         |
| ----------- | ------------- |
| Network ID  | `bob-mainnet` |
| Chain ID    | `60808`       |
| Real-time   | No            |
| Traces      | ✓             |
| State Diffs | ✓             |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/bob-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/bob-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/bob-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# BOB Sepolia
Source: https://docs.sqd.dev/en/data/evm/bob-sepolia

Index BOB Sepolia EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value         |
| ----------- | ------------- |
| Network ID  | `bob-sepolia` |
| Chain ID    | `808813`      |
| Real-time   | No            |
| Traces      | ✗             |
| State Diffs | ✗             |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/bob-sepolia
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/bob-sepolia
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/bob-sepolia/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Botanix
Source: https://docs.sqd.dev/en/data/evm/botanix-mainnet

Index Botanix EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value             |
| ----------- | ----------------- |
| Network ID  | `botanix-mainnet` |
| Chain ID    | `3637`            |
| Real-time   | No                |
| Traces      | ✓                 |
| State Diffs | ✓                 |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/botanix-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/botanix-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/botanix-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Botanix Testnet
Source: https://docs.sqd.dev/en/data/evm/botanix-testnet

Index Botanix Testnet EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value             |
| ----------- | ----------------- |
| Network ID  | `botanix-testnet` |
| Chain ID    | `3636`            |
| Real-time   | No                |
| Traces      | ✓                 |
| State Diffs | ✓                 |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/botanix-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/botanix-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/botanix-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Camp Network Testnet v2
Source: https://docs.sqd.dev/en/data/evm/camp-network-testnet-v2

Index Camp Network Testnet v2 EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value                     |
| ----------- | ------------------------- |
| Network ID  | `camp-network-testnet-v2` |
| Chain ID    | `325000`                  |
| Real-time   | No                        |
| Traces      | ✓                         |
| State Diffs | ✓                         |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/camp-network-testnet-v2
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/camp-network-testnet-v2
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/camp-network-testnet-v2/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Canto
Source: https://docs.sqd.dev/en/data/evm/canto

Index Canto EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value   |
| ----------- | ------- |
| Network ID  | `canto` |
| Chain ID    | `7700`  |
| Real-time   | No      |
| Traces      | ✗       |
| State Diffs | ✗       |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/canto
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/canto
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/canto/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Canto Testnet
Source: https://docs.sqd.dev/en/data/evm/canto-testnet

Index Canto Testnet EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value           |
| ----------- | --------------- |
| Network ID  | `canto-testnet` |
| Chain ID    | `7701`          |
| Real-time   | No              |
| Traces      | ✗               |
| State Diffs | ✗               |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/canto-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/canto-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/canto-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Celo Testnet (Alfajores)
Source: https://docs.sqd.dev/en/data/evm/celo-alfajores-testnet

Index Celo Testnet (Alfajores) EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value                    |
| ----------- | ------------------------ |
| Network ID  | `celo-alfajores-testnet` |
| Chain ID    | `44787`                  |
| Real-time   | No                       |
| Traces      | ✗                        |
| State Diffs | ✗                        |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/celo-alfajores-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/celo-alfajores-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/celo-alfajores-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Celo
Source: https://docs.sqd.dev/en/data/evm/celo-mainnet

Index Celo EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value          |
| ----------- | -------------- |
| Network ID  | `celo-mainnet` |
| Chain ID    | `42220`        |
| Real-time   | Yes            |
| Traces      | ✗              |
| State Diffs | ✗              |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/celo-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/celo-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/celo-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Core Blockchain
Source: https://docs.sqd.dev/en/data/evm/core-mainnet

Index Core Blockchain EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value          |
| ----------- | -------------- |
| Network ID  | `core-mainnet` |
| Chain ID    | `1116`         |
| Real-time   | No             |
| Traces      | ✓              |
| State Diffs | ✗              |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/core-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/core-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/core-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Cronos
Source: https://docs.sqd.dev/en/data/evm/cronos-mainnet

Index Cronos EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value            |
| ----------- | ---------------- |
| Network ID  | `cronos-mainnet` |
| Chain ID    | `25`             |
| Real-time   | Yes              |
| Traces      | ✗                |
| State Diffs | ✗                |

## Endpoint

```
https://portal.sqd.dev/datasets/cronos-mainnet
```

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/cronos-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# CrossFi
Source: https://docs.sqd.dev/en/data/evm/crossfi-mainnet

Index CrossFi EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

<Info>First available block for CrossFi networks is 1.</Info>

| Property    | Value             |
| ----------- | ----------------- |
| Network ID  | `crossfi-mainnet` |
| Chain ID    | `4158`            |
| Real-time   | No                |
| Traces      | ✗                 |
| State Diffs | ✗                 |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/crossfi-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/crossfi-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/crossfi-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# CrossFi Testnet
Source: https://docs.sqd.dev/en/data/evm/crossfi-testnet

Index CrossFi Testnet EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

<Info>First available block for CrossFi networks is 1.</Info>

| Property    | Value             |
| ----------- | ----------------- |
| Network ID  | `crossfi-testnet` |
| Chain ID    | `4157`            |
| Real-time   | No                |
| Traces      | ✗                 |
| State Diffs | ✗                 |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/crossfi-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/crossfi-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/crossfi-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Cyber
Source: https://docs.sqd.dev/en/data/evm/cyber-mainnet

Index Cyber EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value           |
| ----------- | --------------- |
| Network ID  | `cyber-mainnet` |
| Chain ID    | `7560`          |
| Real-time   | No              |
| Traces      | ✗               |
| State Diffs | ✗               |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/cyber-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/cyber-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/cyber-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Cyberconnect L2 Testnet
Source: https://docs.sqd.dev/en/data/evm/cyberconnect-l2-testnet

Index Cyberconnect L2 Testnet EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value                     |
| ----------- | ------------------------- |
| Network ID  | `cyberconnect-l2-testnet` |
| Chain ID    | `111557560`               |
| Real-time   | No                        |
| Traces      | ✓                         |
| State Diffs | ✓                         |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/cyberconnect-l2-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/cyberconnect-l2-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/cyberconnect-l2-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Degen Chain
Source: https://docs.sqd.dev/en/data/evm/degen-chain

Index Degen Chain EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value         |
| ----------- | ------------- |
| Network ID  | `degen-chain` |
| Chain ID    | `666666666`   |
| Real-time   | No            |
| Traces      | ✗             |
| State Diffs | ✗             |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/degen-chain
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/degen-chain
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/degen-chain/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# DFK Chain
Source: https://docs.sqd.dev/en/data/evm/dfk-chain

Index DFK Chain EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value       |
| ----------- | ----------- |
| Network ID  | `dfk-chain` |
| Chain ID    | `53935`     |
| Real-time   | No          |
| Traces      | ✗           |
| State Diffs | ✗           |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/dfk-chain
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/dfk-chain
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/dfk-chain/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Dogechain
Source: https://docs.sqd.dev/en/data/evm/dogechain-mainnet

Index Dogechain EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value               |
| ----------- | ------------------- |
| Network ID  | `dogechain-mainnet` |
| Chain ID    | `2000`              |
| Real-time   | No                  |
| Traces      | ✗                   |
| State Diffs | ✗                   |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/dogechain-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/dogechain-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/dogechain-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Dogechain Testnet
Source: https://docs.sqd.dev/en/data/evm/dogechain-testnet

Index Dogechain Testnet EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value               |
| ----------- | ------------------- |
| Network ID  | `dogechain-testnet` |
| Chain ID    | `568`               |
| Real-time   | No                  |
| Traces      | ✗                   |
| State Diffs | ✗                   |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/dogechain-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/dogechain-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/dogechain-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Ethereum Holesky
Source: https://docs.sqd.dev/en/data/evm/ethereum-holesky

Index Ethereum Holesky EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value              |
| ----------- | ------------------ |
| Network ID  | `ethereum-holesky` |
| Chain ID    | `17000`            |
| Real-time   | No                 |
| Traces      | ✓                  |
| State Diffs | ✓                  |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/ethereum-holesky
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/ethereum-holesky
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/ethereum-holesky/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Ethereum Hoodi
Source: https://docs.sqd.dev/en/data/evm/ethereum-hoodi

Index Ethereum Hoodi EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value            |
| ----------- | ---------------- |
| Network ID  | `ethereum-hoodi` |
| Chain ID    | `560048`         |
| Real-time   | No               |
| Traces      | ✗                |
| State Diffs | ✗                |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/ethereum-hoodi
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/ethereum-hoodi
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/ethereum-hoodi/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup />

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Ethereum
Source: https://docs.sqd.dev/en/data/evm/ethereum-mainnet

Index Ethereum EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value              |
| ----------- | ------------------ |
| Network ID  | `ethereum-mainnet` |
| Chain ID    | `1`                |
| Real-time   | Yes                |
| Traces      | ✓                  |
| State Diffs | ✓                  |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/ethereum-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/ethereum-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/ethereum-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Ethereum Sepolia
Source: https://docs.sqd.dev/en/data/evm/ethereum-sepolia

Index Ethereum Sepolia EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value              |
| ----------- | ------------------ |
| Network ID  | `ethereum-sepolia` |
| Chain ID    | `11155111`         |
| Real-time   | Yes                |
| Traces      | ✓                  |
| State Diffs | ✓                  |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/ethereum-sepolia
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/ethereum-sepolia
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/ethereum-sepolia/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Etherlink
Source: https://docs.sqd.dev/en/data/evm/etherlink-mainnet

Index Etherlink EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value               |
| ----------- | ------------------- |
| Network ID  | `etherlink-mainnet` |
| Chain ID    | `42793`             |
| Real-time   | No                  |
| Traces      | ✗                   |
| State Diffs | ✗                   |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/etherlink-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/etherlink-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/etherlink-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Etherlink Shadownet
Source: https://docs.sqd.dev/en/data/evm/etherlink-shadownet

Index Etherlink Shadownet EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value                 |
| ----------- | --------------------- |
| Network ID  | `etherlink-shadownet` |
| Chain ID    | `127823`              |
| Real-time   | No                    |
| Traces      | ✗                     |
| State Diffs | ✗                     |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/etherlink-shadownet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/etherlink-shadownet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/etherlink-shadownet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Etherlink Testnet
Source: https://docs.sqd.dev/en/data/evm/etherlink-testnet

Index Etherlink Testnet EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value               |
| ----------- | ------------------- |
| Network ID  | `etherlink-testnet` |
| Chain ID    | `128123`            |
| Real-time   | No                  |
| Traces      | ✗                   |
| State Diffs | ✗                   |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/etherlink-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/etherlink-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/etherlink-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Exosama Network
Source: https://docs.sqd.dev/en/data/evm/exosama

Index Exosama Network EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value     |
| ----------- | --------- |
| Network ID  | `exosama` |
| Chain ID    | `2109`    |
| Real-time   | No        |
| Traces      | ✓         |
| State Diffs | ✓         |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/exosama
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/exosama
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/exosama/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Flare
Source: https://docs.sqd.dev/en/data/evm/flare-mainnet

Index Flare EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value           |
| ----------- | --------------- |
| Network ID  | `flare-mainnet` |
| Chain ID    | `14`            |
| Real-time   | Yes             |
| Traces      | ✗               |
| State Diffs | ✗               |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/flare-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/flare-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/flare-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Gravity Alpha
Source: https://docs.sqd.dev/en/data/evm/galxe-gravity

Index Gravity Alpha EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value           |
| ----------- | --------------- |
| Network ID  | `galxe-gravity` |
| Chain ID    | `1625`          |
| Real-time   | No              |
| Traces      | ✓               |
| State Diffs | ✓               |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/galxe-gravity
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/galxe-gravity
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/galxe-gravity/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Arbitrum Blueberry
Source: https://docs.sqd.dev/en/data/evm/gelato-arbitrum-blueberry

Index Arbitrum Blueberry EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value                       |
| ----------- | --------------------------- |
| Network ID  | `gelato-arbitrum-blueberry` |
| Chain ID    | `88153591557`               |
| Real-time   | No                          |
| Traces      | ✗                           |
| State Diffs | ✗                           |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/gelato-arbitrum-blueberry
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/gelato-arbitrum-blueberry
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/gelato-arbitrum-blueberry/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Gelato OP Celestia Raspberry
Source: https://docs.sqd.dev/en/data/evm/gelato-opcelestia-raspberry

Index Gelato OP Celestia Raspberry EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value                         |
| ----------- | ----------------------------- |
| Network ID  | `gelato-opcelestia-raspberry` |
| Chain ID    | `123420111`                   |
| Real-time   | No                            |
| Traces      | ✗                             |
| State Diffs | ✗                             |

## Endpoint

```
https://portal.sqd.dev/datasets/gelato-opcelestia-raspberry
```

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/gelato-opcelestia-raspberry/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Gnosis
Source: https://docs.sqd.dev/en/data/evm/gnosis-mainnet

Index Gnosis EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value            |
| ----------- | ---------------- |
| Network ID  | `gnosis-mainnet` |
| Chain ID    | `100`            |
| Real-time   | Yes              |
| Traces      | ✓                |
| State Diffs | ✓                |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/gnosis-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/gnosis-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/gnosis-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# HyperEVM
Source: https://docs.sqd.dev/en/data/evm/hyperevm-mainnet

Index HyperEVM EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value                 |
| ----------- | --------------------- |
| Network ID  | `hyperliquid-mainnet` |
| Chain ID    | `999`                 |
| Real-time   | Yes                   |
| Traces      | ✓                     |
| State Diffs | ✓                     |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/hyperliquid-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/hyperliquid-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/hyperliquid-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# HyperEVM Testnet
Source: https://docs.sqd.dev/en/data/evm/hyperevm-testnet

Index HyperEVM Testnet EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value                 |
| ----------- | --------------------- |
| Network ID  | `hyperliquid-testnet` |
| Chain ID    | `998`                 |
| Real-time   | No                    |
| Traces      | ✗                     |
| State Diffs | ✗                     |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/hyperliquid-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/hyperliquid-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/hyperliquid-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Immutable zkEVM
Source: https://docs.sqd.dev/en/data/evm/immutable-zkevm-mainnet

Index Immutable zkEVM EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value                     |
| ----------- | ------------------------- |
| Network ID  | `immutable-zkevm-mainnet` |
| Chain ID    | `13371`                   |
| Real-time   | No                        |
| Traces      | ✗                         |
| State Diffs | ✗                         |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/immutable-zkevm-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/immutable-zkevm-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/immutable-zkevm-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Immutable zkEVM Testnet
Source: https://docs.sqd.dev/en/data/evm/immutable-zkevm-testnet

Index Immutable zkEVM Testnet EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value                     |
| ----------- | ------------------------- |
| Network ID  | `immutable-zkevm-testnet` |
| Chain ID    | `13473`                   |
| Real-time   | No                        |
| Traces      | ✗                         |
| State Diffs | ✗                         |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/immutable-zkevm-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/immutable-zkevm-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/immutable-zkevm-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Delegate
Source: https://docs.sqd.dev/en/network/delegate

Delegate SQD tokens to worker nodes on SQD Network and earn rewards — eligibility, commitment period, withdrawal rules, and how to maximize APY.

<Info>
  By delegating, you commit your tokens for approximately 14 days (100,000
  Ethereum blocks) plus the time until the end of the current
  epoch (20 minutes or less). Withdrawal is not
  possible during this period.
</Info>

The easiest way to participate in SQD Network is to delegate your `SQD` tokens to worker nodes. This guide shows you how to delegate and maximize your rewards.

## How to Delegate

<Steps>
  <Step title="Connect to the SQD Network app" icon="link">
    Go to [network.subsquid.io](https://network.subsquid.io) and connect your wallet.

    <Check>
      You're connected when your wallet address appears in the interface.
    </Check>
  </Step>

  <Step title="Choose a worker" icon="magnifying-glass">
    Navigate to the "Dashboard" tab to view all available workers.

    Review worker performance metrics:

    * **Uptime percentage** - Higher is better
    * **Delegation capacity** - More capacity means more room for your delegation
    * **APR** - Expected annual percentage return

    <Tip>
      Look for workers with high uptime and available delegation capacity for
      optimal rewards.
    </Tip>
  </Step>

  <Step title="Delegate your tokens" icon="coins">
    1. Click the "Delegate" button next to your chosen worker
    2. Enter the amount of `SQD` tokens you want to delegate
    3. Click "Confirm" to proceed

    <Check>The form will show you the expected lock period and rewards.</Check>
  </Step>

  <Step title="Confirm the transaction" icon="check">
    Confirm the transaction in your wallet and wait for it to complete on-chain.

    <Check>
      Transaction confirmed! Your delegation is now active and earning rewards.
    </Check>
  </Step>

  <Step title="Monitor your delegation" icon="chart-line">
    Go to the "Delegate" tab to view your active delegations.

    <Note>
      The "Undelegate" button will be inactive until the current
      epoch ends.
    </Note>
  </Step>
</Steps>

## Undelegating Your Tokens

<Steps>
  <Step title="Wait for the lockup period" icon="clock">
    Wait for your delegation lockup period to expire. The period is approximately **14 days** (100,000 Ethereum blocks), starting at the first block following the end of the epoch when you delegated.

    <Warning>
      You cannot undelegate your tokens until the lockup period has passed.
    </Warning>
  </Step>

  <Step title="Access your delegations" icon="wallet">
    1. Go to [network.subsquid.io](https://network.subsquid.io)
    2. Navigate to the "Delegate" tab
    3. You should see all your active delegations

    <Info>
      An inactive "Undelegate" button indicates the delegation lockup period has not
      yet passed.
    </Info>
  </Step>

  <Step title="Initiate undelegation" icon="arrow-up-from-bracket">
    1. Click the "Undelegate" button next to your delegation
    2. Enter the `SQD` amount you want to withdraw
    3. Click "Confirm" and approve the transaction in your wallet

    <Check>
      Your tokens will be returned to your wallet once the transaction completes.
    </Check>
  </Step>
</Steps>

## Maximizing Your Rewards

Choose high-quality but less popular workers for optimal rewards. Look for workers with:

* **High uptime percentage** (90%+)
* **Available delegation capacity**
* **Consistent performance history**

<Tip>
  Workers with high uptime and available delegation capacity typically provide
  the best returns.
</Tip>

### Optimal Delegation Amount

Both workers and their delegators earn the most rewards when the total delegated `SQD` count is around **30,000 tokens**. Delegating significantly more or less than this optimal amount may result in lower rewards.

<Note>
  The optimal delegation amount may change based on network parameters. Monitor
  your delegation performance regularly.
</Note>


# SQD Network
Source: https://docs.sqd.dev/en/network/overview

SQD Network is a decentralized data lake serving historical blockchain data for 200+ chains — workers store chunks and gateways stream them at near-zero cost.

<Frame>
  <img alt="SQD Network - Access blockchain data through decentralized infrastructure" />
</Frame>

SQD Network is a distributed query engine and data lake that powers the SQD ecosystem. It serves historical blockchain data for 200+ networks, enabling you to access data at near-zero cost with granular filtering and efficient batch processing.

<Info>
  SQD Network provides superior data access compared to conventional chain node
  RPCs through its rich streaming and filtering capabilities, allowing you to
  query multiple blocks simultaneously.
</Info>

## How SQD Network Works

SQD Network stores blockchain data in a distributed data lake, making it available through **SQD Portal**, a high-performance data access interface that fetches data chunks from multiple worker nodes in parallel.

Portal exposes a streaming HTTP API optimized for data extraction at scale. It handles pagination, finality tracking, and reorg compensation automatically, so you can focus on building your application rather than managing infrastructure complexity.

You can access the network in three ways:

1. **Use managed Portal endpoints** - SQD-hosted endpoints provide instant access to 200+ networks with zero setup
2. **Self-host Portal** - Run your own Portal instance alongside your nodes for complete data sovereignty
3. **Use the SDK** - Build production indexers with TypeScript SDKs that abstract Portal's API into a developer-friendly interface

<Tip>
  Learn more about Portal and how it transforms blockchain data access in the
  [Portal overview](/en/portal/overview).
</Tip>

## Network Options

SQD Network operates in two configurations, both freely accessible and using the same Portal API:

<div>
  <table>
    <thead>
      <tr>
        <th>
          Feature
        </th>

        <th>
          Permissionless Network
        </th>

        <th>
          Private Network
        </th>
      </tr>
    </thead>

    <tbody>
      <tr>
        <td>Status</td>
        <td>Mainnet (Portal in public beta)</td>
        <td>Production-ready</td>
      </tr>

      <tr>
        <td>Infrastructure</td>
        <td>Fully decentralized</td>
        <td>Operated by Subsquid Labs GmbH</td>
      </tr>

      <tr>
        <td>Participation</td>
        <td>Anyone can run workers or delegate</td>
        <td>Permissioned</td>
      </tr>

      <tr>
        <td>Networks supported</td>
        <td>Almost all</td>
        <td>200+ networks</td>
      </tr>
    </tbody>
  </table>
</div>

<Note>
  Both networks are free to query. See the [complete list of supported
  networks](/en/data/networks/evm) for gateway URLs and network details.
</Note>

## Participate in the Permissionless Network

The decentralized network is now in mainnet, and you can participate in two ways:

<CardGroup>
  <Card title="Run a Worker Node" icon="server" href="/en/network/worker">
    Contribute storage and compute resources to the network and earn SQD tokens
  </Card>

  <Card title="Delegate SQD Tokens" icon="coins" href="/en/network/delegate">
    Reward high-performing workers and share in their profits
  </Card>
</CardGroup>

## Get Started with SQD Network

Ready to build with SQD Network? Choose your path:

<CardGroup>
  <Card title="Portal API" icon="plug" href="/en/portal/evm/quickstart">
    Query blockchain data directly via HTTP API
  </Card>

  <Card title="Build with SDK" icon="code" href="/en/sdk/overview">
    Create production indexers with TypeScript SDKs
  </Card>

  <Card title="Browse Networks" icon="globe" href="/en/data/networks/evm">
    View all 200+ supported networks and gateway URLs
  </Card>
</CardGroup>


# Run a Worker
Source: https://docs.sqd.dev/en/network/worker

Run a SQD Network worker node to serve historical blockchain data and earn SQD token rewards — hardware requirements, setup steps, and operating practices.

<Warning>
  We strongly advise against using Hetzner for running SQD Network worker nodes
  due to reliability concerns.
</Warning>

Running a worker node allows you to contribute storage and compute resources to SQD Network and earn `SQD` token rewards.

## Prerequisites

Before you begin, ensure you have the following:

### Hardware Requirements

* **CPU:** 4 vCPU
* **RAM:** 16GB
* **Storage:** 1TB SSD
* **Network:** Stable 24/7 Internet connection (minimum 1Gbit)

### Network Configuration

* Public IP address
* Two ports open for incoming traffic:
  * **UDP port** for P2P communication (default: 12345)
  * **TCP port** for Prometheus metrics (default: 9090)

### Software Requirements

* Docker
* Docker Compose

### Financial Requirements

* **100,000 `SQD` tokens** (in your wallet or vesting contract)
* **Arbitrum ETH** for gas fees

<Info>
  The `SQD` tokens should be available in your Primary Wallet, either directly or
  through a vesting contract. With sufficient funds, one Primary Wallet can
  register multiple workers.
</Info>

<Tip>
  Use a browser-compatible wallet like MetaMask for your Primary Wallet.
</Tip>

### Setup Options

You can run a worker using either:

* Pre-built Docker image (recommended)
* Build from source code

<Note>
  Docker is required for both options as the configuration script uses it.
</Note>

## Setup Instructions

<Steps>
  <Step title="Choose data and key storage locations" icon="folder">
    Choose your storage locations:

    1. **Data directory** - Select a location that can accommodate at least 1TB
    2. **Key file location** - Must be outside the data directory

    <Warning>
      Protect your key file! Ensure it won't be deleted accidentally and cannot be
      accessed by unauthorized parties. See [consequences of losing your
      key](#what-are-the-consequences-of-losing-my-key-file--getting-it-stolen).
    </Warning>

    <Note>
      Do not create the data folder manually, the setup script will create it.
    </Note>
  </Step>

  <Step title="Download the setup script" icon="download">
    Create a new directory for installation files and download the setup script:

    ```bash theme={"system"}
    curl -fO https://cdn.subsquid.io/worker/setup_worker.sh
    ```

    Make the script executable:

    ```bash theme={"system"}
    chmod +x ./setup_worker.sh
    ```

    <Check>Setup script is ready to run.</Check>
  </Step>

  <Step title="Run the setup script" icon="terminal">
    Execute the setup script with your chosen paths:

    ```bash theme={"system"}
    ./setup_worker.sh <DATA_DIR_PATH> <KEY_PATH>
    ```

    The script will:

    * Prompt you for a UDP port for P2P communication
    * Ask if you want to set your public IP address statically

    <Tip>
      Most users should not set the IP address manually, automatic IP discovery is
      more robust.
    </Tip>

    **What the script creates:**

    * Data directory at `<DATA_DIR_PATH>`
    * Key file at `<KEY_PATH>`
    * `.env` file with default configuration
    * `docker-compose.yaml` file for running the worker

    <Info>
      View all files the script downloads on
      [GitHub](https://github.com/subsquid/cdn/tree/main/src/worker).
    </Info>
  </Step>

  <Step title="Copy your peer ID" icon="id-card">
    The script's output will end with a line like this:

    ```
    Your peer ID is: 12D3KooWPfotj6qQapKeWg4RZysUjEeswLEpiSjgv16LgNTyYFTW. Now you can register it on chain.
    ```

    **Copy your peer ID**, you'll need it for on-chain worker registration.

    <Check>Setup complete! You're ready to register your worker.</Check>
  </Step>

  <Step title="Optional: Customize configuration" icon="gear">
    If needed, edit the generated `.env` file to:

    * Set a custom Prometheus port
    * Use your own RPC provider
    * Adjust other configuration parameters

    <Note>Most users can skip this step and use the default configuration.</Note>
  </Step>
</Steps>

## Worker Registration

<Info>
  By registering a worker, you lock your tokens in the smart contract.
  Once you want to withdraw them, you'll have to wait for approximately 14 days
  (100,000 Ethereum blocks) plus the time until the end of the current
  epoch (up to 20 minutes).
</Info>

Before running your worker node, you must register it on-chain.

<Steps>
  <Step title="Connect your wallet" icon="wallet">
    1. Go to [network.subsquid.io](https://network.subsquid.io)
    2. Connect your Primary Wallet (the one holding your 100,000 `SQD` tokens)

    <Check>Your wallet address should appear in the interface.</Check>
  </Step>

  <Step title="Access worker registration" icon="plus">
    Navigate to the "Workers" tab and click the "Add Worker" button.

    The worker registration form will appear.
  </Step>

  <Step title="Fill the registration form" icon="pen-to-square">
    Complete the registration form:

    1. **Funding source:** Choose either:

       * "Wallet" - Use `SQD` tokens directly from your wallet
       * "Vesting contract" - Use `SQD` tokens from a vesting contract

    2. **Peer ID:** Enter the peer ID you copied during [setup](#setup-instructions)

    3. Submit the form by confirming the transaction in your wallet

    <Warning>
      Double-check your peer ID before submitting, incorrect IDs will prevent your
      worker from functioning properly.
    </Warning>
  </Step>

  <Step title="Wait for activation" icon="hourglass">
    1. Go to the "Workers" tab
    2. Monitor your worker's status

    Your worker will change to "Offline" or "Active" status at the beginning of the next epoch (within approximately 20 minutes).

    <Check>
      Worker registered! Proceed to running your worker once the status updates.
    </Check>
  </Step>
</Steps>

## Running Your Worker

<Note>
  Ensure you're in the folder where you executed `setup_worker.sh` during the
  setup process.
</Note>

<Tabs>
  <Tab title="Docker (Recommended)">
    ### Run with Docker Compose

    Start your worker in detached mode:

    ```bash theme={"system"}
    docker compose up -d
    ```

    ### Monitor Logs

    Check the worker container logs:

    ```bash theme={"system"}
    docker compose logs
    ```

    <Check>
      After a few minutes, the worker should begin outputting information about
      downloaded data chunks.
    </Check>

    <Tip>Use `docker compose logs -f` to follow logs in real-time.</Tip>
  </Tab>

  <Tab title="Build from Source">
    ### Install Prerequisites

    Install required dependencies (Rust, Git, Protobuf, etc.):

    ```bash theme={"system"}
    apt install curl git build-essential pkg-config protobuf-compiler libssl-dev libsqlite3-dev
    curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
    ```

    ### Load Configuration

    Load the configuration files generated during setup:

    ```bash theme={"system"}
    source .env
    ```

    ### Clone the Worker Repository

    Clone the worker repository (this may take a few minutes):

    ```bash theme={"system"}
    git clone --depth 1 https://github.com/subsquid/worker-rs.git
    ```

    ### Build the Worker

    Navigate to the repository and build:

    ```bash theme={"system"}
    cd worker-rs
    cargo build --release
    ```

    ### Run the Worker

    Start the worker:

    ```bash theme={"system"}
    cargo run --release
    ```

    <Tip>
      Consider daemonizing with `systemd` or `openrc` to ensure the worker restarts
      automatically after server reboots.
    </Tip>

    <Check>
      After some time, the worker will start downloading data and serving queries.
    </Check>
  </Tab>
</Tabs>

## Updating Your Worker

Periodically, you'll need to update your worker to the latest version. Follow these steps:

<Warning>Always back up your key file before updating.</Warning>

<Tabs>
  <Tab title="Docker Update">
    1. Navigate to your installation files folder
    2. Fetch the updated `docker-compose.yaml`:

    ```bash theme={"system"}
    curl -fO https://cdn.subsquid.io/worker/docker-compose.yaml
    ```

    3. Apply the update:

    ```bash theme={"system"}
    docker compose up -d
    ```

    <Check>Update complete! Your worker is now running the latest version.</Check>
  </Tab>

  <Tab title="Source Build Update">
    1. Note the new worker version from the announcement
    2. Navigate to your installation files folder
    3. Enter the `worker-rs` subfolder
    4. Fetch new commits:

    ```bash theme={"system"}
    git fetch --unshallow
    ```

    5. Check out the new version:

    ```bash theme={"system"}
    git checkout <new_version_tag>
    ```

    6. Rebuild the worker:

    ```bash theme={"system"}
    cargo build --release
    ```

    7. Restart the worker process

    <Check>Update complete! Your worker is now running the latest version.</Check>
  </Tab>
</Tabs>

<Info>
  You don't need to erase the data folder or re-register your worker during
  updates.
</Info>

<Note>
  Some releases may require deviations from this procedure. Always read release
  notes carefully.
</Note>

## Understanding Worker Jailing

Jailing is a scheduler-side mechanism that ensures data chunk availability across the network. The scheduler identifies potentially unreliable workers and guarantees that each chunk exists on multiple reliable nodes.

<Info>
  Jailed workers continue serving queries from downloaded chunks and earning
  rewards. Jailing is temporary, workers are automatically unjailed within
  approximately 20 minutes once they're reliable again.
</Info>

### Impact on Rewards

While occasional jailing is normal, frequent jailing may reduce your rewards. Monitor your worker's status and address any underlying issues.

### Common Jailing Reasons

Check your worker logs to identify which issue applies:

**Stale Worker** - Worker didn't download assigned chunks in the last 900 seconds

* Most common jailing reason
* Check worker logs for the root cause
* Often related to [download timeouts](#i-see-failed-to-download-chunk--operation-timed-out-in-the-worker-logs)

**Inactive Worker** - Worker didn't send pings for over 600 seconds

* May occur after extended downtime
* Scheduler considers the worker offline and stops assigning chunks

**Unreachable Worker** (Currently disabled)

* Was used to detect P2P network connectivity issues
* Implementation challenges led to this check being disabled

<Tip>
  If your worker stays online consistently and maintains good connectivity, it's
  unlikely to be jailed.
</Tip>

## Troubleshooting

<AccordionGroup>
  <Accordion title="Where do I find my peer ID?">
    Your peer ID can be found in several places:

    1. **Setup script output** - Printed when you run `setup_worker.sh` (see [Setup Instructions](#setup-instructions))

    2. **Worker logs:**
       * **Docker setup:** Run `docker compose logs`
       * **Source build:** Check the output of the worker process

    Look for a log line like this:

    ```
    2024-05-28T07:43:55.591440Z  INFO subsquid_worker::transport::p2p: Local peer ID: <PEER_ID>
    ```
  </Accordion>

  <Accordion title="I see &#x22;Failed to download chunk ... operation timed out&#x22; in the worker logs">
    This error indicates connection issues. Adjust timeout settings based on your connection quality:

    **If you encounter this frequently:**

    1. Set `S3_TIMEOUT` to `180` in your `.env` file
    2. If the issue persists, also set:
       * `CONCURRENT_DOWNLOADS` to `1`
       * `S3_READ_TIMEOUT` to `30`

    These adjustments reduce concurrent downloads and increase timeout thresholds, helping with slower or less stable connections.
  </Accordion>

  <Accordion title="I see &#x22;Unable to get assignment: deadline has elapsed&#x22; in the worker logs">
    This is a known issue in version 2.0.2.

    **Solution:**

    Set `ASSIGNMENT_FETCH_TIMEOUT_SEC=90` in your `.env` file and restart the worker.
  </Accordion>

  <Accordion title="I see an &#x22;Insufficient peers...&#x22; error in the worker logs">
    This error can be safely ignored. It doesn't affect worker functionality or rewards.
  </Accordion>

  <Accordion title="Can I move my worker to another server?">
    Yes! You can migrate your worker to a new server:

    1. Copy the key file (at `<KEY_PATH>`) to the new server
    2. Set up the new data directory
    3. Start the worker on the new server
    4. Make sure the worker on the old server is not running anymore

    <Info>
      You don't need to re-register your worker when moving to a new server.
    </Info>
  </Accordion>

  <Accordion title="I have just started my worker but see no logs">
    This is normal behavior. Wait a few minutes, the logs will start showing data download activity once the worker begins syncing.
  </Accordion>

  <Accordion title="Should I build from source or run with Docker?">
    **Docker (Recommended)** - Easier setup, suitable for most users

    **Build from Source** - Suitable for experienced Linux/Unix users who need more control or customization

    <Tip>
      Choose Docker unless you have specific requirements that necessitate building
      from source.
    </Tip>
  </Accordion>

  <Accordion title="How do I check that my worker is updated to the latest version?">
    Find your worker in the [network app](https://network.subsquid.io/) and look at the "version" column.
    If the version is outdated, there will be a warning sign besides it.
  </Accordion>

  <Accordion title="Which Linux distro is recommended?">
    We recommend **Ubuntu 22.04 LTS** for optimal compatibility and stability.
  </Accordion>

  <Accordion title="I see &#x22;error from daemon in stream: Error grabbing logs&#x22;">
    This is a Docker issue, not a worker problem.

    **Resources for resolution:**

    * [GitHub issue discussion](https://github.com/docker/for-linux/issues/140)
    * [Stack Overflow thread](https://stackoverflow.com/questions/46843711/docker-error-grabbing-logs-invalid-character-x00-looking-for-beginning-of-v)
  </Accordion>

  <Accordion title="I'm getting 'Clone succeeded, but checkout failed' when cloning the worker repo">
    Manually check out the correct version:

    ```bash theme={"system"}
    cd worker-rs
    git checkout v2.0.0
    ```
  </Accordion>

  <Accordion title="What are the consequences of losing my key file / getting it stolen?">
    **If you lose your key file:**

    * You won't be able to run your worker until you generate and register a new one

    **If your key file is stolen:**

    * The perpetrator can cause connectivity issues for your worker

    **Recovery steps:**

    1. Unregister your worker on the ["Workers" tab](https://network.subsquid.io/workers)
    2. [Generate](#setup-instructions) a new key file
    3. [Register](#worker-registration) your new peer ID

    <Warning>
      Protect your key file! Store it securely and restrict access to prevent these
      issues.
    </Warning>
  </Accordion>
</AccordionGroup>


# Pricing Update: April 2026
Source: https://docs.sqd.dev/en/other/pricing-update-april-2026

Pricing update notice for SQD Cloud effective April 1, 2026 — 20% hosting price increase and revised RPC add-on pricing at $5 per 1M requests.

<Info>
  **Effective Date:** April 1, 2026

  Cloud hosting prices will increase by 20% and RPC add-on pricing will update to \$5 per 1M requests.
</Info>

## What's changing?

We're updating our pricing structure effective **April 1, 2026**. Here's a summary of the changes:

* **Cloud Hosting:** 20% increase on all compute resources (API, Processor, Database, Collocated Squids)
* **Storage:** 20% increase (`$0.50` → `$0.60` per GB/month)
* **RPC Add-on:** Overage pricing increases from `$2.00` to `$5.00` per 1M requests
* **Free Tier:** Unchanged - 2M RPC requests per month remain free

### API & Processor Compute

| Profile | Current Price/hr (until March 31) | New Price/hr (April 1) |
| ------- | --------------------------------- | ---------------------- |
| small   | \$0.04                            | \$0.048                |
| medium  | \$0.08                            | \$0.096                |
| large   | \$0.15                            | \$0.18                 |
| xlarge  | \$0.30                            | \$0.36                 |
| 2xlarge | \$0.60                            | \$0.72                 |

### Database Compute

| Profile | Current Price/hr (until March 31) | New Price/hr (April 1) |
| ------- | --------------------------------- | ---------------------- |
| small   | \$0.08                            | \$0.096                |
| medium  | \$0.16                            | \$0.192                |
| large   | \$0.33                            | \$0.396                |
| xlarge  | \$0.66                            | \$0.792                |
| 2xlarge | \$1.32                            | \$1.584                |

### Collocated Squids

* **Current:** $0.02/hr (~$15/month)
* **New (April 1):** $0.024/hr (~$17/month)

### Storage

* **Current:** \$0.50/GB/month
* **New (April 1):** \$0.60/GB/month

### RPC Requests

* **Free Tier:** 2M requests/month (unchanged)
* **Overage - Current:** \$2.00 per 1M requests
* **Overage - New (April 1):** \$5.00 per 1M requests

## When does this take effect?

The new pricing will be effective **April 1, 2026**. All usage from that date forward will be billed at the new rates.

* **Until March 31, 2026:** Current pricing remains in effect
* **From April 1, 2026:** New pricing applies to all usage

<Tip>
  You can review your current usage and estimate the impact of these changes in your [Cloud Dashboard billing section](https://app.subsquid.io/billing/invoices).
</Tip>

## Need help?

We understand that pricing changes may raise questions. Our team is here to help:

* **Schedule a call:** [Book time with our team](https://calendly.com/t-tyrie-subsquid/30min) to discuss your usage and explore options
* **Email support:** Reach out to [contact@subsquid.io](mailto:contact@subsquid.io) with any questions
* **Dashboard:** Review your usage patterns and projected costs in your [billing dashboard](https://app.subsquid.io/billing/invoices)

For the latest pricing details, visit our [Pricing Overview](/en/cloud/pricing/overview) page.


# EVM Portal API Reference
Source: https://docs.sqd.dev/en/portal/evm/api

Quick reference for the SQD Portal EVM HTTP API — endpoints, request schemas, and code samples for querying blocks, transactions, logs, traces, and state.

This page provides a quick reference for the Portal API. For the complete OpenAPI specification with all fields and options:

<Card title="Complete EVM API Specification" icon="book-open" href="/en/portal/evm/api">
  View the full OpenAPI documentation with all available fields, filters, and
  detailed schemas →
</Card>

## Base URL

All Portal requests are made to:

```
https://portal.sqd.dev/datasets/{network}
```

Replace `{network}` with your target blockchain (e.g., `ethereum-mainnet`, `polygon-mainnet`, `arbitrum-one`).

## Core Endpoints

<CardGroup>
  <Card title="POST /stream" icon="wave-pulse">
    Stream blockchain data with filters. Includes unfinalized blocks.
  </Card>

  <Card title="POST /finalized-stream" icon="check-circle">
    Stream only finalized blocks. Use for production pipelines requiring
    certainty.
  </Card>

  <Card title="GET /metadata" icon="info-circle">
    Get dataset information: block range, finality rules, and network details.
  </Card>
</CardGroup>

## Installation

```bash theme={"system"}
npm install @subsquid/portal-client
```

## Quick Start Examples

<CodeGroup>
  ```bash curl theme={"system"}
  curl --compressed -X POST 'https://portal.sqd.dev/datasets/ethereum-mainnet/stream' \
    -H 'Content-Type: application/json' \
    -d '{
      "type": "evm",
      "fromBlock": 18000000,
      "toBlock": 18001000,
      "fields": {
        "block": { "number": true, "timestamp": true },
        "log": { "address": true, "topics": true, "data": true }
      }
    }'
  ```

  ```typescript TypeScript theme={"system"}
  import { DataSource } from "@subsquid/portal-client";

  const dataSource = new DataSource({
    network: "ethereum-mainnet",
  });

  const blocks = await dataSource.getBlocks({
    from: 18000000,
    to: 18001000,
    fields: {
      block: { number: true, timestamp: true },
      log: { address: true, topics: true, data: true },
    },
  });
  ```

  ```python Python theme={"system"}
  import requests

  url = "https://portal.sqd.dev/datasets/ethereum-mainnet/stream"
  headers = {"Content-Type": "application/json"}
  payload = {
      "type": "evm",
      "fromBlock": 18000000,
      "toBlock": 18001000,
      "fields": {
          "block": {"number": True, "timestamp": True},
          "log": {"address": True, "topics": True, "data": True}
      }
  }

  response = requests.post(url, headers=headers, json=payload)
  ```
</CodeGroup>

## Available Fields

### Block Fields

Select which block header fields to retrieve:

```typescript theme={"system"}
fields: {
  block: {
    number: true,          // Block number
    hash: true,            // Block hash
    parentHash: true,      // Parent block hash
    timestamp: true,       // Unix timestamp
    size: true,            // Block size in bytes
    gasUsed: true,         // Total gas used
    gasLimit: true,        // Gas limit
    baseFeePerGas: true,   // Base fee (EIP-1559)
    stateRoot: true,       // State root hash
    transactionsRoot: true,// Transactions root
    receiptsRoot: true,    // Receipts root
    logsBloom: true,       // Logs bloom filter
    difficulty: true,      // Difficulty
    totalDifficulty: true, // Total difficulty
    miner: true,           // Block producer address
    nonce: true,           // Block nonce
    mixHash: true,         // Mix hash
    extraData: true,       // Extra data
  }
}
```

### Transaction Fields

Select transaction data fields:

```typescript theme={"system"}
fields: {
  transaction: {
    hash: true,            // Transaction hash
    from: true,            // Sender address
    to: true,              // Recipient address (null for deployments)
    value: true,           // ETH value transferred
    input: true,           // Transaction input data
    nonce: true,           // Sender nonce
    gas: true,             // Gas limit
    gasPrice: true,        // Gas price (legacy)
    maxFeePerGas: true,    // Max fee (EIP-1559)
    maxPriorityFeePerGas: true, // Priority fee (EIP-1559)
    gasUsed: true,         // Actual gas used
    cumulativeGasUsed: true, // Cumulative gas in block
    effectiveGasPrice: true, // Effective gas price paid
    status: true,          // Status (1 = success, 0 = failure)
    type: true,            // Transaction type
    chainId: true,         // Chain ID
    v: true,               // Signature V
    r: true,               // Signature R
    s: true,               // Signature S
    yParity: true,         // Y parity (EIP-2930/1559)
    contractAddress: true, // Deployed contract address
    transactionIndex: true,// Index in block
  }
}
```

### Log Fields

Select event log fields:

```typescript theme={"system"}
fields: {
  log: {
    address: true,         // Contract address
    topics: true,          // Event topics (topic0-topic3)
    data: true,            // Event data
    logIndex: true,        // Log index in block
    transactionIndex: true,// Transaction index
    transactionHash: true, // Transaction hash
  }
}
```

### Trace Fields

Select call trace fields:

```typescript theme={"system"}
fields: {
  trace: {
    type: true,            // Trace type (call, delegatecall, etc.)
    from: true,            // Caller address
    to: true,              // Callee address
    value: true,           // Value transferred
    input: true,           // Call input data
    output: true,          // Call output data
    gas: true,             // Gas provided
    gasUsed: true,         // Gas actually used
    error: true,           // Error message (if failed)
    revertReason: true,    // Revert reason (if reverted)
    createResultAddress: true, // Created contract address
    createResultCode: true,    // Created contract code
  }
}
```

### State Diff Fields

Select state change fields:

```typescript theme={"system"}
fields: {
  stateDiff: {
    address: true,         // Contract address
    key: true,             // Storage key
    kind: true,            // Change type (=, +, *, -)
    prev: true,            // Previous value
    next: true,            // New value
  }
}
```

## Filtering Options

### Log Filters

Filter logs by contract address and event topics:

```typescript theme={"system"}
logs: [
  {
    address: ["0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48"], // USDC
    topic0: [
      "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef",
    ], // Transfer
    topic1: ["0x..."], // Optional: first indexed param
    topic2: ["0x..."], // Optional: second indexed param
    topic3: ["0x..."], // Optional: third indexed param
  },
  // Multiple filter objects = OR logic
  {
    address: ["0x..."],
    topic0: ["0x..."],
  },
];
```

**Filter logic:**

* Multiple addresses in one object = OR
* Multiple topics in one object = AND
* Multiple filter objects = OR

### Transaction Filters

Filter transactions by sender, recipient, or function signature:

```typescript theme={"system"}
transactions: [
  {
    from: ["0x..."], // Sender addresses
    to: ["0x..."], // Recipient addresses
    sighash: ["0xa9059cbb"], // Function signature (first 4 bytes)
  },
];
```

### Trace Filters

Filter traces by type, caller, or callee:

```typescript theme={"system"}
traces: [
  {
    type: ["call", "delegatecall", "staticcall"],
    callFrom: ["0x..."], // Caller address
    callTo: ["0x..."], // Callee address
    callSighash: ["0x..."], // Function signature
  },
];
```

### State Diff Filters

Filter state changes by contract address:

```typescript theme={"system"}
stateDiffs: [
  {
    address: ["0x..."], // Contract addresses
  },
];
```

## Network Selection

Specify the EVM network when creating the data source:

```typescript theme={"system"}
// Ethereum mainnet
const eth = new DataSource({ network: "ethereum-mainnet" });

// Ethereum testnet
const sepolia = new DataSource({ network: "ethereum-sepolia" });

// Layer 2 networks
const arbitrum = new DataSource({ network: "arbitrum-one" });
const optimism = new DataSource({ network: "optimism-mainnet" });
const base = new DataSource({ network: "base-mainnet" });
const polygon = new DataSource({ network: "polygon-mainnet" });

// Other EVM chains
const bsc = new DataSource({ network: "binance-mainnet" });
const avalanche = new DataSource({ network: "avalanche-mainnet" });
```

<Card title="View All Networks" icon="database" href="/en/data/networks/evm">
  See complete list of 100+ supported EVM networks
</Card>

## Query Patterns

### Pattern 1: Range Query

Query a specific block range:

```typescript theme={"system"}
const blocks = await dataSource.getBlocks({
  from: 18000000,
  to: 18100000,
  fields: {
    /* fields */
  },
  logs: [
    /* filters */
  ],
});
```

### Pattern 2: Batch Processing

Process large ranges in batches:

```typescript theme={"system"}
const BATCH_SIZE = 10000;

for (let i = startBlock; i < endBlock; i += BATCH_SIZE) {
  const blocks = await dataSource.getBlocks({
    from: i,
    to: Math.min(i + BATCH_SIZE, endBlock),
    fields: {
      /* fields */
    },
  });

  // Process blocks
  await processBlocks(blocks);
}
```

### Pattern 3: Multi-Contract Query

Query multiple contracts simultaneously:

```typescript theme={"system"}
const blocks = await dataSource.getBlocks({
  from: 18000000,
  to: 18100000,
  fields: {
    log: { address: true, topics: true, data: true },
  },
  logs: [
    {
      address: [
        "0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48", // USDC
        "0xdAC17F958D2ee523a2206206994597C13D831ec7", // USDT
        "0x6B175474E89094C44Da98b954EedeAC495271d0F", // DAI
      ],
      topic0: [
        "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef",
      ],
    },
  ],
});
```

## Performance Optimization

### 1. Request Only Needed Fields

```typescript theme={"system"}
// Good: Minimal fields
fields: {
  log: {
    address: true,
    topics: true,
  }
}

// Avoid: Requesting unnecessary fields
fields: {
  log: {
    address: true,
    topics: true,
    data: true,        // Only if needed
    transactionHash: true, // Only if needed
    blockNumber: true,     // Only if needed
  }
}
```

### 2. Use Specific Filters

```typescript theme={"system"}
// Good: Filter at Portal level
logs: [
  {
    address: ["0xSpecificContract"],
    topic0: ["0xSpecificEvent"],
  },
];

// Avoid: Fetching everything and filtering in code
logs: []; // Gets all logs
```

### 3. Optimal Batch Sizes

```typescript theme={"system"}
// Good: 10k-50k blocks per query for most use cases
const BATCH_SIZE = 10000;

// Adjust based on:
// - More filters = larger batches OK
// - More fields = smaller batches
// - More activity (mainnet) = smaller batches
```

### 4. Parallel Queries

```typescript theme={"system"}
// Process multiple ranges in parallel
const ranges = [
  { from: 18000000, to: 18100000 },
  { from: 18100000, to: 18200000 },
  { from: 18200000, to: 18300000 },
];

const results = await Promise.all(
  ranges.map((range) =>
    dataSource.getBlocks({
      from: range.from,
      to: range.to,
      fields: {
        /* fields */
      },
    })
  )
);
```

## Stream continuation

A single `/stream` response is a **batch**, not the complete range. The server may close the connection at any point: when a worker's range ends, when a connection is recycled, or at the current dataset head. Clients that treat one HTTP response as the whole query will silently drop blocks.

To stream a range to completion, loop:

```typescript theme={"system"}
async function streamRange(from: number, to: number | undefined, body: any) {
  let currentFrom = from;
  while (to === undefined || currentFrom <= to) {
    const res = await fetch(
      "https://portal.sqd.dev/datasets/ethereum-mainnet/stream",
      {
        method: "POST",
        headers: { "Content-Type": "application/json" },
        body: JSON.stringify({ ...body, fromBlock: currentFrom, toBlock: to }),
      }
    );

    if (res.status === 204) break; // range is above the dataset head
    if (!res.ok) throw new Error(`HTTP ${res.status}: ${await res.text()}`);

    const lines = (await res.text()).trim().split("\n").filter(Boolean);
    if (lines.length === 0) break;

    let lastBlock = currentFrom - 1;
    for (const line of lines) {
      const block = JSON.parse(line);
      // ... process block ...
      lastBlock = block.header.number;
    }
    currentFrom = lastBlock + 1;
  }
}
```

Key points:

* Read the last block's `header.number`, issue the next request with `fromBlock = lastNumber + 1`.
* Keep `toBlock`, `fields`, and filters identical across retries.
* A `204 No Content` response means the range is above the dataset head. For historical queries, you're done; for open-ended queries, wait and retry.

## Handling reorgs near the chain head

When a request starts at or near the chain tip, the block you asked for may no longer be on the canonical chain by the time Portal processes the query. To detect this, pass `parentBlockHash` with each request (the hash of the block immediately before `fromBlock`):

```typescript theme={"system"}
const body = {
  type: "evm",
  fromBlock: lastBlock.number + 1,
  parentBlockHash: lastBlock.hash, // hash of lastBlock, i.e. parent of fromBlock
  fields: { block: { number: true, hash: true, parentHash: true } },
  logs: [
    /* filters */
  ],
};
```

If the chain has reorged away from `parentBlockHash`, Portal returns **HTTP 409** with a list of blocks that **are** on the current canonical chain:

```json theme={"system"}
{
  "previousBlocks": [
    { "number": 24928291, "hash": "0x129bc9..." },
    { "number": 24928290, "hash": "0x8af2be..." }
  ]
}
```

Client recovery:

1. Find a block in your local state whose `(number, hash)` matches one of `previousBlocks`.
2. Roll back your pipeline to that block.
3. Resume the stream with `fromBlock = matched.number + 1` and `parentBlockHash = matched.hash`.

Streams opened against `/finalized-stream` (instead of `/stream`) never return 409: only finalized blocks are served, so there's nothing to reorg. Use that endpoint when you don't need real-time data.

## Error Handling

```typescript theme={"system"}
try {
  const blocks = await dataSource.getBlocks({
    from: 18000000,
    to: 18100000,
    fields: {
      /* fields */
    },
  });
} catch (error) {
  if (error.code === "RATE_LIMIT_EXCEEDED") {
    // HTTP 429: Public Portal rate limit. Honor Retry-After if set.
    console.error("Rate limit exceeded, waiting...");
    await new Promise((resolve) => setTimeout(resolve, 10000));
  } else if (error.code === "CONFLICT") {
    // HTTP 409: chain reorg. See "Handling reorgs near the chain head" above.
    const { previousBlocks } = error.body;
    // Roll back to a common ancestor and resume.
  } else if (error.code === "INVALID_BLOCK_RANGE") {
    // HTTP 400: fromBlock is below the dataset's start_block, or the query is malformed.
    console.error("Invalid block range");
  } else if (error.code === "NETWORK_ERROR") {
    // Transport error, retry with backoff.
    console.error("Network error, retrying...");
  } else {
    // HTTP 500: do not retry. HTTP 503: retry later, honor Retry-After.
    console.error("Unknown error:", error);
  }
}
```

## Common Use Cases

### Use Case 1: Event Monitoring

Monitor specific smart contract events:

```typescript theme={"system"}
const blocks = await dataSource.getBlocks({
  from: latestProcessedBlock + 1,
  fields: {
    block: { number: true, timestamp: true },
    log: { address: true, topics: true, data: true },
  },
  logs: [
    {
      address: [CONTRACT_ADDRESS],
      topic0: [EVENT_SIGNATURE],
    },
  ],
});
```

### Use Case 2: Transaction Tracking

Track transactions to/from specific addresses:

```typescript theme={"system"}
const blocks = await dataSource.getBlocks({
  from: startBlock,
  to: endBlock,
  fields: {
    transaction: { hash: true, from: true, to: true, value: true },
  },
  transactions: [
    {
      from: [WATCHED_ADDRESS],
    },
  ],
});
```

### Use Case 3: Analytics Pipeline

Build analytics on blockchain data:

```typescript theme={"system"}
const blocks = await dataSource.getBlocks({
  from: startBlock,
  to: endBlock,
  fields: {
    block: { number: true, timestamp: true, gasUsed: true },
    transaction: { gasUsed: true, value: true },
  },
});

// Aggregate statistics
const stats = blocks.map((block) => ({
  block: block.header.number,
  txCount: block.transactions.length,
  totalGas: block.header.gasUsed,
  totalValue: block.transactions.reduce(
    (sum, tx) => sum + BigInt(tx.value),
    0n
  ),
}));
```

## Request Structure Reference

Every Portal request follows this structure:

```typescript theme={"system"}
{
  type: "evm",              // Chain type
  fromBlock: number,        // Starting block (inclusive)
  toBlock?: number,         // Ending block (inclusive, optional)
  fields: {                 // What data to return
    block?: { ... },
    transaction?: { ... },
    log?: { ... },
    trace?: { ... },
    stateDiff?: { ... }
  },
  logs?: [...],            // Filter by event logs
  transactions?: [...],    // Filter by transactions
  traces?: [...],          // Filter by call traces
  stateDiffs?: [...]       // Filter by state changes
}
```

## Response Format

Portal returns **newline-delimited JSON (NDJSON)**, with one block per line:

```json theme={"system"}
{"header": {"number": 18000000, ...}, "logs": [...], "transactions": [...]}
{"header": {"number": 18000001, ...}, "logs": [...], "transactions": [...]}
{"header": {"number": 18000002, ...}, "logs": [...], "transactions": [...]}
```

This format enables constant-memory streaming of arbitrary ranges.

## Common Patterns

<AccordionGroup>
  <Accordion title="Batch Processing">
    Process large block ranges by splitting into manageable chunks:

    ```typescript theme={"system"}
    const BATCH_SIZE = 10000;
    for (let from = startBlock; from < endBlock; from += BATCH_SIZE) {
      const to = Math.min(from + BATCH_SIZE, endBlock);
      const blocks = await fetch(/* ... */);
    }
    ```
  </Accordion>

  <Accordion title="Real-time Streaming">
    Stream new blocks as they arrive using /finalized-stream:

    ```typescript theme={"system"}
    const response = await fetch(
      'https://portal.sqd.dev/datasets/ethereum-mainnet/finalized-stream',
      { method: 'POST', body: JSON.stringify({
        type: 'evm',
        fromBlock: latestProcessedBlock + 1,
        fields: { /* ... */ }
      })}
    );
    ```
  </Accordion>

  <Accordion title="Error Handling">
    Handle rate limits and network errors gracefully:

    ```typescript theme={"system"}
    try {
      const response = await fetch(/* ... */);
      if (!response.ok) {
        if (response.status === 429) {
          // Rate limited - wait and retry
          await sleep(10000);
        }
      }
    } catch (error) {
      // Network error - implement retry logic
    }
    ```
  </Accordion>
</AccordionGroup>

## Complete API Specification

For the complete OpenAPI specification with all available fields, options, and detailed schemas:

<Card title="Full EVM API Specification" icon="file-code" href="/en/portal/evm/api">
  View complete OpenAPI documentation with all endpoints, fields, and options →
</Card>

## Next Steps

<CardGroup>
  <Card title="View Examples" icon="code" href="/en/portal/evm/quickstart">
    Explore practical Portal examples for common use cases
  </Card>

  <Card title="Supported Networks" icon="database" href="/en/data/networks/evm">
    See complete list of 100+ supported EVM networks
  </Card>

  <Card title="Quickstart Guide" icon="rocket" href="/en/portal/evm/quickstart">
    Make your first Portal request in 5 minutes
  </Card>
</CardGroup>


# Monitor Contract Deployments
Source: https://docs.sqd.dev/en/portal/evm/examples/contract-deployments

Track new EVM smart contract deployments through the SQD Portal — sample queries to detect contract creation transactions across any supported chain.

Find all new smart contract deployments in a block range to track protocol launches, analyze deployment patterns, or monitor for security research.

## Use Case

Contract deployment monitoring helps you:

* Track new protocol and dApp launches
* Analyze deployment patterns and trends
* Monitor for security research and auditing
* Build contract discovery tools

## Code Example

<CodeGroup>
  ```bash curl theme={"system"}
  curl --compress -X POST 'https://portal.sqd.dev/datasets/ethereum-mainnet/stream' \
    -H 'Content-Type: application/json' \
    -d '{
      "type": "evm",
      "fromBlock": 20000000,
      "toBlock": 20000003,
      "fields": {
        "block": {
          "number": true,
          "timestamp": true
        },
        "trace": {
          "type": true,
          "createFrom": true,
          "createResultAddress": true
        }
      },
      "traces": [{
        "type": ["create"]
      }]
    }'
  ```

  ```typescript TypeScript theme={"system"}
  import { DataSource } from "@subsquid/portal-client";

  const dataSource = new DataSource({
    network: "ethereum-mainnet",
  });

  const blocks = await dataSource.getBlocks({
    from: 20000000,
    to: 20000003,
    fields: {
      block: { number: true, timestamp: true },
      trace: {
        type: true,
        createFrom: true,
        createResultAddress: true,
      },
    },
    traces: [
      {
        type: ["create"],
      },
    ],
  });

  // Process deployments from traces
  // Traces with type "create" indicate contract deployments
  for (const block of blocks) {
    for (const trace of block.traces) {
      if (trace.type === "create") {
        console.log({
          blockNumber: block.header.number,
          timestamp: block.header.timestamp,
          deployer: trace.createFrom,
          contractAddress: trace.result?.address || trace.createResultAddress,
          type: trace.type,
        });
      }
    }
  }
  ```

  ```python Python theme={"system"}
  import requests
  import json

  url = "https://portal.sqd.dev/datasets/ethereum-mainnet/stream"
  headers = {"Content-Type": "application/json"}

  payload = {
      "type": "evm",
      "fromBlock": 20000000,
      "toBlock": 20000003,
      "fields": {
          "block": {
              "number": True,
              "timestamp": True
          },
          "trace": {
              "type": True,
              "createFrom": True,
              "createResultAddress": True
          }
      },
      "traces": [{
          "type": ["create"]
      }]
  }

  response = requests.post(url, headers=headers, json=payload)

  # Process deployments from traces
  # Traces with type "create" indicate contract deployments
  for line in response.text.strip().split('\n'):
      block = json.loads(line)
      for trace in block.get('traces', []):
          if trace.get('type') == 'create':
              print({
                  "blockNumber": block['header']['number'],
                  "timestamp": block['header']['timestamp'],
                  "deployer": trace.get('createFrom'),
                  "contractAddress": (trace.get('result') or {}).get('address') or trace.get('createResultAddress'),
                  "type": trace.get('type')
              })
  ```
</CodeGroup>

<Info>Try it yourself with the interactive query interface below:</Info>

<QueryInterfaceContractDeployments />

## Key Parameters

| Parameter             | Description                                          |
| --------------------- | ---------------------------------------------------- |
| `type`                | Trace type: `"create"` for contract deployments      |
| `createResultAddress` | Address of the newly deployed contract               |
| `createFrom`          | Deployer address (caller of CREATE)                  |
| `trace`               | Internal transaction trace showing contract creation |

<Note>
  Use traces with `type: ["create"]` to filter for contract deployments. The `createResultAddress` field contains the newly deployed contract address.
</Note>

## Expected Output

```json theme={"system"}
{
  "header": {
    "number": 20000001,
    "timestamp": 1705173443
  },
  "traces": [
    {
      "type": "create",
      "createFrom": "0xa9d1e08c7793af67e9d92fe308d5697fb81d3e43",
      "result": {
        "address": "0x742d35Cc6634C0532925a3b844Bc9e7595f0bEb7"
      }
    }
  ]
}
```

## Filter by Deployer

Track deployments from a specific address:

<CodeGroup>
  ```bash curl theme={"system"}
  curl --compress -X POST 'https://portal.sqd.dev/datasets/ethereum-mainnet/stream' \
    -H 'Content-Type: application/json' \
    -d '{
      "type": "evm",
      "fromBlock": 20000000,
      "toBlock": 20000003,
      "fields": {
        "transaction": {
          "hash": true,
          "from": true,
          "contractAddress": true
        }
      },
      "transactions": [{
        "from": ["0x1234567890123456789012345678901234567890"]
      }]
    }'
  ```

  ```typescript TypeScript theme={"system"}
  const DEPLOYER = "0x1234567890123456789012345678901234567890";

  const blocks = await dataSource.getBlocks({
    from: 20000000,
    to: 20000003,
    fields: {
      transaction: { hash: true, from: true, contractAddress: true },
    },
    transactions: [
      {
        from: [DEPLOYER],
      },
    ],
  });

  // Filter for deployments (contractAddress is populated)
  for (const block of blocks) {
    for (const tx of block.transactions) {
      if (tx.contractAddress) {
        console.log({
          deployer: tx.from,
          contractAddress: tx.contractAddress,
          txHash: tx.hash,
        });
      }
    }
  }
  ```

  ```python Python theme={"system"}
  deployer = "0x1234567890123456789012345678901234567890"

  payload = {
      "type": "evm",
      "fromBlock": 20000000,
      "toBlock": 20000003,
      "fields": {
          "transaction": {
              "hash": True,
              "from": True,
              "contractAddress": True
          }
      },
      "transactions": [{
          "from": [deployer]
      }]
  }

  response = requests.post(url, headers=headers, json=payload)

  # Filter for deployments (contractAddress is populated)
  for line in response.text.strip().split('\n'):
      block = json.loads(line)
      for tx in block.get('transactions', []):
          if tx.get('contractAddress'):
              print({
                  "deployer": tx['from'],
                  "contractAddress": tx['contractAddress'],
                  "txHash": tx['hash']
              })
  ```
</CodeGroup>

## Analyze Bytecode Size

Group deployments by bytecode size to find patterns:

<CodeGroup>
  ```typescript TypeScript theme={"system"}
  const deployments = [];

  for (const block of blocks) {
    for (const tx of block.transactions) {
      if (tx.contractAddress) {
        const bytecodeSize = tx.input.length / 2 - 1; // Convert hex to bytes
        
        deployments.push({
          address: tx.contractAddress,
          deployer: tx.from,
          bytecodeSize,
          block: block.header.number,
        });
      }
    }
  }

  // Find large contracts (>24KB, near the limit)
  const largeContracts = deployments.filter(d => d.bytecodeSize > 24000);
  console.log(`Found ${largeContracts.length} contracts near the 24KB limit`);
  ```
</CodeGroup>

<Tip>
  Ethereum has a 24KB contract size limit. Contracts approaching this limit may use proxy patterns or optimizations.
</Tip>

## Detect Proxy Patterns

Identify potential proxy deployments by bytecode patterns:

<CodeGroup>
  ```typescript TypeScript theme={"system"}
  // Simple heuristic: small bytecode often indicates a proxy
  const MIN_PROXY_SIZE = 50;   // Minimal proxy is ~45 bytes
  const MAX_PROXY_SIZE = 500;  // Most proxies are under 500 bytes

  for (const block of blocks) {
    for (const tx of block.transactions) {
      if (tx.contractAddress) {
        const bytecodeSize = tx.input.length / 2 - 1;
        
        if (bytecodeSize >= MIN_PROXY_SIZE && bytecodeSize <= MAX_PROXY_SIZE) {
          console.log({
            possibleProxy: tx.contractAddress,
            bytecodeSize,
            deployer: tx.from,
          });
        }
      }
    }
  }
  ```
</CodeGroup>

## Track Factory Deployments

Monitor contracts deployed by factory contracts:

<CodeGroup>
  ```typescript TypeScript theme={"system"}
  // Track contracts created via factory patterns
  // Factory deployments appear as internal transactions (traces)

  const blocks = await dataSource.getBlocks({
    from: 20000000,
    to: 20000003,
    fields: {
      trace: {
        type: true,
        from: true,
        createResultAddress: true,
        createResultCode: true,
      },
    },
    traces: [
      {
        type: ["create"],
        callFrom: ["0xYourFactoryAddress"],
      },
    ],
  });

  for (const block of blocks) {
    for (const trace of block.traces) {
      if (trace.createResultAddress) {
        console.log({
          factory: trace.callFrom || trace.createFrom,
          deployed: trace.createResultAddress,
          type: trace.type,
        });
      }
    }
  }
  ```
</CodeGroup>

<Warning>
  Factory-deployed contracts don't appear as regular transactions. Use trace queries with `type: ["create"]` instead.
</Warning>

## Performance Tips

1. **Request minimal fields**: Omit `input` if you don't need bytecode
2. **Use block ranges**: Process 10k-50k blocks at a time
3. **Filter by deployer**: When tracking specific addresses
4. **Post-process bytecode**: Analyze bytecode in your application

## Related Examples

<CardGroup>
  <Card title="Query Transactions" icon="arrow-right-arrow-left" href="/en/portal/evm/examples/query-transactions">
    Monitor transaction activity
  </Card>

  <Card title="Query Traces" icon="diagram-project" href="/en/portal/evm/examples/query-traces">
    Track factory deployments with traces
  </Card>

  <Card title="Query Event Logs" icon="file-lines" href="/en/portal/evm/examples/query-logs">
    Monitor contract events
  </Card>

  <Card title="API Reference" icon="book" href="/en/portal/evm/api">
    View complete API docs
  </Card>
</CardGroup>


# Index EVM DEX Swaps
Source: https://docs.sqd.dev/en/portal/evm/examples/dex-swaps

Track EVM DEX trading activity through the SQD Portal — sample queries for Uniswap, SushiSwap, and other decentralized exchange swap events.

Track Uniswap V3 swap events to extract real-time price data, calculate trading volumes, and analyze on-chain liquidity dynamics. Unlike centralized exchanges, DEX data is verifiable and tamper-proof, ideal for building trustless price oracles and analytics.

## Use Case

DEX swap indexing unlocks powerful analytics:

* **Price Discovery**: Extract real-time token prices from swap amounts without relying on third-party APIs
* **Volume Analysis**: Calculate 24h trading volumes, identify trending pairs, and detect unusual activity
* **Liquidity Tracking**: Monitor pool depth changes and identify liquidity migration patterns
* **MEV Detection**: Analyze sandwich attacks, arbitrage opportunities, and front-running patterns by correlating swaps within blocks

## Code Example

<CodeGroup>
  ```bash curl theme={"system"}
  curl --compress -X POST 'https://portal.sqd.dev/datasets/ethereum-mainnet/stream' \
    -H 'Content-Type: application/json' \
    -d '{
      "type": "evm",
      "fromBlock": 12369621,
      "toBlock": 12400000,
      "fields": {
        "block": {
          "number": true,
          "timestamp": true
        },
        "log": {
          "address": true,
          "topics": true,
          "data": true,
          "transactionHash": true
        }
      },
      "logs": [{
        "topic0": ["0xc42079f94a6350d7e6235f29174924f928cc2ac818eb64fed8004e115fbcca67"]
      }]
    }'
  ```

  ```typescript TypeScript theme={"system"}
  import { DataSource } from "@subsquid/portal-client";

  const dataSource = new DataSource({
    network: "ethereum-mainnet",
  });

  // Uniswap V3 Swap event signature
  const SWAP_TOPIC = "0xc42079f94a6350d7e6235f29174924f928cc2ac818eb64fed8004e115fbcca67";

  const blocks = await dataSource.getBlocks({
    from: 12369621, // Uniswap V3 deployment block
    to: 12400000,
    fields: {
      block: { number: true, timestamp: true },
      log: {
        address: true,
        topics: true,
        data: true,
        transactionHash: true,
      },
    },
    logs: [
      {
        topic0: [SWAP_TOPIC],
      },
    ],
  });

  // Process swaps
  for (const block of blocks) {
    for (const log of block.logs) {
      const sender = `0x${log.topics[1].slice(26)}`;
      const recipient = `0x${log.topics[2].slice(26)}`;
      
      console.log({
        blockNumber: block.header.number,
        timestamp: block.header.timestamp,
        pool: log.address,
        sender,
        recipient,
        txHash: log.transactionHash,
        // Decode amounts from data field
      });
    }
  }
  ```

  ```python Python theme={"system"}
  import requests
  import json

  url = "https://portal.sqd.dev/datasets/ethereum-mainnet/stream"
  headers = {"Content-Type": "application/json"}

  # Uniswap V3 Swap event
  swap_topic = "0xc42079f94a6350d7e6235f29174924f928cc2ac818eb64fed8004e115fbcca67"

  payload = {
      "type": "evm",
      "fromBlock": 12369621,  # Uniswap V3 deployment
      "toBlock": 12400000,
      "fields": {
          "block": {
              "number": True,
              "timestamp": True
          },
          "log": {
              "address": True,
              "topics": True,
              "data": True,
              "transactionHash": True
          }
      },
      "logs": [{
          "topic0": [swap_topic]
      }]
  }

  response = requests.post(url, headers=headers, json=payload)

  for line in response.text.strip().split('\n'):
      block = json.loads(line)
      for log in block.get('logs', []):
          sender = "0x" + log['topics'][1][26:]
          recipient = "0x" + log['topics'][2][26:]
          
          print({
              "blockNumber": block['header']['number'],
              "timestamp": block['header']['timestamp'],
              "pool": log['address'],
              "sender": sender,
              "recipient": recipient,
              "txHash": log['transactionHash']
          })
  ```
</CodeGroup>

<Info>Try it yourself with the interactive query interface below:</Info>

<QueryInterfaceDexSwaps />

## Key Parameters

| Parameter   | Description                                                                         |
| ----------- | ----------------------------------------------------------------------------------- |
| `topic0`    | Swap event signature (`Swap(address,address,int256,int256,uint160,uint128,int24)`)  |
| `topics[1]` | Sender address (who initiated the swap)                                             |
| `topics[2]` | Recipient address (who receives the output)                                         |
| `data`      | Swap amounts and other parameters (amount0, amount1, sqrtPriceX96, liquidity, tick) |
| `address`   | Pool contract address                                                               |

<Note>
  The Swap event signature varies by DEX. Uniswap V3 uses a different signature than Uniswap V2 or other DEXs.
</Note>

## Expected Output

```json theme={"system"}
{
  "header": {
    "number": 12369650,
    "timestamp": 1620054321
  },
  "logs": [
    {
      "address": "0x8ad599c3a0ff1de082011efddc58f1908eb6e6d8",
      "topics": [
        "0xc42079f94a6350d7e6235f29174924f928cc2ac818eb64fed8004e115fbcca67",
        "0x00000000000000000000000068b3465833fb72a70ecdf485e0e4c7bd8665fc45",
        "0x00000000000000000000000028c6c06298d514db089934071355e5743bf21d60"
      ],
      "data": "0x...",
      "transactionHash": "0x123..."
    }
  ]
}
```

## Track Specific Pool

Monitor swaps for a specific trading pair:

<CodeGroup>
  ```bash curl theme={"system"}
  curl --compress -X POST 'https://portal.sqd.dev/datasets/ethereum-mainnet/stream' \
    -H 'Content-Type: application/json' \
    -d '{
      "type": "evm",
      "fromBlock": 18000000,
      "toBlock": 18010000,
      "fields": {
        "log": {
          "address": true,
          "topics": true,
          "data": true
        }
      },
      "logs": [{
        "address": ["0x8ad599c3A0ff1De082011EFDDc58f1908eb6e6D8"],
        "topic0": ["0xc42079f94a6350d7e6235f29174924f928cc2ac818eb64fed8004e115fbcca67"]
      }]
    }'
  ```

  ```typescript TypeScript theme={"system"}
  const USDC_WETH_POOL = "0x8ad599c3A0ff1De082011EFDDc58f1908eb6e6D8"; // Uniswap V3 USDC/WETH

  const blocks = await dataSource.getBlocks({
    from: 18000000,
    to: 18010000,
    fields: {
      log: { address: true, topics: true, data: true },
    },
    logs: [
      {
        address: [USDC_WETH_POOL],
        topic0: ["0xc42079f94a6350d7e6235f29174924f928cc2ac818eb64fed8004e115fbcca67"],
      },
    ],
  });
  ```

  ```python Python theme={"system"}
  usdc_weth_pool = "0x8ad599c3A0ff1De082011EFDDc58f1908eb6e6D8"

  payload = {
      "type": "evm",
      "fromBlock": 18000000,
      "toBlock": 18010000,
      "fields": {
          "log": {
              "address": True,
              "topics": True,
              "data": True
          }
      },
      "logs": [{
          "address": [usdc_weth_pool],
          "topic0": ["0xc42079f94a6350d7e6235f29174924f928cc2ac818eb64fed8004e115fbcca67"]
      }]
  }
  ```
</CodeGroup>

## Uniswap V2 Swaps

For Uniswap V2, use the V2 Swap event signature:

<CodeGroup>
  ```typescript TypeScript theme={"system"}
  // Uniswap V2 Swap event: Swap(address,uint256,uint256,uint256,uint256,address)
  const V2_SWAP_TOPIC = "0xd78ad95fa46c994b6551d0da85fc275fe613ce37657fb8d5e3d130840159d822";

  const blocks = await dataSource.getBlocks({
    from: 18000000,
    to: 18010000,
    fields: {
      log: { address: true, topics: true, data: true },
    },
    logs: [
      {
        topic0: [V2_SWAP_TOPIC],
      },
    ],
  });
  ```
</CodeGroup>

<Tip>
  Different DEX protocols use different event signatures. Check the contract ABI to find the correct signature for your target DEX.
</Tip>

## Decode Swap Amounts and Calculate Prices

The real value in DEX data comes from decoding swap amounts to calculate prices. Here's how to extract and interpret Uniswap V3 swap data:

<CodeGroup>
  ```typescript TypeScript theme={"system"}
  import { ethers } from "ethers";

  // Uniswap V3 Swap event ABI
  const swapEventAbi = [
    "event Swap(address indexed sender, address indexed recipient, int256 amount0, int256 amount1, uint160 sqrtPriceX96, uint128 liquidity, int24 tick)"
  ];

  const iface = new ethers.Interface(swapEventAbi);

  for (const block of blocks) {
    for (const log of block.logs) {
      const decoded = iface.parseLog({
        topics: log.topics,
        data: log.data,
      });
      
      const amount0 = decoded.args.amount0;
      const amount1 = decoded.args.amount1;
      const sqrtPriceX96 = decoded.args.sqrtPriceX96;
      
      // Calculate the actual price from sqrtPriceX96
      // price = (sqrtPriceX96 / 2^96)^2
      const price = (Number(sqrtPriceX96) / (2 ** 96)) ** 2;
      
      // Determine swap direction and calculate effective price
      const isBuyingToken0 = amount0 < 0n;
      const effectivePrice = isBuyingToken0
        ? Math.abs(Number(amount1)) / Math.abs(Number(amount0))
        : Math.abs(Number(amount0)) / Math.abs(Number(amount1));
      
      console.log({
        pool: log.address,
        blockNumber: block.header.number,
        timestamp: block.header.timestamp,
        amount0: amount0.toString(),
        amount1: amount1.toString(),
        currentPrice: price,
        effectivePrice: effectivePrice,
        liquidity: decoded.args.liquidity.toString(),
        tick: decoded.args.tick,
      });
    }
  }
  ```

  ```python Python theme={"system"}
  from eth_abi import decode

  # Decode data field (amounts and other params)
  # Uniswap V3: (int256 amount0, int256 amount1, uint160 sqrtPriceX96, uint128 liquidity, int24 tick)
  types = ['int256', 'int256', 'uint160', 'uint128', 'int24']

  for line in response.text.strip().split('\n'):
      block = json.loads(line)
      for log in block.get('logs', []):
          decoded = decode(types, bytes.fromhex(log['data'][2:]))
          amount0, amount1, sqrtPriceX96, liquidity, tick = decoded
          
          print({
              "amount0": amount0,
              "amount1": amount1,
              "sqrtPriceX96": sqrtPriceX96,
              "liquidity": liquidity,
              "tick": tick
          })
  ```
</CodeGroup>

## Advanced Analysis: Detecting MEV Sandwich Attacks

Sandwich attacks are a common MEV strategy where an attacker places trades before and after a victim's transaction to profit from price impact. By analyzing swap sequences within blocks, you can detect these patterns:

```typescript theme={"system"}
// Query swaps with transaction details to detect sandwich attacks
const blocks = await dataSource.getBlocks({
  from: 18000000,
  to: 18010000,
  fields: {
    block: { number: true, timestamp: true },
    transaction: { hash: true, from: true, to: true, transactionIndex: true },
    log: { address: true, topics: true, data: true, transactionHash: true, logIndex: true },
  },
  logs: [{ topic0: [SWAP_TOPIC] }],
  includeAllBlocks: false,
});

// Detect sandwich attacks by analyzing swap sequences
for (const block of blocks) {
  // Group swaps by pool address
  const poolSwaps = new Map();
  
  for (const log of block.logs) {
    const tx = block.transactions.find(t => t.hash === log.transactionHash);
    if (!tx) continue;
    
    if (!poolSwaps.has(log.address)) {
      poolSwaps.set(log.address, []);
    }
    
    poolSwaps.get(log.address).push({
      txIndex: tx.transactionIndex,
      txHash: tx.hash,
      trader: tx.from,
      log: log,
    });
  }
  
  // Check for sandwich pattern: same trader with swaps before and after victim
  for (const [pool, swaps] of poolSwaps) {
    if (swaps.length < 3) continue;
    
    swaps.sort((a, b) => a.txIndex - b.txIndex);
    
    for (let i = 0; i < swaps.length - 2; i++) {
      const first = swaps[i];
      const middle = swaps[i + 1];
      const last = swaps[i + 2];
      
      // Sandwich pattern: same trader surrounds victim's trade
      if (first.trader === last.trader && middle.trader !== first.trader) {
        console.log({
          block: block.header.number,
          pool: pool,
          attacker: first.trader,
          victim: middle.trader,
          frontrunTx: first.txHash,
          victimTx: middle.txHash,
          backrunTx: last.txHash,
        });
      }
    }
  }
}
```

<Warning>
  MEV detection requires analyzing transaction ordering within blocks. Make sure to request `transaction` fields and `transactionIndex` to properly sequence trades.
</Warning>

## Advanced Analysis: Multi-Hop Swaps and Arbitrage

Many swaps route through multiple pools via aggregators like 1inch or Uniswap's Universal Router. By combining swap events with traces, you can reconstruct complete arbitrage paths and calculate profitability:

```typescript theme={"system"}
// Query both swap events and traces to reconstruct multi-hop routes
const blocks = await dataSource.getBlocks({
  from: 18000000,
  to: 18010000,
  fields: {
    block: { number: true, timestamp: true },
    transaction: { hash: true, from: true, to: true },
    log: { address: true, topics: true, data: true, transactionHash: true },
    trace: { callFrom: true, callTo: true, callValue: true, transactionHash: true },
  },
  logs: [{ topic0: [SWAP_TOPIC] }],
  traces: [{ type: ["call"] }],
});

// Group swaps by transaction to identify arbitrage opportunities
const swapsByTx = new Map();
for (const block of blocks) {
  for (const log of block.logs) {
    if (!swapsByTx.has(log.transactionHash)) {
      swapsByTx.set(log.transactionHash, {
        swaps: [],
        traces: [],
        tx: block.transactions.find(t => t.hash === log.transactionHash),
      });
    }
    swapsByTx.get(log.transactionHash).swaps.push(log);
  }
  
  // Add traces for each transaction
  for (const trace of block.traces) {
    if (swapsByTx.has(trace.transactionHash)) {
      swapsByTx.get(trace.transactionHash).traces.push(trace);
    }
  }
}

// Analyze multi-hop swaps and detect circular arbitrage
for (const [txHash, data] of swapsByTx) {
  if (data.swaps.length > 1) {
    const pools = data.swaps.map(s => s.address);
    const uniquePools = new Set(pools);
    
    // Circular arbitrage: starts and ends with same token
    if (uniquePools.size >= 3) {
      console.log({
        txHash,
        trader: data.tx?.from,
        pools: pools,
        hopCount: data.swaps.length,
        type: 'potential-arbitrage',
      });
    }
  }
}
```

<Tip>
  Combine swap events with traces to capture the full execution path, including intermediate token approvals and transfers that don't emit swap events.
</Tip>

## Performance Tips

1. **Filter by pool address**: Query specific high-volume pools (USDC/WETH, WETH/USDT) instead of all swaps
2. **Use block ranges**: Process 10k-50k block batches for optimal throughput
3. **Batch decode**: Decode multiple events in parallel rather than sequentially
4. **Cache pool metadata**: Store token decimals and symbols to avoid repeated lookups
5. **Combine with traces**: Track multi-hop swaps and MEV activity by analyzing call traces alongside events

## Related Examples

<CardGroup>
  <Card title="Query Event Logs" icon="file-lines" href="/en/portal/evm/examples/query-logs">
    General event log querying
  </Card>

  <Card title="Query Traces" icon="diagram-project" href="/en/portal/evm/examples/query-traces">
    Track multi-hop swaps with traces
  </Card>

  <Card title="Track NFT Transfers" icon="image" href="/en/portal/evm/examples/nft-transfers">
    Monitor NFT marketplace activity
  </Card>

  <Card title="API Reference" icon="book" href="/en/portal/evm/api">
    View complete API docs
  </Card>
</CardGroup>


# Track NFT Transfers
Source: https://docs.sqd.dev/en/portal/evm/examples/nft-transfers

Monitor ERC-721 and ERC-1155 NFT transfers through the SQD Portal — sample queries for tracking ownership changes, mints, and burns at scale.

Monitor ERC-721 Transfer events across multiple NFT collections to track collection activity, analyze whale movements, or build NFT analytics platforms.

## Use Case

NFT transfer tracking enables you to:

* Build NFT transfer trackers and analytics
* Monitor collection activity and floor prices
* Track whale movements across collections
* Analyze trading patterns and market trends

## Code Example

<CodeGroup>
  ```bash curl theme={"system"}
  curl --compressed -X POST 'https://portal.sqd.dev/datasets/ethereum-mainnet/stream' \
    -H 'Content-Type: application/json' \
    -d '{
      "type": "evm",
      "fromBlock": 18000000,
      "toBlock": 18010000,
      "fields": {
        "block": {
          "number": true,
          "timestamp": true
        },
        "log": {
          "address": true,
          "topics": true,
          "transactionHash": true
        }
      },
      "logs": [{
        "address": [
          "0xBC4CA0EdA7647A8aB7C2061c2E118A18a936f13D",
          "0x60E4d786628Fea6478F785A6d7e704777c86a7c6",
          "0xb47e3cd837dDF8e4c57F05d70Ab865de6e193BBB"
        ],
        "topic0": ["0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef"]
      }]
    }'
  ```

  ```typescript TypeScript theme={"system"}
  import { DataSource } from "@subsquid/portal-client";

  const dataSource = new DataSource({
    network: "ethereum-mainnet",
  });

  // Popular NFT collections
  const NFT_CONTRACTS = [
    "0xBC4CA0EdA7647A8aB7C2061c2E118A18a936f13D", // BAYC
    "0x60E4d786628Fea6478F785A6d7e704777c86a7c6", // MAYC
    "0xb47e3cd837dDF8e4c57F05d70Ab865de6e193BBB", // CryptoPunks
  ];

  const TRANSFER_TOPIC = "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef";

  const blocks = await dataSource.getBlocks({
    from: 18000000,
    to: 18010000,
    fields: {
      block: { number: true, timestamp: true },
      log: {
        address: true,
        topics: true,
        transactionHash: true,
      },
    },
    logs: [
      {
        address: NFT_CONTRACTS,
        topic0: [TRANSFER_TOPIC],
      },
    ],
  });

  // Process NFT transfers
  for (const block of blocks) {
    for (const log of block.logs) {
      const from = `0x${log.topics[1].slice(26)}`;
      const to = `0x${log.topics[2].slice(26)}`;
      const tokenId = BigInt(log.topics[3]);
      
      console.log({
        blockNumber: block.header.number,
        collection: log.address,
        from,
        to,
        tokenId: tokenId.toString(),
        txHash: log.transactionHash,
      });
    }
  }
  ```

  ```python Python theme={"system"}
  import requests
  import json

  url = "https://portal.sqd.dev/datasets/ethereum-mainnet/stream"
  headers = {"Content-Type": "application/json"}

  # Popular NFT collections
  nft_contracts = [
      "0xBC4CA0EdA7647A8aB7C2061c2E118A18a936f13D",  # BAYC
      "0x60E4d786628Fea6478F785A6d7e704777c86a7c6",  # MAYC
      "0xb47e3cd837dDF8e4c57F05d70Ab865de6e193BBB"   # CryptoPunks
  ]

  payload = {
      "type": "evm",
      "fromBlock": 18000000,
      "toBlock": 18010000,
      "fields": {
          "block": {
              "number": True,
              "timestamp": True
          },
          "log": {
              "address": True,
              "topics": True,
              "transactionHash": True
          }
      },
      "logs": [{
          "address": nft_contracts,
          "topic0": ["0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef"]
      }]
  }

  response = requests.post(url, headers=headers, json=payload)

  for line in response.text.strip().split('\n'):
      block = json.loads(line)
      for log in block.get('logs', []):
          from_address = "0x" + log['topics'][1][26:]
          to_address = "0x" + log['topics'][2][26:]
          token_id = int(log['topics'][3], 16)
          
          print({
              "blockNumber": block['header']['number'],
              "collection": log['address'],
              "from": from_address,
              "to": to_address,
              "tokenId": token_id,
              "txHash": log['transactionHash']
          })
  ```
</CodeGroup>

<Info>Try it yourself with the interactive query interface below:</Info>

<QueryInterfaceNftTransfers />

## Key Parameters

| Parameter   | Description                                                        |
| ----------- | ------------------------------------------------------------------ |
| `address`   | NFT collection contract addresses. Multiple addresses use OR logic |
| `topic0`    | Transfer event signature (same for ERC-721 and ERC-20)             |
| `topics[1]` | From address (indexed parameter)                                   |
| `topics[2]` | To address (indexed parameter)                                     |
| `topics[3]` | Token ID (indexed parameter in ERC-721)                            |

<Tip>
  ERC-721 Transfer events have the token ID in `topics[3]`. ERC-20 transfers don't have this field (amount is in `data` instead).
</Tip>

## Expected Output

```json theme={"system"}
{
  "header": {
    "number": 18000123,
    "timestamp": 1697544779
  },
  "logs": [
    {
      "address": "0xbc4ca0eda7647a8ab7c2061c2e118a18a936f13d",
      "topics": [
        "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef",
        "0x000000000000000000000000a9d1e08c7793af67e9d92fe308d5697fb81d3e43",
        "0x00000000000000000000000028c6c06298d514db089934071355e5743bf21d60",
        "0x0000000000000000000000000000000000000000000000000000000000000001"
      ],
      "transactionHash": "0x123..."
    }
  ]
}
```

## Track Specific Wallet Activity

Monitor NFT transfers for a specific wallet address:

<CodeGroup>
  ```bash curl theme={"system"}
  curl --compressed -X POST 'https://portal.sqd.dev/datasets/ethereum-mainnet/stream' \
    -H 'Content-Type: application/json' \
    -d '{
      "type": "evm",
      "fromBlock": 18000000,
      "toBlock": 18010000,
      "fields": {
        "log": {
          "address": true,
          "topics": true,
          "transactionHash": true
        }
      },
      "logs": [{
        "address": ["0xBC4CA0EdA7647A8aB7C2061c2E118A18a936f13D"],
        "topic0": ["0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef"],
        "topic2": ["0x00000000000000000000000028c6c06298d514db089934071355e5743bf21d60"]
      }]
    }'
  ```

  ```typescript TypeScript theme={"system"}
  const WALLET = "0x28c6c06298d514db089934071355e5743bf21d60";

  const blocks = await dataSource.getBlocks({
    from: 18000000,
    to: 18010000,
    fields: {
      log: { address: true, topics: true, transactionHash: true },
    },
    logs: [
      {
        address: ["0xBC4CA0EdA7647A8aB7C2061c2E118A18a936f13D"], // BAYC
        topic0: ["0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef"],
        topic2: [`0x000000000000000000000000${WALLET.slice(2)}`], // To address
      },
    ],
  });
  ```

  ```python Python theme={"system"}
  wallet = "0x28c6c06298d514db089934071355e5743bf21d60"

  payload = {
      "type": "evm",
      "fromBlock": 18000000,
      "toBlock": 18010000,
      "fields": {
          "log": {
              "address": True,
              "topics": True,
              "transactionHash": True
          }
      },
      "logs": [{
          "address": ["0xBC4CA0EdA7647A8aB7C2061c2E118A18a936f13D"],
          "topic0": ["0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef"],
          "topic2": [f"0x000000000000000000000000{wallet[2:]}"]
      }]
  }
  ```
</CodeGroup>

## Track Minting Events

Monitor NFT mints (transfers from null address):

<CodeGroup>
  ```typescript TypeScript theme={"system"}
  const blocks = await dataSource.getBlocks({
    from: 18000000,
    to: 18010000,
    fields: {
      log: { address: true, topics: true },
    },
    logs: [
      {
        address: ["0xBC4CA0EdA7647A8aB7C2061c2E118A18a936f13D"],
        topic0: ["0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef"],
        topic1: ["0x0000000000000000000000000000000000000000000000000000000000000000"], // From null
      },
    ],
  });
  ```
</CodeGroup>

<Note>
  Minting is represented as a transfer from the zero address (`0x000...000`).
</Note>

## ERC-1155 Transfers

For ERC-1155 NFTs, use different event signatures:

<CodeGroup>
  ```typescript TypeScript theme={"system"}
  // ERC-1155 TransferSingle event
  const TRANSFER_SINGLE = "0xc3d58168c5ae7397731d063d5bbf3d657854427343f4c083240f7aacaa2d0f62";

  // ERC-1155 TransferBatch event
  const TRANSFER_BATCH = "0x4a39dc06d4c0dbc64b70af90fd698a233a518aa5d07e595d983b8c0526c8f7fb";

  const blocks = await dataSource.getBlocks({
    from: 18000000,
    to: 18010000,
    fields: {
      log: { address: true, topics: true, data: true },
    },
    logs: [
      {
        address: ["0x..."], // ERC-1155 contract
        topic0: [TRANSFER_SINGLE, TRANSFER_BATCH],
      },
    ],
  });
  ```
</CodeGroup>

## Performance Tips

1. **Filter by collection**: Specify exact contract addresses
2. **Use topic filters**: Filter by from/to addresses at Portal level
3. **Batch processing**: Query 10k-50k blocks at a time
4. **Minimal fields**: Request only needed fields

## Related Examples

<CardGroup>
  <Card title="Query Event Logs" icon="file-lines" href="/en/portal/evm/examples/query-logs">
    General event log querying
  </Card>

  <Card title="Index DEX Swaps" icon="repeat" href="/en/portal/evm/examples/dex-swaps">
    Track DEX trading activity
  </Card>

  <Card title="Query Transactions" icon="arrow-right-arrow-left" href="/en/portal/evm/examples/query-transactions">
    Monitor transaction activity
  </Card>

  <Card title="API Reference" icon="book" href="/en/portal/evm/api">
    View complete API docs
  </Card>
</CardGroup>


# Query Event Logs
Source: https://docs.sqd.dev/en/portal/evm/examples/query-logs

Track EVM smart contract event logs — ERC-20 transfers, custom events — through the SQD Portal with topic filters and address-based selectors at scale.

Monitor USDC Transfer events on Ethereum mainnet to build a token tracker or analyze transfer patterns.

## Use Case

Event logs are emitted by smart contracts when specific actions occur. This example tracks all USDC Transfer events, which you can use to:

* Build token transfer trackers
* Analyze trading patterns
* Monitor wallet activity
* Track liquidity flows

## Code Example

<CodeGroup>
  ```bash curl theme={"system"}
  curl --compressed -X POST 'https://portal.sqd.dev/datasets/ethereum-mainnet/stream' \
    -H 'Content-Type: application/json' \
    -d '{
      "type": "evm",
      "fromBlock": 18000000,
      "toBlock": 18100000,
      "fields": {
        "block": {
          "number": true,
          "timestamp": true
        },
        "log": {
          "address": true,
          "topics": true,
          "data": true,
          "transactionHash": true,
          "logIndex": true
        }
      },
      "logs": [{
        "address": ["0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48"],
        "topic0": ["0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef"]
      }]
    }'
  ```

  ```typescript TypeScript theme={"system"}
  import { DataSource } from "@subsquid/portal-client";

  const dataSource = new DataSource({
    network: "ethereum-mainnet",
  });

  // USDC contract address
  const USDC_ADDRESS = "0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48";
  // Transfer(address,address,uint256) event signature
  const TRANSFER_TOPIC =
    "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef";

  const blocks = await dataSource.getBlocks({
    from: 18000000,
    to: 18100000,
    fields: {
      block: { number: true, timestamp: true },
      log: {
        address: true,
        topics: true,
        data: true,
        transactionHash: true,
        logIndex: true,
      },
    },
    logs: [
      {
        address: [USDC_ADDRESS],
        topic0: [TRANSFER_TOPIC],
      },
    ],
  });

  // Process the logs
  for (const block of blocks) {
    for (const log of block.logs) {
      const from = `0x${log.topics[1].slice(26)}`;
      const to = `0x${log.topics[2].slice(26)}`;

      console.log({
        blockNumber: block.header.number,
        timestamp: block.header.timestamp,
        txHash: log.transactionHash,
        from,
        to,
        // Decode amount from data field as needed
      });
    }
  }
  ```

  ```python Python theme={"system"}
  import requests
  import json

  url = "https://portal.sqd.dev/datasets/ethereum-mainnet/stream"
  headers = {"Content-Type": "application/json"}

  payload = {
      "type": "evm",
      "fromBlock": 18000000,
      "toBlock": 18100000,
      "fields": {
          "block": {
              "number": True,
              "timestamp": True
          },
          "log": {
              "address": True,
              "topics": True,
              "data": True,
              "transactionHash": True,
              "logIndex": True
          }
      },
      "logs": [{
          "address": ["0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48"],
          "topic0": ["0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef"]
      }]
  }

  response = requests.post(url, headers=headers, json=payload)

  for line in response.text.strip().split('\n'):
      block = json.loads(line)
      for log in block.get('logs', []):
          from_address = "0x" + log['topics'][1][26:]
          to_address = "0x" + log['topics'][2][26:]

          print({
              "blockNumber": block['header']['number'],
              "timestamp": block['header']['timestamp'],
              "txHash": log['transactionHash'],
              "from": from_address,
              "to": to_address
          })
  ```
</CodeGroup>

<Info>Try it yourself with the interactive query interface below:</Info>

<QueryInterfaceQueryLogs />

## Key Parameters

| Parameter | Description                                                                                             |
| --------- | ------------------------------------------------------------------------------------------------------- |
| `address` | Contract address to filter logs. Array supports multiple addresses (OR logic)                           |
| `topic0`  | Event signature (keccak256 hash of event definition). For Transfer: `Transfer(address,address,uint256)` |
| `topic1`  | First indexed parameter (from address in Transfer events)                                               |
| `topic2`  | Second indexed parameter (to address in Transfer events)                                                |
| `topics`  | Array containing all event topics. topic\[0] is signature, topic\[1-3] are indexed params               |
| `data`    | Non-indexed event parameters (amount in Transfer events)                                                |

<Tip>
  To get the event signature, compute
  `keccak256("Transfer(address,address,uint256)")`. The result is
  `0xddf252ad...`.
</Tip>

## Expected Output

Each line in the response represents one block containing matching logs:

```json theme={"system"}
{
  "header": {
    "number": 18000123,
    "timestamp": 1697544779
  },
  "logs": [
    {
      "address": "0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48",
      "topics": [
        "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef",
        "0x000000000000000000000000a9d1e08c7793af67e9d92fe308d5697fb81d3e43",
        "0x00000000000000000000000028c6c06298d514db089934071355e5743bf21d60"
      ],
      "data": "0x0000000000000000000000000000000000000000000000000000000002faf080",
      "transactionHash": "0x123...",
      "logIndex": 45
    }
  ]
}
```

## Filtering Multiple Contracts

Query logs from multiple ERC-20 tokens:

<CodeGroup>
  ```bash curl theme={"system"}
  curl --compressed -X POST 'https://portal.sqd.dev/datasets/ethereum-mainnet/stream' \
    -H 'Content-Type: application/json' \
    -d '{
      "type": "evm",
      "fromBlock": 18000000,
      "toBlock": 18010000,
      "fields": {
        "log": {
          "address": true,
          "topics": true,
          "data": true
        }
      },
      "logs": [{
        "address": [
          "0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48",
          "0xdAC17F958D2ee523a2206206994597C13D831ec7",
          "0x6B175474E89094C44Da98b954EedeAC495271d0F"
        ],
        "topic0": ["0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef"]
      }]
    }'
  ```

  ```typescript TypeScript theme={"system"}
  const blocks = await dataSource.getBlocks({
    from: 18000000,
    to: 18010000,
    fields: {
      log: { address: true, topics: true, data: true },
    },
    logs: [
      {
        address: [
          "0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48", // USDC
          "0xdAC17F958D2ee523a2206206994597C13D831ec7", // USDT
          "0x6B175474E89094C44Da98b954EedeAC495271d0F", // DAI
        ],
        topic0: [
          "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef",
        ],
      },
    ],
  });
  ```

  ```python Python theme={"system"}
  payload = {
      "type": "evm",
      "fromBlock": 18000000,
      "toBlock": 18010000,
      "fields": {
          "log": {
              "address": True,
              "topics": True,
              "data": True
          }
      },
      "logs": [{
          "address": [
              "0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48",  # USDC
              "0xdAC17F958D2ee523a2206206994597C13D831ec7",  # USDT
              "0x6B175474E89094C44Da98b954EedeAC495271d0F"   # DAI
          ],
          "topic0": ["0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef"]
      }]
  }
  ```
</CodeGroup>

## Performance Tips

1. **Request only needed fields**: Don't request `data` if you only need addresses
2. **Use specific filters**: Filter by contract address and event signature
3. **Batch appropriately**: Query 10k-50k blocks at a time for best performance
4. **Filter by indexed parameters**: Use topic1, topic2, topic3 to narrow results

## Related Examples

<CardGroup>
  <Card title="Query Transactions" icon="arrow-right-arrow-left" href="/en/portal/evm/examples/query-transactions">
    Monitor transaction activity
  </Card>

  <Card title="Track NFT Transfers" icon="image" href="/en/portal/evm/examples/nft-transfers">
    Monitor NFT collection events
  </Card>

  <Card title="Index DEX Swaps" icon="repeat" href="/en/portal/evm/examples/dex-swaps">
    Track DEX trading activity
  </Card>

  <Card title="API Reference" icon="book" href="/en/portal/evm/api">
    View complete API docs
  </Card>
</CardGroup>


# Query Traces
Source: https://docs.sqd.dev/en/portal/evm/examples/query-traces

Analyze EVM internal transactions and call traces through the SQD Portal — sample queries for tracing nested contract calls and reverted operations.

Track internal transactions (traces) to analyze complex contract interactions and value flows through multiple contracts.

## Use Case

Call traces help you:

* Analyze DEX trades with multiple hops
* Track value flows through proxy contracts
* Monitor internal contract calls
* Debug complex contract interactions

## Code Example

<CodeGroup>
  ```bash curl theme={"system"}
  curl --compress -X POST 'https://portal.sqd.dev/datasets/ethereum-mainnet/stream' \
    -H 'Content-Type: application/json' \
    -d '{
      "type": "evm",
      "fromBlock": 18000000,
      "toBlock": 18010000,
      "fields": {
        "block": {
          "number": true
        },
        "trace": {
          "type": true,
          "callFrom": true,
          "callTo": true,
          "callValue": true,
          "callInput": true,
          "callResultOutput": true,
          "callResultGasUsed": true,
          "error": true
        }
      },
      "traces": [{
        "type": ["call"]
      }]
    }'
  ```

  ```typescript TypeScript theme={"system"}
  import { DataSource } from "@subsquid/portal-client";

  const dataSource = new DataSource({
    network: "ethereum-mainnet",
  });

  const blocks = await dataSource.getBlocks({
    from: 18000000,
    to: 18010000,
    fields: {
      block: { number: true },
      trace: {
        type: true,
        callFrom: true,
        callTo: true,
        callValue: true,
        callInput: true,
        callResultOutput: true,
        callResultGasUsed: true,
        error: true,
      },
    },
    traces: [
      {
        type: ["call"],
      },
    ],
  });

  // Process traces
  for (const block of blocks) {
    for (const trace of block.traces) {
      if (!trace.error) {
        console.log({
          blockNumber: block.header.number,
          type: trace.type,
          from: trace.callFrom,
          to: trace.callTo,
          value: trace.callValue,
          gasUsed: trace.callResultGasUsed,
        });
      }
    }
  }
  ```

  ```python Python theme={"system"}
  import requests
  import json

  url = "https://portal.sqd.dev/datasets/ethereum-mainnet/stream"
  headers = {"Content-Type": "application/json"}

  payload = {
      "type": "evm",
      "fromBlock": 18000000,
      "toBlock": 18010000,
      "fields": {
          "block": {
              "number": True
          },
          "trace": {
              "type": True,
              "callFrom": True,
              "callTo": True,
              "callValue": True,
              "callInput": True,
              "callResultOutput": True,
              "callResultGasUsed": True,
              "error": True
          }
      },
      "traces": [{
          "type": ["call"]
      }]
  }

  response = requests.post(url, headers=headers, json=payload)

  for line in response.text.strip().split('\n'):
      block = json.loads(line)
      for trace in block.get('traces', []):
          if not trace.get('error'):
              print({
                  "blockNumber": block['header']['number'],
                  "type": trace['type'],
                  "from": trace['callFrom'],
                  "to": trace['callTo'],
                  "value": trace['callValue'],
                  "gasUsed": trace['callResultGasUsed']
              })
  ```
</CodeGroup>

<Info>Try it yourself with the interactive query interface below:</Info>

<QueryInterfaceQueryTraces />

## Key Parameters

| Parameter           | Description                                                            |
| ------------------- | ---------------------------------------------------------------------- |
| `type`              | Trace type: `call` (contract calls) or `create` (contract deployments) |
| `callFrom`          | Caller address                                                         |
| `callTo`            | Callee address                                                         |
| `callValue`         | ETH value transferred in the call                                      |
| `callInput`         | Call input data                                                        |
| `callResultOutput`  | Call output data                                                       |
| `callResultGasUsed` | Gas used by the call                                                   |
| `error`             | Error message if trace failed                                          |

<Info>
  Portal supports two trace types: `call` for regular contract calls and `create` for contract deployments. Other EVM trace types like `staticcall`, `delegatecall`, and `callcode` are normalized to `call` type.
</Info>

## Expected Output

```json theme={"system"}
{
  "header": {
    "number": 18000123
  },
  "traces": [
    {
      "type": "call",
      "callFrom": "0x1234...",
      "callTo": "0x5678...",
      "callValue": "1000000000000000000",
      "callInput": "0xa9059cbb...",
      "callResultOutput": "0x0000000000000000000000000000000000000000000000000000000000000001",
      "callResultGasUsed": "45123",
      "error": null
    }
  ]
}
```

## Filter by Specific Contract

Track all calls to or from a specific contract:

<CodeGroup>
  ```bash curl theme={"system"}
  curl --compress -X POST 'https://portal.sqd.dev/datasets/ethereum-mainnet/stream' \
    -H 'Content-Type: application/json' \
    -d '{
      "type": "evm",
      "fromBlock": 18000000,
      "toBlock": 18010000,
      "fields": {
        "trace": {
          "type": true,
          "callFrom": true,
          "callTo": true,
          "callValue": true
        }
      },
      "traces": [{
        "callTo": ["0x68b3465833fb72A70ecDF485E0e4C7bD8665Fc45"]
      }]
    }'
  ```

  ```typescript TypeScript theme={"system"}
  const blocks = await dataSource.getBlocks({
    from: 18000000,
    to: 18010000,
    fields: {
      trace: { type: true, callFrom: true, callTo: true, callValue: true },
    },
    traces: [
      {
        callTo: ["0x68b3465833fb72A70ecDF485E0e4C7bD8665Fc45"], // Uniswap Router
      },
    ],
  });
  ```

  ```python Python theme={"system"}
  payload = {
      "type": "evm",
      "fromBlock": 18000000,
      "toBlock": 18010000,
      "fields": {
          "trace": {
              "type": True,
              "callFrom": True,
              "callTo": True,
              "callValue": True
          }
      },
      "traces": [{
          "callTo": ["0x68b3465833fb72A70ecDF485E0e4C7bD8665Fc45"]
      }]
  }
  ```
</CodeGroup>

<Note>
  Traces include both successful and failed calls. Check the `error` field to filter failed traces.
</Note>

## Analyze Value Flows

Track ETH transfers through internal calls:

<CodeGroup>
  ```typescript TypeScript theme={"system"}
  const blocks = await dataSource.getBlocks({
    from: 18000000,
    to: 18010000,
    fields: {
      trace: { type: true, callFrom: true, callTo: true, callValue: true },
    },
    traces: [
      {
        type: ["call"],
      },
    ],
  });

  // Filter traces with value transfers
  for (const block of blocks) {
    for (const trace of block.traces) {
      if (BigInt(trace.callValue) > 0n) {
        console.log(`${trace.callFrom} → ${trace.callTo}: ${trace.callValue} wei`);
      }
    }
  }
  ```
</CodeGroup>

## Performance Tips

1. **Filter by type**: Specify trace types to reduce data volume
2. **Use callTo/callFrom**: Filter by specific contracts
3. **Request minimal fields**: Omit `input` and `output` if not needed
4. **Post-filter errors**: Filter failed traces in your code if needed

<Warning>
  Traces can be very numerous. Use specific filters to avoid processing millions of traces.
</Warning>

## Related Examples

<CardGroup>
  <Card title="Query Transactions" icon="arrow-right-arrow-left" href="/en/portal/evm/examples/query-transactions">
    Monitor transaction activity
  </Card>

  <Card title="Index DEX Swaps" icon="repeat" href="/en/portal/evm/examples/dex-swaps">
    Track DEX trading with traces
  </Card>

  <Card title="Query Event Logs" icon="file-lines" href="/en/portal/evm/examples/query-logs">
    Track smart contract events
  </Card>

  <Card title="API Reference" icon="book" href="/en/portal/evm/api">
    View complete API docs
  </Card>
</CardGroup>


# Query EVM Transactions
Source: https://docs.sqd.dev/en/portal/evm/examples/query-transactions

Monitor EVM wallet activity and transaction flows through the SQD Portal — sample queries for filtering by sender, recipient, and value across chains.

Retrieve all transactions sent to a specific address to track wallet activity, exchange flows, or smart contract interactions.

## Use Case

Transaction queries help you:

* Track incoming transactions to wallets or exchanges
* Monitor smart contract interactions
* Analyze transaction patterns
* Build transaction history feeds

## Code Example

<CodeGroup>
  ```bash curl theme={"system"}
  curl --compressed -X POST 'https://portal.sqd.dev/datasets/ethereum-mainnet/stream' \
    -H 'Content-Type: application/json' \
    -d '{
      "type": "evm",
      "fromBlock": 18000000,
      "toBlock": 18010000,
      "fields": {
        "block": {
          "number": true,
          "timestamp": true
        },
        "transaction": {
          "hash": true,
          "from": true,
          "to": true,
          "value": true,
          "input": true,
          "gasUsed": true,
          "status": true
        }
      },
      "transactions": [{
        "to": ["0xdAC17F958D2ee523a2206206994597C13D831ec7"]
      }]
    }'
  ```

  ```typescript TypeScript theme={"system"}
  import { DataSource } from "@subsquid/portal-client";

  const dataSource = new DataSource({
    network: "ethereum-mainnet",
  });

  const TARGET_ADDRESS = "0xdAC17F958D2ee523a2206206994597C13D831ec7"; // USDT

  const blocks = await dataSource.getBlocks({
    from: 18000000,
    to: 18010000,
    fields: {
      block: { number: true, timestamp: true },
      transaction: {
        hash: true,
        from: true,
        to: true,
        value: true,
        input: true,
        gasUsed: true,
        status: true,
      },
    },
    transactions: [
      {
        to: [TARGET_ADDRESS],
      },
    ],
  });

  // Process transactions
  for (const block of blocks) {
    for (const tx of block.transactions) {
      console.log({
        blockNumber: block.header.number,
        txHash: tx.hash,
        from: tx.from,
        to: tx.to,
        value: tx.value,
        gasUsed: tx.gasUsed,
        success: tx.status === 1,
      });
    }
  }
  ```

  ```python Python theme={"system"}
  import requests
  import json

  url = "https://portal.sqd.dev/datasets/ethereum-mainnet/stream"
  headers = {"Content-Type": "application/json"}

  payload = {
      "type": "evm",
      "fromBlock": 18000000,
      "toBlock": 18010000,
      "fields": {
          "block": {
              "number": True,
              "timestamp": True
          },
          "transaction": {
              "hash": True,
              "from": True,
              "to": True,
              "value": True,
              "input": True,
              "gasUsed": True,
              "status": True
          }
      },
      "transactions": [{
          "to": ["0xdAC17F958D2ee523a2206206994597C13D831ec7"]
      }]
  }

  response = requests.post(url, headers=headers, json=payload)

  for line in response.text.strip().split('\n'):
      block = json.loads(line)
      for tx in block.get('transactions', []):
          print({
              "blockNumber": block['header']['number'],
              "txHash": tx['hash'],
              "from": tx['from'],
              "to": tx['to'],
              "value": tx['value'],
              "gasUsed": tx['gasUsed'],
              "success": tx['status'] == 1
          })
  ```
</CodeGroup>

<Info>Try it yourself with the interactive query interface below:</Info>

<QueryInterfaceQueryTransactions />

## Key Parameters

| Parameter | Description                                                |
| --------- | ---------------------------------------------------------- |
| `from`    | Filter transactions by sender address                      |
| `to`      | Filter transactions by recipient address                   |
| `sighash` | Filter by function signature (first 4 bytes of input data) |
| `status`  | Transaction status: 1 = success, 0 = failure               |
| `value`   | ETH amount transferred (in wei)                            |
| `input`   | Transaction calldata                                       |
| `gasUsed` | Actual gas consumed                                        |

## Expected Output

```json theme={"system"}
{
  "header": {
    "number": 18000123,
    "timestamp": 1697544779
  },
  "transactions": [
    {
      "hash": "0x123...",
      "from": "0xa9d1e08c7793af67e9d92fe308d5697fb81d3e43",
      "to": "0xdac17f958d2ee523a2206206994597c13d831ec7",
      "value": "0",
      "input": "0xa9059cbb000000000000000000000000...",
      "gasUsed": "54234",
      "status": 1
    }
  ]
}
```

## Filter by Function Signature

Query only specific function calls (e.g., ERC-20 `transfer` function):

<CodeGroup>
  ```bash curl theme={"system"}
  curl --compressed -X POST 'https://portal.sqd.dev/datasets/ethereum-mainnet/stream' \
    -H 'Content-Type: application/json' \
    -d '{
      "type": "evm",
      "fromBlock": 18000000,
      "toBlock": 18010000,
      "fields": {
        "transaction": {
          "hash": true,
          "from": true,
          "to": true,
          "input": true
        }
      },
      "transactions": [{
        "to": ["0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48"],
        "sighash": ["0xa9059cbb"]
      }]
    }'
  ```

  ```typescript TypeScript theme={"system"}
  const blocks = await dataSource.getBlocks({
    from: 18000000,
    to: 18010000,
    fields: {
      transaction: { hash: true, from: true, to: true, input: true },
    },
    transactions: [
      {
        to: ["0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48"], // USDC
        sighash: ["0xa9059cbb"], // transfer(address,uint256)
      },
    ],
  });
  ```

  ```python Python theme={"system"}
  payload = {
      "type": "evm",
      "fromBlock": 18000000,
      "toBlock": 18010000,
      "fields": {
          "transaction": {
              "hash": True,
              "from": True,
              "to": True,
              "input": True
          }
      },
      "transactions": [{
          "to": ["0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48"],
          "sighash": ["0xa9059cbb"]
      }]
  }
  ```
</CodeGroup>

<Tip>
  Function signature is computed as `keccak256("transfer(address,uint256)")[0:4]`. The result is `0xa9059cbb`.
</Tip>

## Monitor Outgoing Transactions

Track transactions from a specific wallet:

<CodeGroup>
  ```bash curl theme={"system"}
  curl --compressed -X POST 'https://portal.sqd.dev/datasets/ethereum-mainnet/stream' \
    -H 'Content-Type: application/json' \
    -d '{
      "type": "evm",
      "fromBlock": 18000000,
      "toBlock": 18010000,
      "fields": {
        "transaction": {
          "hash": true,
          "from": true,
          "to": true,
          "value": true
        }
      },
      "transactions": [{
        "from": ["0x28C6c06298d514Db089934071355E5743bf21d60"]
      }]
    }'
  ```

  ```typescript TypeScript theme={"system"}
  const blocks = await dataSource.getBlocks({
    from: 18000000,
    to: 18010000,
    fields: {
      transaction: { hash: true, from: true, to: true, value: true },
    },
    transactions: [
      {
        from: ["0x28C6c06298d514Db089934071355E5743bf21d60"],
      },
    ],
  });
  ```

  ```python Python theme={"system"}
  payload = {
      "type": "evm",
      "fromBlock": 18000000,
      "toBlock": 18010000,
      "fields": {
          "transaction": {
              "hash": True,
              "from": True,
              "to": True,
              "value": True
          }
      },
      "transactions": [{
          "from": ["0x28C6c06298d514Db089934071355E5743bf21d60"]
      }]
  }
  ```
</CodeGroup>

## Performance Tips

1. **Combine filters**: Use both `from` and `to` for bidirectional tracking
2. **Filter by sighash**: Reduces results to specific function calls
3. **Request minimal fields**: Omit `input` if you don't need calldata
4. **Check status**: Filter failed transactions in post-processing if needed

## Related Examples

<CardGroup>
  <Card title="Query Event Logs" icon="file-lines" href="/en/portal/evm/examples/query-logs">
    Track smart contract events
  </Card>

  <Card title="Query Traces" icon="diagram-project" href="/en/portal/evm/examples/query-traces">
    Analyze internal transactions
  </Card>

  <Card title="Contract Deployments" icon="rocket" href="/en/portal/evm/examples/contract-deployments">
    Monitor new contract deployments
  </Card>

  <Card title="API Reference" icon="book" href="/en/portal/evm/api">
    View complete API docs
  </Card>
</CardGroup>


# Query State Diffs
Source: https://docs.sqd.dev/en/portal/evm/examples/state-diffs

Track EVM storage state changes for smart contracts through the SQD Portal — sample queries for analyzing slot writes, account changes, and code updates.

Track storage state changes for smart contracts to monitor specific storage slots, analyze contract state evolution, or track ownership changes.

## Use Case

State diff tracking enables you to:

* Monitor specific storage slots (e.g., owner, balance mappings)
* Track contract state evolution over time
* Analyze state changes without replaying transactions
* Build state-based analytics

## Code Example

<CodeGroup>
  ```bash curl theme={"system"}
  curl --compressed -X POST 'https://portal.sqd.dev/datasets/ethereum-mainnet/stream' \
    -H 'Content-Type: application/json' \
    -d '{
      "type": "evm",
      "fromBlock": 19000000,
      "toBlock": 19000003,
      "fields": {
        "block": {
          "number": true,
          "timestamp": true
        },
        "stateDiff": {
          "address": true,
          "key": true,
          "kind": true,
          "prev": true,
          "next": true
        }
      },
      "stateDiffs": [{
        "address": ["0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48"]
      }]
    }'
  ```

  ```typescript TypeScript theme={"system"}
  import { DataSource } from "@subsquid/portal-client";

  const dataSource = new DataSource({
    network: "ethereum-mainnet",
  });

  const blocks = await dataSource.getBlocks({
    from: 19000000,
    to: 19000003,
    fields: {
      block: { number: true, timestamp: true },
      stateDiff: {
        address: true,
        key: true,
        kind: true,
        prev: true,
        next: true,
      },
    },
    stateDiffs: [
      {
        address: ["0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48"], // USDC
      },
    ],
  });

  // Process state changes
  for (const block of blocks) {
    if (block.stateDiffs) {
      for (const diff of block.stateDiffs) {
        console.log({
          blockNumber: block.header.number,
          contract: diff.address,
          storageKey: diff.key,
          changeType: diff.kind, // "=", "+", "*", "-"
          previousValue: diff.prev,
          newValue: diff.next,
        });
      }
    }
  }
  ```

  ```python Python theme={"system"}
  import requests
  import json

  url = "https://portal.sqd.dev/datasets/ethereum-mainnet/stream"
  headers = {"Content-Type": "application/json"}

  payload = {
      "type": "evm",
      "fromBlock": 19000000,
      "toBlock": 19000003,
      "fields": {
          "block": {
              "number": True,
              "timestamp": True
          },
          "stateDiff": {
              "address": True,
              "key": True,
              "kind": True,
              "prev": True,
              "next": True
          }
      },
      "stateDiffs": [{
          "address": ["0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48"]  # USDC
      }]
  }

  response = requests.post(url, headers=headers, json=payload)

  for line in response.text.strip().split('\n'):
      block = json.loads(line)
      if 'stateDiffs' in block:
          for diff in block['stateDiffs']:
              print({
                  "blockNumber": block['header']['number'],
                  "contract": diff['address'],
                  "storageKey": diff['key'],
                  "changeType": diff['kind'],
                  "previousValue": diff.get('prev'),
                  "newValue": diff.get('next')
              })
  ```
</CodeGroup>

<Info>Try it yourself with the interactive query interface below:</Info>

<QueryInterfaceStateDiffs />

## Key Parameters

| Parameter | Description                                                              |
| --------- | ------------------------------------------------------------------------ |
| `address` | Contract address to track state changes                                  |
| `key`     | Storage slot key (keccak256 hash for mappings)                           |
| `kind`    | Change type: `=` (unchanged), `+` (added), `*` (modified), `-` (removed) |
| `prev`    | Previous value (null for new slots)                                      |
| `next`    | New value (null for removed slots)                                       |

<Note>
  State diffs capture all storage changes that occurred in a block, providing a complete picture of contract state evolution.
</Note>

## Expected Output

```json theme={"system"}
{
  "header": {
    "number": 18000123
  },
  "stateDiffs": [
    {
      "address": "0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48",
      "key": "0x1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef",
      "kind": "*",
      "prev": "0x0000000000000000000000000000000000000000000000000000000000000064",
      "next": "0x00000000000000000000000000000000000000000000000000000000000000c8"
    }
  ]
}
```

## Filter by Contract

Track state changes for a specific contract:

<CodeGroup>
  ```bash curl theme={"system"}
  curl --compressed -X POST 'https://portal.sqd.dev/datasets/ethereum-mainnet/stream' \
    -H 'Content-Type: application/json' \
    -d '{
      "type": "evm",
      "fromBlock": 19000000,
      "toBlock": 19000003,
      "fields": {
        "stateDiff": {
          "address": true,
          "key": true,
          "kind": true,
          "prev": true,
          "next": true
        }
      },
      "stateDiffs": [{
        "address": ["0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48"]
      }]
    }'
  ```

  ```typescript TypeScript theme={"system"}
  const USDC_CONTRACT = "0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48";

  const blocks = await dataSource.getBlocks({
    from: 19000000,
    to: 19000003,
    fields: {
      stateDiff: {
        address: true,
        key: true,
        kind: true,
        prev: true,
        next: true,
      },
    },
    stateDiffs: [
      {
        address: [USDC_CONTRACT],
      },
    ],
  });
  ```

  ```python Python theme={"system"}
  usdc_contract = "0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48"

  payload = {
      "type": "evm",
      "fromBlock": 19000000,
      "toBlock": 19000003,
      "fields": {
          "stateDiff": {
              "address": True,
              "key": True,
              "kind": True,
              "prev": True,
              "next": True
          }
      },
      "stateDiffs": [{
          "address": [usdc_contract]
      }]
  }
  ```
</CodeGroup>

## Understanding Change Types

The `kind` field indicates the type of state change:

| Kind | Meaning   | Description                                   |
| ---- | --------- | --------------------------------------------- |
| `=`  | Unchanged | Storage slot was accessed but not modified    |
| `+`  | Added     | New storage slot created                      |
| `*`  | Modified  | Existing storage slot value changed           |
| `-`  | Removed   | Storage slot deleted (SSTORE with zero value) |

<CodeGroup>
  ```typescript TypeScript theme={"system"}
  // Filter only modifications
  for (const block of blocks) {
    if (block.stateDiffs) {
      for (const diff of block.stateDiffs) {
        if (diff.kind === "*") {
          console.log(`Modified at block ${block.header.number}`);
          console.log(`  ${diff.prev} → ${diff.next}`);
        }
      }
    }
  }
  ```
</CodeGroup>

## Track Specific Storage Slots

Monitor a specific storage slot (e.g., owner variable):

<CodeGroup>
  ```typescript TypeScript theme={"system"}
  // Example: Track owner changes
  // Owner is typically at slot 0 for many contracts
  const OWNER_SLOT = "0x0000000000000000000000000000000000000000000000000000000000000000";

  const blocks = await dataSource.getBlocks({
    from: 19000000,
    to: 19000003,
    fields: {
      block: { number: true, timestamp: true },
      stateDiff: {
        address: true,
        key: true,
        prev: true,
        next: true,
      },
    },
    stateDiffs: [
      {
        address: ["0xYourContractAddress"],
      },
    ],
  });

  for (const block of blocks) {
    if (block.stateDiffs) {
      for (const diff of block.stateDiffs) {
        if (diff.key === OWNER_SLOT) {
          console.log({
            block: block.header.number,
            timestamp: block.header.timestamp,
            previousOwner: diff.prev,
            newOwner: diff.next,
          });
        }
      }
    }
  }
  ```
</CodeGroup>

<Tip>
  For mappings, the storage key is computed as `keccak256(abi.encode(mappingKey, slot))`. You'll need to compute this off-chain to track specific mapping entries.
</Tip>

## Monitor Balance Changes

Track balance mapping changes for ERC-20 tokens:

<CodeGroup>
  ```typescript TypeScript theme={"system"}
  import { ethers } from "ethers";

  // Compute storage key for balance mapping
  function getBalanceSlot(address: string, balanceMappingSlot: number): string {
    // For Solidity: mapping(address => uint256) balances;
    // Key = keccak256(abi.encode(address, slot))
    return ethers.keccak256(
      ethers.AbiCoder.defaultAbiCoder().encode(
        ["address", "uint256"],
        [address, balanceMappingSlot]
      )
    );
  }

  const WALLET = "0x28C6c06298d514Db089934071355E5743bf21d60";
  const BALANCE_SLOT = 1; // Balance mapping slot for USDC

  const balanceKey = getBalanceSlot(WALLET, BALANCE_SLOT);

  // Query state diffs and filter for this specific key
  const blocks = await dataSource.getBlocks({
    from: 19000000,
    to: 19000003,
    fields: {
      stateDiff: { address: true, key: true, prev: true, next: true },
    },
    stateDiffs: [
      {
        address: ["0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48"], // USDC
      },
    ],
  });

  for (const block of blocks) {
    if (block.stateDiffs) {
      for (const diff of block.stateDiffs) {
        if (diff.key === balanceKey) {
          const prevBalance = BigInt(diff.prev || "0");
          const nextBalance = BigInt(diff.next || "0");
          console.log(`Balance change: ${prevBalance} → ${nextBalance}`);
        }
      }
    }
  }
  ```
</CodeGroup>

## Performance Tips

1. **Filter by contract**: Always specify contract addresses to reduce data volume
2. **Post-filter keys**: Filter specific storage keys in your application
3. **Understand data volume**: State diffs can be very large for active contracts
4. **Combine with logs**: Use event logs to identify interesting blocks, then query state diffs

<Warning>
  State diffs can generate massive amounts of data, especially for busy contracts. Always filter by specific contract addresses.
</Warning>

## Common Use Cases

### Track Governance Changes

Monitor governance parameter changes:

```typescript theme={"system"}
// Filter state changes for governance contracts
// Look for proposal state changes, quorum updates, etc.
```

### Monitor Oracle Updates

Track price oracle state updates:

```typescript theme={"system"}
// Track specific storage slots for price data
// Useful for MEV analysis or price feed monitoring
```

### Audit Storage Access

Analyze which storage slots are frequently accessed:

```typescript theme={"system"}
const slotAccessCounts = new Map<string, number>();

for (const block of blocks) {
  if (block.stateDiffs) {
    for (const diff of block.stateDiffs) {
      const key = `${diff.address}:${diff.key}`;
      slotAccessCounts.set(key, (slotAccessCounts.get(key) || 0) + 1);
    }
  }
}
```

## Related Examples

<CardGroup>
  <Card title="Query Event Logs" icon="file-lines" href="/en/portal/evm/examples/query-logs">
    Track smart contract events
  </Card>

  <Card title="Query Transactions" icon="arrow-right-arrow-left" href="/en/portal/evm/examples/query-transactions">
    Monitor transaction activity
  </Card>

  <Card title="Query Traces" icon="diagram-project" href="/en/portal/evm/examples/query-traces">
    Analyze internal transactions
  </Card>

  <Card title="API Reference" icon="book" href="/en/portal/evm/api">
    View complete API docs
  </Card>
</CardGroup>


# EVM Portal API
Source: https://docs.sqd.dev/en/portal/evm/overview

Access raw EVM blockchain data — blocks, transactions, logs, traces, state diffs — through the SQD Portal HTTP API for high-throughput data streaming.

Portal provides direct access to raw EVM blockchain data across 100+ networks. Query blocks, transactions, logs, traces, and state diffs with performance that scales beyond traditional RPC limitations.

## EVM-Specific Features

Portal supports the full range of EVM data types:

<CardGroup>
  <Card title="Event Logs" icon="file-lines">
    Query smart contract events with topic filtering. Track ERC-20 transfers, NFT mints, and custom events.
  </Card>

  <Card title="Transactions" icon="arrow-right-arrow-left">
    Access transaction details including input data, gas usage, and execution
    status. Filter by sender, recipient, or function signature.
  </Card>

  <Card title="Call Traces" icon="diagram-project">
    Deep inspection of internal calls, contract creations, and execution paths.
    Essential for DeFi analytics.
  </Card>

  <Card title="State Diffs" icon="code-compare">
    Track storage changes at the contract level. Monitor state transitions and storage slot modifications.
  </Card>
</CardGroup>

## Quick Start

Get started with Portal in 5 minutes:

<CodeGroup>
  ```bash curl theme={"system"}
  curl --compressed -X POST 'https://portal.sqd.dev/datasets/ethereum-mainnet/stream' \
    -H 'Content-Type: application/json' \
    -d '{
      "type": "evm",
      "fromBlock": 18000000,
      "toBlock": 18001000,
      "fields": {
        "block": { "number": true, "timestamp": true },
        "log": { "address": true, "topics": true, "data": true }
      }
    }'
  ```

  ```typescript TypeScript theme={"system"}
  import { DataSource } from "@subsquid/portal-client";

  const dataSource = new DataSource({
    network: "ethereum-mainnet",
  });

  const blocks = await dataSource.getBlocks({
    from: 18000000,
    to: 18001000,
    fields: {
      block: { number: true, timestamp: true },
      log: { address: true, topics: true, data: true },
    },
  });
  ```

  ```python Python theme={"system"}
  import requests

  url = "https://portal.sqd.dev/datasets/ethereum-mainnet/stream"
  headers = {"Content-Type": "application/json"}
  payload = {
      "type": "evm",
      "fromBlock": 18000000,
      "toBlock": 18001000,
      "fields": {
          "block": {"number": True, "timestamp": True},
          "log": {"address": True, "topics": True, "data": True}
      }
  }

  response = requests.post(url, headers=headers, json=payload)
  ```
</CodeGroup>

## Common Use Cases

<CardGroup>
  <Card title="DeFi Analytics" icon="chart-line" href="/en/portal/evm/examples/dex-swaps">
    Track DEX swaps, lending protocol events, and liquidity changes. Use call traces to analyze complex DeFi interactions.
  </Card>

  <Card title="NFT Indexing" icon="image" href="/en/portal/evm/examples/nft-transfers">
    Monitor NFT mints, transfers, and marketplace activity. Track ERC-721 and
    ERC-1155 events across collections.
  </Card>

  <Card title="Wallet & Transaction Monitoring" icon="wallet" href="/en/portal/evm/examples/query-transactions">
    Track wallet activity, monitor specific addresses, and analyze transaction
    patterns. Essential for compliance and security.
  </Card>

  <Card title="Contract Deployment Tracking" icon="code" href="/en/portal/evm/examples/contract-deployments">
    Monitor new contract deployments, track factory patterns, and analyze deployment trends across networks.
  </Card>
</CardGroup>

## Access Methods

Choose how you want to access Portal:

<AccordionGroup>
  <Accordion title="Public Portal (Free)" icon="globe">
    **Best for:** Development and testing

    * Free access with rate limits (20 requests per 10 seconds)
    * No setup required
    * Access to all supported networks

    **Endpoint:** `https://portal.sqd.dev`
  </Accordion>

  <Accordion title="Cloud Portal" icon="cloud">
    **Best for:** Production applications - Higher rate limits - Integrated with
    SQD Cloud deployment - Pay-as-you-go pricing - Production-ready infrastructure

    <Card title="Learn More" icon="cloud" href="/en/cloud/overview">
      Explore Cloud Portal
    </Card>
  </Accordion>

  <Accordion title="Self-Hosted Portal" icon="server">
    **Best for:** Maximum sovereignty

    * No rate limits
    * Full control over infrastructure
    * Run next to your own node
    * Open-source implementation

    <Card title="Self-Hosting Guide" icon="server" href="/en/portal/self-hosting">
      Set up your Portal instance
    </Card>
  </Accordion>
</AccordionGroup>

## Next Steps

<CardGroup>
  <Card title="Quickstart" icon="rocket" href="/en/portal/evm/quickstart">
    Make your first Portal request in 5 minutes
  </Card>

  <Card title="API Reference" icon="book" href="/en/portal/evm/api">
    Complete API reference with all fields and filters
  </Card>

  <Card title="Examples" icon="code" href="/en/portal/evm/examples/query-logs">
    Practical examples for common use cases
  </Card>
</CardGroup>

<Info>
  **Enterprise Custom Development**

  For enterprise clients requiring custom indexer development, please contact our team to discuss your specific requirements. [Schedule a Consultation →](https://calendly.com/t-tyrie-subsquid/30min)
</Info>


# EVM Portal Quickstart
Source: https://docs.sqd.dev/en/portal/evm/quickstart

Make your first SQD Portal request for EVM data in 5 minutes — install the client, query a block range, and stream transaction logs from Ethereum.

Make your first Portal request to access raw EVM blockchain data in under 5 minutes. No setup required, just HTTP requests.

## What You'll Build

In this quickstart, you'll:

1. Make a simple Portal request using curl (no installation required)
2. Query recent Ethereum blocks
3. Filter event logs from a smart contract

<Info>
  This guide uses the **Public Portal** endpoint, which is free and
  rate-limited. Perfect for getting started.
</Info>

## Step 1: Your First Request

Let's query the last 1,000 blocks from Ethereum mainnet. Copy and paste this command:

<Tabs>
  <Tab title="curl">
    ```bash theme={"system"}
    curl --compressed -X POST 'https://portal.sqd.dev/datasets/ethereum-mainnet/stream' \
      -H 'Content-Type: application/json' \
      -d '{
        "type": "evm",
        "fromBlock": 18000000,
        "toBlock": 18001000,
        "includeAllBlocks": true,
        "fields": {
          "block": {
            "number": true,
            "timestamp": true,
            "gasUsed": true
          }
        }
      }'
    ```

    <Check>
      You should see a stream of JSON objects, one per block, showing block numbers,
      timestamps, and gas usage.
    </Check>

    <Note>
      `includeAllBlocks: true` tells Portal to return every block in the range.
      Without it (the default on EVM), Portal returns **only blocks matching your
      data filters** (`logs`, `transactions`, `traces`, `stateDiffs`). This request
      has no filters, so without `includeAllBlocks` it would return just the
      boundary blocks of the underlying workers, not the full range. See [Stream
      semantics](#stream-semantics) below.
    </Note>

    <Info>Try it yourself with the interactive query interface below:</Info>

    <QueryInterfaceQuickstart />
  </Tab>

  <Tab title="TypeScript">
    First, install the client:

    ```bash theme={"system"}
    npm install @subsquid/portal-client
    ```

    Then run this code:

    ```typescript theme={"system"}
    import { DataSource } from "@subsquid/portal-client";

    const dataSource = new DataSource({
      network: "ethereum-mainnet",
    });

    const blocks = await dataSource.getBlocks({
      from: 18000000,
      to: 18001000,
      includeAllBlocks: true,
      fields: {
        block: {
          number: true,
          timestamp: true,
          gasUsed: true,
        },
      },
    });

    console.log(`Retrieved ${blocks.length} blocks`);
    blocks.forEach((block) => {
      console.log(`Block ${block.header.number}: ${block.header.gasUsed} gas used`);
    });
    ```
  </Tab>

  <Tab title="Python">
    First, install requests:

    ```bash theme={"system"}
    pip install requests
    ```

    Then run this code:

    ```python theme={"system"}
    import requests
    import json

    url = "https://portal.sqd.dev/datasets/ethereum-mainnet/stream"
    headers = {"Content-Type": "application/json"}
    payload = {
        "type": "evm",
        "fromBlock": 18000000,
        "toBlock": 18001000,
        "includeAllBlocks": True,
        "fields": {
            "block": {
                "number": True,
                "timestamp": True,
                "gasUsed": True
            }
        }
    }

    response = requests.post(url, headers=headers, json=payload)

    # Response is newline-delimited JSON
    for line in response.text.strip().split('\n'):
        block = json.loads(line)
        print(f"Block {block['header']['number']}: {block['header']['gasUsed']} gas used")
    ```
  </Tab>
</Tabs>

## Step 2: Understanding the Request

Let's break down what you just sent:

```typescript theme={"system"}
{
  "type": "evm",              // Chain type (EVM, Solana, etc.)
  "fromBlock": 18000000,      // Starting block (inclusive)
  "toBlock": 18001000,        // Ending block (inclusive)
  "fields": {                 // What data to return
    "block": {
      "number": true,         // Block number
      "timestamp": true,      // Block timestamp
      "gasUsed": true         // Total gas used in block
    }
  }
}
```

<Tip>
  Portal only returns the fields you request. This keeps responses fast and
  bandwidth-efficient.
</Tip>

## Step 3: Understanding the Response

Portal returns **newline-delimited JSON (NDJSON)**. Each line is a complete JSON object representing one block. With `includeAllBlocks: true` the request above returns 1001 blocks (one per line). The first and last look like this:

```json theme={"system"}
{"header": {"number": 18000000, "timestamp": 1693066895, "gasUsed": "0xf7e9ab"}}
{"header": {"number": 18000001, "timestamp": 1693066907, "gasUsed": "0x1068c5a"}}
...
{"header": {"number": 18001000, "timestamp": 1693078967, "gasUsed": "0xae0606"}}
```

<Note>
  This format enables **constant-memory streaming** of massive ranges. You can
  process millions of blocks without loading everything into RAM.
</Note>

## Stream semantics

Two behaviors are easy to miss on your first Portal request. Both are enforced by the API, and clients that ignore them will silently drop data.

### `includeAllBlocks`

Portal returns **only blocks that match your data filters** (`logs`, `transactions`, `traces`, `stateDiffs`).

* If you set filters (e.g. `logs: [{ address: [...] }]`), Portal returns only blocks containing matching items. You rarely need `includeAllBlocks`.
* If you want **every** block in the range regardless of filters (for analytics, backfilling a full block table, etc.), set `"includeAllBlocks": true`.
* Defaults differ by chain: `false` on EVM, Solana, and Substrate; `true` on Bitcoin. Check the dataset's [API spec](/en/api/evm/stream) if in doubt.

Without filters and without `includeAllBlocks`, Portal returns just the worker-range boundary blocks. On EVM that's typically 2–4 blocks per query. If you see far fewer blocks than expected, this is why.

### The stream can end before `toBlock`

A single HTTP response can be cut short at any point: when a worker's range ends, when a connection is recycled, or at the dataset's current head. Clients must treat one response as a **batch**, not the entire range. To continue:

1. Read the last block number (`N`) in the response.
2. Issue a new request with `fromBlock = N + 1` (and the same `toBlock`, filters, and fields).
3. Loop until you receive HTTP 204 (range is above dataset height) or you reach your `toBlock`.

If you're streaming near the chain head, also set `parentBlockHash` on each subsequent request and handle HTTP 409 (reorg). See the [full API reference](/en/portal/evm/api#stream-continuation).

## Step 4: Filter Event Logs

Track USDC Transfer events:

<CodeGroup>
  ```bash curl theme={"system"}
  curl --compressed -X POST 'https://portal.sqd.dev/datasets/ethereum-mainnet/stream' \
    -H 'Content-Type: application/json' \
    -d '{
      "type": "evm",
      "fromBlock": 18000000,
      "toBlock": 18010000,
      "fields": {
        "log": {
          "address": true,
          "topics": true,
          "data": true,
          "transactionHash": true
        }
      },
      "logs": [{
        "address": ["0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48"],
        "topic0": ["0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef"]
      }]
    }'
  ```

  ```typescript TypeScript theme={"system"}
  import { DataSource } from "@subsquid/portal-client";

  const dataSource = new DataSource({
    network: "ethereum-mainnet",
  });

  const blocks = await dataSource.getBlocks({
    from: 18000000,
    to: 18010000,
    fields: {
      log: {
        address: true,
        topics: true,
        data: true,
        transactionHash: true,
      },
    },
    logs: [
      {
        address: ["0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48"], // USDC
        topic0: [
          "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef",
        ], // Transfer
      },
    ],
  });

  for (const block of blocks) {
    for (const log of block.logs) {
      console.log(`Transfer in tx ${log.transactionHash}`);
    }
  }
  ```

  ```python Python theme={"system"}
  import requests
  import json

  url = "https://portal.sqd.dev/datasets/ethereum-mainnet/stream"
  headers = {"Content-Type": "application/json"}
  payload = {
      "type": "evm",
      "fromBlock": 18000000,
      "toBlock": 18010000,
      "fields": {
          "log": {
              "address": True,
              "topics": True,
              "data": True,
              "transactionHash": True
          }
      },
      "logs": [{
          "address": ["0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48"],
          "topic0": ["0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef"]
      }]
  }

  response = requests.post(url, headers=headers, json=payload)

  for line in response.text.strip().split('\n'):
      block = json.loads(line)
      for log in block.get('logs', []):
          print(f"Transfer in tx {log['transactionHash']}")
  ```
</CodeGroup>

<Check>
  You should see Transfer events from the USDC contract. Portal filtered the
  data before sending, saving bandwidth.
</Check>

## What You Learned

In 5 minutes, you:

* ✓ Made HTTP requests to Portal
* ✓ Queried arbitrary block ranges
* ✓ Filtered smart contract events
* ✓ Processed newline-delimited JSON responses

## Rate Limits

The Public Portal is rate-limited:

* **20 requests per 10 seconds**
* Perfect for development and testing

<CardGroup>
  <Card title="Cloud Portal" icon="cloud" href="/en/cloud/overview">
    Production-ready managed access with higher limits
  </Card>

  <Card title="Self-Host Portal" icon="server" href="/en/portal/self-hosting">
    Run your own Portal instance with no rate limits
  </Card>
</CardGroup>

## Next Steps

<CardGroup>
  <Card title="API Reference" icon="book" href="/en/portal/evm/api">
    Complete reference with all fields and filters
  </Card>

  <Card title="View Examples" icon="code" href="/en/portal/evm/examples/query-logs">
    Practical examples for common use cases
  </Card>

  <Card title="Use with SDK" icon="brackets-curly" href="/en/sdk/overview">
    Build type-safe indexers with Portal as the data source
  </Card>
</CardGroup>

### Popular Examples

<CardGroup>
  <Card title="Query Event Logs" icon="file-lines" href="/en/portal/evm/examples/query-logs">
    Track smart contract events
  </Card>

  <Card title="Query Transactions" icon="arrow-right-arrow-left" href="/en/portal/evm/examples/query-transactions">
    Monitor wallet activity
  </Card>

  <Card title="Track NFT Transfers" icon="image" href="/en/portal/evm/examples/nft-transfers">
    Index NFT collection activity
  </Card>

  <Card title="Index DEX Swaps" icon="repeat" href="/en/portal/evm/examples/dex-swaps">
    Build DEX analytics
  </Card>
</CardGroup>


# Portal for EVM & Substrate
Source: https://docs.sqd.dev/en/portal/migration/cloud-portal-evm

Migrate SQD Cloud squids from v2 archive gateways to the Portal API as primary data source — step-by-step guide for EVM and Substrate indexers.

<Info>
  SQD Network portals are currently in closed beta. Please report any bugs or suggestions to the [SQD Portal Closed Beta chat](https://t.me/+JHrJZPz34kRjYmFk) or [Squid Devs](https://t.me/HydraDevs).
</Info>

This guide walks you through replacing a gateway of the open private version of SQD Network with a portal of the permissionless SQD Network as the primary data source for your Cloud squids.

## Portal types

The SQD team operates two independent portals that serve the needs of Cloud users:

* **Cloud Portal**: Only visible from within the Cloud, ensuring stable performance for squids deployed there
* **Public Portal**: Can be accessed from anywhere for easy experimentation and local development

<Info>
  We're currently experimenting with tightening the data request complexity limits. If you see an HTTP 400 error with a message like this:

  ```
  Couldn't parse query: query contains X item requests, but only 50 is allowed
  ```

  where `X` is some number above 50, or any other HTTP 400 response, please let us know.
</Info>

## Migration steps

### Step 1: Install the portal-api processor

Navigate to your squid's folder and install the `portal-api` version of the processor package:

<CodeGroup>
  ```bash EVM theme={"system"}
  npm i @subsquid/evm-processor@portal-api
  ```

  ```bash Substrate theme={"system"}
  npm i @subsquid/substrate-processor@portal-api
  ```
</CodeGroup>

### Step 2: Configure your data source

Follow the network-specific instructions from the Cloud:

1. [Enable the preview mode](https://app.subsquid.io/preview)
2. Navigate to [the Portal's page](https://app.subsquid.io/portal)
3. Click on the tile of your network to see the instructions

Once you're done, your squid will use the Cloud Portal when deployed and the Public Portal for local runs.

### Step 3: Re-sync with zero downtime

Evaluate the performance of your new data source by re-syncing your squid. Follow the zero-downtime update procedure:

1. Deploy your squid into a new slot
2. Wait for it to sync, observing the improved data fetching
3. Assign your production tag to the new deployment to redirect the GraphQL requests there

See [Slots and Tags](/en/cloud/resources/slots-and-tags#zero-downtime-updates) for details.


# SQD Portal
Source: https://docs.sqd.dev/en/portal/overview

Stream historical and live blockchain data via the SQD Portal API — HTTP-native, reorg-safe, constant memory, free public endpoints across 225+ chains.

<div>
  <div>
    <div>
      <h1>
        The first meaningful innovation in on-chain data access since RPC
      </h1>

      <p>
        RPC was built for transactions, not data extraction. SQD Portal is an
        HTTP interface designed for data pipelines, query arbitrary block ranges,
        handle reorgs natively, and stream terabytes with constant memory usage.
      </p>

      <div>
        <a href="/en/portal/evm/quickstart">
          Build with Portal API

          <svg>
            <path />
          </svg>
        </a>

        <a href="/en/sdk/overview">
          Use with SDK
        </a>

        <a href="/en/portal/self-hosting">
          Self-host Portal
        </a>
      </div>
    </div>
  </div>
</div>

## The Problem with RPC

Modern blockchain data engineering faces a fundamental challenge: RPC was designed for wallets and transaction submission, not for extracting and processing large volumes of historical data.

Traditional RPC-based approaches struggle with:

* **Backfill failures**: Extracting even a single wallet's history can timeout on high-history chains like BSC or Solana
* **Manual pagination**: Block-by-block iteration breaks down on high-TPS chains where throughput exceeds what sequential requests can handle
* **DIY reorg handling**: You're responsible for detecting and compensating for chain reorganizations
* **Limited scalability**: Teams resort to centralized data warehouses with vendor lock-in just to query their own data

Portal solves these problems with an HTTP interface purpose-built for data extraction at scale.

## How Portal Works

<Frame>
  <video aria-label="Portal architecture diagram showing data flow from blockchain nodes through Portal API to data pipelines" />
</Frame>

Portal sits between blockchain nodes and your data pipeline, providing an HTTP API optimized for high-throughput data extraction. It handles the complexity of pagination, finality tracking, and reorg compensation so you don't have to.

## Core Capabilities

### Arbitrary block ranges

Query any block range in a single HTTP request, no manual pagination or block-by-block iteration. Essential for backfills and historical analysis on any chain.

```http theme={"system"}
POST /stream
fromBlock: 18000000
toBlock: 18100000
```

Portal handles the pagination internally and streams results back to you efficiently, even for ranges spanning millions of blocks.

### Native finalization awareness

Portal tracks chain-specific finality rules and communicates reorgs as structured rollback events in the stream. Use `/finalized-stream` for production-safe ingestion that automatically handles chain reorganizations.

<Tip>
  Portal handles reorg detection and compensation automatically, no need to
  implement your own rollback logic.
</Tip>

### HTTP streaming responses

All Portal endpoints return newline-delimited JSON (NDJSON), enabling constant-memory processing of massive block ranges. Process terabytes of data without loading everything into RAM.

```bash theme={"system"}
curl --compressed -X POST https://v2.archive.subsquid.io/network/ethereum-mainnet/stream \
  -H "Content-Type: application/json" \
  -d '{"fromBlock": 18000000, "toBlock": 18001000}' \
  | while IFS= read -r line; do
      echo "$line" | jq '.header.number'
    done
```

### Deployment flexibility

Portal is an interface specification, not a single implementation. Use managed SQD endpoints, self-host alongside your own nodes, or connect to the decentralized SQD Network, your code stays the same.

## Portal vs Alternatives

<div>
  <table>
    <thead>
      <tr>
        <th>
          Capability
        </th>

        <th>
          RPC
        </th>

        <th>
          Subgraphs
        </th>

        <th>
          Centralized SaaS
        </th>

        <th>
          Portal
        </th>
      </tr>
    </thead>

    <tbody>
      <tr>
        <td>Block-range queries</td>
        <td>⚠️ Manual</td>
        <td>✓ DSL-bound</td>
        <td>✓ Proprietary</td>
        <td>✓ Native</td>
      </tr>

      <tr>
        <td>Finality & reorgs</td>
        <td>✗ DIY</td>
        <td>⚠️ Hidden</td>
        <td>⚠️ Opaque</td>
        <td>✓ Explicit</td>
      </tr>

      <tr>
        <td>Streaming</td>
        <td>✗ Limited</td>
        <td>⚠️ Framework-specific</td>
        <td>⚠️ Custom</td>
        <td>✓ HTTP/NDJSON</td>
      </tr>

      <tr>
        <td>Vendor lock-in</td>
        <td>✓ Low</td>
        <td>⚠️ Schema-coupled</td>
        <td>✗ High</td>
        <td>✓ None</td>
      </tr>

      <tr>
        <td>Self-hostable</td>
        <td>✓ Yes</td>
        <td>⚠️ Complex</td>
        <td>✗ No</td>
        <td>✓ Yes</td>
      </tr>

      <tr>
        <td>Scales on high-TPS chains</td>
        <td>✗ No</td>
        <td>⚠️ Partial</td>
        <td>⚠️ Vendor-locked</td>
        <td>✓ Yes</td>
      </tr>
    </tbody>
  </table>
</div>

Portal provides the raw data extraction layer that works with any processing framework or storage backend, giving you complete control over your data pipeline.

## Deployment Options

Choose how you want to run Portal based on your requirements, all options use the same API interface.

<AccordionGroup>
  <Accordion title="Managed Endpoints" icon="cloud">
    SQD-hosted Portal endpoints provide instant access to 200+ networks with zero setup.

    **Best for**: Getting started quickly, production apps without infrastructure overhead, and teams who want to focus on building rather than operations.

    **Key benefits**:

    * Instant access, no infrastructure to manage
    * Production-ready with high availability
    * Free tier available for development

    <Card title="Start Building" icon="rocket" href="/en/portal/evm/quickstart">
      Make your first Portal request in 5 minutes
    </Card>
  </Accordion>

  <Accordion title="Self-Hosted Portal" icon="server">
    Run Portal alongside your own nodes for complete sovereignty over your data access.

    **Best for**: Private or custom networks, teams with strict data sovereignty requirements, and maximum control over infrastructure.

    **Key benefits**:

    * Complete control over data access
    * No external dependencies
    * Works with any EVM or Solana chain
    * Open-source implementation you can audit and modify

    <Card title="Self-Hosting Guide" icon="server" href="/en/portal/self-hosting">
      Learn how to deploy your own Portal instance
    </Card>
  </Accordion>
</AccordionGroup>

## Use Cases

Portal excels at scenarios where RPC falls short:

* **Full contract history extraction**: Query all events for a contract across its entire lifetime
* **Wallet transaction history**: Retrieve complete transaction history for any address, even on high-volume chains
* **Historical backfills**: Synchronize years of blockchain history without timeouts or rate limits
* **High-TPS monitoring**: Keep up with fast chains like Solana where block-by-block RPC polling can't keep pace
* **Multi-chain indexing**: Build data pipelines that span multiple networks using a consistent interface

## Next Steps

<CardGroup>
  <Card title="EVM Quickstart" icon="play" href="/en/portal/evm/quickstart">
    Query Ethereum and EVM chains in 5 minutes
  </Card>

  <Card title="Solana Quickstart" icon="play" href="/en/portal/solana/quickstart">
    Access Solana data with Portal
  </Card>

  <Card title="Use with SDK" icon="code" href="/en/sdk/overview">
    Build production data pipelines with TypeScript SDKs
  </Card>
</CardGroup>


# Pricing
Source: https://docs.sqd.dev/en/portal/pricing

SQD Portal pricing — free public endpoints, paid tiers for higher throughput, and bespoke plans for private datasets and self-hosted deployments.

<Info>
  **Coming Soon**

  Pricing details for SQD Portal are currently being finalized.\
  If you're interested in learning more about our pricing plans and options, please reach out to schedule a call with our team.

  <Button href="https://calendly.com/t-tyrie-subsquid/30min">
    Schedule a Call
  </Button>
</Info>


# Self-Host a Portal
Source: https://docs.sqd.dev/en/portal/self-hosting

Run your own SQD Portal instance to access decentralized network data without relying on managed services — Docker, Kubernetes, and bare-metal setups.

<Info>
  Portal is currently in closed beta. Report bugs or suggestions
  in the SQD Portal chat or to [Squid Devs](https://t.me/HydraDevs).
</Info>

<Warning>
  If you plan to use your portal to index Solana, read the [Using your Portal -
  On Solana](#on-solana) section before committing tokens.
</Warning>

Running your own Portal instance enables you to access the [permissionless SQD Network](/en/network/overview) data without relying on centralized services. You can operate a private Portal instance for personal use or deploy a high-throughput public Portal.

## Prerequisites

Before you begin, ensure you have:

### Software Requirements

* Working Docker installation

### Financial Requirements

* **Minimum 1,000,000 `SQD` tokens**
  * Tokens remain locked while your Portal instance is active
  * More tokens = more bandwidth
  * Tokens can be in your wallet or SQD-issued vesting contract
* **Arbitrum ETH** for gas fees

<Info>
  For the minimum 1,000,000 `SQD`, you'll get enough bandwidth for most tasks.
  See [Token requirements and compute
  units](#token-requirements-and-compute-units) for details.
</Info>

### Hardware Requirements

* **Minimum 4GB unused RAM**

<Note>Hardware requirements scale with your expected usage patterns.</Note>

## Portal Setup

<Steps>
  <Step title="Lock SQD tokens" icon="lock">
    1. Go to [network.subsquid.io](https://network.subsquid.io) and connect your wallet

       * Use MetaMask (recommended) or another supported wallet
       * Ensure your wallet holds the tokens or is the beneficiary of your vesting contract

    2. Navigate to the [portals page](https://network.subsquid.io/portals) and click "Lock"

    3. Specify:

       * Amount of `SQD` to lock
       * Duration of the lockup period

    4. Click "Confirm" and approve the transaction in your wallet

    <Check>
      The page will update to show your locked `SQD` amount and lockup duration.
    </Check>

    ### Enable Auto-Extension (Optional)

    <Tip>
      By default, your Portal instance stops working when the lock period ends.
      Enable auto-extension to keep your Portal instance running continuously.
    </Tip>

    Click the "Auto Extension" switch and confirm the transaction. This automatically relocks your `SQD` when the current period ends.

    <Warning>
      With auto-extension enabled, you must unlock first, then wait for the current
      period to end before withdrawing tokens.
    </Warning>
  </Step>

  <Step title="Generate a peer ID" icon="id-card">
    SQD Network operates as a decentralized peer-to-peer system. Your Portal instance needs a private key and public peer ID to participate.

    Generate your peer ID:

    ```bash theme={"system"}
    docker run --rm subsquid/rpc-node:0.2.5 keygen > <KEY_FILE_PATH>
    ```

    The command creates a key file at `<KEY_FILE_PATH>` and outputs:

    ```
    Your peer ID: THIS IS WHAT YOU NEED TO COPY
    ```

    **Copy your peer ID**, you'll need it for registration.

    <Warning>
      Protect your key file! Ensure it won't be deleted accidentally and cannot be
      accessed by unauthorized parties. See [consequences of losing your
      key](#what-are-the-consequences-of-losing-my-key-file--getting-it-stolen).
    </Warning>
  </Step>

  <Step title="Register your portal" icon="pen-to-square">
    1. Go to the [portals page](https://network.subsquid.io/portals)

    2. Click the "Add portal" button

    3. Fill the portal registration form:

       * Enter your peer ID from the previous step
       * If making your Portal instance public, enable "Publicly available" and complete additional fields

    4. Click "Confirm" and approve the transaction in your wallet

    <Check>Your Portal instance is now registered on-chain.</Check>
  </Step>

  <Step title="Run your portal" icon="rocket">
    ### Clone the Repository

    ```bash theme={"system"}
    git clone https://github.com/subsquid/sqd-portal
    cd sqd-portal
    ```

    ### Configure Environment

    Copy the mainnet configuration:

    ```bash theme={"system"}
    cp mainnet.env .env
    ```

    Set your key file path:

    ```bash theme={"system"}
    echo KEY_PATH=<KEY_FILE_PATH> >> .env
    ```

    <Warning>
      Double-check the key path! An incorrect path will cause the system to generate
      a new random key, resulting in your Portal instance attempting to operate with
      an unregistered peer ID.
    </Warning>

    ### Start the Portal

    Choose your preferred method:

    <Tabs>
      <Tab title="Docker (Recommended)">
        ```bash theme={"system"}
        docker compose up -d
        ```

        Watch the logs:

        ```bash theme={"system"}
        docker compose logs -f
        ```
      </Tab>

      <Tab title="Build from Source">
        ```bash theme={"system"}
        cargo run --release
        ```

        <Note>Tested on `cargo-1.82.0`. May not work on older versions.</Note>

        **If you get a "Could not find protoc" error:**

        Install the Protobuf compiler:

        ```bash theme={"system"}
        apt-get install protobuf-compiler
        ```
      </Tab>
    </Tabs>

    <Check>Your Portal instance is now running!</Check>
  </Step>

  <Step title="Wait for activation" icon="hourglass">
    Your stake will become active at the beginning of the next epoch, within approximately 20 minutes.

    The portals page will update to show your lockup is active.

    <Check>Portal activated! You can now use it to access SQD Network data.</Check>
  </Step>
</Steps>

## Using your portal

Portal serves a new version of API that's not compatible with `@latest` SDK and the old gateways. Update your packages with

```bash theme={"system"}
npx --yes npm-check-updates --filter "@subsquid/*" --target "@portal-api" --upgrade
```

then freeze the versions of `@portal-api` packages at the new ones, either by removing any version range specifiers (`^`, `~`, `<`, `>`, `>=`, `<=`) preceding the package versions or by running

```bash theme={"system"}
sed -i -e 's/[\^~=<>]*\([0-9\.]*-portal-api\.[0-9a-f]\{6\}\)/\1/g' package.json
```

to do the same thing automatically. Finally, reinstall all packages from scratch:

```bash theme={"system"}
rm -r node_modules package-lock.json
npm i
```

A working Portal instance exposes its dataset-specific APIs via URLs such as this one:

```

http://&lt;host&gt;:8000/datasets/&lt;dataset-slug&gt;

```

Here, `<dataset-slug>` is the last path segment of the network-specific gateway URL found on the per-chain listings ([EVM](/en/data/evm), [Substrate](/en/data/substrate), [Solana](/en/data/solana), [Other](/en/data/other)). For example, a local Portal instance will expose its Ethereum dataset API at

```

http://127.0.0.1:8000/datasets/ethereum-mainnet

```

### On EVM

You should be able to simply replace the `setGateway` call with `setPortal` and get exactly the same behavior as before:

```diff theme={"system"}
+  .setPortal('http://127.0.0.1:8000/datasets/ethereum-mainnet')
-  .setGateway('https://v2.archive.subsquid.io/network/ethereum-mainnet')
```

If your squid [uses RPC to ingest unfinalized blocks](/en/sdk/squid-sdk/resources/unfinalized-blocks), `@subsquid/evm-processor@portal-api` will smoothly transition to that regime as it catches up to the network.

### On Solana

Solana SDK in its `@portal-api` version is capable of ingesting real-time data from Portal, but is no longer capable of ingesting it from RPC. Since real-time Solana data is not yet available for private Portal instances, that means that *at the moment it is not possible to get real-time Solana data if the private Portal instance is used as the main data source*. This will be fixed soon, but as of 2025-03-04 this is one limitation we have.

If you just want to try ingesting historical Solana data you can take a look at the `portal-api` branch of our Solana template:

```bash theme={"system"}
git clone https://github.com/subsquid-labs/solana-example/
cd solana-example
git checkout portal-api
npm i
npm run build
docker compose up -d
npx squid-typeorm-migration apply
node -r dotenv/config lib/main.js
```

If the last command fails due to an HTTP 400 response, try raising the lowest block at described in [this comment](https://github.com/subsquid-labs/solana-example/blob/6ed90f7ec346a56effe29fde91eb918e149a558b/src/main.ts#L15-L17).

### Migrating an Existing Squid

<Accordion title="Full migration instructions">
  **A.** Replace all existing data sources with the Portal instance:

  ```diff theme={"system"}
  +  .setPortal('https://portal.tethys.sqd.dev/datasets/solana-beta')
  -  .setGateway('https://v2.archive.subsquid.io/network/solana-mainnet')
  -  .setRpc({
  -    client: new SolanaRpcClient({
  -      url: process.env.SOLANA_NODE
  -    })
  -  })
  ```

  Also, please remove any mentions of `SolanaRpcClient`, for example:

  ```diff theme={"system"}
  -import \{DataSourceBuilder, SolanaRpcClient\} from '@subsquid/solana-stream'
  +import \{DataSourceBuilder\} from '@subsquid/solana-stream'
  ```

  **B.** Replace any block height literals with **slot number** literals.

  ```diff theme={"system"}
  +  .setBlockRange(\{from: 325000000\})
  -  .setBlockRange(\{from: 303262650\})
  ```

  A convenient converter from block heights to slot numbers is TBA; for now bisecting the block range is your best bet. To get block height by slot number use this script:

  ```bash theme={"system"}
  SLOT_NUMBER=320276063 curl --request POST --url https: "//api.mainnet-beta.solana.com --header 'accept: application/json' --header 'content-type: application/json' --data "\{\"id\": 1, \"jsonrpc\": \"2.0\", \"method\": \"getBlock\", \"params\": [$\{SLOT_NUMBER\}, \{\"encoding\": \"jsonParsed\", \"maxSupportedTransactionVersion\": 0\}]\}" | jq | grep blockHeight"
  ```

  **C.** If you used the `slot` field of block headers anywhere in your code, replace it with `.number`:

  ```diff theme={"system"}
  -  slot: block.header.slot,
  +  slot: block.header.number,
  ```

  **D.** If you need the block height (for example to stay compatible with your old code) request it in the `.setFields` call:

  ```diff theme={"system"}
    .setFields({
      block: { // block header fields
        timestamp: true,
  +      height: true
      },
  ```
</Accordion>

## Token Requirements and Compute Units

The minimum token locking requirement for a Portal instance is 1,000,000 `SQD`. This should be enough for most use cases. If you want to know the exact capabilities this gives you, read on.

The rate limiting mechanism of SQD Network relies on the concept of a *compute unit*, or CU for short. CUs do not map directly to the amount of data fetched by a Portal instance; instead, they (roughly) represent the amount of work that the network does internally while serving the Portal instance's requests.

<Accordion title="Understanding compute units (CU)">
  SQD Network datasets are partitioned by block number. Dataset chunks are randomly distributed among [worker nodes](/en/network/worker).

  When a Portal instance receives a data request, the following happens:

  1. The Portal instance forwards the request to several workers that hold the chunks of the relevant dataset

  2. The workers execute the request separately on each chunk

  3. Workers send the results back to the Portal instance. For lightweight queries, they send one response for each dataset chunk; however, if any response exceeds 100 Mbytes, it's split into several parts

  4. The Portal instance concatenates the workers' replies and serves a continuous stream of data to the user

  Each response made by any of the workers in step 3 spends exactly 1 CU.
</Accordion>

The more `SQD` you lock and the greater the lockup period is, the more CUs you get. Currently, each locked `SQD` generates 1-3 CU at the beginning of each epoch. Here's how the number of CUs per `SQD` changes with the lockup period:

In principle, any valid amount of locked `SQD`s generates an infinite amount of CUs. However, if the rate at which your queries consume CUs exceeds the rate at which they are produced, your app will be throttled. To avoid that, you may want to understand how much CUs your queries spend.

Currently there is no tool available for estimating the number of CUs that a query needs before actually running it. However, on EVM you can use the following formula to get an [order of magnitude](https://en.wikipedia.org/wiki/Order_of_magnitude) estimate:

```

10^-5 _ range_length_blocks _ transactions_per_block

```

Notes:

* This assumes lightweight queries - that is, queries that fetch much less data than the total dataset size. For heavyweight queries multiply the estimate by a factor of 2-5.
* This is a rough estimate. Multiply it by ten to get a somewhat safe figure. If you want to minimize your `SQD` lockup, start at that safe figure, then measure the actual amount of CU you spend and reduce the lockup accordingly.
* If your network has an Etherscan-style explorer, you can estimate the `transactions_per_block` by visiting its front page, reading the "Transactions" stat and dividing it by the "Last finalized block" height.

<Accordion title="How the estimate is calculated">
  For a lightweight query, the amount of CU spent is determined by how many dataset chunks the network needs to examine to process it. The ingester creates chunks of roughly the same size within each dataset. Since the amount of data per block is roughly proportional to the number of transactions in that block, we can assume that the number of chunks in any given range is proportional to the number of transactions per block.

  Extrapolating from the Ethereum dataset:

  ```

  num*chunks_in_range = range_length * (eth*chunks / eth_height) * (chain_txs_per_block / eth_txs_per_block)

  ```

  Where:

  * `eth_chunks` = 3.1e4
  * `eth_height` = 2.1e7
  * `eth_txs_per_block` = 1.2e2

  Multiplying all the known values together and rounding to one decimal digit, we get the `1e-5` coefficient of the final formula.

  **Important assumption:** This assumes all EVM datasets have the same chunk size as Ethereum. In reality, chunk sizes vary between 50-1000 Mbytes. Ethereum's chunk size is roughly 500 Mbytes, so expect the estimate to be off by a factor of 0.5-10, which is within the "order of magnitude" definition.

  **Heavyweight queries:** Scale the same way but may spend more than one CU per chunk. The heaviest possible queries (fetching the whole dataset) on Ethereum consume roughly 5 CUs per chunk.
</Accordion>

Now, if your queries consume `X` CUs each and you run them once per `Y` epochs, you need to lock up at least this much `SQD`:

```

X / (Y \* boost_factor)

```

Here, `boost_factor` is a multiplier ranging from 1 to 3 depending on the lockup length (see the graph above).

## High throughput portals

The recommended way to make a Portal instance that can serve a large number of requests is deploy multiple Portal instances associated with a single wallet:

1. Use a single wallet to register as many peer IDs as you need Portal instances.

2. Make one `SQD` lockup for all your Portal instances. See [Token requirements and compute units](#token-requirements-and-compute-units) to get an idea of how large your stake should be.

3. Run Portal instances in parallel, balancing traffic between the instances.

If you plan to automate running your Portal instances, you may find [this helm chart](https://github.com/subsquid/sqd-portal/tree/master/chart) useful.

## Troubleshooting

<AccordionGroup>
  <Accordion title="What are the consequences of losing my key file / getting it stolen?">
    **If you lose your key file:**

    * You won't be able to run your Portal instance until you generate and register a new one

    **If your key file is stolen:**

    * The perpetrator can cause connectivity issues, effectively creating downtime for your Portal instance

    **Recovery steps:**

    1. Unregister your Portal instance on the ["Portals" tab](https://network.subsquid.io/portals)
    2. Generate a new key file
    3. Register the new portal peer ID

    <Warning>
      Always protect your key file! Store it securely and restrict access.
    </Warning>
  </Accordion>

  <Accordion title="My portal is slower than I expected. How do I examine its throttling stats?">
    Portal performance metrics are exposed at the `/metrics` endpoint.

    For a local Portal instance, check throttling statistics:

    ```bash theme={"system"}
    curl --compressed http://127.0.0.1:8000/metrics | grep throttled
    ```

    **Interpreting results:**

    Lower values of `portal_stream_throttled_ratio_sum` indicate better performance.

    <Tip>
      If throttling is high, consider locking more SQD tokens to increase your
      bandwidth allocation.
    </Tip>
  </Accordion>
</AccordionGroup>


# Accessing a Portal
Source: https://docs.sqd.dev/en/portal/self-hosting/accessing-portal

Choose between self-hosted and managed SQD Portal endpoints — connection strings, authentication, and tradeoffs for production data pipelines.

When building data pipelines with Portal, you have two options for accessing Portal endpoints:

## Option 1: Self-Hosted Portal

Run your own Portal instance for complete control over your data access infrastructure.

**Best for:**

* Private or custom networks
* Teams with strict data sovereignty requirements
* Maximum control over infrastructure
* No external dependencies

**Key benefits:**

* Complete control over data access
* No rate limits (subject to your own infrastructure)
* Works with any EVM or Solana chain
* Open-source implementation you can audit and modify

<Card title="Self-Hosting Guide" icon="server" href="/en/portal/self-hosting">
  Learn how to deploy your own Portal instance
</Card>

## Option 2: Managed Portal

Use SQD-hosted Portal endpoints for instant access without infrastructure management.

**Best for:**

* Getting started quickly
* Production apps without infrastructure overhead
* Teams who want to focus on building rather than operations

**Key benefits:**

* Instant access, no infrastructure to manage
* Production-ready with high availability
* Free tier available for development
* Access to 200+ networks with zero setup

<Card title="Portal Overview" icon="rocket" href="/en/portal/overview">
  Learn more about managed Portal endpoints
</Card>

## Choosing the Right Option

Both options use the same Portal API interface, so your code remains the same regardless of which option you choose. The decision depends on your specific requirements:

* **Choose self-hosted** if you need complete control, have custom infrastructure requirements, or want to avoid external dependencies
* **Choose managed** if you want to get started quickly, prefer not to manage infrastructure, or need production-ready endpoints immediately

<Tip>
  You can start with managed endpoints for development and switch to self-hosted later if needed. The API interface is identical, so migration is straightforward.
</Tip>

## Next Steps

<CardGroup>
  <Card title="Self-Hosting Guide" icon="server" href="/en/portal/self-hosting">
    Set up your own Portal instance
  </Card>

  <Card title="Portal Quickstart" icon="rocket" href="/en/portal/evm/quickstart">
    Get started with managed endpoints
  </Card>
</CardGroup>


# Hosting your indexer
Source: https://docs.sqd.dev/en/sdk/self-hosting

Deploy and run a Squid SDK indexer locally or on-premises with Docker — configure processor, GraphQL API, and Postgres in a single image for any host.

<Tip>
  Many tips from our [best practices guide](/en/cloud/resources/best-practices) apply to self-hosted squids, too. Check it out if you intend to use your squid in production.
</Tip>

To deploy a squid locally or on-premises, use the following Dockerfile template to build a single image for both `api` and `processor` services:

```dockerfile title="Dockerfile" theme={"system"}
FROM node:20-alpine AS node
FROM node AS node-with-gyp
FROM node-with-gyp AS builder
WORKDIR /squid
ADD package.json .
ADD package-lock.json .
# remove if needed
ADD assets assets
# remove if needed
ADD db db
# remove if needed
ADD schema.graphql .
RUN npm ci
ADD tsconfig.json .
ADD src src
RUN npm run build
FROM node-with-gyp AS deps
WORKDIR /squid
ADD package.json .
ADD package-lock.json .
RUN npm ci --production
FROM node AS squid
WORKDIR /squid
COPY --from=deps /squid/package.json .
COPY --from=deps /squid/package-lock.json .
COPY --from=deps /squid/node_modules node_modules
COPY --from=builder /squid/lib lib
# remove if no assets folder
COPY --from=builder /squid/assets assets
# remove if no db folder
COPY --from=builder /squid/db db
# remove if no schema.graphql is in the root
COPY --from=builder /squid/schema.graphql schema.graphql
# remove if no commands.json is in the root
ADD commands.json .
RUN echo -e "loglevel=silent\\nupdate-notifier=false" > /squid/.npmrc
RUN npm i -g @subsquid/commands && mv $(which squid-commands) /usr/local/bin/sqd
ENV PROCESSOR_PROMETHEUS_PORT 3000
```

Then build an image with

```bash theme={"system"}
docker buildx build . -t my-squid
```

or with

```bash theme={"system"}
docker build . -t my-squid
```

if you're using an older Docker version.

## Sample compose file

Once can the run the squid services with the freshly built image. Here is a sample `docker-compose` file:

```yaml theme={"system"}
services:
  db:
    image: postgres:15
    environment:
      POSTGRES_DB: squid
      POSTGRES_PASSWORD: postgres
    healthcheck:
      test: ["CMD-SHELL", "pg_isready", "-d", "squid"]
      interval: 5s
      timeout: 5s
      retries: 5
    # Uncomment for logging all SQL statements
    # command: ["postgres", "-c", "log_statement=all"]
  api:
    image: my-squid
    environment:
      - DB_NAME=squid
      - DB_PORT=5432
      - DB_HOST=db
      - DB_PASS=postgres
      - GQL_PORT=4350
    ports:
      # GraphQL endpoint at port 4350
      - "4350:4350"
    command: ["sqd", "serve:prod"]
    depends_on:
      db:
        condition: service_healthy
  processor:
    image: my-squid
    environment:
      - DB_NAME=squid
      - DB_PORT=5432
      - DB_HOST=db
      - DB_PASS=postgres
      # any other variables that your squid processor may be using
    ports:
      # prometheus metrics exposed at port 3000
      - "3000:3000"
    command: ["sqd", "process:prod"]
    depends_on:
      db:
        condition: service_healthy
```

Note that `sqd serve:prod` and `sqd process:prod` commands are defined in the [`commands.json` file](/en/cloud/reference/cli/commands-json).


# Stream Finalized Blocks
Source: https://docs.sqd.dev/en/api/hyperliquid-fills/finalized-stream

/en/api/catalog/hyperliquid/openapi-fills.yaml post /finalized-stream
Returns only finalized blocks matching the query. Same request format as /stream but never returns unfinalized blocks, so there are no chain reorganizations to handle (no 409 responses).




# Dataset Metadata
Source: https://docs.sqd.dev/en/api/hyperliquid-fills/metadata

/en/api/catalog/hyperliquid/openapi-fills.yaml get /metadata
Retrieves metadata describing the dataset, including its name, aliases, start block, and real-time status.



# Stream Blocks
Source: https://docs.sqd.dev/en/api/hyperliquid-fills/stream

/en/api/catalog/hyperliquid/openapi-fills.yaml post /stream
Returns blocks matching the query. May include unfinalized blocks. Response is a JSON array of blocks containing the requested fields. Set toBlock to limit the range, or omit it to stream up to the dataset height. Supports gzip compression via Accept-Encoding and Content-Encoding headers.




# Latest Finalized Block
Source: https://docs.sqd.dev/en/api/solana/finalized-head

/en/api/catalog/solana/openapi.yaml get /finalized-head
Returns the slot number and hash of the highest finalized block available in the dataset, or null if no blocks are available. Useful for diagnotics.



# Stream Finalized Blocks
Source: https://docs.sqd.dev/en/api/solana/finalized-stream

/en/api/catalog/solana/openapi.yaml post /finalized-stream
Returns only finalized blocks matching the query. Same request format as /stream but never returns unfinalized blocks, so there are no chain reorganizations to handle (no 409 responses).




# Latest Block
Source: https://docs.sqd.dev/en/api/solana/head

/en/api/catalog/solana/openapi.yaml get /head
Returns the slot number and hash of the highest block available in the dataset, or null if no blocks are available. Takes real-time data into account. Useful for diagnotics.



# Introduction
Source: https://docs.sqd.dev/en/api/solana/introduction



1. [POST your data request to `/stream`](./stream) or [`/finalized-stream`](./finalized-stream) and you shall receive a [JSON lines](https://jsonlines.org) stream:
   ```bash theme={"system"}
   curl https://portal.sqd.dev/datasets/solana-mainnet/stream \
     -X POST \
     -d '{
       "type": "solana",
       "fromBlock": 325000000,
       "toBlock": 325000001,
       "fields": {
         "block": {"number": true},
         "instruction": {"data": true}
       },
       "instructions": [
         {"programId": [
           "MoonCVVNZFSYkqNXP6bxHLPL6QQJiMagDL3qcqUQTrG"
         ]}
       ]
     }'
   ```
   outputs
   ```jsonl theme={"system"}
   {"header":{"number":325000000},"instructions":[{"data":"XJqfG9ATWCDLLmxbNnKxcQ6KRHveSXFyrM8JwmyievueP"}]}
   {"header":{"number":325000001}}
   ```
2. By default the Portal will only include blocks with data matching the request filters. However, the stream will always include the first and the last blocks of the block range it covered, even if they have no matching data.
3. The portal may end stream before it reaches `toBlock`. You can omit `toBlock` for open-ended streaming.
4. To get the rest of the data, the client must adjust the `fromBlock` field to the number of the last received block **plus one**. When streaming unfinalized data from `/stream`, they must also include `parentBlockHash` into the request.

## Handling forks on `/stream`

5. Chain forks / reorgs sometimes make some of the data from `/stream` obsolete. The portal will detect that using the `parentBlockHash` field of the request and respond with an HTTP 409 containing a sample of new consensus blocks. When that happens, the client must roll back any changes to its own state made due to orphan blocks' data, then send another `POST /stream` that starts from an earlier block to attempt to fetch the updated chain.
6. Post-rollback request must contain `parentBlockHash`. If the Portal detects that the block is still not in the updated chain, it'll return a 409 again. Be prepared to process several of these in a row.
7. To make state management easier for the client, Portal may return `X-Sqd-Finalized-Head-Number` and `X-Sqd-Finalized-Head-Hash` headers with each response from `/stream`. The block from these headers and blocks below it won't ever reorganize.
8. That doesn't mean that `X-Sqd-Finalized-Head-Number` is a non-decreasing value: Portals can be behind load balancers and the value may decrease when the load balancer switches to a lagging instance. The client should trust the guarantees provided by either instance and treat the highest block returned with these headers as a high water mark for finalized blocks.


# Dataset Metadata
Source: https://docs.sqd.dev/en/api/solana/metadata

/en/api/catalog/solana/openapi.yaml get /metadata
Retrieves metadata describing the dataset, including its name, aliases, start block, and real-time status.



# Stream Blocks
Source: https://docs.sqd.dev/en/api/solana/stream

/en/api/catalog/solana/openapi.yaml post /stream
Returns blocks matching the query. May include unfinalized blocks. Response is a JSON array of blocks containing the requested fields. Set toBlock to limit the range, or omit it to stream up to the dataset height. Supports gzip compression via Accept-Encoding and Content-Encoding headers.




# MemeCore Testnet (Formicarium)
Source: https://docs.sqd.dev/en/data/evm/formicarium-testnet

Index MemeCore Testnet (Formicarium) EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value                 |
| ----------- | --------------------- |
| Network ID  | `formicarium-testnet` |
| Chain ID    | `43521`               |
| Real-time   | No                    |
| Traces      | ✓                     |
| State Diffs | ✓                     |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/formicarium-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/formicarium-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/formicarium-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Ink
Source: https://docs.sqd.dev/en/data/evm/ink-mainnet

Index Ink EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value         |
| ----------- | ------------- |
| Network ID  | `ink-mainnet` |
| Chain ID    | `57073`       |
| Real-time   | Yes           |
| Traces      | ✓             |
| State Diffs | ✓             |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/ink-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/ink-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/ink-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Katana
Source: https://docs.sqd.dev/en/data/evm/katana-mainnet

Private SQD Portal access for Katana blockchain data — request a dedicated dataset for high-throughput indexing and bespoke streaming.

<Info>
  Katana is a private SQD Portal network. Data access requires a custom arrangement with our team.
</Info>

| Property     | Value            |
| ------------ | ---------------- |
| Network ID   | `katana-mainnet` |
| Chain ID     | `747474`         |
| Availability | Private          |

## Get access

Reach out to our BD team to discuss Katana data access — pricing, dataset scope, and onboarding.

<Card title="Talk to our team" icon="calendar" href="https://calendly.com/t-tyrie-subsquid/30min">
  Schedule a 30-min call about Katana data access
</Card>

## Reference

<CardGroup>
  <Card title="Portal Overview" icon="network-wired" href="/en/portal/overview">
    How SQD Portal works
  </Card>

  <Card title="Pricing" icon="dollar-sign" href="/en/portal/pricing">
    Plans and enterprise options
  </Card>
</CardGroup>


# Kyoto Testnet
Source: https://docs.sqd.dev/en/data/evm/kyoto-testnet

Index Kyoto Testnet EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value           |
| ----------- | --------------- |
| Network ID  | `kyoto-testnet` |
| Chain ID    | `1998`          |
| Real-time   | No              |
| Traces      | ✗               |
| State Diffs | ✗               |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/kyoto-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/kyoto-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/kyoto-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Linea
Source: https://docs.sqd.dev/en/data/evm/linea-mainnet

Index Linea EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value           |
| ----------- | --------------- |
| Network ID  | `linea-mainnet` |
| Chain ID    | `59144`         |
| Real-time   | Yes             |
| Traces      | ✓               |
| State Diffs | ✗               |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/linea-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/linea-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/linea-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# LUKSO
Source: https://docs.sqd.dev/en/data/evm/lukso-mainnet

Index LUKSO EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value           |
| ----------- | --------------- |
| Network ID  | `lukso-mainnet` |
| Chain ID    | `42`            |
| Real-time   | No              |
| Traces      | ✗               |
| State Diffs | ✗               |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/lukso-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/lukso-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/lukso-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Manta Pacific
Source: https://docs.sqd.dev/en/data/evm/manta-pacific

Index Manta Pacific EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value           |
| ----------- | --------------- |
| Network ID  | `manta-pacific` |
| Chain ID    | `169`           |
| Real-time   | No              |
| Traces      | ✗               |
| State Diffs | ✗               |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/manta-pacific
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/manta-pacific
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/manta-pacific/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Manta Pacific Sepolia
Source: https://docs.sqd.dev/en/data/evm/manta-pacific-sepolia

Index Manta Pacific Sepolia EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value                   |
| ----------- | ----------------------- |
| Network ID  | `manta-pacific-sepolia` |
| Chain ID    | `3441006`               |
| Real-time   | No                      |
| Traces      | ✗                       |
| State Diffs | ✗                       |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/manta-pacific-sepolia
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/manta-pacific-sepolia
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/manta-pacific-sepolia/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Mantle
Source: https://docs.sqd.dev/en/data/evm/mantle-mainnet

Index Mantle EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value            |
| ----------- | ---------------- |
| Network ID  | `mantle-mainnet` |
| Chain ID    | `5000`           |
| Real-time   | No               |
| Traces      | ✗                |
| State Diffs | ✗                |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/mantle-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/mantle-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/mantle-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Mantle Sepolia
Source: https://docs.sqd.dev/en/data/evm/mantle-sepolia

Index Mantle Sepolia EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value            |
| ----------- | ---------------- |
| Network ID  | `mantle-sepolia` |
| Chain ID    | `5003`           |
| Real-time   | No               |
| Traces      | ✗                |
| State Diffs | ✗                |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/mantle-sepolia
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/mantle-sepolia
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/mantle-sepolia/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# MegaETH
Source: https://docs.sqd.dev/en/data/evm/megaeth-mainnet

Index MegaETH EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value             |
| ----------- | ----------------- |
| Network ID  | `megaeth-mainnet` |
| Chain ID    | `4326`            |
| Real-time   | Yes               |
| Traces      | ✗                 |
| State Diffs | ✗                 |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/megaeth-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/megaeth-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/megaeth-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Chain-specific tutorials

* [Squid SDK: WETH transfers on MegaETH](https://github.com/subsquid-labs/squid-megaeth-example?tab=readme-ov-file#weth-transfers-on-megaeth)
* [Pipes SDK: fetching WETH transfers on MegaETH](https://github.com/subsquid-labs/pipes-megaeth-example?tab=readme-ov-file#simple-megaeth-pipes)

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# MegaETH Testnet
Source: https://docs.sqd.dev/en/data/evm/megaeth-testnet

Index MegaETH Testnet EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value             |
| ----------- | ----------------- |
| Network ID  | `megaeth-testnet` |
| Chain ID    | `6342`            |
| Real-time   | No                |
| Traces      | ✗                 |
| State Diffs | ✗                 |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/megaeth-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/megaeth-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/megaeth-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# MemeCore Testnet (Insectarium)
Source: https://docs.sqd.dev/en/data/evm/memecore-insectarium

Index MemeCore Testnet (Insectarium) EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value                  |
| ----------- | ---------------------- |
| Network ID  | `memecore-insectarium` |
| Chain ID    | `43522`                |
| Real-time   | No                     |
| Traces      | ✓                      |
| State Diffs | ✓                      |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/memecore-insectarium
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/memecore-insectarium
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/memecore-insectarium/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# MemeCore
Source: https://docs.sqd.dev/en/data/evm/memecore-mainnet

Index MemeCore EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value              |
| ----------- | ------------------ |
| Network ID  | `memecore-mainnet` |
| Chain ID    | `4352`             |
| Real-time   | No                 |
| Traces      | ✓                  |
| State Diffs | ✓                  |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/memecore-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/memecore-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/memecore-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Merlin
Source: https://docs.sqd.dev/en/data/evm/merlin-mainnet

Index Merlin EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value            |
| ----------- | ---------------- |
| Network ID  | `merlin-mainnet` |
| Chain ID    | `4200`           |
| Real-time   | No               |
| Traces      | ✗                |
| State Diffs | ✗                |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/merlin-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/merlin-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/merlin-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Merlin Testnet
Source: https://docs.sqd.dev/en/data/evm/merlin-testnet

Index Merlin Testnet EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value            |
| ----------- | ---------------- |
| Network ID  | `merlin-testnet` |
| Chain ID    | `686868`         |
| Real-time   | No               |
| Traces      | ✗                |
| State Diffs | ✗                |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/merlin-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/merlin-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/merlin-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Metis Andromeda
Source: https://docs.sqd.dev/en/data/evm/metis-mainnet

Index Metis Andromeda EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

<Info>State diffs available except blocks 16861274–16861286.</Info>

| Property    | Value           |
| ----------- | --------------- |
| Network ID  | `metis-mainnet` |
| Chain ID    | `1088`          |
| Real-time   | No              |
| Traces      | ✓               |
| State Diffs | ✗               |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/metis-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/metis-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/metis-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Mode
Source: https://docs.sqd.dev/en/data/evm/mode-mainnet

Index Mode EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value          |
| ----------- | -------------- |
| Network ID  | `mode-mainnet` |
| Chain ID    | `34443`        |
| Real-time   | No             |
| Traces      | ✗              |
| State Diffs | ✗              |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/mode-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/mode-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/mode-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Monad
Source: https://docs.sqd.dev/en/data/evm/monad-mainnet

Index Monad EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value           |
| ----------- | --------------- |
| Network ID  | `monad-mainnet` |
| Chain ID    | `143`           |
| Real-time   | Yes             |
| Traces      | ✗               |
| State Diffs | ✗               |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/monad-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/monad-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/monad-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Monad Testnet
Source: https://docs.sqd.dev/en/data/evm/monad-testnet

Index Monad Testnet EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value           |
| ----------- | --------------- |
| Network ID  | `monad-testnet` |
| Chain ID    | `10143`         |
| Real-time   | Yes             |
| Traces      | ✗               |
| State Diffs | ✗               |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/monad-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/monad-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/monad-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Moonbase Alpha
Source: https://docs.sqd.dev/en/data/evm/moonbase-testnet

Index Moonbase Alpha EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

<Info>Only for EVM data. For Substrate and ink! data use the corresponding Substrate gateway.</Info>

| Property    | Value              |
| ----------- | ------------------ |
| Network ID  | `moonbase-testnet` |
| Chain ID    | `1287`             |
| Real-time   | No                 |
| Traces      | ✗                  |
| State Diffs | ✗                  |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/moonbase-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/moonbase-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/moonbase-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Moonbeam
Source: https://docs.sqd.dev/en/data/evm/moonbeam-mainnet

Index Moonbeam EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

<Info>Only for EVM data. For Substrate and ink! data use the corresponding Substrate gateway.</Info>

| Property    | Value              |
| ----------- | ------------------ |
| Network ID  | `moonbeam-mainnet` |
| Chain ID    | `1284`             |
| Real-time   | No                 |
| Traces      | ✓                  |
| State Diffs | ✗                  |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/moonbeam-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/moonbeam-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/moonbeam-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Moonriver
Source: https://docs.sqd.dev/en/data/evm/moonriver-mainnet

Index Moonriver EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

<Info>Only for EVM data. For Substrate and ink! data use the corresponding Substrate gateway.</Info>

| Property    | Value               |
| ----------- | ------------------- |
| Network ID  | `moonriver-mainnet` |
| Chain ID    | `1285`              |
| Real-time   | No                  |
| Traces      | ✓                   |
| State Diffs | ✗                   |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/moonriver-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/moonriver-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/moonriver-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Moonsama Network
Source: https://docs.sqd.dev/en/data/evm/moonsama

Index Moonsama Network EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value      |
| ----------- | ---------- |
| Network ID  | `moonsama` |
| Chain ID    | `2199`     |
| Real-time   | No         |
| Traces      | ✗          |
| State Diffs | ✗          |

## Endpoint

```
https://portal.sqd.dev/datasets/moonsama
```

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/moonsama/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Naka Chain
Source: https://docs.sqd.dev/en/data/evm/nakachain

Index Naka Chain EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value       |
| ----------- | ----------- |
| Network ID  | `nakachain` |
| Chain ID    | `42225`     |
| Real-time   | No          |
| Traces      | ✗           |
| State Diffs | ✗           |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/nakachain
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/nakachain
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/nakachain/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Neon EVM Devnet
Source: https://docs.sqd.dev/en/data/evm/neon-devnet

Index Neon EVM Devnet EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

<Info>Genesis block for Neon EVM Devnet is 177,455,580.</Info>

| Property    | Value         |
| ----------- | ------------- |
| Network ID  | `neon-devnet` |
| Chain ID    | `245022926`   |
| Real-time   | No            |
| Traces      | ✗             |
| State Diffs | ✗             |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/neon-devnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/neon-devnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/neon-devnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Neon EVM
Source: https://docs.sqd.dev/en/data/evm/neon-mainnet

Index Neon EVM EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value          |
| ----------- | -------------- |
| Network ID  | `neon-mainnet` |
| Chain ID    | `245022934`    |
| Real-time   | No             |
| Traces      | ✗              |
| State Diffs | ✗              |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/neon-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/neon-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/neon-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Neo X Testnet
Source: https://docs.sqd.dev/en/data/evm/neox-testnet

Index Neo X Testnet EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value          |
| ----------- | -------------- |
| Network ID  | `neox-testnet` |
| Chain ID    | `12227332`     |
| Real-time   | No             |
| Traces      | ✓              |
| State Diffs | ✓              |

## Endpoint

```
https://portal.sqd.dev/datasets/neox-testnet
```

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/neox-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# opBNB
Source: https://docs.sqd.dev/en/data/evm/opbnb-mainnet

Index opBNB EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value           |
| ----------- | --------------- |
| Network ID  | `opbnb-mainnet` |
| Chain ID    | `204`           |
| Real-time   | No              |
| Traces      | ✗               |
| State Diffs | ✗               |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/opbnb-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/opbnb-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/opbnb-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# opBNB Testnet
Source: https://docs.sqd.dev/en/data/evm/opbnb-testnet

Index opBNB Testnet EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value           |
| ----------- | --------------- |
| Network ID  | `opbnb-testnet` |
| Chain ID    | `5611`          |
| Real-time   | No              |
| Traces      | ✗               |
| State Diffs | ✗               |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/opbnb-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/opbnb-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/opbnb-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Optimism
Source: https://docs.sqd.dev/en/data/evm/optimism-mainnet

Index Optimism EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

<Info>State diffs available from block 105235063.</Info>

| Property    | Value              |
| ----------- | ------------------ |
| Network ID  | `optimism-mainnet` |
| Chain ID    | `10`               |
| Real-time   | Yes                |
| Traces      | ✓                  |
| State Diffs | ✓                  |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/optimism-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/optimism-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/optimism-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Optimism Sepolia
Source: https://docs.sqd.dev/en/data/evm/optimism-sepolia

Index Optimism Sepolia EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value              |
| ----------- | ------------------ |
| Network ID  | `optimism-sepolia` |
| Chain ID    | `11155420`         |
| Real-time   | No                 |
| Traces      | ✗                  |
| State Diffs | ✗                  |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/optimism-sepolia
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/optimism-sepolia
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/optimism-sepolia/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Ozean Testnet
Source: https://docs.sqd.dev/en/data/evm/ozean-testnet

Index Ozean Testnet EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value           |
| ----------- | --------------- |
| Network ID  | `ozean-testnet` |
| Chain ID    | `7849306`       |
| Real-time   | No              |
| Traces      | ✓               |
| State Diffs | ✓               |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/ozean-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/ozean-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/ozean-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Peaq
Source: https://docs.sqd.dev/en/data/evm/peaq-mainnet

Index Peaq EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value          |
| ----------- | -------------- |
| Network ID  | `peaq-mainnet` |
| Chain ID    | `3338`         |
| Real-time   | No             |
| Traces      | ✗              |
| State Diffs | ✗              |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/peaq-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/peaq-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/peaq-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Plasma
Source: https://docs.sqd.dev/en/data/evm/plasma-mainnet

Index Plasma EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value            |
| ----------- | ---------------- |
| Network ID  | `plasma-mainnet` |
| Chain ID    | `9745`           |
| Real-time   | Yes              |
| Traces      | ✓                |
| State Diffs | ✓                |

## Endpoint

```
https://portal.sqd.dev/datasets/plasma-mainnet
```

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/plasma-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Plasma Testnet
Source: https://docs.sqd.dev/en/data/evm/plasma-testnet

Index Plasma Testnet EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value            |
| ----------- | ---------------- |
| Network ID  | `plasma-testnet` |
| Chain ID    | `9746`           |
| Real-time   | No               |
| Traces      | ✓                |
| State Diffs | ✓                |

## Endpoint

```
https://portal.sqd.dev/datasets/plasma-testnet
```

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/plasma-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Plume
Source: https://docs.sqd.dev/en/data/evm/plume-mainnet

Index Plume EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value           |
| ----------- | --------------- |
| Network ID  | `plume-mainnet` |
| Chain ID    | `98866`         |
| Real-time   | No              |
| Traces      | ✓               |
| State Diffs | ✓               |

## Endpoint

```
https://portal.sqd.dev/datasets/plume-mainnet
```

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/plume-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Plume Testnet
Source: https://docs.sqd.dev/en/data/evm/plume-testnet

Index Plume Testnet EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value           |
| ----------- | --------------- |
| Network ID  | `plume-testnet` |
| Chain ID    | `98867`         |
| Real-time   | No              |
| Traces      | ✓               |
| State Diffs | ✓               |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/plume-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/plume-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/plume-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Polygon Amoy Testnet
Source: https://docs.sqd.dev/en/data/evm/polygon-amoy-testnet

Index Polygon Amoy Testnet EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value                  |
| ----------- | ---------------------- |
| Network ID  | `polygon-amoy-testnet` |
| Chain ID    | `80002`                |
| Real-time   | No                     |
| Traces      | ✗                      |
| State Diffs | ✗                      |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/polygon-amoy-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/polygon-amoy-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/polygon-amoy-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Polygon
Source: https://docs.sqd.dev/en/data/evm/polygon-mainnet

Index Polygon EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value             |
| ----------- | ----------------- |
| Network ID  | `polygon-mainnet` |
| Chain ID    | `137`             |
| Real-time   | Yes               |
| Traces      | ✓                 |
| State Diffs | ✗                 |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/polygon-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/polygon-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/polygon-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Polygon zkEVM Cardona Testnet
Source: https://docs.sqd.dev/en/data/evm/polygon-zkevm-cardona-testnet

Index Polygon zkEVM Cardona Testnet EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value                           |
| ----------- | ------------------------------- |
| Network ID  | `polygon-zkevm-cardona-testnet` |
| Chain ID    | `2442`                          |
| Real-time   | No                              |
| Traces      | ✗                               |
| State Diffs | ✗                               |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/polygon-zkevm-cardona-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/polygon-zkevm-cardona-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/polygon-zkevm-cardona-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Polygon zkEVM
Source: https://docs.sqd.dev/en/data/evm/polygon-zkevm-mainnet

Index Polygon zkEVM EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value                   |
| ----------- | ----------------------- |
| Network ID  | `polygon-zkevm-mainnet` |
| Chain ID    | `1101`                  |
| Real-time   | No                      |
| Traces      | ✗                       |
| State Diffs | ✗                       |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/polygon-zkevm-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/polygon-zkevm-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/polygon-zkevm-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Prom
Source: https://docs.sqd.dev/en/data/evm/prom-mainnet

Index Prom EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value          |
| ----------- | -------------- |
| Network ID  | `prom-mainnet` |
| Chain ID    | `227`          |
| Real-time   | No             |
| Traces      | ✓              |
| State Diffs | ✓              |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/prom-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/prom-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/prom-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Puppynet (Shibarium's Testnet)
Source: https://docs.sqd.dev/en/data/evm/puppynet

Index Puppynet (Shibarium's Testnet) EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value      |
| ----------- | ---------- |
| Network ID  | `puppynet` |
| Chain ID    | `157`      |
| Real-time   | No         |
| Traces      | ✓          |
| State Diffs | ✓          |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/puppynet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/puppynet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/puppynet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Scroll
Source: https://docs.sqd.dev/en/data/evm/scroll-mainnet

Index Scroll EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value            |
| ----------- | ---------------- |
| Network ID  | `scroll-mainnet` |
| Chain ID    | `534352`         |
| Real-time   | No               |
| Traces      | ✓                |
| State Diffs | ✗                |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/scroll-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/scroll-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/scroll-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Scroll Sepolia
Source: https://docs.sqd.dev/en/data/evm/scroll-sepolia

Index Scroll Sepolia EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value            |
| ----------- | ---------------- |
| Network ID  | `scroll-sepolia` |
| Chain ID    | `534351`         |
| Real-time   | No               |
| Traces      | ✓                |
| State Diffs | ✗                |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/scroll-sepolia
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/scroll-sepolia
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/scroll-sepolia/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Shibarium
Source: https://docs.sqd.dev/en/data/evm/shibarium

Index Shibarium EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value       |
| ----------- | ----------- |
| Network ID  | `shibarium` |
| Chain ID    | `109`       |
| Real-time   | No          |
| Traces      | ✓           |
| State Diffs | ✓           |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/shibarium
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/shibarium
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/shibarium/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Shibuya
Source: https://docs.sqd.dev/en/data/evm/shibuya-testnet

Index Shibuya EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

<Info>Only for EVM data. For Substrate and ink! data use the corresponding Substrate gateway.</Info>

| Property    | Value             |
| ----------- | ----------------- |
| Network ID  | `shibuya-testnet` |
| Chain ID    | `81`              |
| Real-time   | No                |
| Traces      | ✗                 |
| State Diffs | ✗                 |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/shibuya-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/shibuya-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/shibuya-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Shiden
Source: https://docs.sqd.dev/en/data/evm/shiden-mainnet

Index Shiden EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

<Info>Only for EVM data. For Substrate and ink! data use the corresponding Substrate gateway.</Info>

| Property    | Value            |
| ----------- | ---------------- |
| Network ID  | `shiden-mainnet` |
| Chain ID    | `336`            |
| Real-time   | No               |
| Traces      | ✗                |
| State Diffs | ✗                |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/shiden-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/shiden-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/shiden-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# SKALE Nebula Hub
Source: https://docs.sqd.dev/en/data/evm/skale-nebula

Index SKALE Nebula Hub EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value          |
| ----------- | -------------- |
| Network ID  | `skale-nebula` |
| Chain ID    | `1482601649`   |
| Real-time   | No             |
| Traces      | ✗              |
| State Diffs | ✗              |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/skale-nebula
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/skale-nebula
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/skale-nebula/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Soneium
Source: https://docs.sqd.dev/en/data/evm/soneium-mainnet

Index Soneium EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value             |
| ----------- | ----------------- |
| Network ID  | `soneium-mainnet` |
| Chain ID    | `1868`            |
| Real-time   | Yes               |
| Traces      | ✗                 |
| State Diffs | ✗                 |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/soneium-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/soneium-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/soneium-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Soneium Minato Testnet
Source: https://docs.sqd.dev/en/data/evm/soneium-minato-testnet

Index Soneium Minato Testnet EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value                    |
| ----------- | ------------------------ |
| Network ID  | `soneium-minato-testnet` |
| Chain ID    | `1946`                   |
| Real-time   | No                       |
| Traces      | ✓                        |
| State Diffs | ✓                        |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/soneium-minato-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/soneium-minato-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/soneium-minato-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Sonic
Source: https://docs.sqd.dev/en/data/evm/sonic-mainnet

Index Sonic EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value           |
| ----------- | --------------- |
| Network ID  | `sonic-mainnet` |
| Chain ID    | `146`           |
| Real-time   | No              |
| Traces      | ✓               |
| State Diffs | ✗               |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/sonic-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/sonic-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/sonic-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Sonic Testnet
Source: https://docs.sqd.dev/en/data/evm/sonic-testnet

Index Sonic Testnet EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value           |
| ----------- | --------------- |
| Network ID  | `sonic-testnet` |
| Chain ID    | `14601`         |
| Real-time   | No              |
| Traces      | ✓               |
| State Diffs | ✗               |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/sonic-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/sonic-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/sonic-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# StratoVM Sepolia
Source: https://docs.sqd.dev/en/data/evm/stratovm-sepolia

Index StratoVM Sepolia EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value              |
| ----------- | ------------------ |
| Network ID  | `stratovm-sepolia` |
| Chain ID    | `93747`            |
| Real-time   | No                 |
| Traces      | ✓                  |
| State Diffs | ✓                  |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/stratovm-sepolia
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/stratovm-sepolia
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/stratovm-sepolia/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Superseed
Source: https://docs.sqd.dev/en/data/evm/superseed-mainnet

Index Superseed EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value               |
| ----------- | ------------------- |
| Network ID  | `superseed-mainnet` |
| Chain ID    | `5330`              |
| Real-time   | No                  |
| Traces      | ✗                   |
| State Diffs | ✗                   |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/superseed-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/superseed-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/superseed-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Superseed Sepolia
Source: https://docs.sqd.dev/en/data/evm/superseed-sepolia

Index Superseed Sepolia EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value               |
| ----------- | ------------------- |
| Network ID  | `superseed-sepolia` |
| Chain ID    | `53302`             |
| Real-time   | No                  |
| Traces      | ✗                   |
| State Diffs | ✗                   |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/superseed-sepolia
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/superseed-sepolia
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/superseed-sepolia/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# TAC
Source: https://docs.sqd.dev/en/data/evm/tac-mainnet

Private SQD Portal access for TAC blockchain data — request a dedicated dataset for high-throughput indexing and bespoke streaming.

<Info>
  TAC is a private SQD Portal network. Data access requires a custom arrangement with our team.
</Info>

| Property     | Value         |
| ------------ | ------------- |
| Network ID   | `tac-mainnet` |
| Chain ID     | `239`         |
| Availability | Private       |

## Get access

Reach out to our BD team to discuss TAC data access — pricing, dataset scope, and onboarding.

<Card title="Talk to our team" icon="calendar" href="https://calendly.com/t-tyrie-subsquid/30min">
  Schedule a 30-min call about TAC data access
</Card>

## Reference

<CardGroup>
  <Card title="Portal Overview" icon="network-wired" href="/en/portal/overview">
    How SQD Portal works
  </Card>

  <Card title="Pricing" icon="dollar-sign" href="/en/portal/pricing">
    Plans and enterprise options
  </Card>
</CardGroup>


# Taiko Alethia
Source: https://docs.sqd.dev/en/data/evm/taiko-mainnet

Index Taiko Alethia EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value           |
| ----------- | --------------- |
| Network ID  | `taiko-mainnet` |
| Chain ID    | `167000`        |
| Real-time   | No              |
| Traces      | ✗               |
| State Diffs | ✗               |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/taiko-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/taiko-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/taiko-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Tanssi
Source: https://docs.sqd.dev/en/data/evm/tanssi

Index Tanssi EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value    |
| ----------- | -------- |
| Network ID  | `tanssi` |
| Chain ID    | `5678`   |
| Real-time   | No       |
| Traces      | ✗        |
| State Diffs | ✗        |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/tanssi
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/tanssi
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/tanssi/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Tempo
Source: https://docs.sqd.dev/en/data/evm/tempo-mainnet

Private SQD Portal access for Tempo blockchain data — request a dedicated dataset for high-throughput indexing and bespoke streaming.

<Info>
  Tempo is a private SQD Portal network. Data access requires a custom arrangement with our team.
</Info>

| Property     | Value           |
| ------------ | --------------- |
| Network ID   | `tempo-mainnet` |
| Chain ID     | `4217`          |
| Availability | Private         |

## Get access

Reach out to our BD team to discuss Tempo data access — pricing, dataset scope, and onboarding.

<Card title="Talk to our team" icon="calendar" href="https://calendly.com/t-tyrie-subsquid/30min">
  Schedule a 30-min call about Tempo data access
</Card>

## Reference

<CardGroup>
  <Card title="Portal Overview" icon="network-wired" href="/en/portal/overview">
    How SQD Portal works
  </Card>

  <Card title="Pricing" icon="dollar-sign" href="/en/portal/pricing">
    Plans and enterprise options
  </Card>
</CardGroup>


# Unichain
Source: https://docs.sqd.dev/en/data/evm/unichain-mainnet

Index Unichain EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value              |
| ----------- | ------------------ |
| Network ID  | `unichain-mainnet` |
| Chain ID    | `130`              |
| Real-time   | Yes                |
| Traces      | ✓                  |
| State Diffs | ✓                  |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/unichain-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/unichain-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/unichain-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Unichain Sepolia
Source: https://docs.sqd.dev/en/data/evm/unichain-sepolia

Index Unichain Sepolia EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value              |
| ----------- | ------------------ |
| Network ID  | `unichain-sepolia` |
| Chain ID    | `1301`             |
| Real-time   | No                 |
| Traces      | ✗                  |
| State Diffs | ✗                  |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/unichain-sepolia
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/unichain-sepolia
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/unichain-sepolia/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Worldchain
Source: https://docs.sqd.dev/en/data/evm/worldchain-mainnet

Private SQD Portal access for Worldchain blockchain data — request a dedicated dataset for high-throughput indexing and bespoke streaming.

<Info>
  Worldchain is a private SQD Portal network. Data access requires a custom arrangement with our team.
</Info>

| Property     | Value                |
| ------------ | -------------------- |
| Network ID   | `worldchain-mainnet` |
| Chain ID     | `480`                |
| Availability | Private              |

## Get access

Reach out to our BD team to discuss Worldchain data access — pricing, dataset scope, and onboarding.

<Card title="Talk to our team" icon="calendar" href="https://calendly.com/t-tyrie-subsquid/30min">
  Schedule a 30-min call about Worldchain data access
</Card>

## Reference

<CardGroup>
  <Card title="Portal Overview" icon="network-wired" href="/en/portal/overview">
    How SQD Portal works
  </Card>

  <Card title="Pricing" icon="dollar-sign" href="/en/portal/pricing">
    Plans and enterprise options
  </Card>
</CardGroup>


# X Layer Testnet (Chain 1952)
Source: https://docs.sqd.dev/en/data/evm/x1-testnet

Index X Layer Testnet (Chain 1952) EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value        |
| ----------- | ------------ |
| Network ID  | `x1-testnet` |
| Chain ID    | `1952`       |
| Real-time   | No           |
| Traces      | ✗            |
| State Diffs | ✗            |

## Endpoint

```
https://portal.sqd.dev/datasets/x1-testnet
```

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/x1-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# X Layer
Source: https://docs.sqd.dev/en/data/evm/xlayer-mainnet

Index X Layer EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value            |
| ----------- | ---------------- |
| Network ID  | `xlayer-mainnet` |
| Chain ID    | `196`            |
| Real-time   | No               |
| Traces      | ✗                |
| State Diffs | ✗                |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/xlayer-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/xlayer-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/xlayer-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# X Layer Testnet (Chain 195)
Source: https://docs.sqd.dev/en/data/evm/xlayer-testnet

Index X Layer Testnet (Chain 195) EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value            |
| ----------- | ---------------- |
| Network ID  | `xlayer-testnet` |
| Chain ID    | `195`            |
| Real-time   | No               |
| Traces      | ✗                |
| State Diffs | ✗                |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/xlayer-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/xlayer-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/xlayer-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# zkLink Nova
Source: https://docs.sqd.dev/en/data/evm/zklink-nova-mainnet

Index zkLink Nova EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value                 |
| ----------- | --------------------- |
| Network ID  | `zklink-nova-mainnet` |
| Chain ID    | `810180`              |
| Real-time   | No                    |
| Traces      | ✓                     |
| State Diffs | ✗                     |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/zklink-nova-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/zklink-nova-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/zklink-nova-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# zkSync
Source: https://docs.sqd.dev/en/data/evm/zksync-mainnet

Index zkSync EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

<Info>State diffs available from block 15500000.</Info>

| Property    | Value            |
| ----------- | ---------------- |
| Network ID  | `zksync-mainnet` |
| Chain ID    | `324`            |
| Real-time   | Yes              |
| Traces      | ✓                |
| State Diffs | ✗                |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/zksync-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/zksync-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/zksync-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# zkSync Sepolia
Source: https://docs.sqd.dev/en/data/evm/zksync-sepolia

Index zkSync Sepolia EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value            |
| ----------- | ---------------- |
| Network ID  | `zksync-sepolia` |
| Chain ID    | `300`            |
| Real-time   | No               |
| Traces      | ✓                |
| State Diffs | ✗                |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/zksync-sepolia
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/zksync-sepolia
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/zksync-sepolia/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Zora
Source: https://docs.sqd.dev/en/data/evm/zora-mainnet

Index Zora EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value          |
| ----------- | -------------- |
| Network ID  | `zora-mainnet` |
| Chain ID    | `7777777`      |
| Real-time   | Yes            |
| Traces      | ✓              |
| State Diffs | ✓              |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/zora-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/zora-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/zora-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Zora Sepolia
Source: https://docs.sqd.dev/en/data/evm/zora-sepolia

Index Zora Sepolia EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value          |
| ----------- | -------------- |
| Network ID  | `zora-sepolia` |
| Chain ID    | `999999999`    |
| Real-time   | No             |
| Traces      | ✗              |
| State Diffs | ✗              |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/zora-sepolia
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/zora-sepolia
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/zora-sepolia/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Solana Networks
Source: https://docs.sqd.dev/en/data/solana

Browse all Solana and SVM-compatible networks supported by SQD — Solana mainnet, devnet, Eclipse, Spray, and other SVM chains with Portal API endpoints.

<Warning>
  **Portal Migration**: We're migrating from v2 archives to Portal. v2 archives
  will be sunset soon. [Get Started with Portal →](/en/portal/migration)
</Warning>


# Eclipse
Source: https://docs.sqd.dev/en/data/solana/eclipse-mainnet

Stream Eclipse via the SQD Portal API — Solana ecosystem blocks, transactions, instructions, balances, rewards, and logs at high throughput.

| Property   | Value             |
| ---------- | ----------------- |
| Network ID | `eclipse-mainnet` |
| Real-time  | No                |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/eclipse-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/eclipse-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/eclipse-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "solana",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "instruction": { "programId": true, "data": true }
    },
    "instructions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/solana/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Transactions">
    | Field              | Type    | Description                   |
    | ------------------ | ------- | ----------------------------- |
    | `index`            | integer | Transaction index in block    |
    | `signature`        | string  | Transaction signature         |
    | `account_keys`     | array   | Account keys involved         |
    | `recent_blockhash` | string  | Recent blockhash used         |
    | `fee`              | integer | Transaction fee               |
    | `success`          | boolean | Whether transaction succeeded |
    | `error`            | object  | Error information if failed   |
  </Accordion>

  <Accordion title="Instructions">
    | Field                | Type    | Description              |
    | -------------------- | ------- | ------------------------ |
    | `index`              | integer | Instruction index        |
    | `transaction_index`  | integer | Parent transaction index |
    | `program_id`         | string  | Program ID               |
    | `accounts`           | array   | Accounts involved        |
    | `data`               | string  | Instruction data         |
    | `inner_instructions` | array   | Inner instructions       |
  </Accordion>

  <Accordion title="Balances">
    | Field     | Type   | Description                |
    | --------- | ------ | -------------------------- |
    | `account` | string | Account address            |
    | `pre`     | bigint | Balance before transaction |
    | `post`    | bigint | Balance after transaction  |
  </Accordion>

  <Accordion title="Token Balances">
    | Field         | Type    | Description                |
    | ------------- | ------- | -------------------------- |
    | `account`     | string  | Token account              |
    | `mint`        | string  | Token mint address         |
    | `owner`       | string  | Owner address              |
    | `pre_amount`  | bigint  | Balance before transaction |
    | `post_amount` | bigint  | Balance after transaction  |
    | `decimals`    | integer | Token decimals             |
  </Accordion>

  <Accordion title="Rewards">
    | Field         | Type    | Description               |
    | ------------- | ------- | ------------------------- |
    | `pubkey`      | string  | Recipient account         |
    | `lamports`    | bigint  | Reward amount in lamports |
    | `reward_type` | string  | Type of reward            |
    | `commission`  | integer | Commission percentage     |
  </Accordion>

  <Accordion title="Logs">
    | Field                 | Type    | Description                  |
    | --------------------- | ------- | ---------------------------- |
    | `transaction_index`   | integer | Parent transaction index     |
    | `instruction_address` | array   | Instruction path             |
    | `program_id`          | string  | Program that emitted the log |
    | `kind`                | string  | Log kind                     |
    | `message`             | string  | Log message                  |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/solana/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Eclipse Testnet
Source: https://docs.sqd.dev/en/data/solana/eclipse-testnet

Stream Eclipse Testnet via the SQD Portal API — Solana ecosystem blocks, transactions, instructions, balances, rewards, and logs at high throughput.

| Property   | Value             |
| ---------- | ----------------- |
| Network ID | `eclipse-testnet` |
| Real-time  | No                |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/eclipse-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/eclipse-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/eclipse-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "solana",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "instruction": { "programId": true, "data": true }
    },
    "instructions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/solana/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Transactions">
    | Field              | Type    | Description                   |
    | ------------------ | ------- | ----------------------------- |
    | `index`            | integer | Transaction index in block    |
    | `signature`        | string  | Transaction signature         |
    | `account_keys`     | array   | Account keys involved         |
    | `recent_blockhash` | string  | Recent blockhash used         |
    | `fee`              | integer | Transaction fee               |
    | `success`          | boolean | Whether transaction succeeded |
    | `error`            | object  | Error information if failed   |
  </Accordion>

  <Accordion title="Instructions">
    | Field                | Type    | Description              |
    | -------------------- | ------- | ------------------------ |
    | `index`              | integer | Instruction index        |
    | `transaction_index`  | integer | Parent transaction index |
    | `program_id`         | string  | Program ID               |
    | `accounts`           | array   | Accounts involved        |
    | `data`               | string  | Instruction data         |
    | `inner_instructions` | array   | Inner instructions       |
  </Accordion>

  <Accordion title="Balances">
    | Field     | Type   | Description                |
    | --------- | ------ | -------------------------- |
    | `account` | string | Account address            |
    | `pre`     | bigint | Balance before transaction |
    | `post`    | bigint | Balance after transaction  |
  </Accordion>

  <Accordion title="Token Balances">
    | Field         | Type    | Description                |
    | ------------- | ------- | -------------------------- |
    | `account`     | string  | Token account              |
    | `mint`        | string  | Token mint address         |
    | `owner`       | string  | Owner address              |
    | `pre_amount`  | bigint  | Balance before transaction |
    | `post_amount` | bigint  | Balance after transaction  |
    | `decimals`    | integer | Token decimals             |
  </Accordion>

  <Accordion title="Rewards">
    | Field         | Type    | Description               |
    | ------------- | ------- | ------------------------- |
    | `pubkey`      | string  | Recipient account         |
    | `lamports`    | bigint  | Reward amount in lamports |
    | `reward_type` | string  | Type of reward            |
    | `commission`  | integer | Commission percentage     |
  </Accordion>

  <Accordion title="Logs">
    | Field                 | Type    | Description                  |
    | --------------------- | ------- | ---------------------------- |
    | `transaction_index`   | integer | Parent transaction index     |
    | `instruction_address` | array   | Instruction path             |
    | `program_id`          | string  | Program that emitted the log |
    | `kind`                | string  | Log kind                     |
    | `message`             | string  | Log message                  |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/solana/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Spray Client
Source: https://docs.sqd.dev/en/data/solana/shreds

Stream Spray Client via the SQD Portal API — Solana ecosystem blocks, transactions, instructions, balances, rewards, and logs at high throughput.

Spray is a WebSocket-based client that streams real-time Solana blockchain data from Geyser endpoints. It provides filtered, low-latency access to blocks, transactions, instructions, and balance changes.

## Installation

Run Spray using Docker:

```bash theme={"system"}
docker pull subsquid/spray:latest
docker run -p 3000:3000 -v $(pwd)/config.yaml:/config.yaml subsquid/spray:latest /config.yaml
```

## Configuration

Create a `config.yaml` file to define your data sources:

```yaml theme={"system"}
sources:
  mainnet:
    url: grpc://your-geyser-endpoint:10000
    x_token: your-token  # optional
    x_access_token: your-access-token  # optional

port: 3000  # optional, defaults to 3000
```

Multiple sources can be configured for redundancy. Spray automatically deduplicates data from multiple sources.

## Subscription API

Spray exposes a JSON-RPC 2.0 WebSocket API on the configured port.

### Subscribe to Data

**Method:** `spraySubscribe`

**Request:**

```json theme={"system"}
{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "spraySubscribe",
  "params": [{
    "includeAllBlocks": false,
    "fields": {
      "block": {
        "number": true,
        "hash": true,
        "timestamp": true
      },
      "transaction": {
        "transactionIndex": true,
        "signatures": true,
        "err": true,
        "fee": true
      },
      "instruction": {
        "programId": true,
        "accounts": true,
        "data": true
      }
    },
    "transactions": [{
      "feePayer": ["address1", "address2"]
    }],
    "instructions": [{
      "programId": ["program1", "program2"],
      "d8": ["0x0102030405060708"],
      "a0": ["account1"]
    }],
    "balances": [{
      "account": ["address1"]
    }],
    "tokenBalances": [{
      "preMint": ["mint1"],
      "postMint": ["mint2"]
    }]
  }]
}
```

### Query Parameters

**Top-level options:**

* `includeAllBlocks` (boolean) - Include all block headers, not just blocks with matching transactions
* `fields` (object) - Specify which fields to include in responses

**Filter arrays:**

* `transactions` - Array of transaction filters
* `instructions` - Array of instruction filters
* `balances` - Array of balance change filters
* `tokenBalances` - Array of token balance change filters

**Transaction filters:**

* `feePayer` - Filter by fee payer addresses
* `mentionsAccount` - Filter by any mentioned account
* `instructions` - Include all instructions
* `logs` - Include transaction logs
* `balances` - Include balance changes
* `tokenBalances` - Include token balance changes

**Instruction filters:**

* `programId` - Filter by program ID
* `discriminator` - Filter by instruction discriminator (variable length)
* `d1`, `d2`, `d4`, `d8` - Filter by fixed-length discriminators (1, 2, 4, 8 bytes)
* `mentionsAccount` - Filter by any account in instruction
* `a0` through `a15` - Filter by specific account positions
* `isCommitted` - Filter by commitment status
* `transaction` - Include parent transaction
* `transactionInstructions` - Include all instructions from transaction
* `innerInstructions` - Include inner instructions
* `parentInstructions` - Include parent instructions

**Balance filters:**

* `account` - Filter by account address
* `transaction` - Include parent transaction
* `transactionInstructions` - Include transaction instructions

**Token balance filters:**

* `account` - Filter by account address
* `preMint`, `postMint` - Filter by token mint (before/after)
* `preProgramId`, `postProgramId` - Filter by token program
* `preOwner`, `postOwner` - Filter by token owner
* `transaction` - Include parent transaction
* `transactionInstructions` - Include transaction instructions

### Response Format

Spray sends JSON messages for each matching block or transaction:

**Block message:**

```json theme={"system"}
{
  "type": "block",
  "slot": 123456789,
  "header": {
    "number": 123456789,
    "hash": "...",
    "timestamp": 1234567890
  }
}
```

**Transaction message:**

```json theme={"system"}
{
  "type": "transaction",
  "slot": 123456789,
  "transactionIndex": 42,
  "transaction": {
    "signatures": ["..."],
    "fee": "5000"
  },
  "instructions": [...],
  "balances": [...],
  "tokenBalances": [...]
}
```

### Unsubscribe

**Method:** `sprayUnsubscribe`

```json theme={"system"}
{
  "jsonrpc": "2.0",
  "id": 2,
  "method": "sprayUnsubscribe",
  "params": ["subscription_id"]
}
```

## Metrics

Spray exposes Prometheus metrics at `/metrics`:

* `spray_blocks_published` - Number of blocks pushed to subscriptions
* `spray_transactions_published` - Number of transactions pushed to subscriptions
* `spray_last_block` - Last published block number
* `spray_active_subscriptions` - Number of active client subscriptions
* `spray_data_source_errors` - Number of data source connection errors
* `spray_mapping_errors` - Number of data mapping errors

## Source Code

Spray is open source and available at [github.com/subsquid/spray](https://github.com/subsquid/spray).


# Solana Devnet
Source: https://docs.sqd.dev/en/data/solana/solana-devnet

Stream Solana Devnet via the SQD Portal API — Solana ecosystem blocks, transactions, instructions, balances, rewards, and logs at high throughput.

| Property   | Value           |
| ---------- | --------------- |
| Network ID | `solana-devnet` |
| Real-time  | Yes             |

## Endpoint

```
https://portal.sqd.dev/datasets/solana-devnet
```

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/solana-devnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "solana",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "instruction": { "programId": true, "data": true }
    },
    "instructions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/solana/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Transactions">
    | Field              | Type    | Description                   |
    | ------------------ | ------- | ----------------------------- |
    | `index`            | integer | Transaction index in block    |
    | `signature`        | string  | Transaction signature         |
    | `account_keys`     | array   | Account keys involved         |
    | `recent_blockhash` | string  | Recent blockhash used         |
    | `fee`              | integer | Transaction fee               |
    | `success`          | boolean | Whether transaction succeeded |
    | `error`            | object  | Error information if failed   |
  </Accordion>

  <Accordion title="Instructions">
    | Field                | Type    | Description              |
    | -------------------- | ------- | ------------------------ |
    | `index`              | integer | Instruction index        |
    | `transaction_index`  | integer | Parent transaction index |
    | `program_id`         | string  | Program ID               |
    | `accounts`           | array   | Accounts involved        |
    | `data`               | string  | Instruction data         |
    | `inner_instructions` | array   | Inner instructions       |
  </Accordion>

  <Accordion title="Balances">
    | Field     | Type   | Description                |
    | --------- | ------ | -------------------------- |
    | `account` | string | Account address            |
    | `pre`     | bigint | Balance before transaction |
    | `post`    | bigint | Balance after transaction  |
  </Accordion>

  <Accordion title="Token Balances">
    | Field         | Type    | Description                |
    | ------------- | ------- | -------------------------- |
    | `account`     | string  | Token account              |
    | `mint`        | string  | Token mint address         |
    | `owner`       | string  | Owner address              |
    | `pre_amount`  | bigint  | Balance before transaction |
    | `post_amount` | bigint  | Balance after transaction  |
    | `decimals`    | integer | Token decimals             |
  </Accordion>

  <Accordion title="Rewards">
    | Field         | Type    | Description               |
    | ------------- | ------- | ------------------------- |
    | `pubkey`      | string  | Recipient account         |
    | `lamports`    | bigint  | Reward amount in lamports |
    | `reward_type` | string  | Type of reward            |
    | `commission`  | integer | Commission percentage     |
  </Accordion>

  <Accordion title="Logs">
    | Field                 | Type    | Description                  |
    | --------------------- | ------- | ---------------------------- |
    | `transaction_index`   | integer | Parent transaction index     |
    | `instruction_address` | array   | Instruction path             |
    | `program_id`          | string  | Program that emitted the log |
    | `kind`                | string  | Log kind                     |
    | `message`             | string  | Log message                  |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/solana/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Solana
Source: https://docs.sqd.dev/en/data/solana/solana-mainnet

Stream Solana via the SQD Portal API — Solana ecosystem blocks, transactions, instructions, balances, rewards, and logs at high throughput.

<Info>Solana on Portal is available from genesis and uses slots, not blocks. V2 archives (legacy) are available starting from block 269,828,500 (block, not slot).</Info>

| Property   | Value            |
| ---------- | ---------------- |
| Network ID | `solana-mainnet` |
| Real-time  | Yes              |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/solana-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/solana-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/solana-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "solana",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "instruction": { "programId": true, "data": true }
    },
    "instructions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/solana/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Transactions">
    | Field              | Type    | Description                   |
    | ------------------ | ------- | ----------------------------- |
    | `index`            | integer | Transaction index in block    |
    | `signature`        | string  | Transaction signature         |
    | `account_keys`     | array   | Account keys involved         |
    | `recent_blockhash` | string  | Recent blockhash used         |
    | `fee`              | integer | Transaction fee               |
    | `success`          | boolean | Whether transaction succeeded |
    | `error`            | object  | Error information if failed   |
  </Accordion>

  <Accordion title="Instructions">
    | Field                | Type    | Description              |
    | -------------------- | ------- | ------------------------ |
    | `index`              | integer | Instruction index        |
    | `transaction_index`  | integer | Parent transaction index |
    | `program_id`         | string  | Program ID               |
    | `accounts`           | array   | Accounts involved        |
    | `data`               | string  | Instruction data         |
    | `inner_instructions` | array   | Inner instructions       |
  </Accordion>

  <Accordion title="Balances">
    | Field     | Type   | Description                |
    | --------- | ------ | -------------------------- |
    | `account` | string | Account address            |
    | `pre`     | bigint | Balance before transaction |
    | `post`    | bigint | Balance after transaction  |
  </Accordion>

  <Accordion title="Token Balances">
    | Field         | Type    | Description                |
    | ------------- | ------- | -------------------------- |
    | `account`     | string  | Token account              |
    | `mint`        | string  | Token mint address         |
    | `owner`       | string  | Owner address              |
    | `pre_amount`  | bigint  | Balance before transaction |
    | `post_amount` | bigint  | Balance after transaction  |
    | `decimals`    | integer | Token decimals             |
  </Accordion>

  <Accordion title="Rewards">
    | Field         | Type    | Description               |
    | ------------- | ------- | ------------------------- |
    | `pubkey`      | string  | Recipient account         |
    | `lamports`    | bigint  | Reward amount in lamports |
    | `reward_type` | string  | Type of reward            |
    | `commission`  | integer | Commission percentage     |
  </Accordion>

  <Accordion title="Logs">
    | Field                 | Type    | Description                  |
    | --------------------- | ------- | ---------------------------- |
    | `transaction_index`   | integer | Parent transaction index     |
    | `instruction_address` | array   | Instruction path             |
    | `program_id`          | string  | Program that emitted the log |
    | `kind`                | string  | Log kind                     |
    | `message`             | string  | Log message                  |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/solana/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Soon Devnet
Source: https://docs.sqd.dev/en/data/solana/soon-devnet

Stream Soon Devnet via the SQD Portal API — Solana ecosystem blocks, transactions, instructions, balances, rewards, and logs at high throughput.

| Property   | Value         |
| ---------- | ------------- |
| Network ID | `soon-devnet` |
| Real-time  | No            |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/soon-devnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/soon-devnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/soon-devnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "solana",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "instruction": { "programId": true, "data": true }
    },
    "instructions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/solana/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Transactions">
    | Field              | Type    | Description                   |
    | ------------------ | ------- | ----------------------------- |
    | `index`            | integer | Transaction index in block    |
    | `signature`        | string  | Transaction signature         |
    | `account_keys`     | array   | Account keys involved         |
    | `recent_blockhash` | string  | Recent blockhash used         |
    | `fee`              | integer | Transaction fee               |
    | `success`          | boolean | Whether transaction succeeded |
    | `error`            | object  | Error information if failed   |
  </Accordion>

  <Accordion title="Instructions">
    | Field                | Type    | Description              |
    | -------------------- | ------- | ------------------------ |
    | `index`              | integer | Instruction index        |
    | `transaction_index`  | integer | Parent transaction index |
    | `program_id`         | string  | Program ID               |
    | `accounts`           | array   | Accounts involved        |
    | `data`               | string  | Instruction data         |
    | `inner_instructions` | array   | Inner instructions       |
  </Accordion>

  <Accordion title="Balances">
    | Field     | Type   | Description                |
    | --------- | ------ | -------------------------- |
    | `account` | string | Account address            |
    | `pre`     | bigint | Balance before transaction |
    | `post`    | bigint | Balance after transaction  |
  </Accordion>

  <Accordion title="Token Balances">
    | Field         | Type    | Description                |
    | ------------- | ------- | -------------------------- |
    | `account`     | string  | Token account              |
    | `mint`        | string  | Token mint address         |
    | `owner`       | string  | Owner address              |
    | `pre_amount`  | bigint  | Balance before transaction |
    | `post_amount` | bigint  | Balance after transaction  |
    | `decimals`    | integer | Token decimals             |
  </Accordion>

  <Accordion title="Rewards">
    | Field         | Type    | Description               |
    | ------------- | ------- | ------------------------- |
    | `pubkey`      | string  | Recipient account         |
    | `lamports`    | bigint  | Reward amount in lamports |
    | `reward_type` | string  | Type of reward            |
    | `commission`  | integer | Commission percentage     |
  </Accordion>

  <Accordion title="Logs">
    | Field                 | Type    | Description                  |
    | --------------------- | ------- | ---------------------------- |
    | `transaction_index`   | integer | Parent transaction index     |
    | `instruction_address` | array   | Instruction path             |
    | `program_id`          | string  | Program that emitted the log |
    | `kind`                | string  | Log kind                     |
    | `message`             | string  | Log message                  |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/solana/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Soon
Source: https://docs.sqd.dev/en/data/solana/soon-mainnet

Stream Soon via the SQD Portal API — Solana ecosystem blocks, transactions, instructions, balances, rewards, and logs at high throughput.

| Property   | Value          |
| ---------- | -------------- |
| Network ID | `soon-mainnet` |
| Real-time  | No             |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/soon-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/soon-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/soon-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "solana",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "instruction": { "programId": true, "data": true }
    },
    "instructions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/solana/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Transactions">
    | Field              | Type    | Description                   |
    | ------------------ | ------- | ----------------------------- |
    | `index`            | integer | Transaction index in block    |
    | `signature`        | string  | Transaction signature         |
    | `account_keys`     | array   | Account keys involved         |
    | `recent_blockhash` | string  | Recent blockhash used         |
    | `fee`              | integer | Transaction fee               |
    | `success`          | boolean | Whether transaction succeeded |
    | `error`            | object  | Error information if failed   |
  </Accordion>

  <Accordion title="Instructions">
    | Field                | Type    | Description              |
    | -------------------- | ------- | ------------------------ |
    | `index`              | integer | Instruction index        |
    | `transaction_index`  | integer | Parent transaction index |
    | `program_id`         | string  | Program ID               |
    | `accounts`           | array   | Accounts involved        |
    | `data`               | string  | Instruction data         |
    | `inner_instructions` | array   | Inner instructions       |
  </Accordion>

  <Accordion title="Balances">
    | Field     | Type   | Description                |
    | --------- | ------ | -------------------------- |
    | `account` | string | Account address            |
    | `pre`     | bigint | Balance before transaction |
    | `post`    | bigint | Balance after transaction  |
  </Accordion>

  <Accordion title="Token Balances">
    | Field         | Type    | Description                |
    | ------------- | ------- | -------------------------- |
    | `account`     | string  | Token account              |
    | `mint`        | string  | Token mint address         |
    | `owner`       | string  | Owner address              |
    | `pre_amount`  | bigint  | Balance before transaction |
    | `post_amount` | bigint  | Balance after transaction  |
    | `decimals`    | integer | Token decimals             |
  </Accordion>

  <Accordion title="Rewards">
    | Field         | Type    | Description               |
    | ------------- | ------- | ------------------------- |
    | `pubkey`      | string  | Recipient account         |
    | `lamports`    | bigint  | Reward amount in lamports |
    | `reward_type` | string  | Type of reward            |
    | `commission`  | integer | Commission percentage     |
  </Accordion>

  <Accordion title="Logs">
    | Field                 | Type    | Description                  |
    | --------------------- | ------- | ---------------------------- |
    | `transaction_index`   | integer | Parent transaction index     |
    | `instruction_address` | array   | Instruction path             |
    | `program_id`          | string  | Program that emitted the log |
    | `kind`                | string  | Log kind                     |
    | `message`             | string  | Log message                  |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/solana/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Soon Testnet
Source: https://docs.sqd.dev/en/data/solana/soon-testnet

Stream Soon Testnet via the SQD Portal API — Solana ecosystem blocks, transactions, instructions, balances, rewards, and logs at high throughput.

| Property   | Value          |
| ---------- | -------------- |
| Network ID | `soon-testnet` |
| Real-time  | No             |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/soon-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/soon-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/soon-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "solana",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "instruction": { "programId": true, "data": true }
    },
    "instructions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/solana/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Transactions">
    | Field              | Type    | Description                   |
    | ------------------ | ------- | ----------------------------- |
    | `index`            | integer | Transaction index in block    |
    | `signature`        | string  | Transaction signature         |
    | `account_keys`     | array   | Account keys involved         |
    | `recent_blockhash` | string  | Recent blockhash used         |
    | `fee`              | integer | Transaction fee               |
    | `success`          | boolean | Whether transaction succeeded |
    | `error`            | object  | Error information if failed   |
  </Accordion>

  <Accordion title="Instructions">
    | Field                | Type    | Description              |
    | -------------------- | ------- | ------------------------ |
    | `index`              | integer | Instruction index        |
    | `transaction_index`  | integer | Parent transaction index |
    | `program_id`         | string  | Program ID               |
    | `accounts`           | array   | Accounts involved        |
    | `data`               | string  | Instruction data         |
    | `inner_instructions` | array   | Inner instructions       |
  </Accordion>

  <Accordion title="Balances">
    | Field     | Type   | Description                |
    | --------- | ------ | -------------------------- |
    | `account` | string | Account address            |
    | `pre`     | bigint | Balance before transaction |
    | `post`    | bigint | Balance after transaction  |
  </Accordion>

  <Accordion title="Token Balances">
    | Field         | Type    | Description                |
    | ------------- | ------- | -------------------------- |
    | `account`     | string  | Token account              |
    | `mint`        | string  | Token mint address         |
    | `owner`       | string  | Owner address              |
    | `pre_amount`  | bigint  | Balance before transaction |
    | `post_amount` | bigint  | Balance after transaction  |
    | `decimals`    | integer | Token decimals             |
  </Accordion>

  <Accordion title="Rewards">
    | Field         | Type    | Description               |
    | ------------- | ------- | ------------------------- |
    | `pubkey`      | string  | Recipient account         |
    | `lamports`    | bigint  | Reward amount in lamports |
    | `reward_type` | string  | Type of reward            |
    | `commission`  | integer | Commission percentage     |
  </Accordion>

  <Accordion title="Logs">
    | Field                 | Type    | Description                  |
    | --------------------- | ------- | ---------------------------- |
    | `transaction_index`   | integer | Parent transaction index     |
    | `instruction_address` | array   | Instruction path             |
    | `program_id`          | string  | Program that emitted the log |
    | `kind`                | string  | Log kind                     |
    | `message`             | string  | Log message                  |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/solana/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# SVM BNB
Source: https://docs.sqd.dev/en/data/solana/svm-bnb-mainnet

Stream SVM BNB via the SQD Portal API — Solana ecosystem blocks, transactions, instructions, balances, rewards, and logs at high throughput.

| Property   | Value             |
| ---------- | ----------------- |
| Network ID | `svm-bnb-mainnet` |
| Real-time  | No                |

## Endpoint

```
https://portal.sqd.dev/datasets/svm-bnb-mainnet
```

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/svm-bnb-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "solana",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "instruction": { "programId": true, "data": true }
    },
    "instructions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/solana/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Transactions">
    | Field              | Type    | Description                   |
    | ------------------ | ------- | ----------------------------- |
    | `index`            | integer | Transaction index in block    |
    | `signature`        | string  | Transaction signature         |
    | `account_keys`     | array   | Account keys involved         |
    | `recent_blockhash` | string  | Recent blockhash used         |
    | `fee`              | integer | Transaction fee               |
    | `success`          | boolean | Whether transaction succeeded |
    | `error`            | object  | Error information if failed   |
  </Accordion>

  <Accordion title="Instructions">
    | Field                | Type    | Description              |
    | -------------------- | ------- | ------------------------ |
    | `index`              | integer | Instruction index        |
    | `transaction_index`  | integer | Parent transaction index |
    | `program_id`         | string  | Program ID               |
    | `accounts`           | array   | Accounts involved        |
    | `data`               | string  | Instruction data         |
    | `inner_instructions` | array   | Inner instructions       |
  </Accordion>

  <Accordion title="Balances">
    | Field     | Type   | Description                |
    | --------- | ------ | -------------------------- |
    | `account` | string | Account address            |
    | `pre`     | bigint | Balance before transaction |
    | `post`    | bigint | Balance after transaction  |
  </Accordion>

  <Accordion title="Token Balances">
    | Field         | Type    | Description                |
    | ------------- | ------- | -------------------------- |
    | `account`     | string  | Token account              |
    | `mint`        | string  | Token mint address         |
    | `owner`       | string  | Owner address              |
    | `pre_amount`  | bigint  | Balance before transaction |
    | `post_amount` | bigint  | Balance after transaction  |
    | `decimals`    | integer | Token decimals             |
  </Accordion>

  <Accordion title="Rewards">
    | Field         | Type    | Description               |
    | ------------- | ------- | ------------------------- |
    | `pubkey`      | string  | Recipient account         |
    | `lamports`    | bigint  | Reward amount in lamports |
    | `reward_type` | string  | Type of reward            |
    | `commission`  | integer | Commission percentage     |
  </Accordion>

  <Accordion title="Logs">
    | Field                 | Type    | Description                  |
    | --------------------- | ------- | ---------------------------- |
    | `transaction_index`   | integer | Parent transaction index     |
    | `instruction_address` | array   | Instruction path             |
    | `program_id`          | string  | Program that emitted the log |
    | `kind`                | string  | Log kind                     |
    | `message`             | string  | Log message                  |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/solana/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# SVM BNB Testnet
Source: https://docs.sqd.dev/en/data/solana/svm-bnb-testnet

Stream SVM BNB Testnet via the SQD Portal API — Solana ecosystem blocks, transactions, instructions, balances, rewards, and logs at high throughput.

| Property   | Value             |
| ---------- | ----------------- |
| Network ID | `svm-bnb-testnet` |
| Real-time  | No                |

## Endpoint

```
https://portal.sqd.dev/datasets/svm-bnb-testnet
```

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/svm-bnb-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "solana",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "instruction": { "programId": true, "data": true }
    },
    "instructions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/solana/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Transactions">
    | Field              | Type    | Description                   |
    | ------------------ | ------- | ----------------------------- |
    | `index`            | integer | Transaction index in block    |
    | `signature`        | string  | Transaction signature         |
    | `account_keys`     | array   | Account keys involved         |
    | `recent_blockhash` | string  | Recent blockhash used         |
    | `fee`              | integer | Transaction fee               |
    | `success`          | boolean | Whether transaction succeeded |
    | `error`            | object  | Error information if failed   |
  </Accordion>

  <Accordion title="Instructions">
    | Field                | Type    | Description              |
    | -------------------- | ------- | ------------------------ |
    | `index`              | integer | Instruction index        |
    | `transaction_index`  | integer | Parent transaction index |
    | `program_id`         | string  | Program ID               |
    | `accounts`           | array   | Accounts involved        |
    | `data`               | string  | Instruction data         |
    | `inner_instructions` | array   | Inner instructions       |
  </Accordion>

  <Accordion title="Balances">
    | Field     | Type   | Description                |
    | --------- | ------ | -------------------------- |
    | `account` | string | Account address            |
    | `pre`     | bigint | Balance before transaction |
    | `post`    | bigint | Balance after transaction  |
  </Accordion>

  <Accordion title="Token Balances">
    | Field         | Type    | Description                |
    | ------------- | ------- | -------------------------- |
    | `account`     | string  | Token account              |
    | `mint`        | string  | Token mint address         |
    | `owner`       | string  | Owner address              |
    | `pre_amount`  | bigint  | Balance before transaction |
    | `post_amount` | bigint  | Balance after transaction  |
    | `decimals`    | integer | Token decimals             |
  </Accordion>

  <Accordion title="Rewards">
    | Field         | Type    | Description               |
    | ------------- | ------- | ------------------------- |
    | `pubkey`      | string  | Recipient account         |
    | `lamports`    | bigint  | Reward amount in lamports |
    | `reward_type` | string  | Type of reward            |
    | `commission`  | integer | Commission percentage     |
  </Accordion>

  <Accordion title="Logs">
    | Field                 | Type    | Description                  |
    | --------------------- | ------- | ---------------------------- |
    | `transaction_index`   | integer | Parent transaction index     |
    | `instruction_address` | array   | Instruction path             |
    | `program_id`          | string  | Program that emitted the log |
    | `kind`                | string  | Log kind                     |
    | `message`             | string  | Log message                  |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/solana/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Solana Portal API Reference
Source: https://docs.sqd.dev/en/portal/solana/api

Quick reference for the SQD Portal Solana HTTP API — endpoints, request schemas, and code samples for querying blocks, transactions, and instructions.

This page provides a quick reference for the Portal API. For the complete OpenAPI specification with all fields and options:

<Card title="Complete Solana API Specification" icon="book-open" href="/en/portal/solana/api">
  View the full OpenAPI documentation with all available fields, filters, and
  detailed schemas →
</Card>

## Base URL

All Portal requests are made to:

```
https://portal.sqd.dev/datasets/{network}
```

Replace `{network}` with your target blockchain (e.g., `solana-mainnet`).

## Core Endpoints

<CardGroup>
  <Card title="POST /stream" icon="wave-pulse">
    Stream blockchain data with filters. Includes unfinalized blocks.
  </Card>

  <Card title="POST /finalized-stream" icon="check-circle">
    Stream only finalized blocks. Use for production pipelines requiring
    certainty.
  </Card>

  <Card title="GET /metadata" icon="info-circle">
    Get dataset information: slot range, finality rules, and network details.
  </Card>
</CardGroup>

## Installation

```bash theme={"system"}
npm install @subsquid/portal-client
```

## Quick Start Examples

<CodeGroup>
  ```bash curl theme={"system"}
  curl --compressed -X POST 'https://portal.sqd.dev/datasets/solana-mainnet/stream' \
    -H 'Content-Type: application/json' \
    -d '{
      "type": "solana",
      "fromBlock": 259984950,
      "toBlock": 259985000,
      "fields": {
        "block": { "number": true, "timestamp": true },
        "instruction": { "programId": true, "accounts": true, "data": true }
      }
    }'
  ```

  ```typescript TypeScript theme={"system"}
  import { DataSource } from "@subsquid/portal-client";

  const dataSource = new DataSource({
    network: "solana-mainnet",
  });

  const blocks = await dataSource.getBlocks({
    from: 259984950,
    to: 259985000,
    fields: {
      block: { number: true, timestamp: true },
      instruction: { programId: true, accounts: true, data: true },
    },
  });
  ```

  ```python Python theme={"system"}
  import requests

  url = "https://portal.sqd.dev/datasets/solana-mainnet/stream"
  headers = {"Content-Type": "application/json"}
  payload = {
      "type": "solana",
      "fromBlock": 259984950,
      "toBlock": 259985000,
      "fields": {
          "block": {"number": True, "timestamp": True},
          "instruction": {"programId": True, "accounts": True, "data": True}
      }
  }

  response = requests.post(url, headers=headers, json=payload)
  ```
</CodeGroup>

## Available Fields

### Block Fields

Select which block header fields to retrieve:

```typescript theme={"system"}
fields: {
  block: {
    number: true,          // Slot number (field name: number)
    hash: true,            // Block hash
    parentNumber: true,    // Parent slot number
    parentHash: true,      // Parent block hash
    height: true,          // Block height
    timestamp: true,       // Unix timestamp
  }
}
```

### Transaction Fields

Select transaction data fields:

```typescript theme={"system"}
fields: {
  transaction: {
    signatures: true,      // Transaction signatures (base58)
    feePayer: true,        // Fee payer address
    err: true,             // Transaction error (null if successful)
    version: true,         // Transaction version
    accountKeys: true,     // Account keys
    recentBlockhash: true, // Recent blockhash
    fee: true,             // Transaction fee
    computeUnitsConsumed: true, // Compute units consumed
  }
}
```

### Instruction Fields

Select instruction data fields:

```typescript theme={"system"}
fields: {
  instruction: {
    programId: true,       // Program address
    accounts: true,        // Account addresses
    data: true,            // Instruction data (base58)
    transactionIndex: true, // Transaction index (use to match with transactions)
    instructionAddress: true,  // Instruction address in call tree
    isCommitted: true,     // Whether instruction succeeded
  }
}
```

### Log Fields

Select log message fields:

```typescript theme={"system"}
fields: {
  log: {
    programId: true,       // Program address
    message: true,         // Log message
    kind: true,            // Log kind (log, data, other)
    transactionIndex: true, // Transaction index (use to match with transactions)
    instructionAddress: true,  // Instruction address
  }
}
```

### Token Balance Fields

Select token balance fields:

```typescript theme={"system"}
fields: {
  tokenBalance: {
    account: true,         // Token account address
    preMint: true,         // Mint before transaction
    postMint: true,        // Mint after transaction
    preAmount: true,       // Balance before transaction
    postAmount: true,      // Balance after transaction
    preOwner: true,        // Owner before transaction
    postOwner: true,       // Owner after transaction
  }
}
```

### Balance Fields

Select SOL balance fields:

```typescript theme={"system"}
fields: {
  balance: {
    account: true,         // Account address
    pre: true,             // Balance before transaction
    post: true,            // Balance after transaction
  }
}
```

### Reward Fields

Select reward fields:

```typescript theme={"system"}
fields: {
  reward: {
    pubkey: true,          // Reward recipient
    lamports: true,        // Reward amount
    rewardType: true,      // Reward type
    postBalance: true,     // Balance after reward
  }
}
```

## Filtering Options

### Instruction Filters

Filter instructions by program, discriminator, or accounts:

```typescript theme={"system"}
instructions: [
  {
    programId: ["whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc"], // Orca Whirlpool
    d8: ["0xf8c69e91e17587c8"], // Swap instruction discriminator
    accounts: ["7qbRF6YsyGuLUVs6Y1q64bdVrfe4ZcUUz1JRdoVNUJnm"], // Specific pool
  },
];
```

**Filter logic:**

* Multiple program IDs in one object = OR
* Multiple discriminators in one object = OR
* Multiple filter objects = OR

### Transaction Filters

Filter transactions by fee payer:

```typescript theme={"system"}
transactions: [
  {
    feePayer: ["YourWalletAddressHere"], // Fee payer addresses
  },
];
```

### Log Filters

Filter logs by program and kind:

```typescript theme={"system"}
logs: [
  {
    programId: ["whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc"],
    kind: ["log"], // log, data, or other
  },
];
```

### Token Balance Filters

Filter token balances by mint or account:

```typescript theme={"system"}
tokenBalances: [
  {
    preMint: ["EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v"], // USDC
    account: ["TokenAccountAddress..."],
  },
];
```

## Network Selection

Specify the Solana network when creating the data source:

```typescript theme={"system"}
// Solana mainnet
const solana = new DataSource({ network: "solana-mainnet" });
```

<Card title="View All Networks" icon="database" href="/en/data/networks/evm">
  See complete list of supported Solana networks
</Card>

## Query Patterns

### Pattern 1: Range Query

Query a specific slot range:

```typescript theme={"system"}
const blocks = await dataSource.getBlocks({
  from: 259984950,
  to: 260000000,
  fields: {
    /* fields */
  },
  instructions: [
    /* filters */
  ],
});
```

### Pattern 2: Batch Processing

Process large ranges in batches:

```typescript theme={"system"}
const BATCH_SIZE = 10000;

for (let i = startSlot; i < endSlot; i += BATCH_SIZE) {
  const blocks = await dataSource.getBlocks({
    from: i,
    to: Math.min(i + BATCH_SIZE, endSlot),
    fields: {
      /* fields */
    },
  });

  // Process blocks
  await processBlocks(blocks);
}
```

### Pattern 3: Multi-Program Query

Query multiple programs simultaneously:

```typescript theme={"system"}
const blocks = await dataSource.getBlocks({
  from: 259984950,
  to: 260000000,
  fields: {
    instruction: { programId: true, accounts: true, data: true },
  },
  instructions: [
    {
      programId: [
        "whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc", // Orca Whirlpool
        "TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA", // SPL Token
      ],
    },
  ],
});
```

## Performance Optimization

### 1. Request Only Needed Fields

```typescript theme={"system"}
// Good: Minimal fields
fields: {
  instruction: {
    programId: true,
    accounts: true,
  }
}

// Avoid: Requesting unnecessary fields
fields: {
  instruction: {
    programId: true,
    accounts: true,
    data: true,        // Only if needed
    transactionIndex: true,     // Only if needed (use to match with transactions)
  }
}
```

### 2. Use Specific Filters

```typescript theme={"system"}
// Good: Filter at Portal level
instructions: [
  {
    programId: ["SpecificProgram"],
    d8: ["SpecificDiscriminator"],
  },
];

// Avoid: Fetching everything and filtering in code
instructions: []; // Gets all instructions
```

### 3. Optimal Batch Sizes

```typescript theme={"system"}
// Good: 10k-50k slots per query for most use cases
const BATCH_SIZE = 10000;

// Adjust based on:
// - More filters = larger batches OK
// - More fields = smaller batches
// - More activity (mainnet) = smaller batches
```

### 4. Parallel Queries

```typescript theme={"system"}
// Process multiple ranges in parallel
const ranges = [
  { from: 259984950, to: 260000000 },
  { from: 260000000, to: 260015000 },
  { from: 260015000, to: 260030000 },
];

const results = await Promise.all(
  ranges.map((range) =>
    dataSource.getBlocks({
      from: range.from,
      to: range.to,
      fields: {
        /* fields */
      },
    })
  )
);
```

## Stream continuation

A single `/stream` response is a **batch**, not the complete range. The server may close the connection at any point: when a worker's range ends, when a connection is recycled, or at the current dataset head. Clients that treat one HTTP response as the whole query will silently drop slots.

To stream a range to completion, loop:

```typescript theme={"system"}
async function streamRange(from: number, to: number | undefined, body: any) {
  let currentFrom = from;
  while (to === undefined || currentFrom <= to) {
    const res = await fetch(
      "https://portal.sqd.dev/datasets/solana-mainnet/stream",
      {
        method: "POST",
        headers: { "Content-Type": "application/json" },
        body: JSON.stringify({ ...body, fromBlock: currentFrom, toBlock: to }),
      }
    );

    if (res.status === 204) break; // range is above the dataset head
    if (!res.ok) throw new Error(`HTTP ${res.status}: ${await res.text()}`);

    const lines = (await res.text()).trim().split("\n").filter(Boolean);
    if (lines.length === 0) break;

    let lastBlock = currentFrom - 1;
    for (const line of lines) {
      const block = JSON.parse(line);
      // ... process block ...
      lastBlock = block.header.number;
    }
    currentFrom = lastBlock + 1;
  }
}
```

Key points:

* Read the last block's `header.number`, issue the next request with `fromBlock = lastNumber + 1`.
* Keep `toBlock`, `fields`, and filters identical across retries.
* A `204 No Content` response means the range is above the dataset head. For historical queries, you're done; for open-ended queries, wait and retry.

## Handling reorgs near the chain head

When a request starts at or near the chain tip, the block you asked for may no longer be on the canonical chain by the time Portal processes the query. To detect this, pass `parentBlockHash` with each request (the hash of the block immediately before `fromBlock`):

```typescript theme={"system"}
const body = {
  type: "solana",
  fromBlock: lastBlock.number + 1,
  parentBlockHash: lastBlock.hash, // hash of lastBlock, i.e. parent of fromBlock
  fields: { block: { number: true, hash: true, parentHash: true } },
  instructions: [
    /* filters */
  ],
};
```

If the chain has reorged away from `parentBlockHash`, Portal returns **HTTP 409** with a list of blocks that **are** on the current canonical chain:

```json theme={"system"}
{
  "previousBlocks": [
    { "number": 259984850, "hash": "A1b2C3..." },
    { "number": 259984849, "hash": "9F8e7D..." }
  ]
}
```

Client recovery:

1. Find a block in your local state whose `(number, hash)` matches one of `previousBlocks`.
2. Roll back your pipeline to that block.
3. Resume the stream with `fromBlock = matched.number + 1` and `parentBlockHash = matched.hash`.

Streams opened against `/finalized-stream` (instead of `/stream`) never return 409: only finalized blocks are served, so there's nothing to reorg. Use that endpoint when you don't need real-time data.

## Error Handling

```typescript theme={"system"}
try {
  const blocks = await dataSource.getBlocks({
    from: 259984950,
    to: 260000000,
    fields: {
      /* fields */
    },
  });
} catch (error) {
  if (error.code === "RATE_LIMIT_EXCEEDED") {
    // HTTP 429: Public Portal rate limit. Honor Retry-After if set.
    console.error("Rate limit exceeded, waiting...");
    await new Promise((resolve) => setTimeout(resolve, 10000));
  } else if (error.code === "CONFLICT") {
    // HTTP 409: chain reorg. See "Handling reorgs near the chain head" above.
    const { previousBlocks } = error.body;
    // Roll back to a common ancestor and resume.
  } else if (error.code === "INVALID_SLOT_RANGE") {
    // HTTP 400: fromBlock is below the dataset's start_block, or the query is malformed.
    console.error("Invalid slot range");
  } else if (error.code === "NETWORK_ERROR") {
    // Transport error, retry with backoff.
    console.error("Network error, retrying...");
  } else {
    // HTTP 500: do not retry. HTTP 503: retry later, honor Retry-After.
    console.error("Unknown error:", error);
  }
}
```

## Common Use Cases

### Use Case 1: Instruction Monitoring

Monitor specific program instructions:

```typescript theme={"system"}
const blocks = await dataSource.getBlocks({
  from: latestProcessedSlot + 1,
  fields: {
    block: { number: true, timestamp: true },
    instruction: { programId: true, accounts: true, data: true },
  },
  instructions: [
    {
      programId: [PROGRAM_ADDRESS],
      d8: [INSTRUCTION_DISCRIMINATOR],
    },
  ],
});
```

### Use Case 2: Transaction Tracking

Track transactions from/to specific addresses:

```typescript theme={"system"}
const blocks = await dataSource.getBlocks({
  from: startSlot,
  to: endSlot,
  fields: {
    transaction: { signatures: true, feePayer: true },
  },
  transactions: [
    {
      feePayer: [WATCHED_ADDRESS],
    },
  ],
});
```

### Use Case 3: Analytics Pipeline

Build analytics on blockchain data:

```typescript theme={"system"}
const blocks = await dataSource.getBlocks({
  from: startSlot,
  to: endSlot,
  fields: {
    block: { number: true, timestamp: true },
    transaction: { fee: true },
  },
});

// Aggregate statistics
const stats = blocks.map((block) => ({
  slot: block.header.number,
  txCount: block.transactions.length,
  totalFees: block.transactions.reduce(
    (sum, tx) => sum + BigInt(tx.fee || 0),
    0n
  ),
}));
```

## Request Structure Reference

Every Portal request follows this structure:

```typescript theme={"system"}
{
  type: "solana",              // Chain type
  fromBlock: number,            // Starting slot number (inclusive, field name: fromBlock)
  toBlock?: number,             // Ending slot number (inclusive, optional, field name: toBlock)
  fields: {                    // What data to return
    block?: { ... },
    transaction?: { ... },
    instruction?: { ... },
    log?: { ... },
    tokenBalance?: { ... },
    balance?: { ... },
    reward?: { ... }
  },
  instructions?: [...],        // Filter by instructions
  transactions?: [...],       // Filter by transactions
  logs?: [...],               // Filter by logs
  tokenBalances?: [...],      // Filter by token balances
  balances?: [...],           // Filter by balances
  rewards?: [...]             // Filter by rewards
}
```

## Response Format

Portal returns **newline-delimited JSON (NDJSON)**, with one block per line:

```json theme={"system"}
{"header": {"number": 259984950, ...}, "instructions": [...], "transactions": [...]}
{"header": {"number": 259984951, ...}, "instructions": [...], "transactions": [...]}
{"header": {"number": 259984952, ...}, "instructions": [...], "transactions": [...]}
```

This format enables constant-memory streaming of arbitrary ranges.

## Common Patterns

<AccordionGroup>
  <Accordion title="Batch Processing">
    Process large slot ranges by splitting into manageable chunks:

    ```typescript theme={"system"}
    const BATCH_SIZE = 10000;
    for (let from = startSlot; from < endSlot; from += BATCH_SIZE) {
      const to = Math.min(from + BATCH_SIZE, endSlot);
      const blocks = await fetch(/* ... */);
    }
    ```
  </Accordion>

  <Accordion title="Real-time Streaming">
    Stream new blocks as they arrive using /finalized-stream:

    ```typescript theme={"system"}
    const response = await fetch(
      'https://portal.sqd.dev/datasets/solana-mainnet/finalized-stream',
      { method: 'POST', body: JSON.stringify({
        type: 'solana',
        fromBlock: latestProcessedSlot + 1,
        fields: { /* ... */ }
      })}
    );
    ```
  </Accordion>

  <Accordion title="Error Handling">
    Handle rate limits and network errors gracefully:

    ```typescript theme={"system"}
    try {
      const response = await fetch(/* ... */);
      if (!response.ok) {
        if (response.status === 429) {
          // Rate limited - wait and retry
          await sleep(10000);
        }
      }
    } catch (error) {
      // Network error - implement retry logic
    }
    ```
  </Accordion>
</AccordionGroup>

## Complete API Specification

For the complete OpenAPI specification with all available fields, options, and detailed schemas:

<Card title="Full Solana API Specification" icon="file-code" href="/en/portal/solana/api">
  View complete OpenAPI documentation with all endpoints, fields, and options →
</Card>

## Next Steps

<CardGroup>
  <Card title="View Examples" icon="code" href="/en/portal/solana/examples/query-instructions">
    Explore practical Portal examples for common use cases
  </Card>

  <Card title="Supported Networks" icon="database" href="/en/data/networks/evm">
    See complete list of supported Solana networks
  </Card>

  <Card title="Quickstart Guide" icon="rocket" href="/en/portal/solana/quickstart">
    Make your first Portal request in 5 minutes
  </Card>
</CardGroup>


# Index Solana DEX Swaps
Source: https://docs.sqd.dev/en/portal/solana/examples/dex-swaps

Track Solana DEX trading activity through the SQD Portal — sample queries for Raydium, Orca, Jupiter, and other Solana decentralized exchanges.

Monitor Orca Whirlpool swap instructions to build DEX analytics dashboards, analyze liquidity patterns, or create price feeds.

## Use Case

DEX swap tracking enables you to:

* Build DEX trading analytics and volume trackers
* Monitor liquidity pool activity
* Analyze trading patterns and arbitrage
* Create real-time price feeds

## Code Example

<CodeGroup>
  ```bash curl theme={"system"}
  curl --compressed -X POST 'https://portal.sqd.dev/datasets/solana-mainnet/stream' \
    -H 'Content-Type: application/json' \
    -d '{
      "type": "solana",
      "fromBlock": 280000000,
      "toBlock": 280000003,
      "fields": {
        "block": {
          "number": true,
          "timestamp": true
        },
        "instruction": {
          "programId": true,
          "accounts": true,
          "data": true,
          "transactionIndex": true
        }
      },
      "instructions": [{
        "programId": ["whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc"],
        "d8": ["0xf8c69e91e17587c8"]
      }]
    }'
  ```

  ```typescript TypeScript theme={"system"}
  import { DataSource } from "@subsquid/portal-client";

  const dataSource = new DataSource({
    network: "solana-mainnet",
  });

  // Orca Whirlpool program ID
  const WHIRLPOOL_PROGRAM = "whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc";
  // Swap instruction discriminator (first 8 bytes)
  const SWAP_D8 = "0xf8c69e91e17587c8";

  const blocks = await dataSource.getBlocks({
    from: 280000000,
    to: 280000003,
    fields: {
      block: { number: true, timestamp: true },
      instruction: {
        programId: true,
        accounts: true,
        data: true,
        transactionIndex: true,
      },
    },
    instructions: [
      {
        programId: [WHIRLPOOL_PROGRAM],
        d8: [SWAP_D8],
      },
    ],
  });

  // Process swaps
  for (const block of blocks) {
    for (const instruction of block.instructions) {
      console.log({
        slot: block.header.number,
        timestamp: block.header.timestamp,
        pool: instruction.accounts[2], // Whirlpool account (typically at index 2)
        txIndex: instruction.transactionIndex,
        // Decode swap amounts from instruction.data as needed
      });
    }
  }
  ```

  ```python Python theme={"system"}
  import requests
  import json

  url = "https://portal.sqd.dev/datasets/solana-mainnet/stream"
  headers = {"Content-Type": "application/json"}

  # Orca Whirlpool program ID
  whirlpool_program = "whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc"
  # Swap instruction discriminator
  swap_d8 = "0xf8c69e91e17587c8"

  payload = {
      "type": "solana",
      "fromBlock": 280000000,
      "toBlock": 280000003,
      "fields": {
          "block": {
              "number": True,
              "timestamp": True
          },
          "instruction": {
              "programId": True,
              "accounts": True,
              "data": True,
              "transactionIndex": True
          }
      },
      "instructions": [{
          "programId": [whirlpool_program],
          "d8": [swap_d8]
      }]
  }

  response = requests.post(url, headers=headers, json=payload)

  for line in response.text.strip().split('\n'):
      block = json.loads(line)
      for instruction in block.get('instructions', []):
          print({
              "slot": block['header']['number'],
              "timestamp": block['header']['timestamp'],
              "pool": instruction['accounts'][2] if len(instruction['accounts']) > 2 else None,
              "txIndex": instruction['transactionIndex']
          })
  ```
</CodeGroup>

<Info>Try it yourself with the interactive query interface below:</Info>

<QueryInterfaceSolanaDexSwaps />

## Key Parameters

| Parameter   | Description                                                 |
| ----------- | ----------------------------------------------------------- |
| `programId` | DEX program address (e.g., Orca Whirlpool)                  |
| `d8`        | 8-byte instruction discriminator for swap instruction       |
| `accounts`  | Instruction account addresses (includes pool, tokens, etc.) |
| `data`      | Instruction data payload (contains swap amounts)            |

<Note>
  Different DEX protocols use different instruction discriminators. Check the
  program IDL to find the correct discriminator for your target DEX.
</Note>

## Expected Output

```json theme={"system"}
{
  "header": {
    "number": 259985000,
    "timestamp": 1697544779
  },
  "instructions": [
    {
      "programId": "whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc",
      "accounts": ["7qbRF6YsyGuLUVs6Y1q64bdVrfe4ZcUUz1JRdoVNUJnm", "..."],
      "data": "base58encodeddata...",
      "transactionIndex": 0
    }
  ]
}
```

## Track Specific Pool

Monitor swaps for a specific trading pair:

<CodeGroup>
  ```bash curl theme={"system"}
  curl --compressed -X POST 'https://portal.sqd.dev/datasets/solana-mainnet/stream' \
    -H 'Content-Type: application/json' \
    -d '{
      "type": "solana",
      "fromBlock": 280000000,
      "toBlock": 280000003,
      "fields": {
        "instruction": {
          "programId": true,
          "accounts": true,
          "data": true
        }
      },
      "instructions": [{
        "programId": ["whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc"],
        "d8": ["0xf8c69e91e17587c8"],
        "accounts": ["7qbRF6YsyGuLUVs6Y1q64bdVrfe4ZcUUz1JRdoVNUJnm"]
      }]
    }'
  ```

  ```typescript TypeScript theme={"system"}
  const USDC_SOL_POOL = "7qbRF6YsyGuLUVs6Y1q64bdVrfe4ZcUUz1JRdoVNUJnm"; // USDC/SOL pool

  const blocks = await dataSource.getBlocks({
    from: 280000000,
    to: 280000003,
    fields: {
      instruction: { programId: true, accounts: true, data: true },
    },
    instructions: [
      {
        programId: ["whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc"],
        d8: ["0xf8c69e91e17587c8"],
        accounts: [USDC_SOL_POOL], // Filter by pool account
      },
    ],
  });
  ```

  ```python Python theme={"system"}
  usdc_sol_pool = "7qbRF6YsyGuLUVs6Y1q64bdVrfe4ZcUUz1JRdoVNUJnm"

  payload = {
      "type": "solana",
      "fromBlock": 280000000,
      "toBlock": 280000003,
      "fields": {
          "instruction": {
              "programId": True,
              "accounts": True,
              "data": True
          }
      },
      "instructions": [{
          "programId": ["whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc"],
          "d8": ["0xf8c69e91e17587c8"],
          "accounts": [usdc_sol_pool]
      }]
  }
  ```
</CodeGroup>

## Decode Swap Amounts

Swap amounts are encoded in the `data` field. Here's how to decode them for Orca Whirlpool:

<Tip>
  Use the Solana typegen tool to generate TypeScript decoders from program IDLs.
  See the [Solana SDK typegen
  documentation](/en/sdk/squid-sdk/solana-indexing/sdk/typegen) for details.
</Tip>

## Performance Tips

1. **Filter by pool address**: Query specific pools instead of all swaps
2. **Use slot ranges**: Process data in 10k-50k slot batches
3. **Decode off-chain**: Decode swap amounts in your application, not in the query
4. **Combine with token balances**: Use token balances to track actual amounts transferred

## Related Examples

<CardGroup>
  <Card title="Query Instructions" icon="code" href="/en/portal/solana/examples/query-instructions">
    General instruction querying
  </Card>

  <Card title="Token Transfers" icon="coins" href="/en/portal/solana/examples/token-transfers">
    Track token movements
  </Card>

  <Card title="Query Transactions" icon="arrow-right-arrow-left" href="/en/portal/solana/examples/query-transactions">
    Monitor wallet activity
  </Card>

  <Card title="API Reference" icon="book" href="/en/portal/solana/api">
    View complete API docs
  </Card>
</CardGroup>


# Monitor Program Deployments
Source: https://docs.sqd.dev/en/portal/solana/examples/program-deployments

Track new Solana program deployments through the SQD Portal — sample queries for detecting BPF program upgrades and new on-chain program publications.

Monitor new program deployments on Solana to track program launches, analyze deployment trends, or build program discovery tools.

## Use Case

Program deployment tracking enables you:

* Track new program launches
* Monitor program upgrade activity
* Analyze deployment trends
* Build program discovery dashboards

## Code Example

<CodeGroup>
  ```bash curl theme={"system"}
  curl --compressed -X POST 'https://portal.sqd.dev/datasets/solana-mainnet/stream' \
    -H 'Content-Type: application/json' \
    -d '{
      "type": "solana",
      "fromBlock": 270000044,
      "toBlock": 270000047,
      "fields": {
        "block": {
          "number": true,
          "timestamp": true
        },
        "transaction": {
          "signatures": true,
          "feePayer": true
        },
        "instruction": {
          "programId": true,
          "accounts": true
        }
      },
      "instructions": [{
        "programId": ["BPFLoaderUpgradeab1e11111111111111111111111"]
      }]
    }'
  ```

  ```typescript TypeScript theme={"system"}
  import { DataSource } from "@subsquid/portal-client";

  const dataSource = new DataSource({
    network: "solana-mainnet",
  });

  // BPF Upgradeable Loader program ID
  const BPF_UPGRADEABLE_LOADER = "BPFLoaderUpgradeab1e11111111111111111111111";

  const blocks = await dataSource.getBlocks({
    from: 270000044,
    to: 270000047,
    fields: {
      block: { number: true, timestamp: true },
      transaction: {
        signatures: true,
        feePayer: true,
      },
      instruction: {
        programId: true,
        accounts: true,
        transactionIndex: true,
      },
    },
    instructions: [
      {
        programId: [BPF_UPGRADEABLE_LOADER],
      },
    ],
  });

  // Process deployments
  for (const block of blocks) {
    for (const instruction of block.instructions) {
      // The program account is typically the first account
      const programAccount = instruction.accounts[0];
      console.log({
        slot: block.header.number,
        timestamp: block.header.timestamp,
        programAccount: programAccount,
        txIndex: instruction.transactionIndex,
        deployer: block.transactions[instruction.transactionIndex]?.feePayer,
      });
    }
  }
  ```

  ```python Python theme={"system"}
  import requests
  import json

  url = "https://portal.sqd.dev/datasets/solana-mainnet/stream"
  headers = {"Content-Type": "application/json"}

  # BPF Upgradeable Loader program ID
  bpf_upgradeable_loader = "BPFLoaderUpgradeab1e11111111111111111111111"

  payload = {
      "type": "solana",
      "fromBlock": 270000044,
      "toBlock": 270000047,
      "fields": {
          "block": {
              "number": True,
              "timestamp": True
          },
          "transaction": {
              "signatures": True,
              "feePayer": True
          },
          "instruction": {
              "programId": True,
              "accounts": True
          }
      },
      "instructions": [{
          "programId": [bpf_upgradeable_loader]
      }]
  }

  response = requests.post(url, headers=headers, json=payload)

  for line in response.text.strip().split('\n'):
      block = json.loads(line)
      for instruction in block.get('instructions', []):
          program_account = instruction['accounts'][0] if instruction['accounts'] else None
          print({
              "slot": block['header']['number'],
              "timestamp": block['header']['timestamp'],
              "programAccount": program_account,
              "txIndex": instruction.get('transactionIndex')
          })
  ```
</CodeGroup>

<Info>Try it yourself with the interactive query interface below:</Info>

<QueryInterfaceSolanaProgramDeployments />

## Key Parameters

| Parameter          | Description                                                   |
| ------------------ | ------------------------------------------------------------- |
| `programId`        | BPF Upgradeable Loader program ID for deployments             |
| `accounts`         | Instruction accounts (first account is typically the program) |
| `transactionIndex` | Transaction index (use to match with transactions array)      |

<Note>
  Program deployments on Solana use the BPF Upgradeable Loader program. Different instruction types within this program indicate deployments, upgrades, or closures.
</Note>

## Expected Output

```json theme={"system"}
{
  "header": {
    "number": 259985000,
    "timestamp": 1697544779
  },
  "instructions": [
    {
      "programId": "BPFLoaderUpgradeab1e11111111111111111111111",
      "accounts": ["NewProgramAddress...", "..."],
      "transactionIndex": 0
    }
  ]
}
```

## Filter by Deployer

Track deployments from specific addresses:

<CodeGroup>
  ```bash curl theme={"system"}
  curl --compressed -X POST 'https://portal.sqd.dev/datasets/solana-mainnet/stream' \
    -H 'Content-Type: application/json' \
    -d '{
      "type": "solana",
      "fromBlock": 270000044,
      "toBlock": 270000047,
      "fields": {
        "transaction": {
          "signatures": true,
          "feePayer": true
        },
        "instruction": {
          "programId": true,
          "accounts": true,
          "transactionIndex": true
        }
      },
      "instructions": [{
        "programId": ["BPFLoaderUpgradeab1e11111111111111111111111"]
      }],
      "transactions": [{
        "feePayer": ["DeployerAddressHere"]
      }]
    }'
  ```

  ```typescript TypeScript theme={"system"}
  const blocks = await dataSource.getBlocks({
    from: 270000044,
    to: 270000047,
    fields: {
      transaction: { signatures: true, feePayer: true },
      instruction: { programId: true, accounts: true, transactionIndex: true },
    },
    instructions: [
      {
        programId: ["BPFLoaderUpgradeab1e11111111111111111111111"],
      },
    ],
    transactions: [
      {
        feePayer: ["DeployerAddressHere"],
      },
    ],
  });
  ```

  ```python Python theme={"system"}
  payload = {
      "type": "solana",
      "fromBlock": 270000044,
      "toBlock": 270000047,
      "fields": {
          "transaction": {
              "signatures": True,
              "feePayer": True
          },
          "instruction": {
              "programId": True,
              "accounts": True,
              "transactionIndex": True
          }
      },
      "instructions": [{
          "programId": ["BPFLoaderUpgradeab1e11111111111111111111111"]
      }],
      "transactions": [{
          "feePayer": ["DeployerAddressHere"]
      }]
  }
  ```
</CodeGroup>

## Performance Tips

1. **Filter by deployer**: Query deployments from specific addresses
2. **Use slot ranges**: Process data in manageable batches
3. **Request minimal fields**: Only request fields you need
4. **Combine filters**: Use both program and transaction filters for precision

## Related Examples

<CardGroup>
  <Card title="Query Instructions" icon="code" href="/en/portal/solana/examples/query-instructions">
    Track program instructions
  </Card>

  <Card title="Query Transactions" icon="arrow-right-arrow-left" href="/en/portal/solana/examples/query-transactions">
    Monitor wallet activity
  </Card>

  <Card title="DEX Swaps" icon="repeat" href="/en/portal/solana/examples/dex-swaps">
    Track DEX trading activity
  </Card>

  <Card title="API Reference" icon="book" href="/en/portal/solana/api">
    View complete API docs
  </Card>
</CardGroup>


# Query Instructions
Source: https://docs.sqd.dev/en/portal/solana/examples/query-instructions

Track Solana program instructions through the SQD Portal — sample queries for filtering by program ID, accounts, and inner instructions across blocks.

Monitor instructions from specific Solana programs to build program analytics or analyze instruction patterns.

## Use Case

Instruction queries help you:

* Track program interactions and usage
* Monitor specific program activity
* Analyze instruction patterns
* Build program analytics dashboards

## Code Example

<CodeGroup>
  ```bash curl theme={"system"}
  curl --compressed -X POST 'https://portal.sqd.dev/datasets/solana-mainnet/stream' \
    -H 'Content-Type: application/json' \
    -d '{
      "type": "solana",
      "fromBlock": 280000000,
      "toBlock": 280000003,
      "fields": {
        "block": {
          "number": true,
          "timestamp": true
        },
        "instruction": {
          "programId": true,
          "accounts": true,
          "data": true,
          "transactionIndex": true
        }
      },
      "instructions": [{
        "programId": ["whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc"]
      }]
    }'
  ```

  ```typescript TypeScript theme={"system"}
  import { DataSource } from "@subsquid/portal-client";

  const dataSource = new DataSource({
    network: "solana-mainnet",
  });

  // Orca Whirlpool program ID
  const WHIRLPOOL_PROGRAM = "whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc";

  const blocks = await dataSource.getBlocks({
    from: 280000000,
    to: 280000003,
    fields: {
      block: { number: true, timestamp: true },
      instruction: {
        programId: true,
        accounts: true,
        data: true,
        transactionIndex: true,
      },
    },
    instructions: [
      {
        programId: [WHIRLPOOL_PROGRAM],
      },
    ],
  });

  // Process instructions
  for (const block of blocks) {
    for (const instruction of block.instructions) {
      console.log({
        slot: block.header.number,
        timestamp: block.header.timestamp,
        programId: instruction.programId,
        accounts: instruction.accounts,
        txIndex: instruction.transactionIndex,
      });
    }
  }
  ```

  ```python Python theme={"system"}
  import requests
  import json

  url = "https://portal.sqd.dev/datasets/solana-mainnet/stream"
  headers = {"Content-Type": "application/json"}

  # Orca Whirlpool program ID
  whirlpool_program = "whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc"

  payload = {
      "type": "solana",
      "fromBlock": 280000000,
      "toBlock": 280000003,
      "fields": {
          "block": {
              "number": True,
              "timestamp": True
          },
          "instruction": {
              "programId": True,
              "accounts": True,
              "data": True,
              "transactionIndex": True
          }
      },
      "instructions": [{
          "programId": [whirlpool_program]
      }]
  }

  response = requests.post(url, headers=headers, json=payload)

  for line in response.text.strip().split('\n'):
      block = json.loads(line)
      for instruction in block.get('instructions', []):
          print({
              "slot": block['header']['number'],
              "timestamp": block['header']['timestamp'],
              "programId": instruction['programId'],
              "accounts": instruction['accounts'],
              "txIndex": instruction['transactionIndex']
          })
  ```
</CodeGroup>

<Info>Try it yourself with the interactive query interface below:</Info>

<QueryInterfaceSolanaQueryInstructions />

## Key Parameters

| Parameter          | Description                                              |
| ------------------ | -------------------------------------------------------- |
| `programId`        | Filter instructions by program address                   |
| `d8`               | Filter by 8-byte instruction discriminator               |
| `accounts`         | Filter by account addresses used in instruction          |
| `data`             | Instruction data payload                                 |
| `transactionIndex` | Transaction index (use to match with transactions array) |

## Expected Output

```json theme={"system"}
{
  "header": {
    "number": 259985000,
    "timestamp": 1697544779
  },
  "instructions": [
    {
      "programId": "whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc",
      "accounts": ["7qbRF6YsyGuLUVs6Y1q64bdVrfe4ZcUUz1JRdoVNUJnm", "..."],
      "data": "base58encodeddata...",
      "transactionIndex": 0
    }
  ]
}
```

## Filter by Instruction Discriminator

Query only specific instruction types (e.g., swap instructions):

<CodeGroup>
  ```bash curl theme={"system"}
  curl --compressed -X POST 'https://portal.sqd.dev/datasets/solana-mainnet/stream' \
    -H 'Content-Type: application/json' \
    -d '{
      "type": "solana",
      "fromBlock": 280000000,
      "toBlock": 280000003,
      "fields": {
        "instruction": {
          "programId": true,
          "data": true
        }
      },
      "instructions": [{
        "programId": ["whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc"],
        "d8": ["0xf8c69e91e17587c8"]
      }]
    }'
  ```

  ```typescript TypeScript theme={"system"}
  const blocks = await dataSource.getBlocks({
    from: 280000000,
    to: 280000003,
    fields: {
      instruction: { programId: true, data: true },
    },
    instructions: [
      {
        programId: ["whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc"], // Orca Whirlpool
        d8: ["0xf8c69e91e17587c8"], // Swap instruction discriminator
      },
    ],
  });
  ```

  ```python Python theme={"system"}
  payload = {
      "type": "solana",
      "fromBlock": 280000000,
      "toBlock": 280000003,
      "fields": {
          "instruction": {
              "programId": True,
              "data": True
          }
      },
      "instructions": [{
          "programId": ["whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc"],
          "d8": ["0xf8c69e91e17587c8"]
      }]
  }
  ```
</CodeGroup>

<Tip>
  Instruction discriminators are typically the first 8 bytes of the instruction data. They identify the specific instruction type within a program.
</Tip>

## Filter by Account

Query instructions involving specific accounts:

<CodeGroup>
  ```bash curl theme={"system"}
  curl --compressed -X POST 'https://portal.sqd.dev/datasets/solana-mainnet/stream' \
    -H 'Content-Type: application/json' \
    -d '{
      "type": "solana",
      "fromBlock": 280000000,
      "toBlock": 280000003,
      "fields": {
        "instruction": {
          "programId": true,
          "accounts": true
        }
      },
      "instructions": [{
        "programId": ["whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc"],
        "accounts": ["7qbRF6YsyGuLUVs6Y1q64bdVrfe4ZcUUz1JRdoVNUJnm"]
      }]
    }'
  ```

  ```typescript TypeScript theme={"system"}
  const blocks = await dataSource.getBlocks({
    from: 280000000,
    to: 280000003,
    fields: {
      instruction: { programId: true, accounts: true },
    },
    instructions: [
      {
        programId: ["whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc"],
        accounts: ["7qbRF6YsyGuLUVs6Y1q64bdVrfe4ZcUUz1JRdoVNUJnm"], // Specific pool
      },
    ],
  });
  ```

  ```python Python theme={"system"}
  payload = {
      "type": "solana",
      "fromBlock": 280000000,
      "toBlock": 280000003,
      "fields": {
          "instruction": {
              "programId": True,
              "accounts": True
          }
      },
      "instructions": [{
          "programId": ["whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc"],
          "accounts": ["7qbRF6YsyGuLUVs6Y1q64bdVrfe4ZcUUz1JRdoVNUJnm"]
      }]
  }
  ```
</CodeGroup>

## Performance Tips

1. **Combine filters**: Use both `programId` and `d8` for specific instruction types
2. **Filter by accounts**: Reduce results to specific program instances
3. **Request minimal fields**: Omit `data` if you don't need instruction payload
4. **Use slot ranges**: Process data in 10k-50k slot batches

## Related Examples

<CardGroup>
  <Card title="Query Transactions" icon="arrow-right-arrow-left" href="/en/portal/solana/examples/query-transactions">
    Track wallet activity
  </Card>

  <Card title="Query Logs" icon="file-lines" href="/en/portal/solana/examples/query-instructions">
    Retrieve program log messages
  </Card>

  <Card title="DEX Swaps" icon="repeat" href="/en/portal/solana/examples/dex-swaps">
    Track DEX trading activity
  </Card>

  <Card title="API Reference" icon="book" href="/en/portal/solana/api">
    View complete API docs
  </Card>
</CardGroup>


# Query Solana Transactions
Source: https://docs.sqd.dev/en/portal/solana/examples/query-transactions

Monitor Solana wallet activity and transaction flows through the SQD Portal — sample queries for filtering by signer, account, and program ID at scale.

Retrieve all transactions sent by or to a specific address to track wallet activity, exchange flows, or program interactions.

## Use Case

Transaction queries help you:

* Track incoming transactions to wallets or exchanges
* Monitor program interactions
* Analyze transaction patterns
* Build transaction history feeds

## Code Example

<CodeGroup>
  ```bash curl theme={"system"}
  curl --compressed -X POST 'https://portal.sqd.dev/datasets/solana-mainnet/stream' \
    -H 'Content-Type: application/json' \
    -d '{
      "type": "solana",
      "fromBlock": 280000000,
      "toBlock": 280000003,
      "fields": {
        "block": {
          "number": true,
          "timestamp": true
        },
        "transaction": {
          "signatures": true,
          "feePayer": true,
          "err": true
        }
      },
      "transactions": [{}]
    }'
  ```

  ```typescript TypeScript theme={"system"}
  import { DataSource } from "@subsquid/portal-client";

  const dataSource = new DataSource({
    network: "solana-mainnet",
  });

  const blocks = await dataSource.getBlocks({
    from: 280000000,
    to: 280000003,
    fields: {
      block: { number: true, timestamp: true },
      transaction: {
        signatures: true,
        feePayer: true,
        err: true,
      },
    },
    transactions: [{}],
  });

  // Process transactions
  for (const block of blocks) {
    for (const tx of block.transactions) {
      console.log({
        slot: block.header.number,
        signatures: tx.signatures,
        feePayer: tx.feePayer,
        success: tx.err === null,
      });
    }
  }
  ```

  ```python Python theme={"system"}
  import requests
  import json

  url = "https://portal.sqd.dev/datasets/solana-mainnet/stream"
  headers = {"Content-Type": "application/json"}

  payload = {
      "type": "solana",
      "fromBlock": 280000000,
      "toBlock": 280000003,
      "fields": {
          "block": {
              "number": True,
              "timestamp": True
          },
          "transaction": {
              "signatures": True,
              "feePayer": True,
              "err": True
          }
      },
      "transactions": [{}]
  }

  response = requests.post(url, headers=headers, json=payload)

  for line in response.text.strip().split('\n'):
      block = json.loads(line)
      for tx in block.get('transactions', []):
          print({
              "slot": block['header']['number'],
              "signatures": tx['signatures'],
              "feePayer": tx['feePayer'],
              "success": tx['err'] is None
          })
  ```
</CodeGroup>

<Info>Try it yourself with the interactive query interface below:</Info>

<QueryInterfaceSolanaQueryTransactions />

## Key Parameters

| Parameter    | Description                              |
| ------------ | ---------------------------------------- |
| `feePayer`   | Filter transactions by fee payer address |
| `signatures` | Transaction signatures (base58-encoded)  |
| `err`        | Transaction error (null if successful)   |

## Expected Output

```json theme={"system"}
{
  "header": {
    "number": 259985000,
    "timestamp": 1697544779
  },
  "transactions": [
    {
      "signatures": ["5j7s8K9L..."],
      "feePayer": "9WzDXwBbmkg8ZTbNMqUxvQRAyrZzDsGYdLVL9zYtAWWM",
      "err": null
    }
  ]
}
```

## Monitor Outgoing Transactions

Track transactions from a specific wallet:

<CodeGroup>
  ```bash curl theme={"system"}
  curl --compressed -X POST 'https://portal.sqd.dev/datasets/solana-mainnet/stream' \
    -H 'Content-Type: application/json' \
    -d '{
      "type": "solana",
      "fromBlock": 280000000,
      "toBlock": 280000003,
      "fields": {
        "transaction": {
          "signatures": true,
          "feePayer": true
        }
      },
      "transactions": [{}]
    }'
  ```

  ```typescript TypeScript theme={"system"}
  const blocks = await dataSource.getBlocks({
    from: 280000000,
    to: 280000003,
    fields: {
      transaction: { signatures: true, feePayer: true },
    },
    transactions: [
      {
        feePayer: ["9WzDXwBbmkg8ZTbNMqUxvQRAyrZzDsGYdLVL9zYtAWWM"],
      },
    ],
  });
  ```

  ```python Python theme={"system"}
  payload = {
      "type": "solana",
      "fromBlock": 280000000,
      "toBlock": 280000003,
      "fields": {
          "transaction": {
              "signatures": True,
              "feePayer": True
          }
      },
      "transactions": [{
          "feePayer": ["9WzDXwBbmkg8ZTbNMqUxvQRAyrZzDsGYdLVL9zYtAWWM"]
      }]
  }
  ```
</CodeGroup>

## Performance Tips

1. **Filter by fee payer**: Reduces results to transactions from specific wallets
2. **Request minimal fields**: Omit unnecessary fields to improve performance
3. **Check error status**: Filter failed transactions in post-processing if needed
4. **Use slot ranges**: Process data in manageable batches

## Related Examples

<CardGroup>
  <Card title="Query Instructions" icon="code" href="/en/portal/solana/examples/query-instructions">
    Track program instructions
  </Card>

  <Card title="Query Logs" icon="file-lines" href="/en/portal/solana/examples/query-instructions">
    Retrieve program log messages
  </Card>

  <Card title="Token Transfers" icon="coins" href="/en/portal/solana/examples/token-transfers">
    Monitor SPL token activity
  </Card>

  <Card title="API Reference" icon="book" href="/en/portal/solana/api">
    View complete API docs
  </Card>
</CardGroup>


# Track Token Transfers
Source: https://docs.sqd.dev/en/portal/solana/examples/token-transfers

Monitor Solana SPL token transfers through the SQD Portal — sample queries for tracking mint and burn events, holder activity, and balance changes.

Track SPL token transfers to build token analytics, monitor wallet activity, or analyze token flow patterns.

## Use Case

Token transfer tracking enables you to:

* Build token transfer analytics and volume trackers
* Monitor wallet token activity
* Analyze transfer patterns
* Track token mints and burns

## Code Example

<CodeGroup>
  ```bash curl theme={"system"}
  curl --compressed -X POST 'https://portal.sqd.dev/datasets/solana-mainnet/stream' \
    -H 'Content-Type: application/json' \
    -d '{
      "type": "solana",
      "fromBlock": 280000000,
      "toBlock": 280000003,
      "fields": {
        "block": {
          "number": true,
          "timestamp": true
        },
        "tokenBalance": {
          "account": true,
          "preMint": true,
          "postMint": true,
          "preAmount": true,
          "postAmount": true,
          "preOwner": true,
          "postOwner": true
        }
      },
      "tokenBalances": [{
        "preMint": ["EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v"]
      }]
    }'
  ```

  ```typescript TypeScript theme={"system"}
  import { DataSource } from "@subsquid/portal-client";

  const dataSource = new DataSource({
    network: "solana-mainnet",
  });

  // USDC mint address
  const USDC_MINT = "EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v";

  const blocks = await dataSource.getBlocks({
    from: 280000000,
    to: 280000003,
    fields: {
      block: { number: true, timestamp: true },
      tokenBalance: {
        account: true,
        preMint: true,
        postMint: true,
        preAmount: true,
        postAmount: true,
        preOwner: true,
        postOwner: true,
      },
    },
    tokenBalances: [
      {
        preMint: [USDC_MINT],
      },
    ],
  });

  // Process token transfers
  for (const block of blocks) {
    for (const balance of block.tokenBalances) {
      if (balance.preAmount !== balance.postAmount) {
        const amount = BigInt(balance.postAmount) - BigInt(balance.preAmount);
        console.log({
          slot: block.header.number,
          timestamp: block.header.timestamp,
          account: balance.account,
          mint: balance.postMint,
          from: balance.preOwner,
          to: balance.postOwner,
          amount: amount.toString(),
        });
      }
    }
  }
  ```

  ```python Python theme={"system"}
  import requests
  import json

  url = "https://portal.sqd.dev/datasets/solana-mainnet/stream"
  headers = {"Content-Type": "application/json"}

  # USDC mint address
  usdc_mint = "EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v"

  payload = {
      "type": "solana",
      "fromBlock": 280000000,
      "toBlock": 280000003,
      "fields": {
          "block": {
              "number": True,
              "timestamp": True
          },
          "tokenBalance": {
              "account": True,
              "preMint": True,
              "postMint": True,
              "preAmount": True,
              "postAmount": True,
              "preOwner": True,
              "postOwner": True
          }
      },
      "tokenBalances": [{
          "preMint": [usdc_mint]
      }]
  }

  response = requests.post(url, headers=headers, json=payload)

  for line in response.text.strip().split('\n'):
      block = json.loads(line)
      for balance in block.get('tokenBalances', []):
          if balance['preAmount'] != balance['postAmount']:
              amount = int(balance['postAmount']) - int(balance['preAmount'])
              print({
                  "slot": block['header']['number'],
                  "timestamp": block['header']['timestamp'],
                  "account": balance['account'],
                  "mint": balance['postMint'],
                  "from": balance['preOwner'],
                  "to": balance['postOwner'],
                  "amount": amount
              })
  ```
</CodeGroup>

<Info>Try it yourself with the interactive query interface below:</Info>

<QueryInterfaceSolanaTokenTransfers />

## Key Parameters

| Parameter    | Description                                |
| ------------ | ------------------------------------------ |
| `preMint`    | Token mint address before the transaction  |
| `postMint`   | Token mint address after the transaction   |
| `preAmount`  | Token balance before the transaction       |
| `postAmount` | Token balance after the transaction        |
| `preOwner`   | Token account owner before the transaction |
| `postOwner`  | Token account owner after the transaction  |
| `account`    | Token account address                      |

## Expected Output

```json theme={"system"}
{
  "header": {
    "number": 259985000,
    "timestamp": 1697544779
  },
  "tokenBalances": [
    {
      "account": "TokenAccountAddress...",
      "preMint": "EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v",
      "postMint": "EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v",
      "preAmount": "1000000",
      "postAmount": "2000000",
      "preOwner": "SenderAddress...",
      "postOwner": "RecipientAddress..."
    }
  ]
}
```

## Track Specific Token Account

Monitor transfers for a specific token account:

<CodeGroup>
  ```bash curl theme={"system"}
  curl --compressed -X POST 'https://portal.sqd.dev/datasets/solana-mainnet/stream' \
    -H 'Content-Type: application/json' \
    -d '{
      "type": "solana",
      "fromBlock": 280000000,
      "toBlock": 280000003,
      "fields": {
        "tokenBalance": {
          "account": true,
          "preAmount": true,
          "postAmount": true
        }
      },
      "tokenBalances": [{
        "account": ["SpecificTokenAccountAddress"]
      }]
    }'
  ```

  ```typescript TypeScript theme={"system"}
  const blocks = await dataSource.getBlocks({
    from: 280000000,
    to: 280000003,
    fields: {
      tokenBalance: { account: true, preAmount: true, postAmount: true },
    },
    tokenBalances: [
      {
        account: ["SpecificTokenAccountAddress"],
      },
    ],
  });
  ```

  ```python Python theme={"system"}
  payload = {
      "type": "solana",
      "fromBlock": 280000000,
      "toBlock": 280000003,
      "fields": {
          "tokenBalance": {
              "account": True,
              "preAmount": True,
              "postAmount": True
          }
      },
      "tokenBalances": [{
          "account": ["SpecificTokenAccountAddress"]
      }]
  }
  ```
</CodeGroup>

## Performance Tips

1. **Filter by mint**: Query specific tokens instead of all transfers
2. **Use slot ranges**: Process data in 10k-50k slot batches
3. **Request minimal fields**: Only request fields you need
4. **Filter by account**: Monitor specific token accounts

## Related Examples

<CardGroup>
  <Card title="Query Instructions" icon="code" href="/en/portal/solana/examples/query-instructions">
    Track program instructions
  </Card>

  <Card title="DEX Swaps" icon="repeat" href="/en/portal/solana/examples/dex-swaps">
    Track DEX trading activity
  </Card>

  <Card title="Query Transactions" icon="arrow-right-arrow-left" href="/en/portal/solana/examples/query-transactions">
    Monitor wallet activity
  </Card>

  <Card title="API Reference" icon="book" href="/en/portal/solana/api">
    View complete API docs
  </Card>
</CardGroup>


# Solana Portal API
Source: https://docs.sqd.dev/en/portal/solana/overview

Access raw Solana blockchain data — blocks, transactions, instructions, balances, rewards — through the SQD Portal HTTP API for high-throughput streaming.

Portal provides direct access to raw Solana blockchain data. Query slots, transactions, instructions, logs, token balances, and rewards with performance that scales beyond traditional RPC limitations.

<Info>
  Looking for the main Portal overview? See [Portal
  Overview](/en/portal/overview) for concepts, deployment options, and
  comparisons.
</Info>

## Solana-Specific Features

Portal supports the full range of Solana data types:

<CardGroup>
  <Card title="Instructions" icon="code">
    Query program instructions with account and discriminator filtering. Track DEX swaps, token transfers, and custom program interactions.
  </Card>

  <Card title="Token Balances" icon="coins">
    Track SPL token balance changes. Monitor token transfers, mints, and burns across accounts.
  </Card>

  <Card title="Transactions" icon="arrow-right-arrow-left">
    Access transaction details including signatures, fee payers, and execution
    status. Filter by signature or fee payer address.
  </Card>

  <Card title="SOL Balances" icon="wallet">
    Track native SOL balance changes across accounts. Monitor transfers, staking rewards, and fee payments.
  </Card>
</CardGroup>

## Quick Start

Get started with Portal in 5 minutes:

<CodeGroup>
  ```bash curl theme={"system"}
  curl --compressed -X POST 'https://portal.sqd.dev/datasets/solana-mainnet/stream' \
    -H 'Content-Type: application/json' \
    -d '{
      "type": "solana",
      "fromBlock": 259984950,
      "toBlock": 259985000,
      "fields": {
        "block": { "number": true, "timestamp": true },
        "instruction": { "programId": true, "accounts": true, "data": true }
      }
    }'
  ```

  ```typescript TypeScript theme={"system"}
  import { DataSource } from "@subsquid/portal-client";

  const dataSource = new DataSource({
    network: "solana-mainnet",
  });

  const blocks = await dataSource.getBlocks({
    from: 259984950,
    to: 259985000,
    fields: {
      block: { number: true, timestamp: true },
      instruction: { programId: true, accounts: true, data: true },
    },
  });
  ```

  ```python Python theme={"system"}
  import requests

  url = "https://portal.sqd.dev/datasets/solana-mainnet/stream"
  headers = {"Content-Type": "application/json"}
  payload = {
      "type": "solana",
      "fromBlock": 259984950,
      "toBlock": 259985000,
      "fields": {
          "block": {"number": True, "timestamp": True},
          "instruction": {"programId": True, "accounts": True, "data": True}
      }
  }

  response = requests.post(url, headers=headers, json=payload)
  ```
</CodeGroup>

## Common Use Cases

<CardGroup>
  <Card title="DEX Analytics" icon="chart-line" href="/en/portal/solana/examples/dex-swaps">
    Track DEX swaps, liquidity pool activity, and trading patterns. Use instructions to analyze complex DeFi interactions.
  </Card>

  <Card title="Token Transfer Tracking" icon="coins" href="/en/portal/solana/examples/token-transfers">
    Monitor SPL token transfers, mints, and burns. Track token movements across
    accounts and programs.
  </Card>

  <Card title="Program Instruction Monitoring" icon="code" href="/en/portal/solana/examples/query-instructions">
    Track program instructions, monitor specific programs, and analyze
    instruction patterns. Essential for program analytics.
  </Card>

  <Card title="Transaction Monitoring" icon="wallet" href="/en/portal/solana/examples/query-transactions">
    Track wallet activity, monitor specific addresses, and analyze transaction
    patterns. Essential for compliance and security.
  </Card>
</CardGroup>

## Access Methods

Choose how you want to access Portal:

<AccordionGroup>
  <Accordion title="Public Portal (Free)" icon="globe">
    **Best for:** Development and testing

    * Free access with rate limits (20 requests per 10 seconds)
    * No setup required
    * Access to all supported networks

    **Endpoint:** `https://portal.sqd.dev`
  </Accordion>

  <Accordion title="Cloud Portal" icon="cloud">
    **Best for:** Production applications - Higher rate limits - Integrated with
    SQD Cloud deployment - Pay-as-you-go pricing - Production-ready infrastructure

    <Card title="Learn More" icon="cloud" href="/en/cloud/overview">
      Explore Cloud Portal
    </Card>
  </Accordion>

  <Accordion title="Self-Hosted Portal" icon="server">
    **Best for:** Maximum sovereignty

    * No rate limits
    * Full control over infrastructure
    * Run next to your own node
    * Open-source implementation

    <Card title="Self-Hosting Guide" icon="server" href="/en/portal/self-hosting">
      Set up your Portal instance
    </Card>
  </Accordion>
</AccordionGroup>

## Next Steps

<CardGroup>
  <Card title="Quickstart" icon="rocket" href="/en/portal/solana/quickstart">
    Make your first Portal request in 5 minutes
  </Card>

  <Card title="API Reference" icon="book" href="/en/portal/solana/api">
    Complete API reference with all fields and filters
  </Card>

  <Card title="Examples" icon="code" href="/en/portal/solana/examples/query-instructions">
    Practical examples for common use cases
  </Card>
</CardGroup>

<Info>
  **Enterprise Custom Development**

  For enterprise clients requiring custom indexer development, please contact our team to discuss your specific requirements. [Schedule a Consultation →](https://calendly.com/t-tyrie-subsquid/30min)
</Info>


# Solana Portal Quickstart
Source: https://docs.sqd.dev/en/portal/solana/quickstart

Make your first SQD Portal request for Solana data in 5 minutes — install the client, query a slot range, and stream transactions or instructions.

Make your first Portal request to access raw Solana blockchain data in under 5 minutes. No setup required, just HTTP requests.

## What You'll Build

In this quickstart, you'll:

1. Make a simple Portal request using curl (no installation required)
2. Query recent Solana slots
3. Filter instructions from a specific program

<Info>
  This guide uses the **Public Portal** endpoint, which is free and
  rate-limited. Perfect for getting started.
</Info>

## Step 1: Your First Request

Let's query 200 slots from Solana mainnet and get block data with transaction counts. Copy and paste this command:

<Tabs>
  <Tab title="curl">
    ```bash theme={"system"}
    curl --compressed -X POST 'https://portal.sqd.dev/datasets/solana-mainnet/stream' \
      -H 'Content-Type: application/json' \
      -d '{
        "type": "solana",
        "fromBlock": 259984800,
        "toBlock": 259984801,
        "fields": {
          "block": {
            "number": true,
            "timestamp": true
          },
          "transaction": {
            "signatures": true,
            "feePayer": true,
            "err": true
          }
        },
        "transactions": [{}]
      }'
    ```

    <Check>
      You should see a stream of JSON objects, one per block, showing slot numbers,
      timestamps, and transaction data including signatures, fee payers, and success
      status.
    </Check>

    <Info>Try it yourself with the interactive query interface below:</Info>

    <QueryInterfaceSolanaQuickstart />
  </Tab>

  <Tab title="TypeScript">
    First, install the client:

    ```bash theme={"system"}
    npm install @subsquid/portal-client
    ```

    Then run this code:

    ```typescript theme={"system"}
    import { DataSource } from "@subsquid/portal-client";

    const dataSource = new DataSource({
      network: "solana-mainnet",
    });

    const blocks = await dataSource.getBlocks({
      from: 259984800,
      to: 259985000,
      fields: {
        block: {
          number: true,
          timestamp: true,
        },
        transaction: {
          signatures: true,
          feePayer: true,
          err: true,
        },
      },
      transactions: [{}],
    });

    console.log(`Retrieved ${blocks.length} blocks`);
    blocks.forEach((block) => {
      const successCount = block.transactions.filter(
        (tx) => tx.err === null
      ).length;
      const failedCount = block.transactions.length - successCount;
      console.log(
        `Slot ${block.header.number}: ${block.transactions.length} transactions (${successCount} succeeded, ${failedCount} failed)`
      );
    });
    ```
  </Tab>

  <Tab title="Python">
    First, install requests:

    ```bash theme={"system"}
    pip install requests
    ```

    Then run this code:

    ```python theme={"system"}
    import requests
    import json

    url = "https://portal.sqd.dev/datasets/solana-mainnet/stream"
    headers = {"Content-Type": "application/json"}
    payload = {
        "type": "solana",
        "fromBlock": 259984800,
        "toBlock": 259985000,
        "fields": {
            "block": {
                "number": True,
                "timestamp": True
            },
            "transaction": {
                "signatures": True,
                "feePayer": True,
                "err": True
            }
        },
        "transactions": [{}]
    }

    response = requests.post(url, headers=headers, json=payload)

    # Response is newline-delimited JSON
    for line in response.text.strip().split('\n'):
        if line.strip():  # Skip empty lines
            block = json.loads(line)
            tx_count = len(block.get('transactions', []))
            success_count = sum(1 for tx in block.get('transactions', []) if tx.get('err') is None)
            failed_count = tx_count - success_count
            print(f"Slot {block['header']['number']}: {tx_count} transactions ({success_count} succeeded, {failed_count} failed)")
    ```
  </Tab>
</Tabs>

## Step 2: Understanding the Request

Let's break down what you just sent:

```typescript theme={"system"}
{
  "type": "solana",              // Chain type (Solana)
  "fromBlock": 259984800,         // Starting slot number (inclusive, field name: fromBlock)
  "toBlock": 259985000,           // Ending slot number (inclusive, field name: toBlock)
  "fields": {                    // What data to return
    "block": {
      "number": true,             // Slot number (field name: number)
      "timestamp": true          // Block timestamp
    },
    "transaction": {
      "signatures": true,         // Transaction signatures
      "feePayer": true,          // Fee payer address
      "err": true                // Error status (null if successful)
    }
  },
  "transactions": [{}]            // Include transactions filter (empty object = all transactions)
}
```

<Tip>
  Portal only returns the fields you request. This keeps responses fast and
  bandwidth-efficient.
</Tip>

## Step 3: Understanding the Response

Portal returns **JSON lines (JSONL)**. Each line is a complete JSON object representing one block. Note that not every slot produces a block, so you'll see gaps in the slot numbers:

```json theme={"system"}
{
  "header": {
    "number": 259984800,
    "timestamp": 1713053593
  },
  "transactions": [
    {
      "signatures": ["5j7s8K9LmN2pQrS3tUvW4xYz5aB6cD7eF8gH9iJ0kL1mN2oP3qR4sT5uV6wX"],
      "feePayer": "9WzDXwBbmkg8ZTbNMqUxvQRAyrZzDsGYdLVL9zYtAWWM",
      "err": null
    }
  ]
}
{
  "header": {
    "number": 259984837,
    "timestamp": 1713053611
  },
  "transactions": [
    {
      "signatures": ["3kL4mN5oP6qR7sT8uV9wX0yZ1aB2cD3eF4gH5iJ6kL7mN8oP9qR0sT"],
      "feePayer": "7qbRF6YsyGuLUVs6Y1q64bdVrfe4ZcUUz1JRdoVNUJnm",
      "err": null
    }
  ]
}
{
  "header": {
    "number": 259984838,
    "timestamp": 1713053612
  },
  "transactions": []
}
```

<Note>
  This format enables **constant-memory streaming** of massive ranges. You can
  process millions of slots without loading everything into RAM.
</Note>

## Stream semantics

Two behaviors are easy to miss on your first Portal request. Both are enforced by the API, and clients that ignore them will silently drop data.

### Filter objects vs `includeAllBlocks`

Portal returns **only blocks matching your data filters** (`instructions`, `transactions`, `logs`, `balances`, and so on). The example above uses `"transactions": [{}]` (an empty filter object) as a catch-all so every block in the range is returned.

* If you set specific filters (e.g. `instructions: [{ programId: [...] }]`), Portal returns only blocks containing matching items.
* If you drop all filters AND don't set `"includeAllBlocks": true`, Portal returns just the worker-range boundary blocks (typically 2–4 per query), not the full range. If you see far fewer blocks than expected, this is why.
* On Solana, `includeAllBlocks` defaults to `false`. (On Bitcoin it defaults to `true`; on EVM and Substrate `false`.)

### The stream can end before `toBlock`

A single HTTP response can be cut short at any point: when a worker's range ends, when a connection is recycled, or at the dataset's current head. Clients must treat one response as a **batch**, not the entire range. To continue:

1. Read the last block number (`N`) in the response.
2. Issue a new request with `fromBlock = N + 1` (and the same `toBlock`, filters, and fields).
3. Loop until you receive HTTP 204 (range is above dataset height) or you reach your `toBlock`.

If you're streaming near the chain head, also set `parentBlockHash` on each subsequent request and handle HTTP 409 (reorg). See the [full API reference](/en/portal/solana/api#stream-continuation).

## Step 4: Filter Instructions

Track instructions from the Orca Whirlpool program. This example queries a larger range to find actual instructions:

<CodeGroup>
  ```bash curl theme={"system"}
  curl --compressed -X POST 'https://portal.sqd.dev/datasets/solana-mainnet/stream' \
    -H 'Content-Type: application/json' \
    -d '{
      "type": "solana",
      "fromBlock": 259984800,
      "toBlock": 259985000,
      "fields": {
        "block": {
          "number": true,
          "timestamp": true
        },
        "instruction": {
          "programId": true,
          "accounts": true,
          "data": true,
          "transactionIndex": true
        }
      },
      "instructions": [{
        "programId": ["whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc"]
      }]
    }'
  ```

  ```typescript TypeScript theme={"system"}
  import { DataSource } from "@subsquid/portal-client";

  const dataSource = new DataSource({
    network: "solana-mainnet",
  });

  const blocks = await dataSource.getBlocks({
    from: 259984800,
    to: 259985000,
    fields: {
      block: {
        number: true,
        timestamp: true,
      },
      instruction: {
        programId: true,
        accounts: true,
        data: true,
        transactionIndex: true,
      },
    },
    instructions: [
      {
        programId: ["whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc"], // Orca Whirlpool
      },
    ],
  });

  for (const block of blocks) {
    if (block.instructions.length > 0) {
      console.log(
        `Slot ${block.header.number}: Found ${block.instructions.length} Whirlpool instructions`
      );
      for (const instruction of block.instructions) {
        console.log(
          `  Instruction from ${instruction.programId} at tx index ${instruction.transactionIndex}`
        );
      }
    }
  }
  ```

  ```python Python theme={"system"}
  import requests
  import json

  url = "https://portal.sqd.dev/datasets/solana-mainnet/stream"
  headers = {"Content-Type": "application/json"}
  payload = {
      "type": "solana",
      "fromBlock": 259984800,
      "toBlock": 259985000,
      "fields": {
          "block": {
              "number": True,
              "timestamp": True
          },
          "instruction": {
              "programId": True,
              "accounts": True,
              "data": True,
              "transactionIndex": True
          }
      },
      "instructions": [{
          "programId": ["whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc"]
      }]
  }

  response = requests.post(url, headers=headers, json=payload)

  for line in response.text.strip().split('\n'):
      if line.strip():  # Skip empty lines
          block = json.loads(line)
          if block.get('instructions'):
              print(f"Slot {block['header']['number']}: Found {len(block['instructions'])} Whirlpool instructions")
              for instruction in block['instructions']:
                  print(f"  Instruction from {instruction['programId']} at tx index {instruction['transactionIndex']}")
  ```
</CodeGroup>

<Check>
  You should see instructions from the Orca Whirlpool program. Portal filtered
  the data before sending, saving bandwidth.
</Check>

## What You Learned

In 5 minutes, you:

* ✓ Made HTTP requests to Portal
* ✓ Queried arbitrary slot ranges
* ✓ Filtered program instructions
* ✓ Processed newline-delimited JSON responses

## Rate Limits

The Public Portal is rate-limited:

* **20 requests per 10 seconds**
* Perfect for development and testing

<CardGroup>
  <Card title="Cloud Portal" icon="cloud" href="/en/cloud/overview">
    Production-ready managed access with higher limits
  </Card>

  <Card title="Self-Host Portal" icon="server" href="/en/portal/self-hosting">
    Run your own Portal instance with no rate limits
  </Card>
</CardGroup>

## Next Steps

<CardGroup>
  <Card title="API Reference" icon="book" href="/en/portal/solana/api">
    Complete reference with all fields and filters
  </Card>

  <Card title="View Examples" icon="code" href="/en/portal/solana/examples/query-instructions">
    Practical examples for common use cases
  </Card>

  <Card title="Use with SDK" icon="brackets-curly" href="/en/sdk/overview">
    Build type-safe indexers with Portal as the data source
  </Card>
</CardGroup>

### Popular Examples

<CardGroup>
  <Card title="Query Instructions" icon="code" href="/en/portal/solana/examples/query-instructions">
    Track program instructions
  </Card>

  <Card title="Query Transactions" icon="arrow-right-arrow-left" href="/en/portal/solana/examples/query-transactions">
    Monitor wallet activity
  </Card>

  <Card title="Track Token Transfers" icon="coins" href="/en/portal/solana/examples/token-transfers">
    Index SPL token activity
  </Card>

  <Card title="Index DEX Swaps" icon="repeat" href="/en/portal/solana/examples/dex-swaps">
    Build DEX analytics
  </Card>
</CardGroup>


# Finalized Dataset Head
Source: https://docs.sqd.dev/en/api/bitcoin/finalized-head

/en/api/catalog/bitcoin/openapi.yaml get /finalized-head
Returns the highest available finalized block number and hash.



# Stream Finalized Blocks
Source: https://docs.sqd.dev/en/api/bitcoin/finalized-stream

/en/api/catalog/bitcoin/openapi.yaml post /finalized-stream
Returns only finalized blocks matching the query. Same request format as /stream but never returns unfinalized blocks, so there are no chain reorganizations to handle (no 409 responses).




# Dataset Head
Source: https://docs.sqd.dev/en/api/bitcoin/head

/en/api/catalog/bitcoin/openapi.yaml get /head
Returns the highest available block number and hash.



# Dataset Metadata
Source: https://docs.sqd.dev/en/api/bitcoin/metadata

/en/api/catalog/bitcoin/openapi.yaml get /metadata
Retrieves metadata describing the dataset, including its name, aliases, start block, and real-time status.



# Stream Blocks
Source: https://docs.sqd.dev/en/api/bitcoin/stream

/en/api/catalog/bitcoin/openapi.yaml post /stream
Returns blocks matching the query. May include unfinalized blocks. Response is a JSON array of blocks containing the requested fields. Set toBlock to limit the range, or omit it to stream up to the dataset height. Supports gzip compression via Accept-Encoding and Content-Encoding headers.




# Latest Finalized Block
Source: https://docs.sqd.dev/en/api/hyperliquid-fills/finalized-head

/en/api/catalog/hyperliquid/openapi-fills.yaml get /finalized-head
Returns the block number and hash of the highest finalized block available in the dataset, or null if no blocks are available. Useful for diagnostics.



# Latest Block
Source: https://docs.sqd.dev/en/api/hyperliquid-fills/head

/en/api/catalog/hyperliquid/openapi-fills.yaml get /head
Returns the block number and hash of the highest block available in the dataset, or null if no blocks are available. Takes real-time data into account. Useful for diagnostics.



# Latest Finalized Block
Source: https://docs.sqd.dev/en/api/hyperliquid-replica-cmds/finalized-head

/en/api/catalog/hyperliquid/openapi-replica-cmds.yaml get /finalized-head
Returns the block number and hash of the highest finalized block available in the dataset, or null if no blocks are available. Useful for diagnostics.



# Stream Finalized Blocks
Source: https://docs.sqd.dev/en/api/hyperliquid-replica-cmds/finalized-stream

/en/api/catalog/hyperliquid/openapi-replica-cmds.yaml post /finalized-stream
Returns only finalized blocks matching the query. Same request format as /stream but never returns unfinalized blocks, so there are no chain reorganizations to handle (no 409 responses).




# Latest Block
Source: https://docs.sqd.dev/en/api/hyperliquid-replica-cmds/head

/en/api/catalog/hyperliquid/openapi-replica-cmds.yaml get /head
Returns the block number and hash of the highest block available in the dataset, or null if no blocks are available. Takes real-time data into account. Useful for diagnostics.



# Dataset Metadata
Source: https://docs.sqd.dev/en/api/hyperliquid-replica-cmds/metadata

/en/api/catalog/hyperliquid/openapi-replica-cmds.yaml get /metadata
Retrieves metadata describing the dataset, including its name, aliases, start block, and real-time status.



# Stream Blocks
Source: https://docs.sqd.dev/en/api/hyperliquid-replica-cmds/stream

/en/api/catalog/hyperliquid/openapi-replica-cmds.yaml post /stream
Returns blocks matching the query. May include unfinalized blocks. Response is a JSON array of blocks containing the requested fields. Set toBlock to limit the range, or omit it to stream up to the dataset height. Supports gzip compression via Accept-Encoding and Content-Encoding headers.




# Finalized Dataset Head
Source: https://docs.sqd.dev/en/api/substrate/finalized-head

en/api/catalog/substrate/openapi.yaml GET /finalized-head
Returns the highest available finalized block number and hash.

<Warning>**Real-time streaming is not supported for Substrate chains.** Only finalized historical data is available. If you need real-time Substrate data, [schedule a call with our team](https://calendly.com/t-tyrie-subsquid/30min).</Warning>


# Stream Finalized Blocks
Source: https://docs.sqd.dev/en/api/substrate/finalized-stream

en/api/catalog/substrate/openapi.yaml POST /finalized-stream
Returns only finalized blocks matching the query. Same request format as /stream but guarantees only finalized data. Supports gzip compression via Accept-Encoding and Content-Encoding headers.

<Warning>**Real-time streaming is not supported for Substrate chains.** Only finalized historical data is available. If you need real-time Substrate data, [schedule a call with our team](https://calendly.com/t-tyrie-subsquid/30min).</Warning>


# Dataset Head
Source: https://docs.sqd.dev/en/api/substrate/head

en/api/catalog/substrate/openapi.yaml GET /head
Returns the highest available block number and hash.

<Warning>**Real-time streaming is not supported for Substrate chains.** Only finalized historical data is available. If you need real-time Substrate data, [schedule a call with our team](https://calendly.com/t-tyrie-subsquid/30min).</Warning>


# Dataset Metadata
Source: https://docs.sqd.dev/en/api/substrate/metadata

/en/api/catalog/substrate/openapi.yaml get /metadata
Retrieves metadata describing the dataset, including its name, aliases, start block, and real-time status.



# Stream Blocks
Source: https://docs.sqd.dev/en/api/substrate/stream

en/api/catalog/substrate/openapi.yaml POST /stream
Returns blocks matching the query. Response is a JSON array of blocks containing the requested fields. Set toBlock to limit the range, or omit it to stream up to the dataset height. Supports gzip compression via Accept-Encoding and Content-Encoding headers.

<Warning>**Real-time streaming is not supported for Substrate chains.** Only finalized historical data is available. If you need real-time Substrate data, [schedule a call with our team](https://calendly.com/t-tyrie-subsquid/30min).</Warning>


# Bitcoin
Source: https://docs.sqd.dev/en/data/bitcoin/bitcoin-mainnet

Index Bitcoin UTXO blockchain data — blocks, transactions, inputs, and outputs — via SQD Portal API for high-throughput historical streaming.

<Info>Full UTXO history: blocks, transactions, inputs (with prevout data), and outputs.</Info>

| Property   | Value             |
| ---------- | ----------------- |
| Network ID | `bitcoin-mainnet` |
| Real-time  | Yes               |

## Endpoint

```
https://portal.sqd.dev/datasets/bitcoin-mainnet
```

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/bitcoin-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "bitcoin",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "size": true }
    },
    "transactions": [{}]
  }'
```

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field        | Type    | Description              |
    | ------------ | ------- | ------------------------ |
    | `number`     | integer | Block height             |
    | `hash`       | string  | Block hash               |
    | `parentHash` | string  | Parent block hash        |
    | `timestamp`  | integer | Block timestamp (Unix)   |
    | `merkleRoot` | string  | Merkle root              |
    | `size`       | integer | Total block size (bytes) |
    | `weight`     | integer | Block weight (BIP 141)   |
  </Accordion>

  <Accordion title="Transactions">
    | Field              | Type    | Description                        |
    | ------------------ | ------- | ---------------------------------- |
    | `transactionIndex` | integer | Index in the block                 |
    | `txid`             | string  | Transaction ID                     |
    | `hash`             | string  | Tx hash (witness-aware)            |
    | `size`             | integer | Total size (bytes)                 |
    | `vsize`            | integer | Virtual size                       |
    | `weight`           | integer | Weight                             |
    | `version`          | integer | Tx version                         |
    | `locktime`         | integer | Earliest time/height for inclusion |
  </Accordion>

  <Accordion title="Inputs">
    | Field                        | Type    | Description                     |
    | ---------------------------- | ------- | ------------------------------- |
    | `transactionIndex`           | integer | Parent transaction index        |
    | `inputIndex`                 | integer | Index of this input             |
    | `txid`                       | string  | Tx ID of the spent output       |
    | `vout`                       | integer | Index of the spent output       |
    | `scriptSigHex`               | string  | ScriptSig in hex                |
    | `prevoutValue`               | number  | Value of the spent output (BTC) |
    | `prevoutScriptPubKeyAddress` | string  | Address of the spent output     |
  </Accordion>

  <Accordion title="Outputs">
    | Field                 | Type    | Description              |
    | --------------------- | ------- | ------------------------ |
    | `transactionIndex`    | integer | Parent transaction index |
    | `outputIndex`         | integer | Index of this output     |
    | `value`               | number  | Value (BTC)              |
    | `scriptPubKeyHex`     | string  | ScriptPubKey in hex      |
    | `scriptPubKeyType`    | string  | Output type              |
    | `scriptPubKeyAddress` | string  | Output address           |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/bitcoin/overview">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# HyperEVM
Source: https://docs.sqd.dev/en/data/evm/hyperevm-mainnet

Index HyperEVM EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value                 |
| ----------- | --------------------- |
| Network ID  | `hyperliquid-mainnet` |
| Chain ID    | `999`                 |
| Real-time   | Yes                   |
| Traces      | ✓                     |
| State Diffs | ✓                     |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/hyperliquid-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/hyperliquid-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/hyperliquid-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>

  <Accordion title="Traces">
    | Field              | Type    | Description                       |
    | ------------------ | ------- | --------------------------------- |
    | `transactionIndex` | integer | Index of parent transaction       |
    | `type`             | string  | call, create, suicide, reward     |
    | `from`             | address | Trace originator address          |
    | `to`               | address | Trace destination address         |
    | `value`            | bigint  | Value transferred                 |
    | `gas`              | bigint  | Gas provided                      |
    | `gasUsed`          | bigint  | Gas actually used                 |
    | `input`            | bytes   | Input data                        |
    | `output`           | bytes   | Output data                       |
    | `error`            | string  | Error message if the trace failed |
  </Accordion>

  <Accordion title="State Diffs">
    | Field              | Type    | Description                                      |
    | ------------------ | ------- | ------------------------------------------------ |
    | `transactionIndex` | integer | Index of parent transaction                      |
    | `address`          | address | Account whose state changed                      |
    | `key`              | string  | Storage slot key (or `balance`, `nonce`, `code`) |
    | `kind`             | string  | balance, code, nonce, storage                    |
    | `prev`             | bytes   | Previous value                                   |
    | `next`             | bytes   | New value                                        |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# HyperEVM Testnet
Source: https://docs.sqd.dev/en/data/evm/hyperevm-testnet

Index HyperEVM Testnet EVM blockchain data — blocks, transactions, logs, traces, state diffs — via SQD Portal API for high-throughput streaming.

| Property    | Value                 |
| ----------- | --------------------- |
| Network ID  | `hyperliquid-testnet` |
| Chain ID    | `998`                 |
| Real-time   | No                    |
| Traces      | ✗                     |
| State Diffs | ✗                     |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/hyperliquid-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/hyperliquid-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/hyperliquid-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "evm",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "transaction": { "hash": true, "from": true, "to": true, "value": true }
    },
    "transactions": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/evm/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field              | Type    | Description                 |
    | ------------------ | ------- | --------------------------- |
    | `number`           | integer | Block number                |
    | `hash`             | string  | Block hash                  |
    | `timestamp`        | integer | Block timestamp (Unix)      |
    | `parentHash`       | string  | Parent block hash           |
    | `gasUsed`          | bigint  | Total gas used in block     |
    | `gasLimit`         | bigint  | Block gas limit             |
    | `baseFeePerGas`    | bigint  | Base fee per gas (EIP-1559) |
    | `stateRoot`        | string  | State root hash             |
    | `transactionsRoot` | string  | Transactions trie root      |
    | `receiptsRoot`     | string  | Receipts trie root          |
  </Accordion>

  <Accordion title="Transactions">
    | Field                  | Type    | Description                                    |
    | ---------------------- | ------- | ---------------------------------------------- |
    | `hash`                 | string  | Transaction hash                               |
    | `from`                 | address | Sender address                                 |
    | `to`                   | address | Recipient address (null for contract creation) |
    | `value`                | bigint  | Transaction value in wei                       |
    | `gasUsed`              | bigint  | Gas used by transaction                        |
    | `gasPrice`             | bigint  | Gas price in wei                               |
    | `maxFeePerGas`         | bigint  | Max fee per gas (EIP-1559)                     |
    | `maxPriorityFeePerGas` | bigint  | Max priority fee (EIP-1559)                    |
    | `input`                | bytes   | Transaction input data                         |
    | `nonce`                | integer | Transaction nonce                              |
    | `status`               | integer | 1 = success, 0 = failure                       |
  </Accordion>

  <Accordion title="Logs">
    | Field              | Type    | Description                           |
    | ------------------ | ------- | ------------------------------------- |
    | `address`          | address | Contract address that emitted the log |
    | `topics`           | array   | Indexed event topics (up to 4)        |
    | `data`             | bytes   | Non-indexed event data                |
    | `logIndex`         | integer | Log index in the block                |
    | `transactionIndex` | integer | Index of parent transaction           |
    | `transactionHash`  | string  | Hash of parent transaction            |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/evm/api">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Hyperliquid
Source: https://docs.sqd.dev/en/data/hyperliquid/hyperliquid-fills

Stream Hyperliquid fill records and block metadata via the SQD Portal API for high-throughput access to HyperCore trading activity.

<Info>HyperCore Fills: trade execution data from Hyperliquid's perpetual futures exchange. Includes position changes (closing-trade PnL), builder routing, and per-user activity.</Info>

| Property   | Value               |
| ---------- | ------------------- |
| Network ID | `hyperliquid-fills` |
| Real-time  | Yes                 |

## Endpoint

```
https://portal.sqd.dev/datasets/hyperliquid-fills
```

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/hyperliquid-fills/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "hyperliquidFills",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "fill": { "user": true, "coin": true, "px": true, "sz": true, "side": true }
    },
    "fills": [{}]
  }'
```

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field       | Type    | Description     |
    | ----------- | ------- | --------------- |
    | `number`    | integer | Block number    |
    | `hash`      | string  | Block hash      |
    | `timestamp` | integer | Block timestamp |
  </Accordion>

  <Accordion title="Fills">
    | Field        | Type    | Description                        |
    | ------------ | ------- | ---------------------------------- |
    | `user`       | string  | Trader address                     |
    | `coin`       | string  | Asset name                         |
    | `px`         | number  | Fill price                         |
    | `sz`         | number  | Fill size                          |
    | `side`       | string  | `A` (ask) or `B` (bid)             |
    | `time`       | integer | Fill timestamp (ms)                |
    | `hash`       | string  | Fill hash                          |
    | `oid`        | bigint  | Order ID                           |
    | `closed_pnl` | number  | Closed PnL                         |
    | `crossed`    | boolean | Whether the order crossed the book |
    | `fee`        | number  | Fee paid                           |
    | `fee_token`  | string  | Fee token                          |
    | `tid`        | bigint  | Trade ID                           |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/hyperliquid/overview">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Hyperliquid Replica Commands
Source: https://docs.sqd.dev/en/data/hyperliquid/hyperliquid-replica-cmds

Stream Hyperliquid Replica Commands data — block metadata and actions — via the SQD Portal API for full HyperCore replica state indexing.

<Info>HyperCore replica command stream — used for advanced infrastructure monitoring and replay.</Info>

| Property   | Value                      |
| ---------- | -------------------------- |
| Network ID | `hyperliquid-replica-cmds` |
| Real-time  | No                         |

## Endpoint

```
https://portal.sqd.dev/datasets/hyperliquid-replica-cmds
```

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/hyperliquid-replica-cmds/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "hyperliquidReplicaCmds",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "action": { "type": true, "user": true, "data": true }
    },
    "actions": [{}]
  }'
```

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field       | Type    | Description     |
    | ----------- | ------- | --------------- |
    | `number`    | integer | Block number    |
    | `hash`      | string  | Block hash      |
    | `timestamp` | integer | Block timestamp |
  </Accordion>

  <Accordion title="Actions">
    | Field  | Type    | Description           |
    | ------ | ------- | --------------------- |
    | `type` | string  | Action type           |
    | `user` | string  | Acting address        |
    | `data` | object  | Action payload        |
    | `hash` | string  | Action hash           |
    | `time` | integer | Action timestamp (ms) |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/portal/hyperliquid/overview">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Substrate Networks
Source: https://docs.sqd.dev/en/data/substrate

Browse all Substrate-based networks supported by SQD — Polkadot, Kusama, parachains, and standalone Substrate chains with gateway and Portal endpoints.

<Warning>
  **Portal Migration**: We're migrating from v2 archives to Portal. V2 archives will be sunset soon. [Get Started with Portal →](/en/portal/migration)
</Warning>


# Acala
Source: https://docs.sqd.dev/en/data/substrate/acala

Index Acala Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value   |
| ---------- | ------- |
| Network ID | `acala` |
| Real-time  | No      |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/acala
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/acala
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/acala/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Acurast Canary
Source: https://docs.sqd.dev/en/data/substrate/acurast-canary

Index Acurast Canary Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value            |
| ---------- | ---------------- |
| Network ID | `acurast-canary` |
| Real-time  | No               |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/acurast-canary
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/acurast-canary
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/acurast-canary/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Agung (Substrate)
Source: https://docs.sqd.dev/en/data/substrate/agung

Index Agung (Substrate) Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value   |
| ---------- | ------- |
| Network ID | `agung` |
| Real-time  | No      |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/agung
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/agung
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/agung/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Aleph Zero
Source: https://docs.sqd.dev/en/data/substrate/aleph-zero

Index Aleph Zero Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value        |
| ---------- | ------------ |
| Network ID | `aleph-zero` |
| Real-time  | No           |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/aleph-zero
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/aleph-zero
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/aleph-zero/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Aleph Zero Testnet
Source: https://docs.sqd.dev/en/data/substrate/aleph-zero-testnet

Index Aleph Zero Testnet Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value                |
| ---------- | -------------------- |
| Network ID | `aleph-zero-testnet` |
| Real-time  | No                   |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/aleph-zero-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/aleph-zero-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/aleph-zero-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Amplitude
Source: https://docs.sqd.dev/en/data/substrate/amplitude

Index Amplitude Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value       |
| ---------- | ----------- |
| Network ID | `amplitude` |
| Real-time  | No          |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/amplitude
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/amplitude
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/amplitude/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Kusama Asset Hub
Source: https://docs.sqd.dev/en/data/substrate/asset-hub-kusama

Index Kusama Asset Hub Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

<Info>Asset Hub Kusama was formerly known as Statemine.</Info>

| Property   | Value              |
| ---------- | ------------------ |
| Network ID | `asset-hub-kusama` |
| Real-time  | No                 |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/asset-hub-kusama
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/asset-hub-kusama
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/asset-hub-kusama/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Paseo Asset Hub
Source: https://docs.sqd.dev/en/data/substrate/asset-hub-paseo

Index Paseo Asset Hub Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value             |
| ---------- | ----------------- |
| Network ID | `asset-hub-paseo` |
| Real-time  | No                |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/asset-hub-paseo
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/asset-hub-paseo
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/asset-hub-paseo/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Polkadot Asset Hub
Source: https://docs.sqd.dev/en/data/substrate/asset-hub-polkadot

Index Polkadot Asset Hub Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

<Info>Asset Hub Polkadot was formerly known as Statemint.</Info>

| Property   | Value                |
| ---------- | -------------------- |
| Network ID | `asset-hub-polkadot` |
| Real-time  | No                   |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/asset-hub-polkadot
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/asset-hub-polkadot
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/asset-hub-polkadot/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Rococo Asset Hub
Source: https://docs.sqd.dev/en/data/substrate/asset-hub-rococo

Index Rococo Asset Hub Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value              |
| ---------- | ------------------ |
| Network ID | `asset-hub-rococo` |
| Real-time  | No                 |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/asset-hub-rococo
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/asset-hub-rococo
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/asset-hub-rococo/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Astar (Substrate)
Source: https://docs.sqd.dev/en/data/substrate/astar-substrate

Index Astar (Substrate) Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value             |
| ---------- | ----------------- |
| Network ID | `astar-substrate` |
| Real-time  | No                |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/astar-substrate
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/astar-substrate
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/astar-substrate/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Avail
Source: https://docs.sqd.dev/en/data/substrate/avail

Index Avail Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

<Info>The Avail Mainnet dataset starts at block 1.</Info>

| Property   | Value   |
| ---------- | ------- |
| Network ID | `avail` |
| Real-time  | No      |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/avail
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/avail
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/avail/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Basilisk
Source: https://docs.sqd.dev/en/data/substrate/basilisk

Index Basilisk Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value      |
| ---------- | ---------- |
| Network ID | `basilisk` |
| Real-time  | No         |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/basilisk
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/basilisk
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/basilisk/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Bifrost Kusama
Source: https://docs.sqd.dev/en/data/substrate/bifrost-kusama

Index Bifrost Kusama Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value            |
| ---------- | ---------------- |
| Network ID | `bifrost-kusama` |
| Real-time  | No               |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/bifrost-kusama
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/bifrost-kusama
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/bifrost-kusama/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Bifrost Polkadot
Source: https://docs.sqd.dev/en/data/substrate/bifrost-polkadot

Index Bifrost Polkadot Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value              |
| ---------- | ------------------ |
| Network ID | `bifrost-polkadot` |
| Real-time  | No                 |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/bifrost-polkadot
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/bifrost-polkadot
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/bifrost-polkadot/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Bittensor (Substrate)
Source: https://docs.sqd.dev/en/data/substrate/bittensor

Index Bittensor (Substrate) Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value       |
| ---------- | ----------- |
| Network ID | `bittensor` |
| Real-time  | No          |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/bittensor
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/bittensor
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/bittensor/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Bittensor Testnet (Substrate)
Source: https://docs.sqd.dev/en/data/substrate/bittensor-testnet

Index Bittensor Testnet (Substrate) Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value               |
| ---------- | ------------------- |
| Network ID | `bittensor-testnet` |
| Real-time  | No                  |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/bittensor-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/bittensor-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/bittensor-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Kusama Bridge Hub
Source: https://docs.sqd.dev/en/data/substrate/bridge-hub-kusama

Index Kusama Bridge Hub Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value               |
| ---------- | ------------------- |
| Network ID | `bridge-hub-kusama` |
| Real-time  | No                  |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/bridge-hub-kusama
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/bridge-hub-kusama
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/bridge-hub-kusama/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Polkadot Bridge Hub
Source: https://docs.sqd.dev/en/data/substrate/bridge-hub-polkadot

Index Polkadot Bridge Hub Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value                 |
| ---------- | --------------------- |
| Network ID | `bridge-hub-polkadot` |
| Real-time  | No                    |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/bridge-hub-polkadot
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/bridge-hub-polkadot
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/bridge-hub-polkadot/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Rococo Bridge Hub
Source: https://docs.sqd.dev/en/data/substrate/bridge-hub-rococo

Index Rococo Bridge Hub Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value               |
| ---------- | ------------------- |
| Network ID | `bridge-hub-rococo` |
| Real-time  | No                  |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/bridge-hub-rococo
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/bridge-hub-rococo
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/bridge-hub-rococo/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Centrifuge
Source: https://docs.sqd.dev/en/data/substrate/centrifuge

Index Centrifuge Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value        |
| ---------- | ------------ |
| Network ID | `centrifuge` |
| Real-time  | No           |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/centrifuge
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/centrifuge
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/centrifuge/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Cere
Source: https://docs.sqd.dev/en/data/substrate/cere

Index Cere Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value  |
| ---------- | ------ |
| Network ID | `cere` |
| Real-time  | No     |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/cere
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/cere
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/cere/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Chainflip
Source: https://docs.sqd.dev/en/data/substrate/chainflip

Index Chainflip Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value       |
| ---------- | ----------- |
| Network ID | `chainflip` |
| Real-time  | No          |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/chainflip
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/chainflip
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/chainflip/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Clover
Source: https://docs.sqd.dev/en/data/substrate/clover

Index Clover Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value    |
| ---------- | -------- |
| Network ID | `clover` |
| Real-time  | No       |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/clover
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/clover
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/clover/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Polkadot Collectives
Source: https://docs.sqd.dev/en/data/substrate/collectives-polkadot

Index Polkadot Collectives Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value                  |
| ---------- | ---------------------- |
| Network ID | `collectives-polkadot` |
| Real-time  | No                     |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/collectives-polkadot
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/collectives-polkadot
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/collectives-polkadot/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Crust
Source: https://docs.sqd.dev/en/data/substrate/crust

Index Crust Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value   |
| ---------- | ------- |
| Network ID | `crust` |
| Real-time  | No      |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/crust
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/crust
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/crust/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Dancebox
Source: https://docs.sqd.dev/en/data/substrate/dancebox

Index Dancebox Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value      |
| ---------- | ---------- |
| Network ID | `dancebox` |
| Real-time  | No         |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/dancebox
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/dancebox
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/dancebox/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Darwinia
Source: https://docs.sqd.dev/en/data/substrate/darwinia

Index Darwinia Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value      |
| ---------- | ---------- |
| Network ID | `darwinia` |
| Real-time  | No         |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/darwinia
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/darwinia
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/darwinia/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Darwinia Crab
Source: https://docs.sqd.dev/en/data/substrate/darwinia-crab

Index Darwinia Crab Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value           |
| ---------- | --------------- |
| Network ID | `darwinia-crab` |
| Real-time  | No              |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/darwinia-crab
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/darwinia-crab
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/darwinia-crab/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Avail (Legacy)
Source: https://docs.sqd.dev/en/data/substrate/data-avail

Index Avail (Legacy) Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value        |
| ---------- | ------------ |
| Network ID | `data-avail` |
| Real-time  | No           |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/data-avail
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/data-avail
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/data-avail/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Eden
Source: https://docs.sqd.dev/en/data/substrate/eden

Index Eden Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value  |
| ---------- | ------ |
| Network ID | `eden` |
| Real-time  | No     |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/eden
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/eden
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/eden/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Enjin Canary Matrixchain
Source: https://docs.sqd.dev/en/data/substrate/enjin-canary-matrix

Index Enjin Canary Matrixchain Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value                 |
| ---------- | --------------------- |
| Network ID | `enjin-canary-matrix` |
| Real-time  | No                    |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/enjin-canary-matrix
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/enjin-canary-matrix
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/enjin-canary-matrix/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Enjin Matrixchain
Source: https://docs.sqd.dev/en/data/substrate/enjin-matrix

Index Enjin Matrixchain Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value          |
| ---------- | -------------- |
| Network ID | `enjin-matrix` |
| Real-time  | No             |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/enjin-matrix
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/enjin-matrix
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/enjin-matrix/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Enjin Relaychain
Source: https://docs.sqd.dev/en/data/substrate/enjin-relay

Index Enjin Relaychain Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value         |
| ---------- | ------------- |
| Network ID | `enjin-relay` |
| Real-time  | No            |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/enjin-relay
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/enjin-relay
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/enjin-relay/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Equilibrium
Source: https://docs.sqd.dev/en/data/substrate/equilibrium

Index Equilibrium Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value         |
| ---------- | ------------- |
| Network ID | `equilibrium` |
| Real-time  | No            |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/equilibrium
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/equilibrium
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/equilibrium/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Foucoco
Source: https://docs.sqd.dev/en/data/substrate/foucoco

Index Foucoco Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value     |
| ---------- | --------- |
| Network ID | `foucoco` |
| Real-time  | No        |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/foucoco
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/foucoco
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/foucoco/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Frequency
Source: https://docs.sqd.dev/en/data/substrate/frequency

Index Frequency Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value       |
| ---------- | ----------- |
| Network ID | `frequency` |
| Real-time  | No          |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/frequency
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/frequency
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/frequency/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Hydration
Source: https://docs.sqd.dev/en/data/substrate/hydradx

Index Hydration Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value     |
| ---------- | --------- |
| Network ID | `hydradx` |
| Real-time  | No        |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/hydradx
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/hydradx
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/hydradx/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Integritee
Source: https://docs.sqd.dev/en/data/substrate/integritee

Index Integritee Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value        |
| ---------- | ------------ |
| Network ID | `integritee` |
| Real-time  | No           |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/integritee
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/integritee
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/integritee/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Interlay
Source: https://docs.sqd.dev/en/data/substrate/interlay

Index Interlay Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value      |
| ---------- | ---------- |
| Network ID | `interlay` |
| Real-time  | No         |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/interlay
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/interlay
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/interlay/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# InvArch
Source: https://docs.sqd.dev/en/data/substrate/invarch-parachain

Index InvArch Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value               |
| ---------- | ------------------- |
| Network ID | `invarch-parachain` |
| Real-time  | No                  |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/invarch-parachain
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/invarch-parachain
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/invarch-parachain/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# InvArch Tinkernet
Source: https://docs.sqd.dev/en/data/substrate/invarch-tinkernet

Index InvArch Tinkernet Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value               |
| ---------- | ------------------- |
| Network ID | `invarch-tinkernet` |
| Real-time  | No                  |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/invarch-tinkernet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/invarch-tinkernet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/invarch-tinkernet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Joystream
Source: https://docs.sqd.dev/en/data/substrate/joystream

Index Joystream Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value       |
| ---------- | ----------- |
| Network ID | `joystream` |
| Real-time  | No          |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/joystream
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/joystream
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/joystream/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Karura
Source: https://docs.sqd.dev/en/data/substrate/karura

Index Karura Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value    |
| ---------- | -------- |
| Network ID | `karura` |
| Real-time  | No       |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/karura
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/karura
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/karura/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Khala
Source: https://docs.sqd.dev/en/data/substrate/khala

Index Khala Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value   |
| ---------- | ------- |
| Network ID | `khala` |
| Real-time  | No      |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/khala
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/khala
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/khala/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# KILT
Source: https://docs.sqd.dev/en/data/substrate/kilt

Index KILT Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value  |
| ---------- | ------ |
| Network ID | `kilt` |
| Real-time  | No     |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/kilt
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/kilt
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/kilt/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Kintsugi
Source: https://docs.sqd.dev/en/data/substrate/kintsugi

Index Kintsugi Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value      |
| ---------- | ---------- |
| Network ID | `kintsugi` |
| Real-time  | No         |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/kintsugi
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/kintsugi
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/kintsugi/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Kusama
Source: https://docs.sqd.dev/en/data/substrate/kusama

Index Kusama Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value    |
| ---------- | -------- |
| Network ID | `kusama` |
| Real-time  | No       |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/kusama
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/kusama
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/kusama/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Heima
Source: https://docs.sqd.dev/en/data/substrate/litentry

Index Heima Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value      |
| ---------- | ---------- |
| Network ID | `litentry` |
| Real-time  | No         |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/litentry
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/litentry
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/litentry/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Moonbase Alpha (Substrate)
Source: https://docs.sqd.dev/en/data/substrate/moonbase-substrate

Index Moonbase Alpha (Substrate) Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value                |
| ---------- | -------------------- |
| Network ID | `moonbase-substrate` |
| Real-time  | No                   |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/moonbase-substrate
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/moonbase-substrate
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/moonbase-substrate/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Moonbeam (Substrate)
Source: https://docs.sqd.dev/en/data/substrate/moonbeam-substrate

Index Moonbeam (Substrate) Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value                |
| ---------- | -------------------- |
| Network ID | `moonbeam-substrate` |
| Real-time  | No                   |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/moonbeam-substrate
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/moonbeam-substrate
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/moonbeam-substrate/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Moonriver (Substrate)
Source: https://docs.sqd.dev/en/data/substrate/moonriver-substrate

Index Moonriver (Substrate) Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value                 |
| ---------- | --------------------- |
| Network ID | `moonriver-substrate` |
| Real-time  | No                    |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/moonriver-substrate
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/moonriver-substrate
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/moonriver-substrate/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Paseo
Source: https://docs.sqd.dev/en/data/substrate/paseo

Index Paseo Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value   |
| ---------- | ------- |
| Network ID | `paseo` |
| Real-time  | No      |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/paseo
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/paseo
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/paseo/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Peaq (Substrate)
Source: https://docs.sqd.dev/en/data/substrate/peaq-mainnet-substrate

Index Peaq (Substrate) Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value                    |
| ---------- | ------------------------ |
| Network ID | `peaq-mainnet-substrate` |
| Real-time  | No                       |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/peaq-mainnet-substrate
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/peaq-mainnet-substrate
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/peaq-mainnet-substrate/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Pendulum
Source: https://docs.sqd.dev/en/data/substrate/pendulum

Index Pendulum Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value      |
| ---------- | ---------- |
| Network ID | `pendulum` |
| Real-time  | No         |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/pendulum
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/pendulum
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/pendulum/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Polkadot People
Source: https://docs.sqd.dev/en/data/substrate/people-chain

Index Polkadot People Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value          |
| ---------- | -------------- |
| Network ID | `people-chain` |
| Real-time  | No             |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/people-chain
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/people-chain
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/people-chain/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Phala
Source: https://docs.sqd.dev/en/data/substrate/phala

Index Phala Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value   |
| ---------- | ------- |
| Network ID | `phala` |
| Real-time  | No      |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/phala
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/phala
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/phala/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Phala Testnet
Source: https://docs.sqd.dev/en/data/substrate/phala-testnet

Index Phala Testnet Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value           |
| ---------- | --------------- |
| Network ID | `phala-testnet` |
| Real-time  | No              |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/phala-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/phala-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/phala-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Picasso
Source: https://docs.sqd.dev/en/data/substrate/picasso

Index Picasso Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value     |
| ---------- | --------- |
| Network ID | `picasso` |
| Real-time  | No        |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/picasso
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/picasso
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/picasso/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Polimec
Source: https://docs.sqd.dev/en/data/substrate/polimec

Index Polimec Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value     |
| ---------- | --------- |
| Network ID | `polimec` |
| Real-time  | No        |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/polimec
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/polimec
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/polimec/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Polkadex
Source: https://docs.sqd.dev/en/data/substrate/polkadex

Index Polkadex Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value      |
| ---------- | ---------- |
| Network ID | `polkadex` |
| Real-time  | No         |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/polkadex
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/polkadex
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/polkadex/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Polkadot
Source: https://docs.sqd.dev/en/data/substrate/polkadot

Index Polkadot Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value      |
| ---------- | ---------- |
| Network ID | `polkadot` |
| Real-time  | No         |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/polkadot
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/polkadot
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/polkadot/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Polymesh
Source: https://docs.sqd.dev/en/data/substrate/polymesh

Index Polymesh Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value      |
| ---------- | ---------- |
| Network ID | `polymesh` |
| Real-time  | No         |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/polymesh
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/polymesh
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/polymesh/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Reef
Source: https://docs.sqd.dev/en/data/substrate/reef

Index Reef Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value  |
| ---------- | ------ |
| Network ID | `reef` |
| Real-time  | No     |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/reef
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/reef
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/reef/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Reef Testnet
Source: https://docs.sqd.dev/en/data/substrate/reef-testnet

Index Reef Testnet Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value          |
| ---------- | -------------- |
| Network ID | `reef-testnet` |
| Real-time  | No             |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/reef-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/reef-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/reef-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Robonomics
Source: https://docs.sqd.dev/en/data/substrate/robonomics

Index Robonomics Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value        |
| ---------- | ------------ |
| Network ID | `robonomics` |
| Real-time  | No           |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/robonomics
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/robonomics
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/robonomics/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Rococo
Source: https://docs.sqd.dev/en/data/substrate/rococo

Index Rococo Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value    |
| ---------- | -------- |
| Network ID | `rococo` |
| Real-time  | No       |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/rococo
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/rococo
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/rococo/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Query Taproot Inputs
Source: https://docs.sqd.dev/en/portal/bitcoin/examples/query-inputs-by-address

Filter Bitcoin transaction inputs by script type with the SQD Portal — examples for tracking Taproot spending and analyzing UTXO addresses.

Query Bitcoin transaction inputs filtered by prevout script type to analyze spending patterns across address types.

## Use Case

Analyze Bitcoin inputs to:

* Track Taproot adoption by monitoring witness\_v1\_taproot inputs
* Analyze UTXO spending patterns by script type
* Monitor value flow from specific address types
* Build spending analytics dashboards

<QueryInterfaceQueryInputsByAddress />

## Sample Response

```json theme={"system"}
{
  "header": {
    "number": 880000,
    "timestamp": 1737337343
  },
  "inputs": [
    {
      "transactionIndex": 9,
      "inputIndex": 0,
      "txid": "80e6a90b473e8fd1e70f3e2844dc24b679728d1c23d581a0f6b849d4e9c1fddf",
      "vout": 0,
      "prevoutValue": 0.68954682,
      "prevoutScriptPubKeyType": "witness_v1_taproot",
      "prevoutScriptPubKeyAddress": "bc1p2vnhpsfmyfz4259zdw9jddvq2nnnktev9gz4ggqnqf3pf660z04q8ar0ka"
    }
  ]
}
```

## Field Descriptions

* `transactionIndex` - Index of the parent transaction within the block
* `inputIndex` - Index of this input within the transaction
* `txid` - Transaction ID of the output being spent
* `vout` - Index of the output being spent in the referenced transaction
* `prevoutValue` - Value of the spent output in BTC
* `prevoutScriptPubKeyType` - Script type of the spent output (e.g. `witness_v1_taproot`)
* `prevoutScriptPubKeyAddress` - Address of the spent output


# Query Outputs by Type
Source: https://docs.sqd.dev/en/portal/bitcoin/examples/query-outputs-by-address

Filter Bitcoin transaction outputs by script type with the SQD Portal — code examples for grouping UTXOs and identifying spend patterns by address.

Query Bitcoin transaction outputs filtered by script type to analyze address activity and UTXO distribution.

## Use Case

Track Bitcoin outputs to:

* Monitor Taproot adoption (witness\_v1\_taproot outputs)
* Analyze UTXO set composition by script type
* Track value flow to specific address types
* Build address balance tracking systems

<QueryInterfaceQueryOutputsByAddress />

## Sample Response

```json theme={"system"}
{
  "header": {
    "number": 880000,
    "timestamp": 1737504179
  },
  "outputs": [
    {
      "transactionIndex": 1,
      "outputIndex": 0,
      "value": 50000,
      "scriptPubKeyType": "witness_v1_taproot",
      "scriptPubKeyAddress": "bc1p..."
    }
  ]
}
```

## Field Descriptions

* `transactionIndex` - Index of the parent transaction within the block
* `outputIndex` - Index of this output within the transaction
* `value` - Output value in satoshis
* `scriptPubKeyType` - Script type (e.g. `pubkeyhash`, `scripthash`, `witness_v0_keyhash`, `witness_v1_taproot`, `nulldata`)
* `scriptPubKeyAddress` - Output address (when applicable)


# Query Bitcoin Transactions
Source: https://docs.sqd.dev/en/portal/bitcoin/examples/query-transactions

Query Bitcoin transactions with full metadata via the SQD Portal — paginate, filter by address or script, and stream historical UTXO transaction data.

Query all Bitcoin transactions in a block range with full transaction metadata including txid, hash, size, and weight.

## Use Case

Analyze Bitcoin transaction activity to:

* Monitor transaction volume and sizes
* Track SegWit adoption via vsize vs size comparison
* Analyze block utilization and fee markets
* Build transaction analytics dashboards

<QueryInterfaceQueryTransactions />

## Sample Response

```json theme={"system"}
{
  "header": {
    "number": 880000,
    "hash": "0000000000000000000320283a032748cef8227873ff4872689bf23f1cda83a5",
    "timestamp": 1737504179
  },
  "transactions": [
    {
      "transactionIndex": 0,
      "txid": "a1b2c3d4e5f6...",
      "hash": "a1b2c3d4e5f6...",
      "size": 250,
      "vsize": 166,
      "weight": 661,
      "version": 2,
      "locktime": 0
    }
  ]
}
```

## Field Descriptions

* `txid` - Transaction ID (double SHA-256 of serialized transaction without witness data)
* `hash` - Transaction hash (includes witness data; differs from txid for SegWit transactions)
* `size` - Total transaction size in bytes (including witness)
* `vsize` - Virtual size (weight / 4), used for fee calculation
* `weight` - Transaction weight (non-witness bytes \* 4 + witness bytes)
* `version` - Transaction version number
* `locktime` - Earliest time or block height at which the transaction can be included


# Bitcoin Portal API
Source: https://docs.sqd.dev/en/portal/bitcoin/overview

Access Bitcoin UTXO data — blocks, transactions, inputs, and outputs — through the SQD Portal HTTP API for high-throughput historical streaming.

Portal provides direct access to Bitcoin block and transaction data. Query blocks, transactions, inputs, and outputs with comprehensive filtering to analyze on-chain activity, track addresses, and monitor the Bitcoin network.

## Supported Networks

## Next Steps

<CardGroup>
  <Card title="Interactive Examples" icon="play" href="/en/portal/bitcoin/examples/query-transactions">
    Try live queries with our interactive examples
  </Card>

  <Card title="OpenAPI Schema" icon="file-code" href="https://beta.docs.sqd.dev/files/bitcoin-openapi.yaml">
    Download the full OpenAPI specification
  </Card>
</CardGroup>

<Info>
  **Enterprise Custom Development**

  For enterprise clients requiring custom indexer development, please contact our team to discuss your specific requirements. [Schedule a Consultation →](https://calendly.com/t-tyrie-subsquid/30min)
</Info>


# Analyze Closing Trades
Source: https://docs.sqd.dev/en/portal/hyperliquid/examples/analyze-closing-trades

Track Hyperliquid position-closing trades and realized PnL through the SQD Portal — sample queries for closeTime, fills, and trader-level performance.

Monitor trades that close positions to analyze realized profits and losses across all traders and assets.

## Use Case

Track position closures to:

* Monitor realized PnL distributions
* Identify profitable trading patterns
* Analyze liquidation events
* Build PnL leaderboards and analytics

<QueryInterfaceAnalyzeClosingTrades />

## Sample Response

```json theme={"system"}
{
  "header": {
    "number": 900000004,
    "timestamp": 1771562718537
  },
  "fills": [
    {
      "user": "0x0bb3e7ca6d66021eecbf23fb09e6dabe44bf1b41",
      "coin": "BTC",
      "px": 67261.0,
      "sz": 0.00305,
      "side": "B",
      "dir": "Close Short",
      "closedPnl": 0.061915,
      "startPosition": -3.23188,
      "fee": 0.0,
      "time": 1771562718537
    }
  ]
}
```

## Field Descriptions

* `dir` - Trade direction: "Close Long" or "Close Short"
* `closedPnl` - Realized profit/loss from closing the position (in USD)
* `startPosition` - Position size before this fill (negative = short, positive = long)
* `px` - Exit price
* `sz` - Size of the position being closed
* `fee` - Trading fee charged
* `time` - Fill timestamp in milliseconds since Unix epoch

## Analysis Tips

* **Net PnL** = `closedPnl` - `fee` (subtract fees for true profitability)
* **Profitable trades** have `closedPnl > 0`
* **Loss-making trades** have `closedPnl < 0`
* Positive `startPosition` indicates closing a long, negative indicates closing a short


# Monitor Builder Activity
Source: https://docs.sqd.dev/en/portal/hyperliquid/examples/monitor-builder-activity

Track Hyperliquid fills routed through specific builders via the SQD Portal — sample queries for monitoring builder addresses, fees, and trade volume.

Monitor trade fills that are routed through specific builder addresses to analyze builder performance and fee structures.

## Use Case

Track builder activity to:

* Monitor builder routing and fee capture
* Analyze builder market share
* Track builder performance metrics
* Build builder comparison dashboards

<QueryInterfaceMonitorBuilderActivity />

## Sample Response

```json theme={"system"}
{
  "header": {
    "number": 900000001,
    "timestamp": 1771562718255
  },
  "fills": [
    {
      "user": "0x588bc56a39fcc6363d6cd3bac1e2da2f857e8698",
      "coin": "SUI",
      "px": 0.93401,
      "sz": 3.4,
      "fee": 0.001688,
      "builderFee": 0.000317,
      "tid": 752320185569099,
      "builder": "0x6530512a6c89c7cfcebc3ba7fcd9ada5f30827a6"
    }
  ]
}
```

## Field Descriptions

* `builder` - Builder wallet address (0x-prefixed hex, lowercase)
* `builderFee` - Additional fee paid to the order builder
* `fee` - Total trading fee charged
* `tid` - Trade ID (unique identifier for the matched trade)
* `coin` - Asset symbol being traded
* `px` - Fill price
* `sz` - Fill size

## Note

Not all fills have builder information - only fills routed through builders will have the `builder` and `builderFee` fields populated.


# Query BTC Fills
Source: https://docs.sqd.dev/en/portal/hyperliquid/examples/query-btc-fills

Track Hyperliquid BTC perpetual futures fills via the SQD Portal — sample queries for analyzing Bitcoin perp trade volume, prices, and trader positions.

Monitor all BTC perpetual futures trades including position changes and realized PnL.

## Use Case

Track Bitcoin trading activity to:

* Monitor BTC market depth and liquidity
* Analyze position changes and PnL
* Track market maker activity
* Build BTC-specific trading analytics

<QueryInterfaceQueryBtcFills />

## Sample Response

```json theme={"system"}
{
  "header": {
    "number": 900000004,
    "timestamp": 1771562718537
  },
  "fills": [
    {
      "user": "0x0bb3e7ca6d66021eecbf23fb09e6dabe44bf1b41",
      "coin": "BTC",
      "px": 67261.0,
      "sz": 0.00305,
      "side": "B",
      "startPosition": -3.23188,
      "dir": "Close Short",
      "closedPnl": 0.061915,
      "fee": 0.0
    }
  ]
}
```

## Field Descriptions

* `coin` - Asset symbol (always "BTC" in this query)
* `px` - Fill price in USD
* `sz` - Fill size in BTC
* `side` - "A" (ask/sell) or "B" (bid/buy)
* `dir` - Trade direction: "Open Long", "Close Long", "Open Short", "Close Short"
* `closedPnl` - Realized profit/loss when closing a position (0 for opening trades)
* `startPosition` - Position size before this fill (negative = short, positive = long)
* `fee` - Trading fee charged


# Track Multi-Asset Trading
Source: https://docs.sqd.dev/en/portal/hyperliquid/examples/track-asset-trading

Monitor Hyperliquid trading across multiple assets via the SQD Portal — sample queries for cross-asset volume, trader portfolios, and market correlations.

Track trading activity across multiple cryptocurrency perpetual futures to analyze market trends and liquidity.

## Use Case

Monitor multi-asset trading to:

* Compare trading volumes across assets
* Identify trending assets
* Analyze cross-asset arbitrage opportunities
* Build market overview dashboards

<QueryInterfaceTrackAssetTrading />

## Sample Response

```json theme={"system"}
{
  "header": {
    "number": 905000003,
    "timestamp": 1771899118437
  },
  "fills": [
    {
      "user": "0xa710948745544a27ddd66a63f408ff11b869797d",
      "coin": "ETH",
      "px": 1943.4,
      "sz": 0.0258,
      "side": "B",
      "dir": "Open Long",
      "fee": 0.02166,
      "crossed": true,
      "tid": 826601424406812
    },
    {
      "user": "0x6a9e7566877b52b13b28230b5543cbccf3685870",
      "coin": "SOL",
      "px": 98.45,
      "sz": 12.5,
      "side": "A",
      "dir": "Close Short",
      "fee": 0.0154,
      "crossed": false,
      "tid": 826601424406813
    }
  ]
}
```

## Field Descriptions

* `coin` - Asset symbol (e.g., "ETH", "BTC", "SOL")
* `px` - Fill price in USD
* `sz` - Fill size in the asset's base unit
* `crossed` - Whether this fill crossed the order book (was a taker fill)
* `tid` - Trade ID (unique identifier for the matched trade)
* `side` - "A" (ask/sell) or "B" (bid/buy)
* `dir` - Trade direction: "Open Long", "Close Long", "Open Short", "Close Short"

## Analysis Tips

* **Notional Volume** = `px` × `sz` (total USD value of the trade)
* **Taker fills** have `crossed: true`, maker fills have `crossed: false`
* Aggregate volumes across assets to identify the most actively traded markets
* Compare taker vs. maker ratios to understand market dynamics


# Track Trader Fills
Source: https://docs.sqd.dev/en/portal/hyperliquid/examples/track-trader-fills

Monitor all Hyperliquid trade fills for specific traders via the SQD Portal — sample queries for trader history, PnL, and per-asset trading patterns.

Track all trade executions for a specific trader address to analyze their trading activity, including entry and exit positions.

## Use Case

Monitor individual trader activity to:

* Track trading patterns and strategies
* Analyze profit and loss over time
* Monitor position sizing and risk management
* Build trader performance analytics

<QueryInterfaceTrackTraderFills />

## Sample Response

```json theme={"system"}
{
  "header": {
    "number": 800000383,
    "timestamp": 1763423588545
  },
  "fills": [
    {
      "user": "0xd3d123f0f62559c62745ef0b631a21a95916b6b1",
      "coin": "STRK",
      "px": 0.19454,
      "sz": 229.7,
      "side": "B",
      "dir": "Open Long",
      "oid": 239170049656,
      "fee": 0.017605
    }
  ]
}
```

## Field Descriptions

* `user` - Trader wallet address (0x-prefixed hex, lowercase)
* `coin` - Asset symbol being traded
* `px` - Fill price as a decimal
* `sz` - Fill size (quantity)
* `side` - "A" (ask/sell) or "B" (bid/buy)
* `dir` - Trade direction: "Open Long", "Close Long", "Open Short", "Close Short"
* `fee` - Fee charged for the fill
* `oid` - Order ID that generated this fill


# Hyperliquid Portal API
Source: https://docs.sqd.dev/en/portal/hyperliquid/overview

Access Hyperliquid HyperCore data — fills, builder activity, trader positions — through the SQD Portal HTTP API for trading analytics and indexing.

Portal provides direct access to Hyperliquid data. Query trade fills with comprehensive filtering to analyze trading activity, PnL, builder routing, and market dynamics.

## Supported Networks

## Next Steps

<CardGroup>
  <Card title="Interactive Examples" icon="play" href="/en/portal/hyperliquid/examples/track-trader-fills">
    Try live queries with our interactive examples
  </Card>

  <Card title="API Reference" icon="book" href="/en/portal/hyperliquid/networks/hypercore-fills">
    Complete API reference with all fields and filters
  </Card>

  <Card title="OpenAPI Schema" icon="file-code" href="https://beta.docs.sqd.dev/files/hyperliquid-openapi.yaml">
    Download the full OpenAPI specification
  </Card>
</CardGroup>

<Info>
  **Enterprise Custom Development**

  For enterprise clients requiring custom indexer development, please contact our team to discuss your specific requirements. [Schedule a Consultation →](https://calendly.com/t-tyrie-subsquid/30min)
</Info>


# Agent Skills
Source: https://docs.sqd.dev/en/ai/agent-skills

Install pre-built Agent Skills for Claude Code, Cursor, and other AI coding agents — build SQD indexers and query onchain blockchain data faster.

Extend your AI coding agent with pre-built skills for SQD products. Skills follow the [Agent Skills](https://agentskills.io/) format and work with any compatible AI coding agent.

<Info>
  Agent Skills are experimental. The format and available skills may change.
</Info>

## Install all skills

```bash theme={"system"}
npx skills add subsquid-labs/skills --all
```

Or install skills individually:

<Tabs>
  <Tab title="Pipes SDK">
    ```bash theme={"system"}
    npx skills add subsquid-labs/skills/pipes-sdk
    ```
  </Tab>

  <Tab title="Portal">
    ```bash theme={"system"}
    npx skills add subsquid-labs/skills/portal
    ```
  </Tab>

  <Tab title="Squid SDK">
    ```bash theme={"system"}
    npx skills add subsquid-labs/skills/squid-sdk/squid-perf
    ```
  </Tab>
</Tabs>

Skills activate automatically once installed. The agent picks the right one based on your task.

## Pipes SDK skill

One skill covering the full lifecycle of blockchain indexers built with the [Pipes SDK](https://github.com/subsquid-labs/pipes-sdk), a lightweight TypeScript framework for EVM, Solana, and Hyperliquid chains.

| Skill       | Use case                                                                                                                                                                                                                  |
| ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `pipes-sdk` | Build, configure, deploy, and troubleshoot Pipes SDK indexers. Covers EVM, Solana, and Hyperliquid scaffolding via `@iankressin/pipes-cli`, runtime error diagnosis, sync performance tuning, and data quality validation |

**Example prompts:**

* "Create a new indexer for USDC transfers on Ethereum"
* "Create an indexer for Uniswap V3 swaps. Help me fetch the ABI and design the schema"
* "Track BTC/ETH/SOL perpetual futures fills on Hyperliquid"
* "My indexer crashed with 'Cannot read properties of undefined (reading 'topic')'"
* "My indexer is syncing slowly, help me optimize it"
* "Deploy my indexer to ClickHouse Cloud"

<Note>
  Pipes SDK requires Node.js LTS (v20 or v22). Avoid v25+ due to zstd decompression bugs that crash large Portal streams.
</Note>

## Portal skill

One skill for querying blockchain data across 210+ chains via [SQD Portal](https://portal.sqd.dev) without infrastructure.

| Skill    | Use case                                                                                                                                                                                                                                         |
| -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `portal` | Query blockchain data across all supported chains. EVM logs/transactions/traces, Solana instructions, Substrate events/calls/extrinsics, Hyperliquid fills, and Bitcoin transactions/inputs/outputs. Includes dataset discovery and verification |

**Example prompts:**

* "Query all USDC transfers on Base between blocks 10M–11M"
* "Find all contracts deployed by 0x123... on Ethereum"
* "Track Jupiter swap instructions on Solana"
* "Query recent Bitcoin transactions and analyze fee patterns"
* "Analyze BTC trading fills on Hyperliquid"
* "Index Polkadot balance transfer events"
* "What's the correct Portal dataset name for Arbitrum?"

## Squid SDK skill

One skill for profiling and comparing sync-time performance of [Squid SDK](https://github.com/subsquid/squid-sdk) deployments.

| Skill        | Use case                                                                                                                                                                                                                                                                                                                                               |
| ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `squid-perf` | Compare sync-time performance across one or more Squid SDK deployments. Fetches logs via the `sqd` CLI, parses per-service progress, and generates a self-contained HTML report plus a Markdown summary with wall-clock, active-time, and downtime breakdowns at log-spaced block breakpoints. Supports single-indexer mode for metrics-only profiling |

**Example prompts:**

* "Compare sync performance of my staging and production Squid deployments"
* "Profile the sync time of my Squid indexer and show me where it slows down"
* "/squid-perf squid-name\@v1 squid-name\@v2"
* "Why is my settlement-arbitrum service syncing slower than yesterday's deployment?"

## Related

* [Agent Skills Repository](https://github.com/subsquid-labs/skills). Browse and contribute on GitHub
* [AI Development Overview](/en/ai/ai-development). All AI resources for SQD
* [Portal MCP Server](/en/ai/mcp-server). Query blockchain data via MCP
* [Pipes SDK Quickstart](/en/sdk/pipes-sdk/evm/quickstart). Get started with Pipes SDK


# AI Development
Source: https://docs.sqd.dev/en/ai/ai-development

Build AI agents on SQD blockchain data via MCP servers, Agent Skills for coding assistants, and LLM-friendly docs across 225+ networks and datasets.

SQD exposes blockchain data from 225+ networks through interfaces designed for AI agents and LLMs: MCP servers for tool-based access, agent skills for coding assistants, and machine-readable documentation files.

## MCP Servers

Connect AI agents to SQD data through [Model Context Protocol](https://modelcontextprotocol.io). No API keys required.

<CardGroup>
  <Card title="Portal MCP Server" icon="database" href="/en/ai/mcp-server">
    29 tools for querying blocks, transactions, logs, and analytics across EVM, Solana, Bitcoin, and Hyperliquid chains.
  </Card>

  <Card title="Documentation MCP Server" icon="book" href="/en/ai/mcp-server-docs">
    Search and retrieve SQD documentation pages from within your AI client.
  </Card>
</CardGroup>

## Agent Skills

Pre-built [Agent Skills](https://agentskills.io/) for AI coding agents. Install with one command:

```bash theme={"system"}
npx skills add subsquid-labs/skills --all
```

<CardGroup>
  <Card title="Available Skills" icon="wand-magic-sparkles" href="/en/ai/agent-skills">
    3 skills covering Pipes SDK, Portal queries, and more
  </Card>

  <Card title="GitHub Repository" icon="github" href="https://github.com/subsquid-labs/skills">
    Source code and contribution guide
  </Card>
</CardGroup>

## OpenAPI Schemas

Machine-readable API specifications for each network type. Use these to give LLMs and code generators structured knowledge of the Portal API request/response format.

<CardGroup>
  <Card title="EVM" icon="file-code" href="/en/ai/evm-openapi">
    Logs, transactions, traces, state diffs
  </Card>

  <Card title="Solana" icon="file-code" href="/en/ai/solana-openapi">
    Instructions, token balances, rewards
  </Card>

  <Card title="Bitcoin" icon="file-code" href="/en/ai/bitcoin-openapi">
    Transactions, inputs, outputs
  </Card>

  <Card title="Substrate" icon="file-code" href="/en/ai/substrate-openapi">
    Events, calls, extrinsics
  </Card>

  <Card title="Hyperliquid Fills" icon="file-code" href="/en/ai/hyperliquid-openapi">
    Trade fills with PnL and fees
  </Card>

  <Card title="Hyperliquid Replica CMDs" icon="file-code" href="/en/ai/hyperliquid-replica-cmds-openapi">
    Orders, cancels, transfers
  </Card>
</CardGroup>

## LLM Documentation Files

Static files for feeding SQD documentation directly to LLMs:

| File                                                     | Description                                     |
| -------------------------------------------------------- | ----------------------------------------------- |
| [llms.txt](https://beta.docs.sqd.dev/llms.txt)           | Lightweight index of all documentation pages    |
| [llms-full.txt](https://beta.docs.sqd.dev/llms-full.txt) | Complete documentation content in a single file |
| [skill.md](/skill.md)                                    | AI skill definition for SQD                     |
| [Available Datasets](https://portal.sqd.dev/datasets)    | Browse all 225+ datasets                        |

## Related

* [Portal API](/en/portal/evm/overview). HTTP API for blockchain data
* [Pipes SDK](/en/sdk/pipes-sdk/evm/quickstart). TypeScript SDK for building indexers


# Available Datasets
Source: https://docs.sqd.dev/en/ai/available-datasets

Browse the live SQD Portal dataset catalog — 225+ blockchain datasets across EVM, Solana, Substrate, Bitcoin, and Hyperliquid with metadata and schemas.



# Bitcoin OpenAPI Schema
Source: https://docs.sqd.dev/en/ai/bitcoin-openapi

Bitcoin OpenAPI specification for the SQD Portal — endpoint definitions, request/response schemas, and code samples for querying blocks, txns, and UTXOs.



# EVM OpenAPI Schema
Source: https://docs.sqd.dev/en/ai/evm-openapi

EVM OpenAPI specification for the SQD Portal — endpoint definitions, request/response schemas, and code samples for querying logs, traces, and transactions.



# Hyperliquid OpenAPI Schema
Source: https://docs.sqd.dev/en/ai/hyperliquid-openapi

Hyperliquid Fills OpenAPI specification for the SQD Portal — endpoints, request/response schemas, and code samples for streaming HyperCore trade fills.



# Hyperliquid Replica CMDs OpenAPI Schema
Source: https://docs.sqd.dev/en/ai/hyperliquid-replica-cmds-openapi

Hyperliquid replica commands OpenAPI specification for the SQD Portal — endpoints and schemas for streaming HyperCore replica state, blocks, and actions.



# llms-full.txt
Source: https://docs.sqd.dev/en/ai/llms-full-txt

Full SQD documentation as a single llms-full.txt file — every page concatenated for direct ingestion into LLM context windows and AI coding tools.



# llms.txt
Source: https://docs.sqd.dev/en/ai/llms-txt

SQD documentation index in llms.txt format — a machine-readable list of all pages so LLMs and AI coding agents can discover and fetch relevant docs on demand.



# Portal MCP Server
Source: https://docs.sqd.dev/en/ai/mcp-server

Query blockchain data from AI agents via the Portal MCP server — 29 tools for blocks, txns, logs, and analytics across EVM, Solana, Bitcoin, and Hyperliquid.

<Info>
  The Portal MCP server is experimental. Available tools and behavior may change.
</Info>

Connect the Portal MCP server to your favorite AI coding clients.

MCP (Model Context Protocol) is an open-source standard for connecting AI applications to external systems. The Portal MCP server gives your agent direct access to blockchain data across 225+ datasets. Discover datasets, query transactions and logs, analyze wallets, and compare chains.

<video>
  <source type="video/mp4" />
</video>

## Connect Portal MCP

* MCP endpoint: `https://portal.sqd.dev/mcp`
* Authentication: Free, no API keys required

<Tabs>
  <Tab title="Claude Code">
    ```bash theme={"system"}
    claude mcp add --transport http sqd-portal https://portal.sqd.dev/mcp
    ```

    Test the connection:

    ```bash theme={"system"}
    claude mcp list
    ```

    See the [Claude Code documentation](https://docs.anthropic.com/en/docs/claude-code/mcp#installing-mcp-servers) for more details.
  </Tab>

  <Tab title="Claude">
    <Info>
      Custom connectors are available on all Claude plans. Free users are limited to one custom connector. For Team and Enterprise plans, only Owners can add connectors.
    </Info>

    <Steps>
      <Step title="Add the Portal MCP server to Claude">
        1. Navigate to the [Connectors](https://claude.ai/settings/connectors) page in Claude settings.
        2. Select **Add custom connector**.
        3. Add the Portal MCP server:
           * Name: `Portal`
           * URL: `https://portal.sqd.dev/mcp`
        4. Select **Add**.
      </Step>

      <Step title="Access the MCP server in your chat">
        1. When using Claude, select the attachments button (the plus icon).
        2. Select the Portal MCP server.
        3. Ask Claude questions about blockchain data from Portal API.
      </Step>
    </Steps>

    See the [Model Context Protocol documentation](https://modelcontextprotocol.io/docs/tutorials/use-remote-mcp-server) for more details.
  </Tab>

  <Tab title="Cursor">
    <Steps>
      <Step title="Open MCP settings">
        1. Use <kbd>Command</kbd> + <kbd>Shift</kbd> + <kbd>P</kbd> (<kbd>Ctrl</kbd> + <kbd>Shift</kbd> + <kbd>P</kbd> on Windows) to open the command palette.
        2. Search for "Open MCP settings".
        3. Select **Add custom MCP**. This opens the `mcp.json` file.
      </Step>

      <Step title="Configure the Portal MCP server">
        In `mcp.json`, add:

        ```json theme={"system"}
        {
          "mcpServers": {
            "sqd-portal": {
              "url": "https://portal.sqd.dev/mcp"
            }
          }
        }
        ```
      </Step>

      <Step title="Test the connection">
        In Cursor's chat, ask "What tools do you have available?" Cursor should show the Portal MCP server as an available tool.
      </Step>
    </Steps>

    See the [Cursor documentation](https://docs.cursor.com/en/context/mcp#installing-mcp-servers) for more details.
  </Tab>

  <Tab title="VS Code">
    Create a `.vscode/mcp.json` file and add:

    ```json theme={"system"}
    {
      "servers": {
        "sqd-portal": {
          "type": "http",
          "url": "https://portal.sqd.dev/mcp"
        }
      }
    }
    ```

    See the [VS Code documentation](https://code.visualstudio.com/docs/copilot/chat/mcp-servers) for more details.
  </Tab>
</Tabs>

## Available tools

23 core tools across EVM, Solana, Bitcoin, Substrate, and Hyperliquid, plus experimental OHLC and debug helpers. All queryable in natural language. Every tool follows the same description style, input schema, response envelope, and routing metadata, and every response includes structured follow-up hints and optional interactive panels for MCP-aware clients.

### Discovery & shared

| Tool                         | Description                                                                                                                                      |
| ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ |
| `portal_list_networks`       | Find the right network or alias across EVM, Solana, Bitcoin, Substrate, and Hyperliquid. First stop when you're not sure which dataset to query. |
| `portal_get_network_info`    | Check if a network is indexed, fresh, caught up, or behind, and see which tables are available.                                                  |
| `portal_get_head`            | Get just the latest or finalized head block or slot for a network.                                                                               |
| `portal_get_recent_activity` | Cross-VM recent-activity feed with chronological paging. Simplest answer to "what's been happening on X lately?".                                |
| `portal_get_wallet_summary`  | One-call wallet analysis across supported VMs. Overview, activity, and assets.                                                                   |
| `portal_get_time_series`     | Chart-ready metric buckets over time with optional compare-to-previous and EVM contract grouping.                                                |

### EVM

| Tool                               | Description                                                                                                       |
| ---------------------------------- | ----------------------------------------------------------------------------------------------------------------- |
| `portal_evm_query_transactions`    | Query raw EVM transactions with optional logs, traces, and state-diff context via include flags.                  |
| `portal_evm_query_logs`            | Query raw EVM logs with address/topic filters and optional inline decoding.                                       |
| `portal_evm_query_token_transfers` | Token-transfer activity on EVM without needing to remember the Transfer event signature.                          |
| `portal_evm_get_analytics`         | Network-wide EVM snapshot with ranked top contracts and compact overview metrics.                                 |
| `portal_evm_get_contract_activity` | Contract-centric summary: recent interactions, unique callers, and optional event activity.                       |
| `portal_evm_get_ohlc`              | **Experimental.** DEX OHLC candles plus a recent-trade tape from Uniswap v2/v3/v4 and Aerodrome Slipstream swaps. |

### Solana

| Tool                               | Description                                                                                                   |
| ---------------------------------- | ------------------------------------------------------------------------------------------------------------- |
| `portal_solana_query_transactions` | Query raw Solana transactions with optional balances, token balances, rewards, logs, and instruction context. |
| `portal_solana_query_instructions` | Query raw instructions with program, account, and Anchor-discriminator filters.                               |
| `portal_solana_get_analytics`      | Solana throughput, fee, wallet activity, and optional top-program snapshot.                                   |

### Bitcoin

| Tool                                | Description                                                                          |
| ----------------------------------- | ------------------------------------------------------------------------------------ |
| `portal_bitcoin_query_transactions` | Query raw Bitcoin transactions with optional inline inputs and outputs.              |
| `portal_bitcoin_get_analytics`      | Bitcoin block cadence, fees, SegWit/Taproot adoption, and address activity snapshot. |

### Hyperliquid

| Tool                               | Description                                     |
| ---------------------------------- | ----------------------------------------------- |
| `portal_hyperliquid_query_fills`   | Query trade fills with PnL, fees, and routing.  |
| `portal_hyperliquid_get_analytics` | Trading volume, top coins, and trader rankings. |
| `portal_hyperliquid_get_ohlc`      | OHLC candles for Hyperliquid markets.           |

### Substrate

| Tool                             | Description                                                                                         |
| -------------------------------- | --------------------------------------------------------------------------------------------------- |
| `portal_substrate_query_calls`   | Query Polkadot-, Kusama-, and other Substrate-style extrinsic calls with pallet and method filters. |
| `portal_substrate_query_events`  | Query Substrate events emitted by runtime pallets.                                                  |
| `portal_substrate_get_analytics` | Substrate network-wide snapshot: block cadence, extrinsic volume, and top pallets.                  |

### Debug

| Tool                                              | Description                                                                                      |
| ------------------------------------------------- | ------------------------------------------------------------------------------------------------ |
| `portal_debug_query_blocks`                       | Inspect raw blocks across VMs for schema debugging.                                              |
| `portal_debug_resolve_time_to_block`              | Resolve a timestamp to a block or slot for any supported network.                                |
| `portal_debug_hyperliquid_query_replica_commands` | Low-level access to Hyperliquid replica commands (orders, cancels, transfers, leverage updates). |

## Known limitations

The MCP server may struggle with memory issues when handling large volumes of data, especially on high-TPS chains like Solana and Hyperliquid. v0.7.5 introduced chunked fetching and lower default limits to mitigate this, but large unfiltered queries can still cause timeouts or incomplete results. Always use filters and keep block ranges tight on dense chains.

## Best practices

1. **Use timeframes for any query**. All query tools support the `timeframe` parameter (e.g., `"24h"`, `"7d"`). No need to calculate block ranges manually
2. **Try specialized tools first**. Use high-level tools like `portal_get_wallet_summary` or `portal_get_recent_transactions` before lower-level query tools
3. **Always filter queries**. Use addresses, topics, or other filters to keep responses under the 50KB cap
4. **Leverage analytics tools**. Use per-chain analytics and time series tools to get summaries without fetching raw data
5. **Mind block ranges**. Keep queries under 10K blocks for logs, 5K for transactions, and 1K for traces for best performance. On high-TPS chains, use even smaller ranges

## Related

* [AI Development Overview](/en/ai/ai-development). Resources for building AI agents with SQD
* [Portal API Reference](/en/portal/evm/overview). HTTP API for blockchain data
* [Documentation MCP Server](/en/ai/mcp-server-docs). Search SQD documentation
* [Pipes SDK](/en/sdk/pipes-sdk/evm/quickstart). TypeScript SDK for custom pipelines
* [llms.txt](https://beta.docs.sqd.dev/llms.txt). Documentation index for LLMs


# Documentation MCP Server
Source: https://docs.sqd.dev/en/ai/mcp-server-docs

Connect Claude, Cursor, and other AI clients to SQD documentation via MCP — search and retrieve any docs page directly from your AI assistant in real time.

## Access the SQD MCP server

**Server URL**: `https://beta.docs.sqd.dev/mcp`

## Use the SQD MCP server

Connect the SQD MCP server to your preferred AI tools.

<Tabs>
  <Tab title="Claude Code">
    ```bash theme={"system"}
    claude mcp add --transport http SQD https://beta.docs.sqd.dev/mcp
    ```

    Test the connection:

    ```bash theme={"system"}
    claude mcp list
    ```

    See the [Claude Code documentation](https://docs.anthropic.com/en/docs/claude-code/mcp#installing-mcp-servers) for more details.
  </Tab>

  <Tab title="Claude">
    <Steps>
      <Step title="Add the SQD MCP server to Claude">
        1. Navigate to the [Connectors](https://claude.ai/settings/connectors) page in Claude settings.
        2. Select **Add custom connector**.
        3. Add the SQD MCP server:
           * Name: `SQD`
           * URL: `https://beta.docs.sqd.dev/mcp`
        4. Select **Add**.
      </Step>

      <Step title="Access the MCP server in your chat">
        1. When using Claude, select the attachments button (the plus icon).
        2. Select the SQD MCP server.
        3. Ask Claude questions about SQD documentation.
      </Step>
    </Steps>

    See the [Model Context Protocol documentation](https://modelcontextprotocol.io/docs/tutorials/use-remote-mcp-server) for more details.
  </Tab>

  <Tab title="Cursor">
    <Steps>
      <Step title="Open MCP settings">
        1. Use <kbd>Command</kbd> + <kbd>Shift</kbd> + <kbd>P</kbd> (<kbd>Ctrl</kbd> + <kbd>Shift</kbd> + <kbd>P</kbd> on Windows) to open the command palette.
        2. Search for "Open MCP settings".
        3. Select **Add custom MCP**. This opens the `mcp.json` file.
      </Step>

      <Step title="Configure the SQD MCP server">
        In `mcp.json`, add:

        ```json theme={"system"}
        {
          "mcpServers": {
            "SQD": {
              "url": "https://beta.docs.sqd.dev/mcp"
            }
          }
        }
        ```
      </Step>

      <Step title="Test the connection">
        In Cursor's chat, ask "What tools do you have available?" Cursor should show the SQD MCP server as an available tool.
      </Step>
    </Steps>

    See the [Cursor documentation](https://docs.cursor.com/en/context/mcp#installing-mcp-servers) for more details.
  </Tab>

  <Tab title="VS Code">
    Create a `.vscode/mcp.json` file and add:

    ```json theme={"system"}
    {
      "servers": {
        "SQD": {
          "type": "http",
          "url": "https://beta.docs.sqd.dev/mcp"
        }
      }
    }
    ```

    See the [VS Code documentation](https://code.visualstudio.com/docs/copilot/chat/mcp-servers) for more details.
  </Tab>
</Tabs>

## Available tools

### `search_sqd_documentation`

Semantic search across SQD documentation. Returns contextual content with page titles and direct links. Best for broad or conceptual questions.

**Parameters:**

* `query` (string, required). Search query

**Example queries:**

* "How do I query EVM logs with Portal API?"
* "Show me Solana indexing examples"
* "What are the supported networks?"

### `query_docs_filesystem_sqd_documentation`

Run read-only shell commands against a virtual filesystem containing all SQD documentation pages and OpenAPI specs. Use this when you need exact keyword matching, regex search, structural exploration, or full page content.

**Parameters:**

* `command` (string, required). A shell command (`rg`, `grep`, `cat`, `head`, `tree`, `ls`, `find`, `jq`, etc.)

**Example commands:**

* `tree / -L 2`. See the top-level directory layout
* `rg -il "batch processing" /`. Find all files mentioning "batch processing"
* `rg -C 3 "addLog" /en/sdk/`. Show matches with context
* `head -80 /en/portal/evm/quickstart.mdx`. Read a specific page
* `cat /en/api/catalog/evm/openapi.yaml | head -50`. Inspect OpenAPI specs

<Tip>
  Start with `search_sqd_documentation` for broad questions, then use `query_docs_filesystem_sqd_documentation` to read specific pages or grep for exact patterns.
</Tip>

## Related

* [Portal API MCP Server](/en/ai/mcp-server) - Query blockchain data through Portal API
* [Portal API Reference](/en/portal/evm/overview) - HTTP API for blockchain data
* [Pipes SDK](/en/sdk/pipes-sdk/evm/quickstart) - TypeScript SDK for custom pipelines
* [llms.txt](https://beta.docs.sqd.dev/llms.txt) - Documentation index for LLMs


# skill.md
Source: https://docs.sqd.dev/en/ai/skill-md

Install the SQD skill.md file in Claude Code or Cursor — context for blockchain indexing, Portal API queries, and SDK patterns when generating code.



# Solana OpenAPI Schema
Source: https://docs.sqd.dev/en/ai/solana-openapi

Solana OpenAPI specification for the SQD Portal — endpoints, request/response schemas, and code samples for querying instructions, transactions, and balances.



# Substrate OpenAPI Schema
Source: https://docs.sqd.dev/en/ai/substrate-openapi

Substrate OpenAPI specification for the SQD Portal — endpoints, schemas, and code samples for streaming finalized blocks, extrinsics, events, and calls.



# Fuel Mainnet Overview
Source: https://docs.sqd.dev/en/data/fuel/fuel-mainnet

Fuel Mainnet data overview — open private network ID, gateway URL, available data fields, and rate limits for SQD-powered Fuel indexing.

<Info>
  Starting today (2025-02-23) all open private network gateways are rate limited
  at 50 requests per 10 seconds per IP. Higher bandwidths will soon become
  available via the public network.
</Info>

## Network Information

* **Network ID**: `fuel-mainnet`
* **Gateway**: Open Private Network

## Endpoints

### Gateway Endpoint

```
https://v2.archive.subsquid.io/network/fuel-mainnet
```

### Usage

```typescript theme={"system"}
import { FuelDataSource } from '@subsquid/fuel-stream'

const dataSource = new FuelDataSource()
  .setGateway("https://v2.archive.subsquid.io/network/fuel-mainnet")
  .setFields({
    block: { height: true },
    transaction: { id: true },
    receipt: { receiptType: true, data: true }
  });
```

## Quick Start

Get started indexing Fuel Mainnet data:

<Card title="Quick Start" icon="terminal" href="/en/sdk/squid-sdk/fuel-indexing/indexing-receipts">
  Quick reference for building your indexer
</Card>

## Related Resources

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/sdk/squid-sdk/fuel-indexing/network-api/fuel-api">
    Complete API documentation
  </Card>

  <Card title="Network Overview" icon="network-wired" href="/en/network/overview">
    Learn about the Subsquid Network
  </Card>
</CardGroup>


# Fuel Testnet Overview
Source: https://docs.sqd.dev/en/data/fuel/fuel-testnet

Fuel Testnet data overview — open private network ID, gateway URL, available data fields, and rate limits for SQD-powered Fuel testnet indexing.

<Info>
  Starting today (2025-02-23) all open private network gateways are rate limited
  at 50 requests per 10 seconds per IP. Higher bandwidths will soon become
  available via the public network.
</Info>

## Network Information

* **Network ID**: `fuel-testnet`
* **Gateway**: Open Private Network

## Endpoints

### Gateway Endpoint

```
https://v2.archive.subsquid.io/network/fuel-testnet
```

### Usage

```typescript theme={"system"}
import { FuelDataSource } from '@subsquid/fuel-stream'

const dataSource = new FuelDataSource()
  .setGateway("https://v2.archive.subsquid.io/network/fuel-testnet")
  .setFields({
    block: { height: true },
    transaction: { id: true },
    receipt: { receiptType: true, data: true }
  });
```

## Quick Start

Get started indexing Fuel Testnet data:

<Card title="Quick Start" icon="terminal" href="/en/sdk/squid-sdk/fuel-indexing/indexing-receipts">
  Quick reference for building your indexer
</Card>

## Related Resources

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/sdk/squid-sdk/fuel-indexing/network-api/fuel-api">
    Complete API documentation
  </Card>

  <Card title="Network Overview" icon="network-wired" href="/en/network/overview">
    Learn about the Subsquid Network
  </Card>
</CardGroup>


# Other Networks
Source: https://docs.sqd.dev/en/data/other

Browse Fuel, Tron, and Starknet networks supported by SQD for blockchain data indexing — gateway endpoints, schemas, and access notes for each chain.

<Warning>
  **Portal Migration**: We're migrating from v2 archives to Portal. V2 archives will be sunset soon. [Get Started with Portal →](/en/portal/migration)
</Warning>


# Starknet Mainnet Overview
Source: https://docs.sqd.dev/en/data/starknet/starknet-mainnet

Starknet Mainnet data overview — open private network ID, gateway URL, available data fields, and rate limits for SQD-powered Starknet indexing.

<Info>
  Starting today (2025-02-23) all open private network gateways are rate limited
  at 50 requests per 10 seconds per IP. Higher bandwidths will soon become
  available via the public network.
</Info>

## Network Information

* **Network ID**: `starknet-mainnet`
* **Gateway**: Open Private Network

## Endpoints

### Gateway Endpoint

```
https://v2.archive.subsquid.io/network/starknet-mainnet
```

### Usage

```typescript theme={"system"}
import { StarknetDataSource } from '@subsquid/starknet-stream'

const dataSource = new StarknetDataSource()
  .setGateway("https://v2.archive.subsquid.io/network/starknet-mainnet")
  .setBlockRange({ from: 0 });
```

## Quick Start

Get started indexing Starknet Mainnet data:

<Card title="Quick Start" icon="terminal" href="/en/sdk/squid-sdk/starknet-indexing/network-api/starknet-api">
  Quick reference for building your indexer
</Card>

## Related Resources

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/reference/starknet-api">
    Complete API documentation
  </Card>

  <Card title="Network Overview" icon="network-wired" href="/en/network/overview">
    Learn about the Subsquid Network
  </Card>
</CardGroup>


# Starknet Sepolia Overview
Source: https://docs.sqd.dev/en/data/starknet/starknet-sepolia

Starknet Sepolia testnet data overview — network ID, gateway URL, available data fields, and rate limits for SQD-powered Starknet testnet indexing.

<Info>
  Starting today (2025-02-23) all open private network gateways are rate limited
  at 50 requests per 10 seconds per IP. Higher bandwidths will soon become
  available via the public network.
</Info>

## Network Information

* **Network ID**: `starknet-sepolia`
* **Gateway**: Open Private Network

## Endpoints

### Gateway Endpoint

```
https://v2.archive.subsquid.io/network/starknet-sepolia
```

### Usage

```typescript theme={"system"}
import { StarknetDataSource } from '@subsquid/starknet-stream'

const dataSource = new StarknetDataSource()
  .setGateway("https://v2.archive.subsquid.io/network/starknet-sepolia")
  .setBlockRange({ from: 0 });
```

## Quick Start

Get started indexing Starknet Sepolia data:

<Card title="Quick Start" icon="terminal" href="/en/sdk/squid-sdk/starknet-indexing/network-api/starknet-api">
  Quick reference for building your indexer
</Card>

## Related Resources

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/reference/starknet-api">
    Complete API documentation
  </Card>

  <Card title="Network Overview" icon="network-wired" href="/en/network/overview">
    Learn about the Subsquid Network
  </Card>
</CardGroup>


# Westend Asset Hub
Source: https://docs.sqd.dev/en/data/substrate/asset-hub-westend

Index Westend Asset Hub Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value               |
| ---------- | ------------------- |
| Network ID | `asset-hub-westend` |
| Real-time  | No                  |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/asset-hub-westend
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/asset-hub-westend
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/asset-hub-westend/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Westend Bridge Hub
Source: https://docs.sqd.dev/en/data/substrate/bridge-hub-westend

Index Westend Bridge Hub Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value                |
| ---------- | -------------------- |
| Network ID | `bridge-hub-westend` |
| Real-time  | No                   |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/bridge-hub-westend
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/bridge-hub-westend
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/bridge-hub-westend/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Westend Collectives
Source: https://docs.sqd.dev/en/data/substrate/collectives-westend

Index Westend Collectives Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value                 |
| ---------- | --------------------- |
| Network ID | `collectives-westend` |
| Real-time  | No                    |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/collectives-westend
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/collectives-westend
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/collectives-westend/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Subspace Gemini 3h
Source: https://docs.sqd.dev/en/data/substrate/gemini-3h

Index Subspace Gemini 3h Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value       |
| ---------- | ----------- |
| Network ID | `gemini-3h` |
| Real-time  | No          |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/gemini-3h
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/gemini-3h
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/gemini-3h/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Rolimec
Source: https://docs.sqd.dev/en/data/substrate/rolimec

Index Rolimec Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value     |
| ---------- | --------- |
| Network ID | `rolimec` |
| Real-time  | No        |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/rolimec
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/rolimec
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/rolimec/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Shibuya (Substrate)
Source: https://docs.sqd.dev/en/data/substrate/shibuya-substrate

Index Shibuya (Substrate) Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value               |
| ---------- | ------------------- |
| Network ID | `shibuya-substrate` |
| Real-time  | No                  |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/shibuya-substrate
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/shibuya-substrate
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/shibuya-substrate/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Shiden (Substrate)
Source: https://docs.sqd.dev/en/data/substrate/shiden-substrate

Index Shiden (Substrate) Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value              |
| ---------- | ------------------ |
| Network ID | `shiden-substrate` |
| Real-time  | No                 |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/shiden-substrate
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/shiden-substrate
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/shiden-substrate/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# SORA
Source: https://docs.sqd.dev/en/data/substrate/sora-mainnet

Index SORA Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value          |
| ---------- | -------------- |
| Network ID | `sora-mainnet` |
| Real-time  | No             |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/sora-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/sora-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/sora-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Subsocial
Source: https://docs.sqd.dev/en/data/substrate/subsocial-parachain

Index Subsocial Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value                 |
| ---------- | --------------------- |
| Network ID | `subsocial-parachain` |
| Real-time  | No                    |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/subsocial-parachain
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/subsocial-parachain
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/subsocial-parachain/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Ternoa
Source: https://docs.sqd.dev/en/data/substrate/ternoa

Index Ternoa Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value    |
| ---------- | -------- |
| Network ID | `ternoa` |
| Real-time  | No       |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/ternoa
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/ternoa
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/ternoa/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Turing Avail
Source: https://docs.sqd.dev/en/data/substrate/turing-avail

Index Turing Avail Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value          |
| ---------- | -------------- |
| Network ID | `turing-avail` |
| Real-time  | No             |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/turing-avail
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/turing-avail
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/turing-avail/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Turing
Source: https://docs.sqd.dev/en/data/substrate/turing-mainnet

Index Turing Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value            |
| ---------- | ---------------- |
| Network ID | `turing-mainnet` |
| Real-time  | No               |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/turing-mainnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/turing-mainnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/turing-mainnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Vara
Source: https://docs.sqd.dev/en/data/substrate/vara

Index Vara Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value  |
| ---------- | ------ |
| Network ID | `vara` |
| Real-time  | No     |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/vara
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/vara
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/vara/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Vara Testnet
Source: https://docs.sqd.dev/en/data/substrate/vara-testnet

Index Vara Testnet Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value          |
| ---------- | -------------- |
| Network ID | `vara-testnet` |
| Real-time  | No             |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/vara-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/vara-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/vara-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Westend
Source: https://docs.sqd.dev/en/data/substrate/westend

Index Westend Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value     |
| ---------- | --------- |
| Network ID | `westend` |
| Real-time  | No        |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/westend
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/westend
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/westend/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Zeitgeist
Source: https://docs.sqd.dev/en/data/substrate/zeitgeist

Index Zeitgeist Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value       |
| ---------- | ----------- |
| Network ID | `zeitgeist` |
| Real-time  | No          |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/zeitgeist
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/zeitgeist
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/zeitgeist/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Zeitgeist Testnet
Source: https://docs.sqd.dev/en/data/substrate/zeitgeist-testnet

Index Zeitgeist Testnet Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value               |
| ---------- | ------------------- |
| Network ID | `zeitgeist-testnet` |
| Real-time  | No                  |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/zeitgeist-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/zeitgeist-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/zeitgeist-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# zkVerify Testnet
Source: https://docs.sqd.dev/en/data/substrate/zkverify-testnet

Index zkVerify Testnet Substrate blockchain data — blocks, extrinsics, events, and calls — via SQD Portal API for finalized historical streaming.

| Property   | Value              |
| ---------- | ------------------ |
| Network ID | `zkverify-testnet` |
| Real-time  | No                 |

## Endpoint

<Tabs>
  <Tab title="Portal (recommended)">
    ```
    https://portal.sqd.dev/datasets/zkverify-testnet
    ```
  </Tab>

  <Tab title="v2 archives (legacy)">
    <Warning>
      Will be sunset soon. [Migrate to Portal →](/en/portal/migration)
    </Warning>

    ```
    https://v2.archive.subsquid.io/network/zkverify-testnet
    ```
  </Tab>
</Tabs>

## Quick start

```bash theme={"system"}
curl --compressed -X POST \
  'https://portal.sqd.dev/datasets/zkverify-testnet/stream' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "substrate",
    "fromBlock": 0,
    "toBlock": 100,
    "fields": {
      "block": { "number": true, "timestamp": true },
      "event": { "name": true, "args": true }
    },
    "events": [{}]
  }'
```

For full SDK examples:

<CardGroup>
  <Card title="Pipes SDK Quickstart" icon="bars-staggered" href="/en/sdk/pipes-sdk/substrate/quickstart">
    Streaming pipeline indexer
  </Card>

  <Card title="Squid SDK Quickstart" icon="server" href="/en/sdk/squid-sdk/quickstart">
    GraphQL API indexer
  </Card>
</CardGroup>

## Schema

<AccordionGroup>
  <Accordion title="Blocks">
    | Field             | Type    | Description                   |
    | ----------------- | ------- | ----------------------------- |
    | `number`          | integer | Block number                  |
    | `hash`            | string  | Block hash                    |
    | `parent_hash`     | string  | Parent block hash             |
    | `timestamp`       | integer | Block timestamp               |
    | `state_root`      | string  | State root hash               |
    | `extrinsics_root` | string  | Extrinsics root hash          |
    | `spec_version`    | integer | Runtime specification version |
  </Accordion>

  <Accordion title="Extrinsics">
    | Field       | Type    | Description                 |
    | ----------- | ------- | --------------------------- |
    | `index`     | integer | Extrinsic index in block    |
    | `hash`      | string  | Extrinsic hash              |
    | `version`   | integer | Extrinsic version           |
    | `signature` | object  | Signature information       |
    | `success`   | boolean | Whether extrinsic succeeded |
    | `error`     | object  | Error information if failed |
    | `call`      | object  | Call data                   |
  </Accordion>

  <Accordion title="Events">
    | Field             | Type    | Description                                  |
    | ----------------- | ------- | -------------------------------------------- |
    | `index`           | integer | Event index                                  |
    | `phase`           | string  | ApplyExtrinsic, Finalization, Initialization |
    | `extrinsic_index` | integer | Associated extrinsic index                   |
    | `name`            | string  | Event name                                   |
    | `args`            | object  | Event arguments                              |
    | `topics`          | array   | Event topics                                 |
  </Accordion>

  <Accordion title="Calls">
    | Field     | Type    | Description                 |
    | --------- | ------- | --------------------------- |
    | `name`    | string  | Call name (pallet.method)   |
    | `args`    | object  | Call arguments              |
    | `origin`  | object  | Call origin                 |
    | `success` | boolean | Whether call succeeded      |
    | `error`   | object  | Error information if failed |
  </Accordion>
</AccordionGroup>

## Reference

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/api/substrate/finalized-stream">
    Field definitions and query syntax
  </Card>

  <Card title="Portal Migration" icon="arrow-right" href="/en/portal/migration">
    Migrate from v2 archives to Portal
  </Card>
</CardGroup>


# Tron Mainnet Overview
Source: https://docs.sqd.dev/en/data/tron/tron-mainnet

Tron Mainnet data overview — open private network ID, gateway URL, available data fields, and rate limits for SQD-powered Tron blockchain indexing.

<Info>
  Starting today (2025-02-23) all open private network gateways are rate limited
  at 50 requests per 10 seconds per IP. Higher bandwidths will soon become
  available via the public network.
</Info>

## Network Information

* **Network ID**: `tron-mainnet`
* **Gateway**: Open Private Network

## Endpoints

### Gateway Endpoint

```
https://v2.archive.subsquid.io/network/tron-mainnet
```

### Usage

```typescript theme={"system"}
import { TronBatchProcessor } from '@subsquid/tron-processor'

const processor = new TronBatchProcessor()
  .setGateway("https://v2.archive.subsquid.io/network/tron-mainnet")
  .setBlockRange({ from: 0 });
```

## Quick Start

Get started indexing Tron Mainnet data:

<Card title="Quick Start" icon="terminal" href="/en/sdk/squid-sdk/tron-indexing/indexing-usdt-transfers">
  Quick reference for building your indexer
</Card>

## Related Resources

<CardGroup>
  <Card title="API Reference" icon="code" href="/en/sdk/squid-sdk/tron-indexing/network-api/tron-api">
    Complete API documentation
  </Card>

  <Card title="Network Overview" icon="network-wired" href="/en/network/overview">
    Learn about the Subsquid Network
  </Card>
</CardGroup>


# Getting SQD Tokens
Source: https://docs.sqd.dev/en/network/procuring-sqd

Buy SQD tokens to delegate, stake, or pay for SQD Network resources — verified token contracts on Ethereum and Arbitrum, plus DEX and CEX listings.

Participating in SQD Network requires SQD tokens. This guide shows you all the ways to acquire them.

## Before You Start

<Warning>
  Always verify you're using the correct SQD token contract addresses to avoid
  scams.
</Warning>

### Official Token Contracts

<Tabs>
  <Tab title="Arbitrum">
    **Contract Address:** `0x1337420dED5ADb9980CFc35f8f2B054ea86f8aB1`

    [View on Arbiscan](https://arbiscan.io/address/0x1337420dED5ADb9980CFc35f8f2B054ea86f8aB1)
  </Tab>

  <Tab title="Base">
    **Contract Address:** `0xd4554BEa546EFa83C1E6B389ecac40EA999B3E78`

    [View on Basescan](https://basescan.org/address/0xd4554bea546efa83c1e6b389ecac40ea999b3e78)
  </Tab>

  <Tab title="Ethereum">
    **Contract Address:** `0x1337420dED5ADb9980CFc35f8f2B054ea86f8aB1`

    [View on Etherscan](https://etherscan.io/token/0x1337420ded5adb9980cfc35f8f2b054ea86f8ab1)
  </Tab>
</Tabs>

<Info>
  To interact with SQD Network contracts, your tokens must be on **Arbitrum
  One**.
</Info>

### Supported Wallets

Use the [SQD Network app](https://network.subsquid.io) to interact with contracts. Connect with any of these wallets:

<CardGroup>
  <Card title="MetaMask" icon="wallet" href="https://metamask.io">
    Most popular Ethereum wallet
  </Card>

  <Card title="Rainbow" icon="palette" href="https://rainbow.me">
    User-friendly mobile wallet
  </Card>

  <Card title="Coinbase Wallet" icon="bitcoin" href="https://www.coinbase.com/wallet">
    Secure self-custody wallet
  </Card>

  <Card title="WalletConnect" icon="link" href="https://walletconnect.network">
    Connect any compatible wallet
  </Card>
</CardGroup>

## Purchase SQD Tokens

<Tabs>
  <Tab title="DEXes (Recommended)">
    ### Buy on Arbitrum

    <Card title="PancakeSwap (Best Rates)" icon="arrow-right" href="https://pancakeswap.finance/?chain=arb&outputCurrency=0x1337420dED5ADb9980CFc35f8f2B054ea86f8aB1">
      Buy SQD directly on Arbitrum One
    </Card>

    <Tip>
      The WETH/SQD pool on PancakeSwap Arbitrum is the best-funded SQD pool across
      all DEXes, typically offering the best rates and lowest slippage.
    </Tip>

    ### Buy on Base

    <Card title="Aerodrome Finance" icon="arrow-right" href="https://aerodrome.finance/swap?from=0x833589fcd6edb6e08f4c7c32d4f71b54bda02913&to=0xd4554bea546efa83c1e6b389ecac40ea999b3e78&chain0=8453&chain1=8453">
      Buy SQD on Base network
    </Card>

    <Warning>
      If you purchase on Base, you'll need to bridge your tokens to Arbitrum One to
      use them with SQD Network.
    </Warning>

    <AccordionGroup>
      <Accordion title="Bridge from Base to Arbitrum">
        <Steps>
          <Step title="Choose your bridge option">
            Select between fast bridges or native bridges based on your priorities:

            * **Fast bridges:** Quick transfers (minutes to hours)
            * **Native bridges:** Lower fees but longer wait times (up to 7 days)
          </Step>

          <Step title="Fast bridge option">
            Use [IceCreamSwap bridge](https://icecreamswap.com/bridge?chain=base) for quick transfers.

            <Check>
              Your tokens will arrive within minutes to a few hours.
            </Check>
          </Step>

          <Step title="Native bridge option (Lower fees)">
            For minimal fees, use the native bridges:

            1. Bridge from [Base → Ethereum](https://superbridge.app/base)
            2. Then bridge from [Ethereum → Arbitrum](https://bridge.arbitrum.io/?destinationChain=arbitrum-one\&sourceChain=ethereum\&token=0x1337420ded5adb9980cfc35f8f2b054ea86f8ab1)

            <Note>
              Native bridges require up to 7 days for transfers but offer the lowest fees.
            </Note>
          </Step>
        </Steps>
      </Accordion>
    </AccordionGroup>
  </Tab>

  <Tab title="Centralized Exchanges">
    ### Major Exchanges

    Purchase SQD on these centralized exchanges and withdraw to your wallet:

    <CardGroup>
      <Card title="Bybit" icon="building-columns" href="https://www.bybit.com/en-US/trade/spot/SQD/USDT">
        Trade SQD/USDT
      </Card>

      <Card title="MEXC" icon="building-columns" href="https://www.mexc.com/exchange/SQD_USDT">
        Trade SQD/USDT
      </Card>

      <Card title="KuCoin" icon="building-columns" href="https://www.kucoin.com/trade/SQD-USDT">
        Trade SQD/USDT
      </Card>

      <Card title="Gate.io" icon="building-columns" href="https://www.gate.io/trade/SQD_USDT">
        Trade SQD/USDT
      </Card>

      <Card title="CoinEx" icon="building-columns" href="https://www.coinex.com/en/exchange/sqd-usdt">
        Trade SQD/USDT
      </Card>
    </CardGroup>

    <Info>
      View the complete list of exchanges supporting SQD on our [CoinMarketCap
      page](https://coinmarketcap.com/currencies/subsquid/).
    </Info>

    <Warning>
      After purchasing on a centralized exchange, make sure to withdraw your tokens
      to Arbitrum One network to interact with SQD Network contracts.
    </Warning>
  </Tab>
</Tabs>


# Tokenomics
Source: https://docs.sqd.dev/en/network/tokenomics

SQD token utility, distribution, and economic model — staking, delegation, and governance roles for the SQD Network ecosystem and worker rewards.

SQD is the ERC-20 protocol token that is native to the SQD Network ecosystem. The token smart contract is to be deployed on the Ethereum mainnet and bridged to Arbitrum One. This strategy seeks to ensure the blockchain serves as a reliable, censorship-resistant, and verifiably impartial ledger, facilitating reward settlements and managing access to network resources.

## Token Utility

The SQD token is a critical component of the SQD ecosystem. Use cases for the SQD token are focused on streamlining and securing network operations in a permissionless manner:

* Alignment of incentives for infrastructure providers: SQD is used to reward node operators that contribute computation and storage resources to the network.
* Curation of network participants: Via delegation, the SQD token design includes built-in curation of nodes, facilitating permissionless selection of trustworthy operators for rewards.
* Fair resource consumption: By locking SQD tokens, consumers of data from the decentralized data lake may increase rate limits.
* Network decision making: SQD tokenholders can participate in governance, and are enabled to vote on protocol changes and other proposals.

The SQD token’s innovative curation component allows the SQD community to delegate SQD to Node Operators of their choice, ensuring trustlessness. SQD’s utility as a tool for adjusting rate limits is unique in increasing trustless performance, by locking SQD tokens, without having to pay a centralized provider for quicker or more efficient data access.

| Category             | % of Total Supply | Description                                                                                                                                                             |
| -------------------- | ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Pre-Seed Backers     | 12                | SQD’s earliest backers from the beginning of 2021. 6-month lockup after TGE, with a 20% release followed by a 24-month linear vesting for the remaining 80%.            |
| Seed Backers         | 16.3              | Early backers from late 2021. 6-month lockup after TGE, with a 20% release followed by a 24-month linear vesting for the remaining 80%.                                 |
| Strategic I Backers  | 4.6               | Strategic backers from early 2022. 12-month linear vesting after TGE.                                                                                                   |
| Strategic II Backers | 2                 | Strategic backers from 2022. 6-month lockup after TGE, with a 25% release followed by an 18-month linear vesting for the remaining 75%.                                 |
| Team                 | 15                | 6-month lockup after TGE, with a 20% release followed by a 24-month linear vesting for the remaining 80%.                                                               |
| Reserved Treasury    | 28.1              | Allocated for network development, grant programs, bounties, and ecosystem activities. 36-month linear vesting after TGE.                                               |
| Liquid Treasury      | 5                 | Liquid tokens for market making, liquidity provision on CEX/DEX, other ecosystem activities and events.                                                                 |
| Worker Rewards       | 10                | Allocated to reward network node operators. 84-month vesting after TGE.                                                                                                 |
| Community Sale       | 5                 | 20% release at TGE, followed by a 6-month linear vesting.                                                                                                               |
| Testnet Participants | 1                 | Allocated to reward early adopters of the network who participated in the testnet on CoinList. 6-month linear vesting after TGE.                                        |
| Testnet Worker       | 1                 | Allocated to reward early node operators in the network who participated in the testnet on CoinList. 12-months lockup after TGE, followed by a 24-month linear vesting. |

\*Please note that all figures are approximate based on date estimates, and may be impacted by alternate dates and other events.


# Whitepaper
Source: https://docs.sqd.dev/en/network/whitepaper

Detailed technical design overview of the SQD Network architecture and tokenomics — economic model, worker roles, gateway protocol, and governance design.

<Warning>
  The SQD whitepaper and any other documents attached or linked to this
  statement are intended only to provide a broad overview of the general
  direction of SQD and the SQD Network. The whitepaper and these documents are
  intended for information purposes only and may not be incorporated into any
  contract. These materials are not and should not be considered to offer a
  commitment to deliver any material, code, functionality, token, and should not
  be relied upon in making token purchase decisions or any kind of investment.
  All details related to the development, timing, or issue of any SQD feature or
  token remain at the sole discretion of Subsquid Labs GmbH.
</Warning>

## Intro

Blockchain, while initially designed solely for peer-to-peer transfers (Bitcoin), has since evolved into a general
purpose global execution runtime. The introduction of smart contract blockchains (Ethereum being the first) has enabled the creation of decentralized Apps (dApps) that are now catering to users in countless verticals, including:

* Decentralized Finance (DeFi) and tokenized real-world assets (RWA)
* Decentralized Social application (DeSoc)
* Decentralized Gaming with peer-to-peer game economies (GameFi)
* Decentralized physical infrastructure (DePIN)
* Decentralized creator economy (Music, Videos, NFTs)

One of the key fundamentals that has allowed countless dApps to thrive is the emergent infrastructure addressing the challenges of scaling, storage, and retrieval of data. We also refer to this as the "Web3 tech stack". The below all form part of the stack:

* Scalable execution (L2 chains, Rollups, Bridges, Solana)
* Bringing off-chain and real-world data on-chain (Oracles like Chainlink)
* Permanent and long-term storage for large data (Arweave, Filecoin)
* Application-specific databases and APIs (TheGraph, POKT, Kwil, WeaveDB)

<Frame>
  <img alt="Web3 tech stack diagram" />
</Frame>

However, there is still one part missing. One of the biggest challenges dApp developers face is accessing data at scale. It's currently highly complex to query and aggregate
the exponentially growing amount of data produced by blockchains (transactions and state data), applications
(decoded smart contract states), and any relevant off-chain data (prices, data kept in IPFS and Arweave, pruned EIP4844 data). In Web2, data produced by applications (such as logs or client data) is
stored in centralized data lakes like BigQuery, Snowflake, Apache, and Iceberg to facilitate access. Storing Web3 data in similar centralized data lakes would defeat the purpose of open, resilient access. Still, Web3 app data, if aggregated, filtered, and easily extractable, will serve as a catalyst to unlock the potential of the industry in a multi-chain paradigm.

Instead of replicating Web2 approaches the permissionless and decentralized nature of Web3 requires a fundamentally different solution with the following properties:

* Infinite horizontal scalability: the capacity of the lake should grow indefinitely as new nodes join
* Permissionless data access: the data can be uploaded and queried without any gatekeeping or centralized control
* Credible neutrality: no barrier for new nodes to join the network
* Trust-minimized queries: the data can be audited, and all clients can verify the query result
* Low maintenance cost: the price per query should be negligible

The SQD Network is built to satisfy all the above properties, thanks to its architecture where:

* The raw data is uploaded into permanent storage by the data providers, which act as oracles but for "big data"
* The data is compressed and distributed among the network nodes
* Node operators have to bond a security deposit, which can be slashed for byzantine behavior
* Each node efficiently queries the local data with DuckDB
* Any query can be verified by submitting a signed response to an on-chain smart contract

With the recent advances in in-process databases like DuckDB and an ever-decreasing price of quality SSD storage, the operational cost of a single node handling 1Tb of data is as low as 35/mo \[1] and will likely decline further in the future.

In the first stage, the SQD Network only supports simple queries that can be resolved by querying the local data of a suitable node. This already provides a fundamentally new primitive for on-chain data availability, fully eliminating the need to maintain expensive archival blockchain nodes to access historical blockchain data.

After the bootstrapping period, the network will be expanded with an additional query planning and execution layer supporting general-purpose SQL queries inspired by Apache Calcite and DataFusion.

## Design Overview

The below actors all participate in the SQD Network:

* Data Providers
* Workers
* Scheduler
* Logs Collector
* Rewards manager
* Data Consumers

<Frame>
  <img alt="SQD Network actors diagram" />
</Frame>

### Data Providers

Data Providers produce data to be served by SQD Network. Currently, the focus is on on-chain data, and data providers are blockchains and L2s. At the moment, SQD only supports EVM- and Substrate chains, but we plan on adding support for Cosmos and Solana in the near future.

Data providers are responsible for ensuring the quality and timely provision of data. During the bootstrapping phase, Subsquid Labs GmbH acts as the sole data provider for the SQD Network, serving as a proxy for chains from which the data is ingested block-by-block. The ingested data is validated by comparing hashes. It's then split into small compressed chunks and saved into persistent storage, from which the chunks are randomly distributed between the workers.

<Frame>
  <img alt="Data provider workflow diagram" />
</Frame>

The metadata is saved on-chain and is updated by the data provider each time a new piece of the data is uploaded. The metadata describes the schema, the data layout, the reserved storage, and the desired replication factor to be accommodated by the network. You can find the metadata structure in the Appendix.

The data provider is incentivized to provide consistent and valid data, and it is their responsibility to make the data available in persistent storage. During the bootstrap phase, the persistent storage used by Subsquid Labs is an S3-compatible service with backups pinned to IPFS. As the network matures, more data providers and storage options will be added, with data providers being vetted by on-chain governance and known trusted parties.

Data providers pay on-chain subscription fees to the network to make data available and to have it served by workers. These fees are sent to the SQD Network treasury.

### Scheduler

The scheduler is responsible for distributing the data chunks submitted by the data providers among the workers. The scheduler listens to updates of the data sets, as well as updates of the worker sets, and sends requests to the workers to download new chunks and/or redistribute the existing data chunks based on the capacity and the target redundancy for each dataset. Once a worker receives an update request , it downloads the missing data chunks from the corresponding persistent storage.

<Frame>
  <img alt="Scheduler workflow diagram" />
</Frame>

### Workers

Workers contribute the storage and compute resources to the network. They serve the data in a peer-to-peer manner for consumption and receive SQD tokens as compensation. Each worker has to be registered on-chain by bonding 100,000 SQD tokens, which can be slashed if the worker provably violates the protocol rule. SQD holders can also delegate to a specific worker to signal the reliability of the worker and earn a portion of the rewards.

The rewards are distributed each epoch and depend on:

* The previous number of epochs the worker stayed online
* The amount of data served to clients
* The number of delegated tokens
* Fairness
* Liveness during the epoch

Workers submit ping information along with their signed logs of the executed query requests to the Validator, thus committing to the results returned to the clients.

### Logs collector

The Logs Collector's sole responsibility is to collect the liveness pings and the query execution logs from the workers, batch them, and save them into public persistent storage. The logs are signed by the workers' P2P identities and pinned to IPFS. The data is stored for at least 6 months and used by other network participants.

### Reward Manager

The Reward Manager is responsible for calculating and submitting worker rewards on-chain for each epoch. The rewards depend on:

* Worker liveness during the epoch
* Delegated tokens
* Served queries (in bytes; both scanned and returned sizes are accounted for)
* Liveness since the registration

The Reward Manager accesses the logs, calculates the rewards, and submits a claimable commitment on-chain for each epoch. Each worker then claims their reward individually. The rewards may expire after an extended period of time.

### Data Consumers

To query the network, data consumers have to operate a gateway or use an externally provided service (public or private). Each gateway is bound to an on-chain address. The number of requests a gateway can submit to the network is capped by a number calculated based on the amount of locked SQD tokens. Consequently, the more tokens are locked by the gateway operator, the more bandwidth it is allowed to consume.

One can think of this mechanism as if the locked SQD yields virtual "compute units" (CU) based on the period the tokens are locked. All queries cost the same price of 1 CU (until complex SQL queries are implemented).

The query cap is calculated by:

* Calculating the virtual yield in SQD on the locked tokens (in SQD)
* Multiplying by the current CU price (in CU/SQD)

### Boosters

The locking mechanism has an additional booster design to incentivize gateway operators to lock their tokens for longer periods of time for an increase in CU. The longer the lock period, the more CUs are allocated per SQD/yr. The APY is calculated as `BASE_VIRTUAL_APY * BOOSTER`.

<Frame>
  <img alt="Booster design diagram" />
</Frame>

At the launch of the network, the parameters are set to be `BASE_VIRTUAL_APY = 12%` and `1SQD = 4000 CU`.

For example, if a gateway operator locks 100 SQD for 2 months, the virtual yield is 2SQD, which means it can perform 8000 queries (8000 CU).
If 100 SQD are locked for 3 years, the virtual yield is `12% * 3` APY, so the operator gets CUs worth 108 of SQD, that is it can submit up to `432000` queries to the network within the period of 3 years.

## Query Validation

The SQD Network provides economic guarantees for the validity of the queried data, with the added possibility of validating specific queries on-chain. All query responses are signed by the worker who executed the query, acting as a commitment to the query response. Anyone can submit such a response on-chain, and if it is deemed incorrect, the worker bond is slashed. The smart contract validation logic may be dataset-specific depending on the nature of the data being queried, with the following options:

* Proof by Authority: a white-listed set of on-chain identities decides on the validity of the response.
* Optimistic on-chain: after the validation request is submitted, anyone can submit a claim proving the query response is incorrect. For example, assuming the original query was "Return transactions matching the filter `X` in the block range `[Y, Z]`" and the response is some set of transactions `T.` During the validation window, anyone can submit a Merkle proof for some transaction `t` matching the filter `X` yet not in `T.` If no such proofs are submitted during the decision window, the response is considered valid.
* Zero-Knowledge: a zero-knowledge proof that the response exactly matches the requests. The zero-knowledge proof is generated off-chain by a prover and is validated on-chain by the smart contract.

Since submitting each query for on-chain validation on-chain is costly and not feasible, clients opt-in to submit query responses in an off-chain queue, together with the metadata such as response latency. Then, independent searchers scan the queue and submit suspicious queries for on-chain validation. If the query is deemed invalid, the submitter gets a portion of the slashed bond as a reward, thus incentivizing searchers to efficiently scan the queue for malicious responses.

<Frame>
  <img alt="Query validation process diagram" />
</Frame>

## SQD Token

SQD is the ERC-20 protocol token that is native to the SQD Network ecosystem. The token smart contract is to be deployed on the Ethereum mainnet and bridged to Arbitrum One. This strategy seeks to ensure the blockchain serves as a reliable, censorship-resistant, and verifiably impartial ledger, facilitating reward settlements and managing access to network resources.

The SQD token is a critical component of the SQD ecosystem. Use cases for the SQD token are focused on streamlining and securing network operations in a permissionless manner:

1. Alignment of incentives for infrastructure providers: SQD is used to reward node operators that contribute computation and storage resources to the network.

2. Curation of network participants: Via delegation, the SQD token design includes built-in curation of nodes, facilitating permissionless selection of trustworthy operators for rewards.

3. Fair resource consumption: By locking SQD tokens, consumers of data from the decentralized data lake may increase rate limits.

4. Network decision making: SQD tokenholders can participate in governance, and are enabled to vote on protocol changes and other proposals.

The SQD token’s innovative curation component allows the SQD community to delegate SQD to Node Operators of their choice, ensuring trustlessness. SQD’s utility as a tool for adjusting rate limits is unique in increasing trustless performance, by locking SQD tokens, without having to pay a centralized provider for quicker or more efficient data access.

## Appendix I -- Metadata

The metadata has the following structure:

```ts theme={"system"}
interface DataSet {
  dataset_id: string;
  // how many times the dataset should be replicated
  replication_factor: int;
  // a unique URL for permanent storage. Can be s3:// bucket
  permanent_storage: string;
  // how much space should for a single replica.
  // the total space required by a dataset is replication_factor * reserved_space
  reserved_space: int;
  // dataset-specific additional metadata
  extra_meta?: string;
  // dataset-specific state description
  state?: string;
  // reserved for future use, must be false
  premium: boolean;
}
```

The on-chain state of the dataset should be updated by the data provider periodically, and the total space must not exceed `reserved_space`. The dataset can be in the following states:

```ts theme={"system"}
enum DatasetState {
  // submitted by the data provider
  SUBMITTED,
  // the data being distributed by the network
  IN_PREPARATION,
  // served by the network
  ACTIVE,
  // if the subscription fee
  // has not been provided for the duration of an epoch
  COOLDOWN,
  // if the dataset is no longer served by the network
  // and will be deleted
  DISABLED,
}
```

The `Scheduler` changes the state to `IN_PREPARATION` and `ACTIVE` from `SUBMITTED`. The `COOLDOWN` and `DISABLED` states are activated automatically if subscription payments aren't made.

At the initial stage of the network, the switch to disabling datasets is granted to Subsquid Labs GmbH, which is going to be replaced by auto payments at a later stage.

## Appendix II -- Rewards

The network rewards are paid out to workers and delegators for each epoch. The Reward Manager submits an on-chain claim commitment, from which each participant can claim.

The rewards are allocated from the rewards pool. Each epoch, the rewards pool unlocks `APY_D * S * EPOCH_LENGTH` in rewards, where `EPOCH_LENGTH` is the length of the epoch in days, `S` is the total (bonded + delegated) amount of staked `SQD` during the epoch and `APY_D` is the (variable) base reward rate calculated for the epoch.

### Rewards pool

The SQD supply is fixed for the initial pool, and the rewards are distributed from a pool, to which 10% of the supply is allocated at TGE. The reward pool has a protective mechanism which caps the amount of rewards distributed per epoch, and it is subject to change via governance.

During the initial 3-year bootstrapping phase, the reward cap and the total supply of SQD is fixed. Afterwards, the reward cap drops significantly until the governance motion concludes on the inflation schedule and new tokens are minted to replenish the reward pool.

Unlike most projects who fix the inflation schedule before the launch, postponing this decision leaves a lot more flexibility and allows the community to analyze the historical 3-year data to make an informed decision on the future issuance of SQD.

### Reward rate

The reward rate depends on two factors: utilization of the network and staked supply. The network utilization rate is defined as

```
u_rate = (target_capacity - actual_capacity)/target_capacity
```

The target capacity is calculated as

```
target_capacity = sum([d.reserved_space * d.replication_factor])
```

where the sum is over the non-disabled datasets `d`.

The actual capacity is calculated as

```
actual_capacity = num_of_active_workers() * WORKER_CAPCITY * CHURN
```

The `WORKER_CAPACITY` is a fixed storage per worker, set to 1TB. `CHURN` is a discounting factor to account for the churn, set to 0.9.

The target APR (365-day-based rate) is then calculated as:

```
rAPR = MIN(base_apr(u_rate), APR_CAP(total_staked))
```

The `base_apr` is projected to be around 20% in the equilibrium state, when the actual worker capacity matches the desired network capacity, set externally. It is increased up to 70% to incentivize more workers to join the network until the target capacity is reached:

<Frame>
  <img alt="APR calculation chart" />
</Frame>

The `APR_CAP` cut-off is added to cap the total rewards per epoch.
One defines first

```
total_staked = bonded + delegated
```

to be the total amount of reward-earning SQD locked by workers (bonded) and delegators (delegated) respectively.

We then define

```
APR_CAP(total_staked) = 0.3 * INITIAL_POOL_SIZE / total_staked
```

It is set so that after the 3-year bootstrapping period at most 90% of the initial reward pool is spent.

### Worker reward rate

For each epoch, `rAPR` is calculated, and the total of

```
R = rAPR/365 * total_staked * EPOCH_LENGTH
```

is unlocked from the rewards pool.

The rewards are then distributed between the workers and delegators, and the leftovers are split between the burn and the treasury.

For a single worker and stakers for this token, the maximal reward is `rAPR/365 * (bond + staked) * EPOCH_LENGTH`. It is split into the worker liveness reward and the worker traffic reward.

Let `S[i]` be the stake for `i`-th worker and `T[i]` be the traffic units (defined below) processed by the worker. We define the relative weights as

```
s[i] = S[i]/sum(S[i])
t_scanned[i] = T_scanned[i]/sum(T_scanned[i])
t_e[i] = T_e[i]/sum(T_e[i])
t[i] = sqrt(t_scanned[i] * t_e[i])
```

In other words, `s[i]` and `t[i]` correspond to the contribution of the `i`-th worker to the total stake and to total traffic, respectively.

The traffic weight `t[i]` is a geometric average of the normalized scanned (`t_scanned[i]`) and the egress (`t_e[i]`) traffic processed by the worker. It is calculated by aggregating the logs of the queries processed by the worker during the epoch, and for each processed query, the worker reports the response size (egress) and the number of scanned data chunks.

The max potential yield for the epoch is given by `rAPR` described above:

```
r_max = rAPR/365 * EPOCH_LENGTH
```

The actual yield `r[i]` for the `i`-th worker is discounted:

```
r[i] = r_max * D_liveness * D_traffic(t_i, s_i) * D_tenure
```

`D_traffic` is a Cobb-Douglas type discount factor defined as

```
D_traffic(t_i, s_i) = min( (t_i/s_i)^alpha, 1 )
```

with the elasticity parameter alpha set to 0.1.

It has the following properties:

* Always in the interval `[0, 1]`
* Goes to zero as `t_i` goes to zero
* Neutral (i.e., close to 1) when `s_i ~ t_i`, that is, the stake contribution is fair (proportional to the traffic contribution)
* Excess traffic contributes only sub-linearly to the reward

`D_liveness` is a liveness factor calculated as the percentage of the time the worker is self-reported as online. A worker sends a ping message every 10 seconds, and if there are no pings within a minute, the worker is deemed offline for this period of time. The liveness factor is the percentage of the time (with minute-based granularity) the network is live. We suggest a piecewise linear function with the following properties:

* It is 0 below a reasonably high threshold (set to 0.8)
* Sharply increases to near 1 in the "intermediary" regime 0.8-0.9
* The penalty around 1 is diminishing

<Frame>
  <img alt="Liveness factor function chart" />
</Frame>

Finally, `D_tenure` is a long-range liveness factor incentivizing consistent liveness across the epochs. The rationale is that

* The probability of a worker failure decreases with the time the worker is live thus freshly spawned workers are rewarded less
* The discount for freshly spawned workers discourages the churn among workers and incentivizes longer-term commitments

<Frame>
  <img alt="Tenure factor function chart" />
</Frame>

### Distribution between the worker and delegators

The total claimable reward for the `i`-th worker and the stakers is calculated simply as `r[i] * s[i]`. Clearly, `s[i]` is the sum of the (fixed) bond `b[i]` and the (variable) delegated stake `d[i]`. Thus, the delegator rewards account for `r[i] * d[i]`. This extra reward part is split between the worker and the delegators:

* The worker gets: `r[i] * b[i] + 0.5 * r[i] * s[i]`
* The delegators get `0.5 * r[i] * s[i]`, effectively attaining `0.5 * r[i]` as the effectual reward rate.

The rationale for this split is:

* Make the worker accountable for `r[i]`
* Incentivize the worker to attract stakers (the additional reward part)
* Incentivize the stakers to stake for a worker with high liveness (and, in general, high `r[i]`)

At an equilibrium, the stakers will get a 10% annual yield, while workers get anything between 20-30% depending on the staked funds. Note that the maximal stake is limited by the bond size.

## References

`[1]` [SuperDAO Growth Trends report](https://superdao.notion.site/Web3-Growth-Trends-2023-Superdao-Report-1b51a98ad10644afba94c1d9df1e5f99)

`[2]` Based on the estimate that read-only RPC queries constitute roughly 90% of the RPC provider traffic


# Case studies
Source: https://docs.sqd.dev/en/other/case-studies

Real-world case studies of SQD in production — how PancakeSwap, leading DEXs, gaming, AI, and DeFi teams use SQD to power blockchain data infrastructure.

Explore how leading Web3 projects across DeFi, gaming, privacy, AI, and social sectors leverage SQD's data infrastructure to power their applications. From top DEXs to metaverses and AI platforms, discover real-world implementations and success stories.

<Tabs>
  <Tab title="All">
    <CardGroup>
      <Card title="Large-Scale Multichain Data for Top DEX PancakeSwap" icon="chart-mixed" href="https://blog.sqd.dev/case-study-how-sqd-supports-pancakeswap-with-large-scale-access-to-multichain-data/">
        **Category:** DeFi

        Discover how PancakeSwap leverages SQD for multichain data indexing.
      </Card>

      <Card title="Data-Powered Growth for Hydration's Liquidity" icon="droplet" href="https://blog.sqd.dev/case-study-how-hydration-uses-sqd-data-access-to-facilitate-its-defi-success/">
        **Category:** DeFi

        Learn how Hydration uses SQD to power their liquidity protocol.
      </Card>

      <Card title="How Polimec uses SQD data to provide compliant DeFi" icon="shield-check" href="https://blog.sqd.dev/how-polimec-uses-sqd-data-to-provide-compliant-defi/">
        **Category:** DeFi

        See how Polimec leverages SQD for compliant DeFi operations.
      </Card>

      <Card title="How GMX Accelerated Cross-Chain Indexing with SQD" icon="chart-line" href="https://blog.sqd.dev/how-gmx-accelerated-cross-chain-indexing-with-sqd/">
        **Category:** DeFi

        Learn how GMX accelerated cross-chain indexing with SQD.
      </Card>

      <Card title="zkVerify Built Proof-Scale Indexing & Monitoring with SQD" icon="shield-check" href="https://blog.sqd.dev/zkverify-built-proof-scale-indexing-monitoring-with-sqd/">
        **Category:** DeFi

        See how zkVerify built proof-scale indexing and monitoring with SQD.
      </Card>

      <Card title="Curvance Cut Indexing Time from Days to Minutes with SQD" icon="clock" href="https://blog.sqd.dev/curvance-cut-indexing-time-from-days-to-minutes-with-sqd/">
        **Category:** DeFi

        Learn how Curvance dramatically reduced indexing time with SQD.
      </Card>

      <Card title="How a DeFi Builder Cut Indexing Time from Months to Days with SQD" icon="gauge-high" href="https://blog.sqd.dev/how-a-defi-builder-cut-indexing-time-from-months-to-days-with-sqd/">
        **Category:** DeFi

        Discover how a DeFi builder achieved massive time savings with SQD.
      </Card>

      <Card title="Seamless Game Development Across Chains with ChainSafe" icon="gamepad" href="https://blog.sqd.dev/case-study-chainsafe-using-subsquid-to-power-fast-evm-deployments/">
        **Category:** Gaming

        Explore how ChainSafe builds cross-chain games with SQD.
      </Card>

      <Card title="How Skybreach leverages SQD data access in its games" icon="cloud" href="https://blog.sqd.dev/skybreach-case-study/">
        **Category:** Gaming

        Learn how Skybreach integrates SQD data into their gaming platform.
      </Card>

      <Card title="How SQD powers Fantasy.Top's game with data" icon="trophy" href="https://blog.sqd.dev/how-fantasy-top-relies-on-sqd-to-provide-a-seamless-social-gaming-experience/">
        **Category:** Gaming

        Discover how Fantasy.Top uses SQD for real-time game data.
      </Card>

      <Card title="How SQD enables Chillwhales to expand NFT browsing on LUKSO" icon="image" href="https://blog.sqd.dev/chillwhales-x-sqd-partner-case-study/">
        **Category:** Social

        See how Chillwhales enhances NFT browsing with SQD.
      </Card>

      <Card title="How Levr.Bet powers real-time sports betting on Monad with SQD" icon="chart-line" href="https://blog.sqd.dev/levr-bet-x-sqd-partner-case-study/">
        **Category:** Social

        Learn how Levr.Bet delivers real-time sports betting with SQD.
      </Card>

      <Card title="How CoolWallet uses SQD to secure its data back-end" icon="wallet" href="https://blog.sqd.dev/how-coolwallet-uses-sqd-to-secure-its-data-back-end/">
        **Category:** Social

        Discover how CoolWallet secures their data infrastructure with SQD.
      </Card>

      <Card title="How interface uses SQD to enable people to explore Ethereum" icon="magnifying-glass" href="https://blog.sqd.dev/how-interface-uses-sqd-to-enable-people-to-explore-ethereum/">
        **Category:** Social

        Explore how Interface enables Ethereum exploration with SQD.
      </Card>

      <Card title="Crosschain Privacy and Trace Data with RAILGUN" icon="lock" href="https://blog.sqd.dev/case-study-railgun-leveraging-sqd-trace-data-to-enable-privacy-pools/">
        **Category:** Privacy

        Learn how RAILGUN leverages SQD for crosschain privacy.
      </Card>

      <Card title="Data for AI Use Cases with Guru Network" icon="brain" href="https://blog.sqd.dev/case-study-how-guru-network-powers-its-ai-processors-with-data-from-sqd/">
        **Category:** AI

        Discover how Guru Network powers AI applications with SQD data.
      </Card>
    </CardGroup>
  </Tab>

  <Tab title="DeFi">
    <CardGroup>
      <Card title="Large-Scale Multichain Data for Top DEX PancakeSwap" icon="chart-mixed" href="https://blog.sqd.dev/case-study-how-sqd-supports-pancakeswap-with-large-scale-access-to-multichain-data/">
        **Category:** DeFi

        Discover how PancakeSwap leverages SQD for multichain data indexing.
      </Card>

      <Card title="Data-Powered Growth for Hydration's Liquidity" icon="droplet" href="https://blog.sqd.dev/case-study-how-hydration-uses-sqd-data-access-to-facilitate-its-defi-success/">
        **Category:** DeFi

        Learn how Hydration uses SQD to power their liquidity protocol.
      </Card>

      <Card title="How Polimec uses SQD data to provide compliant DeFi" icon="shield-check" href="https://blog.sqd.dev/how-polimec-uses-sqd-data-to-provide-compliant-defi/">
        **Category:** DeFi

        See how Polimec leverages SQD for compliant DeFi operations.
      </Card>

      <Card title="How GMX Accelerated Cross-Chain Indexing with SQD" icon="chart-line" href="https://blog.sqd.dev/how-gmx-accelerated-cross-chain-indexing-with-sqd/">
        **Category:** DeFi

        Learn how GMX accelerated cross-chain indexing with SQD.
      </Card>

      <Card title="zkVerify Built Proof-Scale Indexing & Monitoring with SQD" icon="shield-check" href="https://blog.sqd.dev/zkverify-built-proof-scale-indexing-monitoring-with-sqd/">
        **Category:** DeFi

        See how zkVerify built proof-scale indexing and monitoring with SQD.
      </Card>

      <Card title="Curvance Cut Indexing Time from Days to Minutes with SQD" icon="clock" href="https://blog.sqd.dev/curvance-cut-indexing-time-from-days-to-minutes-with-sqd/">
        **Category:** DeFi

        Learn how Curvance dramatically reduced indexing time with SQD.
      </Card>

      <Card title="How a DeFi Builder Cut Indexing Time from Months to Days with SQD" icon="gauge-high" href="https://blog.sqd.dev/how-a-defi-builder-cut-indexing-time-from-months-to-days-with-sqd/">
        **Category:** DeFi

        Discover how a DeFi builder achieved massive time savings with SQD.
      </Card>
    </CardGroup>
  </Tab>

  <Tab title="Gaming">
    <CardGroup>
      <Card title="Seamless Game Development Across Chains with ChainSafe" icon="gamepad" href="https://blog.sqd.dev/case-study-chainsafe-using-subsquid-to-power-fast-evm-deployments/">
        **Category:** Gaming

        Explore how ChainSafe builds cross-chain games with SQD.
      </Card>

      <Card title="How Skybreach leverages SQD data access in its games" icon="cloud" href="https://blog.sqd.dev/skybreach-case-study/">
        **Category:** Gaming

        Learn how Skybreach integrates SQD data into their gaming platform.
      </Card>

      <Card title="How SQD powers Fantasy.Top's game with data" icon="trophy" href="https://blog.sqd.dev/how-fantasy-top-relies-on-sqd-to-provide-a-seamless-social-gaming-experience/">
        **Category:** Gaming

        Discover how Fantasy.Top uses SQD for real-time game data.
      </Card>
    </CardGroup>
  </Tab>

  <Tab title="Privacy">
    <CardGroup>
      <Card title="Crosschain Privacy and Trace Data with RAILGUN" icon="lock" href="https://blog.sqd.dev/case-study-railgun-leveraging-sqd-trace-data-to-enable-privacy-pools/">
        **Category:** Privacy

        Learn how RAILGUN leverages SQD for crosschain privacy.
      </Card>
    </CardGroup>
  </Tab>

  <Tab title="AI">
    <CardGroup>
      <Card title="Data for AI Use Cases with Guru Network" icon="brain" href="https://blog.sqd.dev/case-study-how-guru-network-powers-its-ai-processors-with-data-from-sqd/">
        **Category:** AI

        Discover how Guru Network powers AI applications with SQD data.
      </Card>
    </CardGroup>
  </Tab>

  <Tab title="Social">
    <CardGroup>
      <Card title="How SQD enables Chillwhales to expand NFT browsing on LUKSO" icon="image" href="https://blog.sqd.dev/chillwhales-x-sqd-partner-case-study/">
        **Category:** Social

        See how Chillwhales enhances NFT browsing with SQD.
      </Card>

      <Card title="How Levr.Bet powers real-time sports betting on Monad with SQD" icon="chart-line" href="https://blog.sqd.dev/levr-bet-x-sqd-partner-case-study/">
        **Category:** Social

        Learn how Levr.Bet delivers real-time sports betting with SQD.
      </Card>

      <Card title="How CoolWallet uses SQD to secure its data back-end" icon="wallet" href="https://blog.sqd.dev/how-coolwallet-uses-sqd-to-secure-its-data-back-end/">
        **Category:** Social

        Discover how CoolWallet secures their data infrastructure with SQD.
      </Card>

      <Card title="How interface uses SQD to enable people to explore Ethereum" icon="magnifying-glass" href="https://blog.sqd.dev/how-interface-uses-sqd-to-enable-people-to-explore-ethereum/">
        **Category:** Social

        Explore how Interface enables Ethereum exploration with SQD.
      </Card>
    </CardGroup>
  </Tab>
</Tabs>


# Contracts
Source: https://docs.sqd.dev/en/other/contracts

Verified SQD token, SQD Network, and Portal Pool contract addresses across Ethereum, Arbitrum, Base, and other chains — source of truth for integrations.

## Deployed Contracts

### Ethereum Mainnet

| Contract name | Address                                      |
| ------------- | -------------------------------------------- |
| SQD           | `0x1337420dED5ADb9980CFc35f8f2B054ea86f8aB1` |

### Base

| Contract name | Address                                      |
| ------------- | -------------------------------------------- |
| SQD           | `0xd4554bea546efa83c1e6b389ecac40ea999b3e78` |

### BNB Smart Chain

| Contract name | Address                                      |
| ------------- | -------------------------------------------- |
| SQD           | `0xe50e3d1a46070444f44df911359033f2937fcc13` |

### Arbitrum One (production)

| Contract name                  | Address                                      |
| ------------------------------ | -------------------------------------------- |
| SQD                            | `0x1337420dED5ADb9980CFc35f8f2B054ea86f8aB1` |
| Router                         | `0x67F56D27dab93eEb07f6372274aCa277F49dA941` |
| NetworkController              | `0xf5462EF65Ca8a9Cca789c912Bc8ada80b582d68d` |
| Staking                        | `0xb31a0d39d2c69ed4b28d96e12cbf52c5f9ac9a51` |
| WorkerRegistration             | `0x36e2b147db67e76ab67a4d07c293670ebefcae4e` |
| RewardTreasury                 | `0x237abf43bc51fd5c50d0d598a1a4c26e56a8a2a0` |
| DistributedRewardsDistribution | `0x4de282bD18aE4987B3070F4D5eF8c80756362AEa` |
| GatewayRegistry                | `0x8a90a1ce5fa8cf71de9e6f76b7d3c0b72feb8c4b` |
| RewardCalculation              | `0xd3D2C185a30484641C07b60e7d952d7B85516eB5` |
| SoftCap                        | `0xde29d5215c28036ce56091ea91038c94c84c87d0` |
| EqualStrategy                  | `0xa604f84c9c59e223b12c831b35723aa0d7277f8b` |
| SubequalStrategy               | `0xf197094d96f45325ee8bd2c43c5d25c05d66ab62` |
| AllocationsViewer              | `0x88ce6d8d70df9fe049315fd9d6c3d59108c15c4c` |
| VestingFactory                 | `0x1f8f83cd76baeca1cb5c064ad59203c82b4e4ece` |
| TemporaryHoldingFactory        | `0x14926ebf05a904b8e2e2bf05c10ecca9a54d8d0d` |

#### Portal Pool system (Arbitrum One)

Factory and registry are UUPS proxies; pools are beacon proxies (one shared implementation, upgradable in one shot via the beacon).

| Contract name                        | Address                                      |
| ------------------------------------ | -------------------------------------------- |
| PortalPoolFactory (UUPS proxy)       | `0x18184740eBE24881355E33cec620C44E575F2C70` |
| ↳ Factory implementation             | `0xCe5D796769Ba065Bf61a8eFC892A8a835FAe0351` |
| PortalRegistry (UUPS proxy)          | `0x29edE9EB0ad3C02B6A98B0E41bF99Cd709812850` |
| ↳ Registry implementation            | `0xC3725B2584Ad46c52f9eFA6F27d0291E3dbC3045` |
| PortalPoolBeacon                     | `0x16983f5a5816d4B04c92Ab43Fed3B2F212D4e568` |
| ↳ PortalPoolImplementation (current) | `0x2981E64342fc76d531168Cf6754F81422138b3C4` |
| FeeRouterModule                      | `0x59c074ee3dd85125620B4A5b452C008Bc792a787` |

##### Live pools

| Pool             | Address                                      | plSQD receipt token                          |
| ---------------- | -------------------------------------------- | -------------------------------------------- |
| Lambda × SQD     | `0x89ca93e09ec7355a1d6bd410fe0bb4c9b24542db` | `0xF7B057C3b0ee4dd101047B26Fd3964185B7d8cc4` |
| SQD Revenue Pool | `0x438c2a47e82cd445524ce5651ae7e6c1dd386d09` | `0x365709Ef4830B77a6EB4a689F13F57A1C22d8306` |

### Arbitrum Sepolia (testnet)

| Contract name                  | Address                                      |
| ------------------------------ | -------------------------------------------- |
| tSQDArbitrum                   | `0x24f9C46d86c064a6FA2a568F918fe62fC6917B3c` |
| Router                         | `0xD2093610c5d27c201CD47bCF1Df4071610114b64` |
| NetworkController              | `0x68Fc7E375945d8C8dFb0050c337Ff09E962D976D` |
| Staking                        | `0x347E326b8b4EA27c87d5CA291e708cdEC6d65EB5` |
| WorkerRegistration             | `0xCD8e983F8c4202B0085825Cf21833927D1e2b6Dc` |
| RewardTreasury                 | `0x785136e611E15D532C36502AaBdfE8E35008c7ca` |
| DistributedRewardsDistribution | `0x68f9fE3504652360afF430dF198E1Cb7B2dCfD57` |
| GatewayRegistry                | `0xAB46F688AbA4FcD1920F21E9BD16B229316D8b0a` |
| VestingFactory                 | `0x0eD5FB811167De1928322a0fa30Ed7F3c8C370Ca` |
| RewardCalculation              | `0x93D16d5210122c804DE9931b41b3c6FA2649CE3F` |
| EqualStrategy                  | `0x94DF0410BF415765e8e9431d545AF9805859b5Db` |
| SubequalStrategy               | `0x20cA692986D127CE78938E2518cE2F49F105eC48` |
| AllocationsViewer              | `0xC0Af6432947db51e0C179050dAF801F19d40D2B7` |

### Arbitrum Goerli (legacy)

| Contract name                  | Address                                      |
| ------------------------------ | -------------------------------------------- |
| tSQD                           | `0x69f643dCa8B633F97e2fC979E8eBa6cB63B242A9` |
| DistributedRewardsDistribution | `0x69f643dCa8B633F97e2fC979E8eBa6cB63B242A9` |
| RewardTreasury                 | `0x87F1B67c10237CBB32019EF33052B96940994149` |
| Staking                        | `0x99Fa79f673ffa4354e96670999cb67A0d43de4C1` |
| WorkerRegistration             | `0x6867E96A0259E68A571a368C0b8d733Aa56E3915` |
| RewardCalculation              | `0xC60CA978Bf5A9E2374B82D346d1B36Fd35D27991` |
| NetworkController              | `0xF0512AD4f8945Ba47B9100609122B4B2769cA99C` |

## Token Contracts

Official SQD token contract addresses for purchasing and trading:

<Tabs>
  <Tab title="Arbitrum">
    **Contract Address:** `0x1337420dED5ADb9980CFc35f8f2B054ea86f8aB1`

    [View on Arbiscan](https://arbiscan.io/address/0x1337420dED5ADb9980CFc35f8f2B054ea86f8aB1)
  </Tab>

  <Tab title="Base">
    **Contract Address:** `0xd4554BEa546EFa83C1E6B389ecac40EA999B3E78`

    [View on Basescan](https://basescan.org/address/0xd4554bea546efa83c1e6b389ecac40ea999b3e78)
  </Tab>

  <Tab title="BNB Smart Chain">
    **Contract Address:** `0xe50e3d1a46070444f44df911359033f2937fcc13`

    [View on BscScan](https://bscscan.com/token/0xe50e3d1a46070444f44df911359033f2937fcc13)
  </Tab>

  <Tab title="Ethereum">
    **Contract Address:** `0x1337420dED5ADb9980CFc35f8f2B054ea86f8aB1`

    [View on Etherscan](https://etherscan.io/token/0x1337420ded5adb9980cfc35f8f2b054ea86f8ab1)
  </Tab>
</Tabs>

<Info>
  To interact with SQD Network contracts, your tokens must be on **Arbitrum
  One**.
</Info>


# SQD Network FAQ
Source: https://docs.sqd.dev/en/other/faq

Frequently asked questions about the SQD Network — participation, tokenomics, worker and gateway operation, decentralization, and how to get started.

## Network Overview

<AccordionGroup>
  <Accordion title="What is SQD Network?">
    SQD Network is a decentralized data lake focused on making blockchain data accessible, regardless of scale. It's a huge and growing compute cluster consisting of 2500+ decentralized workers.

    Each worker has 1 TB of storage and is equipped with a powerful query engine, allowing them to serve queries to the Network. [Worker nodes](../network/worker) store the data and run queries on their local data chunks, while data access nodes (called *Portals*) request partial query results from multiple workers, concatenate the outputs, and expose a stream-based API.

    Decentralization means that the workers are not controlled by a single party but are supported by independent operators across the globe.

    <Frame>
      <img alt="SQD Network architecture diagram showing decentralized workers" />
    </Frame>
  </Accordion>

  <Accordion title="What's Portal?">
    Portal, formerly known as Gateway, is a service opening a window to the Network for the user.

    It communicates with decentralized workers via *libp2p*, providing a simple and efficient HTTP API for streaming data.

    You can think about it as a **bridge** connecting centralized and decentralized worlds.
  </Accordion>

  <Accordion title="What's hotblocks?">
    While static archival data is served from the Network allowing very high *throughput*, the "last-mile" real-time data requires low *latency*.

    That's why this data is stored together with the portal and receives live updates from RPC nodes.

    The portal then combines both data sources to provide a single stream of data that has both high throughput and low latency.
  </Accordion>

  <Accordion title="Can I run my own Portal?">
    Yes! But to access SQD Network, you have to lock SQD tokens. The more SQD is locked, the higher rate limit you get.

    For most people it's easier to use one of the existing Portals with flexible payment options provided by its operators.

    For example, we're providing a public Portal for free to test your code before going to production.
  </Accordion>

  <Accordion title="What data is available?">
    Currently, SQD Network supports **raw blockchain data** for 200+ networks.

    It means that you can request which fields you need, set filters, and get results from the entire blockchain history downloaded within **minutes or hours** (depending on the amount of extracted data) as opposed to days or months, as it would be with direct RPC requests.

    The aggregation (calculating token balances, prices, etc) is then done on the client. We also provide a convenient SDK for that.
  </Accordion>

  <Accordion title="What exactly are we releasing at the end of 2025?">
    We've already been running a centralized version of the Network called "v2 archives".

    Switch to the Network gives multiple benefits:

    * hundreds of times bigger compute cluster. Petabytes of storage, thousands of CPUs
    * 10x faster Rust query engine
    * much faster data streaming enabled by fetch-ahead parallelism
    * "hotblocks" included. No need to bring your own RPCs anymore
    * permissionless access
    * much bigger datasets, like the full history of Solana and Eclipse
  </Accordion>

  <Accordion title="What is a decentralized data lake?">
    A decentralized data lake is a distributed system that stores and organizes large amounts of raw data in its native format until it's needed. SQD Network is an example of a decentralized data lake, providing scalable and efficient access to blockchain data.
  </Accordion>

  <Accordion title="What are some use cases for SQD's decentralized data lake?">
    SQD's decentralized data lake powers various blockchain applications:

    * Indexing blockchain data for quick access
    * Blockchain explorers
    * Analytics platforms
    * Multi-chain aggregators
    * Real-time and historical data APIs
    * dApp backends and more
  </Accordion>

  <Accordion title="Is SQD production-ready? Can I trust it for my dApp?">
    Yes! SQD is production-ready and reliable for your dApp. It has been effectively used in over 500 use cases across ecosystems, proving its trustworthiness and utility within the Web3 developer community.

    <Check>
      SQD Network has powered production applications for hundreds of projects.
    </Check>
  </Accordion>
</AccordionGroup>

## Participation

<AccordionGroup>
  <Accordion title="How can I participate in SQD Network?">
    You can participate in SQD Network in several ways:

    1. **Run a worker node** - [Set up a worker](../network/worker) to contribute storage and compute resources to the network and earn `SQD` tokens as rewards
    2. **Delegate tokens** - [Delegate your `SQD` tokens](../network/delegate) to high-performing workers and earn a share of their rewards

    <Tip>
      Non-technical users can still earn rewards by delegating tokens to reliable
      workers.
    </Tip>
  </Accordion>

  <Accordion title="What are the hardware requirements to run a worker node?">
    To run a single worker, you will need:

    **Hardware:**

    * 4 vCPU
    * 16GB RAM
    * 1TB SSD
    * Stable 24/7 Internet connection (at least 1Gbit)

    **Network:**

    * Public IP address
    * Two ports open for incoming traffic:
      * One UDP port for P2P communication (default: 12345)
      * One TCP port for Prometheus metrics (default: 9090)

    **Financial:**

    * 100,000 `SQD` tokens (in your wallet or vesting contract)
    * Some Arbitrum ETH for gas fees

    <Info>
      A single account can register multiple workers. See the [worker setup
      guide](../network/worker) for detailed instructions.
    </Info>
  </Accordion>

  <Accordion title="I can't code. How can I participate in SQD Network?">
    Non-technical participants can contribute in multiple ways:

    **Delegation** - [Delegate SQD tokens](../network/delegate) to reliable workers and earn a percentage of their rewards.

    **Community Growth** - Help grow the SQD Network by:

    * Cultivating a vibrant community
    * Amplifying the network's reach through social media
    * Sharing information across various platforms
  </Accordion>

  <Accordion title="Does participating in SQD Network require committing my tokens?">
    Yes. When you delegate SQD or register a worker, your tokens will be locked for a period of 100,000 Ethereum blocks (approximately 14 days), starting at the end of the current [epoch](#what-is-an-epoch-in-the-context-of-sqd-network).

    <Warning>Your tokens cannot be withdrawn during the lockup period.</Warning>
  </Accordion>

  <Accordion title="Can I delegate to multiple workers at once?">
    Yes, you can split your SQD tokens and delegate to multiple workers. This can help diversify your risk across different operators.
  </Accordion>
</AccordionGroup>

## Development

<AccordionGroup>
  <Accordion title="How do I deploy a simple squid indexer?">
    You can start your first squid indexer in under 5 minutes using our templates! Learn more in the [development guide](/en/sdk/squid-sdk/quickstart).
  </Accordion>

  <Accordion title="Do I need to know how to code to use SQD?">
    It depends on what you want to do. To delegate tokens and earn rewards, no coding is required. To build indexers or query blockchain data, you'll need programming knowledge. SQD offers three options: Portal API (works with any language), Pipes SDK (TypeScript), and Squid SDK (TypeScript with built-in database and GraphQL).
  </Accordion>

  <Accordion title="What's the difference between Portal API, Pipes SDK, and Squid SDK?">
    Portal API provides raw blockchain data via HTTP in any language. Pipes SDK is a lightweight TypeScript library for custom data pipelines. Squid SDK is a complete TypeScript framework with built-in PostgreSQL and auto-generated GraphQL API. Choose based on your needs: Portal for simple queries, Pipes for custom pipelines, Squid for full dApp backends.
  </Accordion>
</AccordionGroup>

## Data & Chains

<AccordionGroup>
  <Accordion title="What kind of datasets are available from SQD Network?">
    SQD Network provides historical blockchain data optimized for batch access, enabling efficient indexing and querying across multiple blocks.
  </Accordion>

  <Accordion title="What chains are currently supported by SQD Network?">
    SQD Network currently supports:

    * EVM / Ethereum-compatible chains
    * Substrate-based chains
    * Solana
    * Fuel
    * Starknet

    View the [complete list of supported networks](/en/data/evm).
  </Accordion>
</AccordionGroup>

## Technical Details

<AccordionGroup>
  <Accordion title="What is an epoch in the context of SQD Network?">
    An "epoch" is a unit of time that SQD Network uses for internal settlement. It is [defined on-chain](https://arbiscan.io/address/0x4cf58097d790b193d22ed633bf8b15c9bc4f0da7#readContract#F3) and is currently set to 100 L1 (Ethereum) blocks, or approximately 20 minutes.
  </Accordion>

  <Accordion title="What is Tethys?">
    Tethys is the long-running testnet of SQD Network. Its contracts run on Arbitrum Sepolia. For more details, see the [Tethys testnet announcement](https://github.com/subsquid/subsquid-network-contracts/wiki/Tethys-testnet-announcement).
  </Accordion>

  <Accordion title="Why did SQD choose Arbitrum for its contracts?">
    We chose Arbitrum for its:

    * Large, active community
    * High-quality tooling for smart contract deployment
    * Cost-effective transactions
  </Accordion>

  <Accordion title="Where is the data that SQD Network indexes stored?">
    The data indexed by SQD Network is stored in persistent storage. During the bootstrap phase, Subsquid Labs uses an S3-compatible service with backups pinned to IPFS for data redundancy and availability.
  </Accordion>

  <Accordion title="How does SQD ensure the security and privacy of the data accessed through the network?">
    SQD Network employs validators who:

    * Listen to logs of executed queries
    * Resubmit queries to validate responses
    * Trigger an arbitration process to handle any discrepancies

    This validation mechanism ensures data integrity across the network.
  </Accordion>

  <Accordion title="What is the difference between SQD Network and the Graph Network?">
    Learn about the key differences and review a comprehensive feature comparison in our [SQD vs The Graph](/en/sdk/squid-sdk/subsquid-vs-thegraph) guide.
  </Accordion>
</AccordionGroup>

## General

<AccordionGroup>
  <Accordion title="What is SQD's roadmap?">
    View the up-to-date roadmap at [sqd.dev/roadmap](https://www.sqd.dev/roadmap).
  </Accordion>

  <Accordion title="Does SQD Network have an app?">
    Yes! Access the SQD Network app at [network.subsquid.io](https://network.subsquid.io).
  </Accordion>

  <Accordion title="How does SQD Network make blockchain data accessible for free?">
    SQD Network offers data access through two models:

    **Open Private Network** - Free for consumers, funded by fixed subscription fees from blockchain networks providing data.

    **Permissionless Public Network** - Requires locking `SQD` tokens to reserve bandwidth (requests per second). However, SQD provides a free rate-limited portal accessible to everyone, you only need tokens if running your own portal.

    **Indexer Hosting** - [SQD Cloud](../cloud/overview) includes free unlimited access to a dedicated SQD Portal. You only pay for computational resources (CPUs, SSD space, etc.). View our [competitive pricing](../cloud/pricing).

    <Info>
      You can also self-host your indexers on any infrastructure. See our
      [self-hosting guide](/en/sdk/squid-sdk/resources/self-hosting) for details.
    </Info>
  </Accordion>
</AccordionGroup>


# SQD Glossary
Source: https://docs.sqd.dev/en/other/glossary/glossary

Definitions of key terms used across SQD documentation — archives, gateways, workers, Portals, squids, processors, slots, tags, and more.

### `Archives`[​](#archives "Direct link to Archives")

Deprecated term used for [SQD Network](/en/network/overview) and for the data sourcing service of the deprecated FireSquid SDK version. Occasionally refers to a chain-specific endpoint available from either source (e.g. "an Ethereum archive"). The new terminology is:

* "Archives" as an abstract collection of services for some networks is replaced by "[SQD Network](/en/network/overview)" (when referring to data location) or "SQD Network gateway" (when referring to the service)
* "public Archives" are replaced by the [open private version](/en/network/overview#open-private-network) of SQD Network
* "an archive" for a particular network is replaced by "a SQD Network gateway"

List of gateways for the open private SQD Network is available in the [EVM](/en/data/evm), [Substrate](/en/data/substrate), [Solana](/en/data/solana), and [Other](/en/data/other) listings, and via [`sqd gateways`](/en/cloud/reference/cli/gateways).

**Not to be confused with [archive blockchain nodes](https://ethereum.org/developers/docs/nodes-and-clients/archive-nodes)**.

### `archive-registry`[​](#archive-registry "Direct link to archive-registry")

The deprecated NPM package `@subsquid/archive-registry` that was used to look up squid data sources by network aliases (with `lookupArchive()` and a small CLI). We now recommend using raw gateway URLs instead of `lookupArchive()` calls in processor configuration. The exploratory CLI is replaced by [`sqd gateways`](/en/cloud/reference/cli/gateways); per-chain gateway listings are available under the [EVM](/en/data/evm), [Substrate](/en/data/substrate), [Solana](/en/data/solana), and [Other](/en/data/other) sections.

### `Block`[​](#block "Direct link to Block")

An atomic state transition of a blockchain. Typically an ordered collection of transactions.

### `Call`[​](#call "Direct link to Call")

On [Substrate](#substrate), a call is a sub-routine changing the runtime state. An extrinsic consists of a root call which in turn may have sub-calls, thus calls executed by an extrinsic have parent-child relationship. For example, `util.batch` extrinsic has a single root call and multiple child calls. SQD processor is call-based rather than extrinsic based, as normally one is interested in specific calls changing the substrate state, no matter if it was part of a batch extrinsic, or it was wrapped in a sudo or proxy call.

### `Cloud (former Aquarium)`[​](#cloud-former-aquarium "Direct link to Cloud (former Aquarium)")

A [cloud service](/en/cloud/) for deploying [squids](#squid) in a serverless fashion maintained by Subsquid Labs GmbH.

### `Contracts pallet`[​](#contracts-pallet "Direct link to Contracts pallet")

[`Contracts`](https://substrate-developer-hub.github.io/substrate-how-to-guides/docs/pallet-design/add-contracts-pallet/) is a [pallet](#pallet) developed by Parity to execute WASM-based smart contracts.

### `ETL`[​](#etl "Direct link to ETL")

Stands for Extract-Transform-Load. A pipeline to extract data from the source, enrich, transform, normalize and load into the target data store.

### `Event`[​](#event "Direct link to Event")

An operation performed during a blockchain state transition that results in emission of a structured log message. Subsequently, the message can be retrieved from blockchain nodes.

### `EVM`[​](#evm "Direct link to EVM")

Stands for the Ethereum Virtual Machine. An instruction set and the runtime specification originally developed for the Ethereum network which was later adopted by many other chains (e.g. BSC, Polygon).

### `Extrinsic`[​](#extrinsic "Direct link to Extrinsic")

On [Substrate](#substrate), a generalized transaction externally submitted to a Substrate runtime for execution. There are [technical nuances](https://substrate.stackexchange.com/questions/2248/is-a-transaction-an-extrinsic) differentiating transactions and extrinsics.

### `FRAME pallets`[​](#frame-pallets "Direct link to FRAME pallets")

A collection of standard [Substrate](#substrate) pallets maintained by Parity. Currently kept in the [Substrate Repo](https://github.com/paritytech/substrate/tree/master/frame).

### `Frontier pallet`[​](#frontier-pallet "Direct link to Frontier pallet")

[Frontier](https://github.com/polkadot-evm/frontier) is a [Substrate](#substrate) [pallet](#pallet) implementing the [Ethereum Virtual Machine](#evm). In particular, Substrate chains with Frontier pallet support EVM-based smart contracts.

### `GraphQL`[​](#graphql "Direct link to GraphQL")

A declarative query language and an API specification language developed by Facebook as an alternative to REST. See the official [GraphQL docs](https://graphql.org/) for more details.

### `Hot blocks`[​](#hot-blocks "Direct link to Hot blocks")

An alternative term for [unfinalized blocks](/en/sdk/squid-sdk/resources/unfinalized-blocks) that emphasizes the fact that the data extracted from these blocks is volatile. Sometimes used in Squid SDK code and comments.

### `ink!`[​](#ink "Direct link to ink!")

An SDK (software development kit) and a smart-contract language for developing WASM-based smart contracts, maintained by Parity. The contracts developed with ink! are compiled into a WASM blob compatible with the API exposed by the [`Contracts` pallet](#contracts-pallet). More details [here](https://use.ink).

### `OpenReader`[​](#openreader "Direct link to OpenReader")

1. SQD's own open source [GraphQL server](/en/sdk/squid-sdk/reference/openreader-server/overview), built in-house. No longer recommended for new projects running PostgreSQL due to its [limitations](/en/sdk/squid-sdk/reference/openreader-server/overview#limitations). See [Serving GraphQL](/en/sdk/squid-sdk/resources/serving-graphql) to learn more.

2. The GraphQL [schema generation library](https://github.com/subsquid/squid-sdk/tree/master/graphql/openreader) at the heart of 1. Implements [OpenCRUD](https://www.opencrud.org/).

### `Pallet`[​](#pallet "Direct link to Pallet")

A portable module that can be added to a [Substrate](#substrate) runtime. Typically, contains a self-contained implementation of a business logic that can be re-used across multiple chains.

### `Schema file`[​](#schema-file "Direct link to Schema file")

A file describing the target data schema for a squid, normally called `schema.graphql`. The file uses a GraphQL dialect to define entities, properties and relations. See [details here](/en/sdk/squid-sdk/reference/schema-file).

### `State`[​](#state "Direct link to State")

A key-value map defining the internal worldview of an EVM contract of a Substrate runtime at a specific point in time. The consensus algorithm ensures that the honest majority of the nodes agree on the runtime state.

### `Storage`[​](#storage "Direct link to Storage")

On [Substrate](#substrate), a persistent key-value database kept by the chain nodes. It is used to access the current and historical [state](#state). See details on the [Substrate docs page](https://docs.substrate.io/fundamentals/state-transitions-and-storage/)

### `Squid`[​](#squid "Direct link to Squid")

A project consisting of an [ETL](#etl) for extracting and transforming on-chain data (squid processor), and optionally an API to present the data (squid API).

### `Squid processor`[​](#squid-processor "Direct link to Squid processor")

The [ETL](#etl) part of the squid. Extracts on-chain data from an [SQD Network](/en/network/overview) gateway and/or directly from chain RPC, then transforms and optionally enriches it with external data. Saves the result into a target [data sink](/en/sdk/squid-sdk/reference/store).

### `Squid API`[​](#squid-api "Direct link to Squid API")

The data presentation part of the squid. Typically, it's an [OpenReader](#openreader) GraphQL API auto-generated from a schema file.

### `Substrate`[​](#substrate "Direct link to Substrate")

[Substrate](https://substrate.io) is a framework for developing blockchain runtimes. Used to develop the Polkadot, Kusama chains and all the parachains.

### `Substrate Runtime`[​](#substrate-runtime "Direct link to Substrate Runtime")

The code that defines the state transition logic of a blockchain, and by extention its business logic. See details on the [Substrate docs page](https://docs.substrate.io/fundamentals/architecture/).

### `Squid SDK`[​](#squid-sdk "Direct link to Squid SDK")

[Squid SDK](/en/sdk/squid-sdk/overview) is a collection of open-source libraries for building [squids](#squid).

### `Typegen`[​](#typegen "Direct link to Typegen")

A tool generating strongly typed data access classes from a metadata in some format. Squid SDK includes typegen tools:

* for accessing EVM smart contract data based on the contract ABI
* for accessing event, extrinsics and storage data based on [Substrate](#substrate) metadata
* for accessing [ink!](#ink) smart contract data based on the contract metadata

See [this section](/en/sdk/squid-sdk/resources/tools/typegen/generation) for documentation on these tools.

### `WASM`[​](#wasm "Direct link to WASM")

A portable binary code format and an execution environment specification. WASM programs enjoy deterministic outputs and near-native execution speeds, which makes WASM an attractive alternative to EVM.

[Edit this page](https://github.com/subsquid/docs/edit/master/docs/glossary.md)


# Showcase
Source: https://docs.sqd.dev/en/other/showcase

Live products and apps powered by SQD's data infrastructure — showcase of dashboards, terminals, and analytics tools built on the Portal API and Squid SDK.

Discover live products and tools powered by SQD's data infrastructure.

<CardGroup>
  <Card title="Soldexer - Solana Data Room" icon="chart-mixed" href="https://room.soldexer.dev">
    Real-time Solana analytics dashboard with network TPS, validator stats, wallet metrics, and program activity. Powered by SQD Network.
  </Card>

  <Card title="PolyGains - Polymarket Whale Terminal" icon="whale" href="https://polygains.com">
    Polymarket insider and whale trading terminal that monitors unusual activity, detects new wallets and large bets, and tracks on-chain prediction market data.
  </Card>
</CardGroup>