Deployment workflow

Here we show how to deploy a production-ready indexer ("squid") to SQD Cloud. See Development flow and the rest of the Indexing SDK section for more info on how to develop one of these.

Pre-requisites: Docker, Squid CLI

1. Prepare the squid for deployment

Make sure that the squid is ready for deployment. This includes:

• Verifying that the squid is working as expected locally.
• Ensuring that the squid is free of any major performance issues. See Best practices for guidance.
• Updating the squid.yaml file (a.k.a. "deployment manifest") with the correct values for your use case.
• Setting up SQD Cloud secrets if necessary.

2. Register a SQD account

You can register a SQD account by visiting the SQD cloud. Click Create an account and fill in the required information, or sign in with your GitHub or Google account.

3. Edit the squid.yaml file

3.1. Basic configuration

First, set squid name, description, and other metadata in the header section of the squid.yaml file.

If necessary, configure your build options next. If the defaults - NodeJS v20 and npm for package manager - work for you, just create an empty build section.

Finally, edit the deploy section to specify the deployment options.

The resulting configuration file may look like this:

manifest_version: subsquid.io/v0.1
name: sample-squid

build:

deploy:
  processor:
    cmd: ["sqd", "process:prod"]
  api:
    cmd: ["sqd", "serve:prod"]

3.2. Using addons

SQD provides RPC and database addons that can be used with your squid deployment. Enable your addons in the deploy.addons section.

rpc addon

For real time data you can use the rpc addon. First, open the RPC endpoints tab in the SQD cloud sidebar and copy the URL of the chosen endpoint.

Add it to the .env file:

RPC_ARBITRUM_ONE_HTTP=<endpoint-url>

This allows using the Cloud RPC in local runs. To make the same endpoint available in Cloud, enable the addon in the addons section:

deploy:
  addons:
    rpc:
      - arbitrum-one.http

To use this endpoint in your squid, set the RPC endpoint like so in src/main.ts:

import { assertNotNull } from '@subsquid/util-internal'

export const processor = new EvmBatchProcessor().setRpcEndpoint(
  assertNotNull(
    process.env.RPC_ARBITRUM_ONE_HTTP,
    'Required env variable RPC_ARBITRUM_ONE_HTTP is missing'
  )
);
// ...the rest of the processor configuration

This configuration will use the Arbitrum RPC endpoint provided by SQD.

database addon

You can also opt in to use the database addon. Add postgres: to the deploy.addons section:

deploy:
  addons:
    postgres:

You should also configure the scale section for the addon when deploying to production, e.g.:

scale:
  addons:
    postgres:
      storage: 100G
      profile: medium

3.3. Services

Squids come with a GraphQL service out-of-the-box. You can enable or disable the service by adding or removing the deploy.api section of the squid.yaml file. In the scale section you can also set the scale and number of replicas for the service.

deploy:
  api:
    cmd: ["sqd", "serve:prod"]
scale:
  api:
    profile: large
    # load-balance three replicas
    replicas: 3

3.4. Processor scale

Next, set the scale of the indexer processor. You can set the profile to small, medium, or large.

scale:
  processor:
    profile: medium

3.5. Dedicated deployment

By default, all squids are collocated, meaning that the squid shares resources with other collocated squids. In this case, computing resources might not be available at all times.

danger

We strongly discourage using collocated squids in production.

To deploy a dedicated squid, you need to set the dedicated option to true in the scale section of the squid.yaml file.

scale:
  dedicated: true

3.6. The resulting squid.yaml

Here is an example of a squid.yaml file with all the options set:

manifest_version: subsquid.io/v0.1
name: sample-squid

build:

deploy:
  addons:
    postgres:
    rpc:
      - arbitrum-one.http
  processor:
    cmd: ["sqd", "process:prod"]
  api:
    cmd: ["sqd", "serve:prod"]

scale:
  # dedicated deployment
  dedicated: true
  addons:
    postgres:
      storage: 100G
      profile: medium
  processor:
    profile: medium
  api:
    profile: large
    # load-balance three replicas
    replicas: 3

For all deployment options, check out the deployment manifest page.

4. Set any required secrets

If your squid uses any sensitive data such as a private URL or an access key, you need to store it in a SQD Cloud secret. You can do this by going to the Secrets tab in the SQD cloud sidebar and adding the required values.

Alternatively, use sqd secrets.

5. Deploy the squid

To deploy the squid to the cloud, open Squids in the sidebar and press the Deploy a squid button in the SQD cloud.

You will be prompted to install the Squid CLI if you haven't already. Follow the instructions to install the CLI. Next, set up your auth key as shown in the SQD cloud. Type the squid name to be the same as in the squid.yaml file.

Finally, deploy the squid:

sqd deploy --org <your_organization> <path_to_squid_project_root>

6. Monitor the squid

tip

Take a look at logging page for tips on emitting and reading logs.

After deploying the squid, you can monitor its status in SQD Cloud. You can see the logs, metrics, and other information about the squid in the Cloud dashboard.

Open the monitoring tab in the SQD cloud sidebar to see the status of your squid.

Deployed quids are available in the Squid tab.

You can see memory usage, CPU usage, and other metrics in the monitoring tab. Here is an example of the monitoring tab:

7. Use the squid

If your squid uses a database, you'll have direct access. Take a look at the DB access tab of your squid's card in SQD Cloud console.

If your squids serves a GraphQL API, you'll be able to access it via a deployment-based URL or via a permanent squid-specific URL that redirects to a particular deployment. See Production alias.

What's next?

• See how to update or kill a deployed squid version.
• Learn about organizations and pricing.