Deployment manifest

The deployment manifest is named squid.yaml by convention. The manifest defines how the squid should be built, customized and scaled during the deployment to SQD Cloud. It is used together with sqd deploy to deploy or update an existing squid version.

With the introduction of the deployment manifest the add-ons (postgres) and the API services (api) become optional. This allows flexible deployment configuration for analytic use-cases (e.g. transform and write to s3 or BigQuery).

tip

Cloud secrets of the squid's organization are exposed as the secrets context. You can set any variable defined in any of the env: sections mentioned below:

env:
  RPC_ENDPOINT: ${{ secrets.FAST_RPC_ENDPOINT_URL }}

See the Environment variables page for more info.

The manifest header defines the squid metadata

Field	Description
`manifest_version`	Only `subsquid.io/v0.1` is currently supported
`name`	A globally unique squid name. Can only contain lowercase Latin letters (`a-z`) and a dash (`-`). Must be under 20 symbols.
`version`	Squid version. Must be an integer. A squid deployment is canonically identified as `${name}@${version}`.
`description`	(Optional) A short description of the squid

`build:`

Specifies the way the squid is built into a docker image.

Name	Type	Default value
`node_version`	`18 \| 20 \| 21`	`20`
`package_manager`	`auto \| npm \| pnpm \| yarn`	`auto`
`cmd`	list

For a successful build the following files and folders must be present in the root folder of the squid:

/src
tsconfig.json
package.json
commands.json

The db and assets folders are added to the build context if present in the squid folder. See Project structure for more info.

Under the hood, Cloud builds a Docker image and runs a docker container for each service (api, processor, migrate) using the same image. See Self-hosting for instructions on how to build and run the Docker image locally. Even though the squid services (api, processor, migrate) use the same single container image, the exec command is different and can is defined by the deploy: section as explained below.

`cmd:`

Enables overriding the dependencies installation command, e.g.

build:
  cmd:
    - npm
    - install

The default is to use a shrinkwrap-based installation command appropriate for the detected package manager (e.g. npm ci).

`deploy:`

The deploy section may define:

`addons:`

A list of add-on services to be deployed along the squid services.

postgres:

See Postgres add-on.

rpc

See RPC add-on.

`migrate:`

Optional The init container ran before processor and api services are started. If it exits with an error, squid deployment fails.

Name	Description	Type	Default value	Optional
`cmd`	Exec command passed to the squid container. By default runs TypeORM migrations in the `db` folder	`string[]`	`['npx', 'squid-typeorm-migration', 'apply']`	Optional
`env`	A key-value list of env variables to be set	`{k:v}[]`	[]	Optional

`processor:`

A processor service or a list of processor services of the squid.

Name	Description	Type	Default value	Optional
`cmd`	Processor exec command passed to the squid container	`string[]`		Required
`name`	Processor name	`string`		Required with more than one processor
`env`	A key-value list of env variables to be set	`{k:v}[]`	[]	Optional

With a single processor this section may look like this:

  processor:
    cmd: [ "sqd", "process:prod" ]

For the multiprocessor case:

  processor:
    - name: eth-processor
      cmd: [ "sqd", "process:prod:eth" ]
    - name: bsc-processor
      cmd: [ "sqd", "process:prod:bsc" ]

where process:prod:bsc and process:prod:eth are extra sqd commands defined at commands.json:

commands.json
...
  "process:prod:eth": {
    "deps": ["migration:apply"],
    "cmd": ["node", "lib/eth/main.js"]
   },
  "process:prod:bsc": {
     "cmd": ["node", "lib/bsc/main.js"]
   },
...

`api:`

Optional The GraphQL API service of the squid. Automatically provisions a publicly available endpoint https://{org}.subsquid.io/{name}/v/v{version}/graphql and binds it to the server.

Name	Description	Type	Default value	Optional
`cmd`	GraphQL API server exec command passed to the squid container. The server must be listening at port `GRAPHQL_SERVER_PORT`.	`string[]`		Required
`env`	A key-value list of env variables to be set	`{k:v}[]`	[]	Optional

`env:`

Optional A key-value list of deployment-wide (i.e. visible to all services of the squid) env variables to be set

`scale:`

See the Scale the deployment section.

Examples

A minimal example of manifest is below:

squid.yaml
manifest_version: subsquid.io/v0.1
name: sample-squid
version: 1
description: |-
  My sample squid

build: 

deploy:
  addons:
    postgres: 
  processor:
    cmd: [ "sqd", "process:prod" ]
  api:
    cmd: [ "sqd", "serve:prod" ]

An extended version:

squid.yaml
manifest_version: subsquid.io/v0.1
name: sample-squid
version: 1
description: |-
  My advanced squid

build: 

deploy:
  addons:
    postgres: 
  migrate:
    env:
      FOO: bar
    cmd: [ "echo", "skip migrations!" ]
  processor:
    # static and secret-valued env variables
    env:
      SQD_DEBUG: sqd:mapping
      RPC_ENDPOINT: ${{ secrets.ACALA_RPC_ENDPOINT }}
      COINGECKO_API_KEY: ${{ secrets.COINGECKO_API_KEY }}
    cmd: [ "sqd", "process:prod" ]
  api:
    env:
      SQD_DEBUG: '*'
    # custom run command for the GraphQL server
    cmd: [ "npx", "squid-graphql-server", "--subscriptions", "--max-root-fields", "10", "--sql-statement-timeout", "1000" ]

scale:
  addons:
    postgres:
      storage: 100G
      profile: medium
  processor:
    profile: medium
  api:
    profile: large
    # load-balance three replicas
    replicas: 3

Deployment manifest

Header​

build:​

cmd:​

deploy:​

addons:​

migrate:​

processor:​

api:​

env:​

scale:​

Examples​

Header