How to use your dedicated Portal with a key

This guide takes a squid that reads from the public Portal and points it at a dedicated Portal that requires an API key. The data, the queries and the handler stay the same; only the data source configuration changes.

Where are you starting from?

Your squid uses .setGateway() (a v2 squid, @subsquid/solana-stream 0.x). Migrate it to the public Portal first by following Migrate a squid to Portal, then come back here.
Your squid uses .setPortal('https://portal.sqd.dev/...') (@subsquid/solana-stream 1.x). Read on.

Prerequisites

The base URL of your dedicated Portal and an API key for it. See Portal pricing for the plans that come with a key.
@subsquid/solana-stream 1.0.0 or later. All 1.x releases accept the http option shown below and send its headers with every Portal request. The 0.x releases have no .setPortal() at all; that is the v2 case above.

Migration steps

Pass the URL and the key to .setPortal()

Build the dataset URL from your Portal’s base URL and put the key into the x-api-key header. If you followed the Portal migration guide, your .setPortal() call already has an http object; add headers next to retryAttempts:

 const dataSource = new DataSourceBuilder()
   .setPortal({
-    url: 'https://portal.sqd.dev/datasets/solana-mainnet',
+    url: endpointUrl + '/datasets/solana-mainnet',
     http: {
-      retryAttempts: Infinity
+      retryAttempts: Infinity,
+      headers: {
+        'x-api-key': portalKey,
+      },
     },
   })

If you pass the URL as a plain string, switch to the options object form shown above. The dataset slug (solana-mainnet here) is the same as on the public Portal. Portal also accepts the key as Authorization: Bearer <key>; see Presenting an API key.

Passing a PortalClient instance?

If you construct a PortalClient yourself and pass it to .setPortal(), set the same url and http options on its constructor instead.

Read the URL and the key from the environment

Do not hardcode the key. Read both values from environment variables and fail early if either is missing:

import {assertNotNull} from '@subsquid/util-internal'

const endpointUrl = assertNotNull(process.env.PORTAL_URL, 'PORTAL_URL is required')
const portalKey = assertNotNull(
  process.env.PORTAL_KEY,
  `SQD Portal endpoint ${endpointUrl} requires a PORTAL_KEY`
)

@subsquid/util-internal is already installed as a dependency of @subsquid/solana-stream. Add it to your package.json explicitly (npm i @subsquid/util-internal) if you prefer to import only direct dependencies.

Set the variables locally

Add the variables to the .env file of your squid:

.env

PORTAL_URL=https://<your-portal>
PORTAL_KEY=<your-key>

sqd process loads .env automatically. If you run the compiled squid directly, load it with node --require=dotenv/config lib/main.js. Keep .env out of version control once it holds the key.

Set the variables in SQD Cloud

Optional Only for squids deployed to SQD Cloud.Store the key as an organization secret:

sqd secrets set PORTAL_KEY <your-key> -o <organization>

Then expose both variables to the processor in squid.yaml:

squid.yaml

deploy:
  processor:
    cmd: ["sqd", "process:prod"]
    env:
      PORTAL_URL: https://<your-portal>
      PORTAL_KEY: "${{ secrets.PORTAL_KEY }}"

Smoke-test and deploy

Build and start the squid:

npm run build
npx squid-typeorm-migration apply
node --require=dotenv/config lib/main.js

A missing variable stops the squid at startup with the message from assertNotNull. A key that the Portal rejects fails the first request with a 403; the error code tells a missing key from a wrong or expired one (see Error codes).The data served by your dedicated Portal is the same as on the public one, so there is no need to re-sync. Re-deploy the squid without resetting its database.

Complete example

src/main.ts (data source part)

import {DataSourceBuilder} from '@subsquid/solana-stream'
import {assertNotNull} from '@subsquid/util-internal'

const endpointUrl = assertNotNull(process.env.PORTAL_URL, 'PORTAL_URL is required')
const portalKey = assertNotNull(
  process.env.PORTAL_KEY,
  `SQD Portal endpoint ${endpointUrl} requires a PORTAL_KEY`
)

const dataSource = new DataSourceBuilder()
  .setPortal({
    url: endpointUrl + '/datasets/solana-mainnet',
    http: {
      retryAttempts: Infinity,
      headers: {
        'x-api-key': portalKey,
      },
    },
  })
  // .setBlockRange(), .setFields(), .addInstruction() etc. stay as they were
  .build()

The rest of the squid (the run() call and the batch handler) is unchanged. For a full squid to compare against, see the solana-example template.

Pipes SDK

Squid SDK

How to use your dedicated Portal with a key

Where are you starting from?

Prerequisites

Migration steps

Complete example

​Where are you starting from?

​Prerequisites

​Migration steps

​Complete example

Where are you starting from?

Prerequisites

Migration steps

Complete example