OpenTelemetry tracing

The profiler span tree can be exported as OpenTelemetry traces. Pass opentelemetryProfiler() as the source’s profiler option; every batch then produces a trace with one span per pipeline stage, viewable in Jaeger, Tempo, or any OTLP-compatible backend.

import { opentelemetryProfiler } from '@subsquid/pipes/opentelemetry'

@opentelemetry/api is an optional peer dependency. The exporter setup below additionally uses the OTEL Node SDK:

npm install @opentelemetry/sdk-node @opentelemetry/exporter-trace-otlp-http

Setup

The pipe change is a single option — swap profiler: true for opentelemetryProfiler():

const stream = evmPortalStream({
  // ...
  profiler: opentelemetryProfiler(),
})

The rest is one-time OTEL SDK bootstrap at process startup: wire the OTLP exporter, then flush spans before exit. The complete program:

import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http'
import { NodeSDK } from '@opentelemetry/sdk-node'
import { commonAbis, evmEventDecoder, evmPortalStream } from '@subsquid/pipes/evm'
import { opentelemetryProfiler } from '@subsquid/pipes/opentelemetry'

const sdk = new NodeSDK({
  serviceName: 'my-pipe',
  traceExporter: new OTLPTraceExporter({
    // Jaeger OTLP HTTP endpoint (default port 4318)
    url: 'http://localhost:4318/v1/traces',
  }),
})

sdk.start()

async function cli() {
  const stream = evmPortalStream({
    id: 'jaeger-tracing',
    portal: 'https://portal.sqd.dev/datasets/arbitrum-one',
    profiler: opentelemetryProfiler(),
    outputs: evmEventDecoder({
      range: { from: 'latest' },
      events: {
        transfers: commonAbis.erc20.events.Transfer,
      },
    }),
  })

  for await (const { data } of stream) {
    console.log(data.transfers.length)
  }

  // flush remaining spans before the process exits
  await sdk.shutdown()
}

void cli()

To attach pipe spans to an existing trace (for example, when the pipe runs inside a request handler), pass an OTEL context:

profiler: opentelemetryProfiler(requestContext)

Running Jaeger locally

Jaeger supports OTLP natively since v1.35:

docker run --rm --name jaeger \
  -p 16686:16686 \
  -p 4317:4317 \
  -p 4318:4318 \
  cr.jaegertracing.io/jaegertracing/jaeger:2.15.0

Open http://localhost:16686 and search for the service name you configured (my-pipe above). Each batch appears as a trace; the span hierarchy matches the profiler tree: data fetch, transformers (including named decoder spans), and target stages. Full runnable example: 13.jaeger-tracing.example.ts.

Overview

Pipes SDK

Squid SDK

Alternative clients

Setup

Running Jaeger locally

​Setup

​Running Jaeger locally

Setup

Running Jaeger locally