Skip to content

Webhooks

With webhooks (API: callbacks), SEC Blast sends each new filing that matches a subscription to your HTTPS endpoint as a signed POST. An endpoint is a URL that receives events; a subscription is a set of filing filters that sends what matches to one endpoint. The routes on this page manage both and take a v3 API key, sent as on every other route (Authentication). A filing in an event has the shape of a filing in the filing stream. Deliveries come from 67.227.64.202: IPv4 only, TCP port 443, https only.

Endpoints

POST /v3/callbacks/endpoints with url, event_types and an optional description creates an endpoint. The url is https on port 443, with a public DNS name that resolves to a public address; IP addresses, user names and passwords, localhost, and names under .local or .internal are refused. The answer, 201, carries the endpoint's signing secret, whsec_ followed by base64; it is shown only in that answer.

Shell
curl -sS -X POST -H "Authorization: Bearer $SECBLAST_API_KEY" \
  -H "Content-Type: application/json" -d @body.json \
  'https://api.secblast.com/v3/callbacks/endpoints'
JSON
{
  "url": "https://hooks.example.com/secblast",
  "event_types": [
    "filing.new"
  ],
  "description": "8-K alerts"
}

GET /v3/callbacks/endpoints lists the endpoints, each with its health and its deliveries of the last 24 hours. PATCH /v3/callbacks/endpoints/{id} changes url, description, event_types or active; GET reads the endpoint and DELETE deletes it, which also deactivates the subscriptions that deliver to it. POST /v3/callbacks/endpoints/{id}/rotate-secret issues a new secret, shown once; the previous secret keeps signing beside it for expire_previous_in_s seconds.

event_types lists the events the endpoint receives:

EventWhen
filing.newA new filing matches a subscription. data.filing is the filing.
filing.updatedA filing already sent changes: documents added or fetched again, made final by the nightly feed, merged with a redelivery, or linked to the filing it amends. data.change says which. Sent to subscriptions with include_corrections.
filing.correctedSEC corrects a filing already sent. data.changes lists the header fields and documents that changed. Sent to subscriptions with include_corrections.
filing.digestThe filing.new events of a subscription with min_interval_s, gathered over its interval: data.events, data.count and data.truncated.
endpoint.verificationThe endpoint's challenge (see below). Sent to every endpoint, whatever its event_types.

Subscriptions

POST /v3/search/subscriptions with name, query and endpoint_id creates a subscription. query takes the filters of GET /v3/search as JSON members: forms, form_types, form_families, ciks, tickers, sics, states and inc_states, plus items for 8-K item codes. Tickers are resolved to CIKs when the subscription is saved. A query with filters makes a filter subscription; a query with none makes a firehose subscription, which receives every new filing. Both deliver filing.new, and the endpoint must list it in event_types.

JSON
{
  "name": "8-K item 5.02",
  "query": {
    "forms": "8-K",
    "items": "5.02"
  },
  "endpoint_id": "cbe_…",
  "include_corrections": false
}
  • include_corrections true also sends filing.updated and filing.corrected for the filings the subscription has sent.
  • min_interval_s sends at most one request per interval: the events of the interval arrive together as one filing.digest.
  • A query with a filing-date window (filed_from, filed_to) or with status gets 400 naming it: a subscription receives new filings.
  • A query with q, doctypes, primary_only=true or entityName gets 400 invalid_parameter naming that parameter: a subscription matches filings by their filing-level fields.
  • POST /v3/search/subscriptions/{id}/test answers with the events the subscription would have sent for recent filings, each with test true. Nothing is delivered.
  • PATCH /v3/search/subscriptions/{id} changes a subscription; GET reads it and DELETE deletes it.

The answers of GET /v3/callbacks/endpoints and GET /v3/search/subscriptions carry limits: for endpoints, and for filter and firehose subscriptions, the plan's limit and the active ones (used). A plan without webhooks gets 403 plan_required; an endpoint or subscription beyond the plan's limit gets 409 limit_reached with kind. The limits of each plan are on the fees page.

Endpoint verification

A new endpoint is unverified, and so is an endpoint whose url changes or that is activated again. SEC Blast sends it an endpoint.verification event, signed like every event, whose data.challenge is 32 hexadecimal characters:

JSON
{
  "type": "endpoint.verification",
  "data": {
    "challenge": "6f1c0d2e9a8b47c3b5e2f1a0d9c8b7a6"
  }
}

The endpoint is verified when it answers 2xx within 10 seconds and the response body, trimmed of surrounding white space, equals the challenge, or the body is JSON whose challenge field equals it. A leading UTF-8 byte order mark is ignored. A body that contains the challenge among other text does not verify. Either of these bodies verifies:

Text
6f1c0d2e9a8b47c3b5e2f1a0d9c8b7a6

{"challenge": "6f1c0d2e9a8b47c3b5e2f1a0d9c8b7a6"}

Until the endpoint is verified, its deliveries are held. They are sent once it is verified, and dropped after 72 hours.

Verify and test

POST /v3/callbacks/endpoints/{id}/verify sends a new challenge and answers 202 with the endpoint. A challenge sent earlier and not yet delivered is dropped, and an answer to it no longer verifies. An inactive endpoint gets 409 endpoint_disabled; PATCH with active true verifies it again.

POST /v3/callbacks/endpoints/{id}/test with type sends one sample event of a type the endpoint takes, signed like any event, with test true and sample data. It answers 200 with the outcome of its one attempt (delivered, http_status, error_class, duration_ms), or 202 when no attempt was made within 12 seconds; the event is then still sent, and the delivery log has the outcome. A test send is not retried and not billed. While another test of the organization runs, a test gets 409 test_running; an unverified or disabled endpoint gets 409 endpoint_unverified or endpoint_disabled.

The two routes share a limit of 6 requests a minute per organization, with a burst of 3. Beyond it they answer 429 rate_limited with the reason verify_test and Retry-After.

Endpoint health

Each endpoint has health.state, and last_24h: its deliveries of the last 24 hours by state.

StateMeaning
unverifiedWaiting for the answer to its challenge. Deliveries are held.
activeVerified. Deliveries are sent.
failingA delivery failed every attempt within one hour, or 10 attempts in a row failed. Deliveries are still sent; one success makes the endpoint active again.
disabledEvery attempt failed for 72 hours, with at least 20 attempts, or the endpoint answered 410. New deliveries are held for 72 hours, then dropped. PATCH with active true makes the endpoint unverified and sends a challenge; once it is verified, the held deliveries are sent in order.

An endpoint set to active false holds nothing: its deliveries are dropped.

Delivery log, retry and replay

GET /v3/callbacks/deliveries lists the organization's deliveries, newest first, with attempt_count, last_status, last_error and, while an attempt is due, next_attempt_at. It filters by endpoint_id, subscription_id, state, type, accession, and from and to on created_at. GET /v3/callbacks/deliveries/{id} adds the payload and every attempt. Deliveries are kept 30 days after their final state. The list pages with cursor and next_cursor (Pagination); a cursor is opaque and is sent back as it was received.

StateMeaning
pendingWaiting for its next attempt.
heldWaiting for a condition, named in held_reason: an unverified or disabled endpoint, the plan, the monthly transfer, or a digest interval.
deliveringAn attempt is in flight.
deliveredThe endpoint answered 2xx.
failedEvery attempt failed, the endpoint answered 410, or a hold ran out; last_error says which.
droppedThe endpoint was inactive, or a hold on an unverified or disabled endpoint ran out.

POST /v3/callbacks/deliveries/{id}/retry queues one more attempt of a failed delivery now and answers 202. The attempt count goes on from where it was. A delivery can be retried this way twice; after that it gets 409 limit_reached. Only a failed delivery that was never billed can be retried; any other gets 409 invalid_state.

POST /v3/callbacks/endpoints/{id}/replay with from, and optionally to, states and types, queues again the failed and dropped deliveries of one endpoint created in that window, oldest first. The window is at most 30 days. It answers 202 with {replayed, truncated, last_created_at}. One call queues at most 10,000 deliveries; when truncated is true, the next call keeps to, states and types and sets from to last_created_at. Billed deliveries, test sends and verifications are never queued again. A replay of the same endpoint within 60 seconds of the last gets 409 replay_running with Retry-After.

JSON
{
  "replayed": 214,
  "truncated": false,
  "last_created_at": null
}

Retries

An attempt fails when the endpoint does not answer 2xx within 10 seconds; redirects are not followed. A delivery is attempted up to 12 times in total, over about 71 hours, and is then failed. A 410 fails the delivery and disables the endpoint at once. Delivery is at least once: every attempt of a delivery carries the same webhook-id, which is also the body's id.

Signatures

Every request is a POST with Content-Type: application/json and the headers of Standard Webhooks:

HeaderValue
webhook-idThe delivery's id, the same on every attempt.
webhook-timestampUnix seconds of this attempt.
webhook-signatureOne or more space-separated v1,<signature> entries.
X-SecBlast-EventThe event type, as the body's type.

The signature is base64(HMAC-SHA256(key, "<webhook-id>.<webhook-timestamp>.<body>")), where the key is the bytes that the base64 after whsec_ decodes to. It is computed over the raw request body bytes: a body parsed and serialized again is a different byte sequence and does not verify. While a rotated secret's previous value still signs, the header has two entries; the request is valid when one of them matches. webhook-timestamp is the time of each attempt, so a receiver can refuse one more than 5 minutes from its own clock, as the examples below do.

The body of every event except endpoint.verification has these fields:

FieldMeaning
idThe webhook-id of the delivery.
typeThe event type.
schema_version1. Within a version, changes only add fields.
emitted_atWhen the event was made.
testtrue for a test send.
subscriptionThe subscription that produced the event, {id, name}; null for a test send.
stream_seqFor filing.* events, the event's position in the filing stream.
dataThe event's data, by type (see the table of events).

Python, with Flask:

Python
import base64, hashlib, hmac, json, os, time
from flask import Flask, request

app = Flask(__name__)
SECRET = os.environ["SECBLAST_WEBHOOK_SECRET"]  # the whsec_ value from the endpoint's creation
KEY = base64.b64decode(SECRET.removeprefix("whsec_"))

def verified(body: bytes, headers) -> bool:
    msg_id, ts = headers.get("webhook-id", ""), headers.get("webhook-timestamp", "")
    if not ts.isdigit() or abs(time.time() - int(ts)) > 300:
        return False
    mac = hmac.new(KEY, f"{msg_id}.{ts}.".encode() + body, hashlib.sha256).digest()
    expected = base64.b64encode(mac).decode()
    entries = headers.get("webhook-signature", "").split()
    return any(e.startswith("v1,") and hmac.compare_digest(e[3:], expected) for e in entries)

@app.post("/secblast")
def secblast_webhook():
    body = request.get_data()  # the raw body bytes, as signed
    if not verified(body, request.headers):
        return "", 400
    event = json.loads(body)
    if event["type"] == "endpoint.verification":
        return event["data"]["challenge"]
    if event["type"] == "filing.new":
        filing = event["data"]["filing"]
        print(event["id"], filing["accession"], filing["form_type"])
    return "", 204

TypeScript, with Express and node:crypto:

TypeScript
import express from "express";
import { createHmac, timingSafeEqual } from "node:crypto";

type Event = { id: string; type: string; data: { challenge?: string; filing?: { accession: string; form_type: string } } };
const secret = process.env.SECBLAST_WEBHOOK_SECRET!; // the whsec_ value from the endpoint's creation
const key = Buffer.from(secret.replace(/^whsec_/, ""), "base64");

function verified(body: Buffer, h: express.Request["headers"]): boolean {
  const id = String(h["webhook-id"] ?? ""), ts = String(h["webhook-timestamp"] ?? "");
  if (!/^\d+$/.test(ts) || Math.abs(Date.now() / 1000 - Number(ts)) > 300) return false;
  const expected = createHmac("sha256", key).update(`${id}.${ts}.`).update(body).digest();
  return String(h["webhook-signature"] ?? "").split(" ").some((e) => {
    const sig = e.startsWith("v1,") ? Buffer.from(e.slice(3), "base64") : Buffer.alloc(0);
    return sig.length === expected.length && timingSafeEqual(sig, expected);
  });
}

const app = express();
// express.raw keeps the body as the bytes that were signed.
app.post("/secblast", express.raw({ type: "application/json" }), (req, res) => {
  if (!verified(req.body, req.headers)) return res.sendStatus(400);
  const event = JSON.parse(req.body.toString("utf8")) as Event;
  if (event.type === "endpoint.verification") return res.type("text/plain").send(event.data.challenge);
  if (event.type === "filing.new") console.log(event.id, event.data.filing?.accession, event.data.filing?.form_type);
  return res.sendStatus(204);
});

Markdown version: /docs/webhooks.md