Webhooks
With webhooks (API: callbacks), SEC Blast sends each new filing that matches a subscription to your HTTPS endpoint as a signed POST. An endpoint is a URL that receives events; a subscription is a set of filing filters that sends what matches to one endpoint. The routes on this page manage both and take a v3 API key, sent as on every other route (Authentication). A filing in an event has the shape of a filing in the filing stream. Deliveries come from 67.227.64.202: IPv4 only, TCP port 443, https only.
Endpoints
POST /v3/callbacks/endpoints with url, event_types and an optional description creates an endpoint. The url is https on port 443, with a public DNS name that resolves to a public address; IP addresses, user names and passwords, localhost, and names under .local or .internal are refused. The answer, 201, carries the endpoint's signing secret, whsec_ followed by base64; it is shown only in that answer.
curl -sS -X POST -H "Authorization: Bearer $SECBLAST_API_KEY" \
-H "Content-Type: application/json" -d @body.json \
'https://api.secblast.com/v3/callbacks/endpoints'{
"url": "https://hooks.example.com/secblast",
"event_types": [
"filing.new"
],
"description": "8-K alerts"
}GET /v3/callbacks/endpoints lists the endpoints, each with its health and its deliveries of the last 24 hours. PATCH /v3/callbacks/endpoints/{id} changes url, description, event_types or active; GET reads the endpoint and DELETE deletes it, which also deactivates the subscriptions that deliver to it. POST /v3/callbacks/endpoints/{id}/rotate-secret issues a new secret, shown once; the previous secret keeps signing beside it for expire_previous_in_s seconds.
event_types lists the events the endpoint receives:
| Event | When |
|---|---|
filing.new | A new filing matches a subscription. data.filing is the filing. |
filing.updated | A filing already sent changes: documents added or fetched again, made final by the nightly feed, merged with a redelivery, or linked to the filing it amends. data.change says which. Sent to subscriptions with include_corrections. |
filing.corrected | SEC corrects a filing already sent. data.changes lists the header fields and documents that changed. Sent to subscriptions with include_corrections. |
filing.digest | The filing.new events of a subscription with min_interval_s, gathered over its interval: data.events, data.count and data.truncated. |
endpoint.verification | The endpoint's challenge (see below). Sent to every endpoint, whatever its event_types. |
Subscriptions
POST /v3/search/subscriptions with name, query and endpoint_id creates a subscription. query takes the filters of GET /v3/search as JSON members: forms, form_types, form_families, ciks, tickers, sics, states and inc_states, plus items for 8-K item codes. Tickers are resolved to CIKs when the subscription is saved. A query with filters makes a filter subscription; a query with none makes a firehose subscription, which receives every new filing. Both deliver filing.new, and the endpoint must list it in event_types.
{
"name": "8-K item 5.02",
"query": {
"forms": "8-K",
"items": "5.02"
},
"endpoint_id": "cbe_…",
"include_corrections": false
}include_correctionstruealso sendsfiling.updatedandfiling.correctedfor the filings the subscription has sent.min_interval_ssends at most one request per interval: the events of the interval arrive together as onefiling.digest.- A query with a filing-date window (
filed_from,filed_to) or withstatusgets400naming it: a subscription receives new filings. - A query with
q,doctypes,primary_only=trueorentityNamegets400invalid_parameternaming that parameter: a subscription matches filings by their filing-level fields. POST /v3/search/subscriptions/{id}/testanswers with the events the subscription would have sent for recent filings, each withtesttrue. Nothing is delivered.PATCH /v3/search/subscriptions/{id}changes a subscription;GETreads it andDELETEdeletes it.
The answers of GET /v3/callbacks/endpoints and GET /v3/search/subscriptions carry limits: for endpoints, and for filter and firehose subscriptions, the plan's limit and the active ones (used). A plan without webhooks gets 403 plan_required; an endpoint or subscription beyond the plan's limit gets 409 limit_reached with kind. The limits of each plan are on the fees page.
Endpoint verification
A new endpoint is unverified, and so is an endpoint whose url changes or that is activated again. SEC Blast sends it an endpoint.verification event, signed like every event, whose data.challenge is 32 hexadecimal characters:
{
"type": "endpoint.verification",
"data": {
"challenge": "6f1c0d2e9a8b47c3b5e2f1a0d9c8b7a6"
}
}The endpoint is verified when it answers 2xx within 10 seconds and the response body, trimmed of surrounding white space, equals the challenge, or the body is JSON whose challenge field equals it. A leading UTF-8 byte order mark is ignored. A body that contains the challenge among other text does not verify. Either of these bodies verifies:
6f1c0d2e9a8b47c3b5e2f1a0d9c8b7a6
{"challenge": "6f1c0d2e9a8b47c3b5e2f1a0d9c8b7a6"}Until the endpoint is verified, its deliveries are held. They are sent once it is verified, and dropped after 72 hours.
Verify and test
POST /v3/callbacks/endpoints/{id}/verify sends a new challenge and answers 202 with the endpoint. A challenge sent earlier and not yet delivered is dropped, and an answer to it no longer verifies. An inactive endpoint gets 409 endpoint_disabled; PATCH with active true verifies it again.
POST /v3/callbacks/endpoints/{id}/test with type sends one sample event of a type the endpoint takes, signed like any event, with test true and sample data. It answers 200 with the outcome of its one attempt (delivered, http_status, error_class, duration_ms), or 202 when no attempt was made within 12 seconds; the event is then still sent, and the delivery log has the outcome. A test send is not retried and not billed. While another test of the organization runs, a test gets 409 test_running; an unverified or disabled endpoint gets 409 endpoint_unverified or endpoint_disabled.
The two routes share a limit of 6 requests a minute per organization, with a burst of 3. Beyond it they answer 429 rate_limited with the reason verify_test and Retry-After.
Endpoint health
Each endpoint has health.state, and last_24h: its deliveries of the last 24 hours by state.
| State | Meaning |
|---|---|
unverified | Waiting for the answer to its challenge. Deliveries are held. |
active | Verified. Deliveries are sent. |
failing | A delivery failed every attempt within one hour, or 10 attempts in a row failed. Deliveries are still sent; one success makes the endpoint active again. |
disabled | Every attempt failed for 72 hours, with at least 20 attempts, or the endpoint answered 410. New deliveries are held for 72 hours, then dropped. PATCH with active true makes the endpoint unverified and sends a challenge; once it is verified, the held deliveries are sent in order. |
An endpoint set to active false holds nothing: its deliveries are dropped.
Delivery log, retry and replay
GET /v3/callbacks/deliveries lists the organization's deliveries, newest first, with attempt_count, last_status, last_error and, while an attempt is due, next_attempt_at. It filters by endpoint_id, subscription_id, state, type, accession, and from and to on created_at. GET /v3/callbacks/deliveries/{id} adds the payload and every attempt. Deliveries are kept 30 days after their final state. The list pages with cursor and next_cursor (Pagination); a cursor is opaque and is sent back as it was received.
| State | Meaning |
|---|---|
pending | Waiting for its next attempt. |
held | Waiting for a condition, named in held_reason: an unverified or disabled endpoint, the plan, the monthly transfer, or a digest interval. |
delivering | An attempt is in flight. |
delivered | The endpoint answered 2xx. |
failed | Every attempt failed, the endpoint answered 410, or a hold ran out; last_error says which. |
dropped | The endpoint was inactive, or a hold on an unverified or disabled endpoint ran out. |
POST /v3/callbacks/deliveries/{id}/retry queues one more attempt of a failed delivery now and answers 202. The attempt count goes on from where it was. A delivery can be retried this way twice; after that it gets 409 limit_reached. Only a failed delivery that was never billed can be retried; any other gets 409 invalid_state.
POST /v3/callbacks/endpoints/{id}/replay with from, and optionally to, states and types, queues again the failed and dropped deliveries of one endpoint created in that window, oldest first. The window is at most 30 days. It answers 202 with {replayed, truncated, last_created_at}. One call queues at most 10,000 deliveries; when truncated is true, the next call keeps to, states and types and sets from to last_created_at. Billed deliveries, test sends and verifications are never queued again. A replay of the same endpoint within 60 seconds of the last gets 409 replay_running with Retry-After.
{
"replayed": 214,
"truncated": false,
"last_created_at": null
}Retries
An attempt fails when the endpoint does not answer 2xx within 10 seconds; redirects are not followed. A delivery is attempted up to 12 times in total, over about 71 hours, and is then failed. A 410 fails the delivery and disables the endpoint at once. Delivery is at least once: every attempt of a delivery carries the same webhook-id, which is also the body's id.
Signatures
Every request is a POST with Content-Type: application/json and the headers of Standard Webhooks:
| Header | Value |
|---|---|
webhook-id | The delivery's id, the same on every attempt. |
webhook-timestamp | Unix seconds of this attempt. |
webhook-signature | One or more space-separated v1,<signature> entries. |
X-SecBlast-Event | The event type, as the body's type. |
The signature is base64(HMAC-SHA256(key, "<webhook-id>.<webhook-timestamp>.<body>")), where the key is the bytes that the base64 after whsec_ decodes to. It is computed over the raw request body bytes: a body parsed and serialized again is a different byte sequence and does not verify. While a rotated secret's previous value still signs, the header has two entries; the request is valid when one of them matches. webhook-timestamp is the time of each attempt, so a receiver can refuse one more than 5 minutes from its own clock, as the examples below do.
The body of every event except endpoint.verification has these fields:
| Field | Meaning |
|---|---|
id | The webhook-id of the delivery. |
type | The event type. |
schema_version | 1. Within a version, changes only add fields. |
emitted_at | When the event was made. |
test | true for a test send. |
subscription | The subscription that produced the event, {id, name}; null for a test send. |
stream_seq | For filing.* events, the event's position in the filing stream. |
data | The event's data, by type (see the table of events). |
Python, with Flask:
import base64, hashlib, hmac, json, os, time
from flask import Flask, request
app = Flask(__name__)
SECRET = os.environ["SECBLAST_WEBHOOK_SECRET"] # the whsec_ value from the endpoint's creation
KEY = base64.b64decode(SECRET.removeprefix("whsec_"))
def verified(body: bytes, headers) -> bool:
msg_id, ts = headers.get("webhook-id", ""), headers.get("webhook-timestamp", "")
if not ts.isdigit() or abs(time.time() - int(ts)) > 300:
return False
mac = hmac.new(KEY, f"{msg_id}.{ts}.".encode() + body, hashlib.sha256).digest()
expected = base64.b64encode(mac).decode()
entries = headers.get("webhook-signature", "").split()
return any(e.startswith("v1,") and hmac.compare_digest(e[3:], expected) for e in entries)
@app.post("/secblast")
def secblast_webhook():
body = request.get_data() # the raw body bytes, as signed
if not verified(body, request.headers):
return "", 400
event = json.loads(body)
if event["type"] == "endpoint.verification":
return event["data"]["challenge"]
if event["type"] == "filing.new":
filing = event["data"]["filing"]
print(event["id"], filing["accession"], filing["form_type"])
return "", 204TypeScript, with Express and node:crypto:
import express from "express";
import { createHmac, timingSafeEqual } from "node:crypto";
type Event = { id: string; type: string; data: { challenge?: string; filing?: { accession: string; form_type: string } } };
const secret = process.env.SECBLAST_WEBHOOK_SECRET!; // the whsec_ value from the endpoint's creation
const key = Buffer.from(secret.replace(/^whsec_/, ""), "base64");
function verified(body: Buffer, h: express.Request["headers"]): boolean {
const id = String(h["webhook-id"] ?? ""), ts = String(h["webhook-timestamp"] ?? "");
if (!/^\d+$/.test(ts) || Math.abs(Date.now() / 1000 - Number(ts)) > 300) return false;
const expected = createHmac("sha256", key).update(`${id}.${ts}.`).update(body).digest();
return String(h["webhook-signature"] ?? "").split(" ").some((e) => {
const sig = e.startsWith("v1,") ? Buffer.from(e.slice(3), "base64") : Buffer.alloc(0);
return sig.length === expected.length && timingSafeEqual(sig, expected);
});
}
const app = express();
// express.raw keeps the body as the bytes that were signed.
app.post("/secblast", express.raw({ type: "application/json" }), (req, res) => {
if (!verified(req.body, req.headers)) return res.sendStatus(400);
const event = JSON.parse(req.body.toString("utf8")) as Event;
if (event.type === "endpoint.verification") return res.type("text/plain").send(event.data.challenge);
if (event.type === "filing.new") console.log(event.id, event.data.filing?.accession, event.data.filing?.form_type);
return res.sendStatus(204);
});Markdown version: /docs/webhooks.md