52 KiB
PLAN: SMS notifications (/notify-sms)
Status: implemented. Phases 1-15 are code-complete except the four deployment steps noted under phase 12, which need a real Twilio account and a real handset.
Goal
Deliver the daily alertSearch digest by SMS as well as by FCM. A user registers a phone number, proves possession of it with a 6-digit code, authorizes a batch of delegated alertSearch JWTs for the SMS channel, and receives at most one text per local day when that day's retrieval finds updates — at an hour of their choosing, and only until they revoke the authorization.
Every phone action is authorized by a JWT that names the action and the phone number it applies to, and every phone action is recorded.
Scope
In scope: six HTTP endpoints, the middleware chain behind them, five SQLite
tables, a Twilio sender, a per-channel cursor split, and an SMS delivery pass
next to the FCM one. Plus one FCM-side addition: DELETE /notifications/alert-authorization, the twin of the SMS revoke route, because a
user turning alerts off means both channels and only one of them had a way to
say so.
Out of scope: changing how the FCM channel searches or delivers, changing
WAKEUP_PING, MMS, inbound conversational SMS beyond opt-out keywords, and
international sender registration beyond US A2P 10DLC. The notification hour is
an SMS-channel setting; the FCM scheduler does not read it.
Endpoints
Mounted at /notify-sms in src/index.ts, a sibling of /notifications, in a
new src/routes/notifySms.ts.
| Method | Path | Purpose |
|---|---|---|
| GET | /notify-sms/phone |
List this DID's registrations; with ?phoneNumber=, the other DIDs on a number this DID has verified |
| POST | /notify-sms/phone |
Record a phone for the DID, unverified, and text it a 6-digit code |
| PUT | /notify-sms/phone |
Match the code and flip verified to true |
| DELETE | /notify-sms/phone |
Remove the phone entirely |
| POST | /notify-sms/alert-authorization |
Store a delegated JWT batch for the SMS channel |
| DELETE | /notify-sms/alert-authorization |
Remove every SMS batch and JWT for the DID, turning the channel off |
src/index.ts CORS lists ["GET", "POST", "PUT", "OPTIONS"]. DELETE must be
added to that list, or the browser preflight for the delete route fails before
Express ever sees it.
GET /notify-sms/phone
No body. Optional query parameter phoneNumber.
- Without the parameter: this DID's own registrations —
{ phoneNumber, verified, verifiedAt, createdAt }each. Numbers are returned in full; they are the caller's own. - With
phoneNumber: additionallydids, the full DID of every verified registration of that number, only when the calling DID holds a verified registration of it. Otherwise403 SMS_PHONE_NOT_VERIFIED_BY_CALLER, with no count and no identities. - Response
200. A DID with no registrations gets an empty list, not a404.
The verified-registration gate is the same possession rule §Telling the caller who holds the slots applies to PUT: a DID that has verified the number has already proved it holds the handset. A DID that has not, learns nothing — which is why the blocked-at-PUT response carries the list inline rather than telling the caller to come back with a GET they are not yet entitled to make.
POST /notify-sms/phone
Body: { "phoneNumber": "+15555550123" }
- Normalize to E.164. A number that does not normalize is
400SMS_PHONE_INVALID. - If the DID already has this exact number and it is verified, return
200withverified: trueand send nothing. Idempotent, costs no money, and removes the obvious SMS-bombing lever. - Otherwise upsert the
sms_registrationsrow withverified = 0, mint a code, store its HMAC, and send the text. - Code:
crypto.randomInt(0, 1000000)zero-padded to six digits. NotMath.random(). - Stored as
code_hash = HMAC-SHA256(code, SMS_CODE_SECRET). The plaintext code exists in memory and in the outbound message, nowhere else. code_expires_at = now + SMS_CODE_TTL_SEC(default 600).code_attempts = 0.- Send throttle, counted from
sms_phone_log: 3 code sends per phone per hour, 10 per DID per day. Over the limit is429SMS_CODE_RATE_LIMITED. - Response
200:{ success, phoneNumber: "+1555*****23", verified: false, expiresAt }. The response never carries the code.
SMS_DEV_ECHO_CODE exists so a developer with no Twilio account, or no carrier
coverage at the desk, can still exercise POST-then-PUT. With
NODE_ENV=test-local and SMS_DEV_ECHO_CODE=true, the POST response gains
one extra field, devCode, holding the plaintext six digits that would have
been texted. The console SmsSender prints the same message instead of sending
it, so no carrier is involved and no money is spent.
Both conditions are required, and NODE_ENV is checked first. A production
process with SMS_DEV_ECHO_CODE=true set by accident echoes nothing, because
its NODE_ENV is not test-local. Without that pairing the flag is a
one-variable path to handing every verification code back to whoever asked for
it, which defeats the entire point of sending the code out-of-band to a phone
the caller must physically hold.
A DID that wants to re-verify a number it already verified calls DELETE, then POST. That keeps a fumbled re-registration from silently switching off a working alert channel.
PUT /notify-sms/phone
Body: { "phoneNumber": "+15555550123", "code": "483920" }
- Already verified:
200,verified: true, no attempt counted. - No pending code, or
code_expires_atpassed:400SMS_CODE_EXPIRED. code_attempts >= SMS_CODE_MAX_ATTEMPTS(default 5):429SMS_CODE_ATTEMPTS_EXHAUSTED. The code is cleared; recovery is another POST.- Compare with
crypto.timingSafeEqualover the HMACs. - Match:
verified = 1,verified_at,code_hash = NULL,code_attempts = 0. - Miss: increment
code_attempts,400SMS_CODE_MISMATCHwithattemptsRemaining.
DELETE /notify-sms/phone
Body { "phoneNumber": "+15555550123" }, and also accepted as
?phoneNumber=, because a fair number of proxies drop bodies on DELETE.
- Deletes the
sms_registrationsrow matchingWHERE user_id = ? AND phone_e164 = ?. Never the number alone. - Sets
phone_e164 = NULLon that DID'ssms_phone_logrows for that number, leavingphone_hashand the action history intact. "Totally removes the phone" and "keeps an audit trail" are both satisfied: the log says what happened and when, without naming whose number it was. - Response
200{ success: true, deleted: true|false }. Deleting a number that is not registered isdeleted: false, not an error. - Deleting the last verified phone leaves any stored SMS JWT batch in place; the delivery pass finds no verified recipients and sends nothing.
POST /notify-sms/alert-authorization
Body is the FCM twin's shape plus a required UTC hour and an optional zone:
{ batchId, notifyHourUtc, notifyMinuteUtc, timezone?, jwts: [100] }.
- Validation reuses
validateAlertAuthorizationBatch. It verifies each delegated JWT's signature, matchesissto the authenticated DID, requires 100 consecutive sequences and distinct days, and requires each JWT to be valid for the whole UTC day it names —nbfat or before that day's midnight UTC,expat or after the next. The daily run selects by UTC day and may fire at any moment inside it, so a partial window would hand Endorser a credential outside its own validity period. notifyHourUtc(0-23) andnotifyMinuteUtc(0-59) are both required, both integers, and both UTC — the field names carry the frame, so nothing else has to. Absent, out of range, or the wrong type is rejected with the rest of the batch, and one arriving without the other names the missing field. Every batch therefore states its own hour, and no client silently inherits the first tick after midnight UTC — the setting that puts every user on one tick. §Notification hour covers what the value does.timezoneis optional: an IANA name, validated againstIntland stored. Nothing reads it. It may be omitted from a batch that states its hour, but not the reverse. §Notification hour says what it is being kept for.- Requires at least one verified phone for the DID. Without one:
409SMS_NO_VERIFIED_PHONE. Storing 100 credentials for a channel with no reachable address is inventory nobody asked for. - Stores into
sms_alert_authorization_batches/sms_alert_authorization_jwtsthroughsmsAlertAuthorizationDb.replaceUnusedBatch, which isalertAuthorizationDb.replaceUnusedBatchpointed at the SMS tables: drop this user's unused SMS JWTs, drop orphaned SMS batch rows, insert the new batch, in one transaction. - Response mirrors the FCM one plus the stored hour and zone:
{ success, batchId, notifyHourUtc, notifyMinuteUtc, timezone, storedCount, unusedCount }. - The route registers on
POSTand onPUTwith the same handler. The FCM twin isPUT, the semantics are replace-not-append, and an app that reaches forPUTout of symmetry should not get a 404 for its trouble.
DELETE /notify-sms/alert-authorization
No body. The action claim is revoke-alert-search, which binds to no phone
number.
- Removes every SMS batch and every SMS JWT for the DID, consumed rows included, in one transaction, so the SMS scheduler stops listing that identity at all.
- Response
200{ success: true, deletedBatches, deletedJwts }. Nothing stored is zeros, not a404: the caller asked for a state, and that state is what they get. - Registered phone numbers survive. Silencing alerts is not a request to redo
the possession check on return;
DELETE /notify-sms/phoneis the route that forgets a number, andSTOPis the route that blocks one. - The alertSearch cursors survive too, so a later re-authorization resumes where this one stopped rather than replaying months of history.
- Logged as
alert-authorization-deleted. For a DID whose handset is already gone there is no number to log against, sophone_hashholds a hash of the identity instead — the column isNOT NULLand the revocation is worth a row.
DELETE /notifications/alert-authorization is the same operation on the FCM
inventory, authorized the way its PUT twin is: Bearer JWT plus the Endorser
check, no action claim. Device registrations and WAKEUP_PING are untouched.
One phone, several DIDs
Nothing keys off a phone number alone except carrier opt-out. Every other read
and write is scoped by (user_id, phone_e164), so two identities sharing one
handset stay independent:
| Action by DID B | Effect on DID A's registration of the same number |
|---|---|
| POST (register) | None. DID B gets its own row, verified = 0, its own code. |
| PUT (verify) | None. The code is matched against DID B's row only. |
| DELETE | None. The WHERE clause carries both columns. |
| DELETE's log scrub | None. Only DID B's sms_phone_log rows are nulled. |
| Daily alert send | None. Recipients are that DID's own verified rows. |
Verification is per (DID, phone) and has to be. Possession of the handset is
what the code proves, and DID B has not proved it by watching DID A do so.
Three consequences worth stating rather than discovering:
- Two verified DIDs on one handset receive two texts a day, one per identity, because each identity runs its own alertSearch against its own cursor and sees its own results. The per-DID daily cap does not merge them.
- The per-phone send throttle is deliberately cross-DID. Three code sends
per hour is counted from
phone_hash, not fromuser_id, because a per-identity counter is trivially defeated by minting more identities. Two legitimate registrations minutes apart both fit; a bombing run does not. - DELETE forgets a number per DID;
STOPforgets it everywhere. A carrier opt-out arrives with a phone number and no identity attached, and stopping traffic to that handset is not optional, so phase 9 setsverified = 0on every registration of that number regardless of DID. That is the single place in the design where the DID boundary is crossed on purpose.
The cap
SMS_MAX_DIDS_PER_PHONE bounds how many DIDs one handset can carry. Default 5,
configurable up or down without a code change.
The cap counts verified rows only. Counting every row would hand an attacker a registration lock: five POSTs from five throwaway DIDs, never verified, and the handset's actual owner can no longer register it. Unverified rows cost nothing to hold and cannot receive an alert, so they are not what the cap is protecting against.
Checked in two places:
- POST, as an early rejection when the number already has
SMS_MAX_DIDS_PER_PHONEverified registrations:409 SMS_PHONE_DID_LIMIT. Cheaper and clearer than letting someone verify a code and then be told no. - PUT, immediately before flipping
verifiedto 1, counting rows other than this one. This is the check that actually holds: several registrations can clear the POST check while the count sits under the limit, and PUT is the moment a row starts consuming a slot. Same409 SMS_PHONE_DID_LIMIT.
The count reads the phone_e164 index, filtered on verified = 1.
What stops unverified rows from accumulating is not this cap but the per-phone send throttle: every POST that creates one also sends a text, and three texts per phone per hour is the actual brake. The cap governs how many identities can be reached at one number; the throttle governs how fast anyone can try.
Telling the caller who holds the slots
A blocked caller needs to know which DIDs occupy the number, and in the ordinary case the answer is "your own other identities, which you have lost track of." Disclosing that list is gated on proof of possession of the handset, never on the request alone.
| Rejection point | Body |
|---|---|
| POST at the limit | 409 SMS_PHONE_DID_LIMIT, limit and verifiedCount. No identities. |
| PUT at the limit | 409 SMS_PHONE_DID_LIMIT, limit, verifiedCount, and dids: the full DID of every verified registration of that number. |
The asymmetry is the whole design. A POST names any phone number on earth and proves nothing about it. If it answered with identities, the endpoint would be a phone-number-to-identity lookup oracle: feed it numbers, harvest the DIDs of whoever holds them, authenticated by nothing more than possessing some DID of one's own. A PUT that reaches the limit check has already matched a correct six-digit code, which means the caller is holding the handset and can read every message sent to it. Nothing is disclosed that possession did not already grant.
The code is consumed on a limit rejection exactly as it is on success. One code
buys one answer; a fresh answer costs a fresh POST, which the per-phone throttle
meters at three per hour. The rejection is logged as did-limit-disclosed in
sms_phone_log.
sms_action_jwt_use is keyed on the token hash, so two identities acting on one
number never collide there.
Middleware chain
Four stages, in this order, on all five routes:
requireAuth— existing. Bearer JWT, signature verified against the issuer DID,req.did/req.jwtset.requireEndorserAuth— existing.GET {ENDORSER_URL}/api/report/rateLimitswith the same token. This is the "rateLimits" stage: it proves Endorser knows and accepts this DID.requireSmsActionJwt(action)— added by this plan. Confirms the verified token authorizes this specific action on this specific phone.- The route handler itself — the Express callback that does the work
described under §Endpoints: normalize, read and write SQLite, call the
SmsSender, writesms_phone_log, send the response. It is listed as a stage because Express treats it as one more function in the same chain, and because of what that ordering guarantees: by the time the handler's first line runs, the caller is authenticated, accepted by Endorser, and proven to have authorized this exact action on this exact number. The handler performs no authorization checks of its own and never re-readsreq.headers; it trustsreq.didand the validated body, and nothing else.
Stages 1–3 either call next() or send a response and return. A stage that
sends a response ends the chain, so no later stage — the handler included —
runs. That is the whole mechanism preventing an unauthorized request from
reaching a sendSms call.
testMode has no bypass here. There is no local-test path that sends real texts
to real handsets on someone else's dime.
Stage 3, in detail
There is one JWT. The client mints it with the claim inside, sends it as the Bearer token, and the signature over that token is the authorization — the same arrangement Endorser uses. Nothing is sent alongside it.
decodeAndVerifyJwt already returns { issuer, payload, verified }, and
requireAuth keeps only issuer (as req.did) and the raw token string (as
req.jwt), discarding the decoded payload. Stage 3 needs the claim out of that
payload, so requireAuth stops discarding it: req.auth = { did, jwt, payload }.
payload is the verifier's own output for that one token, not a second
document.
Stage 3 reads that object and does not decode the token again. For did:peer
JWANT identities, re-decoding would be wrong, not merely wasteful: the outer
payload of a passkey token is a WebAuthn envelope
(AuthenticationDataB64URL, ClientDataJSONB64URL), and the actual claim sits
base64url-encoded inside clientData.challenge. peerVerifyJwt unwraps it and
that inner object is what decodeAndVerifyJwt returns. A stage that re-parsed
segment two would find the envelope and no claim at all, failing every passkey
user.
Stage 3 does not compare an issuer against req.did. requireAuth assigns
req.did from the verified issuer, so the two are the same value read twice —
a check that cannot fail proves nothing. Identity comes from payload.iss, and
the claim carries no DID of its own to disagree with it.
The one structural guard stage 3 does keep is that req.auth is present.
requireAuth sets it only after verified.verified is true, so its presence
means verification happened. Its absence means the route was mounted without
stage 1, or mounted with requireAuthOrNotificationLocalTest, whose testMode
path sets a synthetic req.did and no req.auth. Either is a wiring mistake
that must not fall through to a handler that sends texts. It is a presence
check, not a second verification, and it carries no timestamp — the signature
already did the verifying.
Checks, each with its own error code:
| Check | Failure |
|---|---|
req.auth present |
500 — route wiring bug, not a client error |
payload.claim is an object |
403 SMS_ACTION_JWT_MISSING_CLAIM |
claim.action equals the route's action |
403 SMS_ACTION_JWT_WRONG_ACTION |
claim.phoneNumber normalizes equal to the body's phone (phone routes) |
403 SMS_ACTION_JWT_PHONE_MISMATCH |
payload.iat within SMS_ACTION_JWT_MAX_AGE_SEC (default 300) |
401 SMS_ACTION_JWT_STALE |
payload.exp, when present, not passed |
401 SMS_ACTION_JWT_EXPIRED |
sha256(jwt) absent from sms_action_jwt_use |
401 SMS_ACTION_JWT_REPLAYED |
On success the hash is inserted into sms_action_jwt_use before the handler
runs, so one token buys one action. A handler that fails afterward does not
release the hash; the client mints a fresh JWT, which it can do freely.
Action claim contract
The app mints the Bearer JWT with a claim this service defines and consumes:
{
"iss": "did:ethr:0x…",
"iat": 1756270800,
"exp": 1756271100,
"claim": {
"@context": "https://giftopia.tech",
"@type": "SmsNotificationAction",
"action": "register-phone",
"phoneNumber": "+15555550123"
}
}
action is one of list-phones, register-phone, verify-phone,
delete-phone, authorize-alert-search, revoke-alert-search. phoneNumber
is required for register-phone, verify-phone and delete-phone, and
optional for list-phones — when the request carries the query parameter, the
claim must carry the matching number
and is the only binding the claim carries. The claim holds no DID — the
authenticated identity is payload.iss — and no batchId.
A batch upload or revocation is bound by action alone. Both act on the DID's
whole inventory rather than on one handset, so there is no number to bind to. Binding it to a batch id would add
nothing: the client invents the id, and the 100 delegated JWTs in the body must
each verify against the authenticated DID, so a stolen bearer token cannot
upload a batch it did not already have the user's signing key to produce. The
worst a replay achieves is replacing that user's own unused batch with that
user's own other batch, which is the endpoint's declared semantics anyway. The
body's batchId is recorded on the alert-authorization-stored row in
sms_phone_log, which is where an id useful for tracing belongs.
@context is https://giftopia.tech with no trailing path, and it is the same
context for the FCM setup claim as for the SMS one. @type is what separates
them, so one context URL covers both channels and any later setup claim without
minting a new namespace per endpoint.
This is a client change. The TimeSafari app sends a plain identity JWT to
/notifications/* and mints no such claim. A /notify-sms call carrying no
claim returns SMS_ACTION_JWT_MISSING_CLAIM.
SMS_REQUIRE_ACTION_CLAIM is the switch for that stage, and it defaults to
false for one reason only: stage 3 is built in phase 11, after the routes it
guards. Between phase 5 and phase 11 the routes exist with the check absent, and
the flag names that gap instead of hiding it. It is not a compatibility flag —
no deployed client calls /notify-sms, so there is nothing to stay compatible
with. SMS_ENABLED (also false by default) keeps the whole surface off the
public internet during that window; the two flags together mean a half-built
/notify-sms cannot be reached by anyone.
Phase 12 flips the default to true. A flag that weakens authorization and has
no expiry date will be found years from now, still false, in a config nobody
has opened since. Aside: "temporary" is the longest-lived word in software,
right after "TODO".
Data model
All tables are added to SCHEMA_SQL in src/db/sqlite.ts with
CREATE TABLE IF NOT EXISTS. No existing table is altered and no data is
migrated.
sms_registrations
| Column | Notes |
|---|---|
id |
TEXT PK, randomUUID() |
user_id |
authenticated DID |
phone_e164 |
normalized number |
verified |
INTEGER 0/1, default 0 |
code_hash |
HMAC of the pending code, NULL once verified |
code_expires_at |
ISO string, NULL once verified |
code_attempts |
INTEGER, default 0 |
last_code_sent_at |
ISO string |
verified_at |
ISO string |
created_at, updated_at |
ISO strings |
UNIQUE (user_id, phone_e164). Indexes on user_id, on phone_e164 (opt-out
keyword lookup arrives with the number, not the DID), and on
(user_id, verified).
sms_phone_log
| Column | Notes |
|---|---|
id |
TEXT PK |
user_id |
DID |
phone_e164 |
nullable; nulled by DELETE |
phone_hash |
HMAC of the number, never nulled |
action |
see list below |
result |
ok / rejected / failed |
detail |
short reason, no code and no full JWT |
jwt_hash |
sha256 of the authorizing token, when there was one |
provider_message_id |
Twilio SID for sends |
created_at |
ISO string |
Actions: register-requested, code-sent, code-send-failed,
verify-succeeded, verify-failed, did-limit-blocked,
did-limit-disclosed, deleted, alert-authorization-stored,
alert-authorization-deleted, alert-sent, alert-send-failed,
recipient-not-allowed, number-blocked, number-unblocked, opt-out.
did-limit-blocked records a POST refused at the limit; did-limit-disclosed
records a PUT refused at the limit, where the DID list left the building.
Indexes on (user_id, created_at), (phone_hash, created_at),
(action, created_at). The throttle counts read the second one.
sms_action_jwt_use
id, jwt_hash (UNIQUE), user_id, action, used_at. Rows older than
SMS_ACTION_JWT_MAX_AGE_SEC × 10 are pruned on each SMS scheduler pass; a token
that stale fails the freshness check anyway, so the row has nothing left to
protect.
sms_alert_authorization_batches / sms_alert_authorization_jwts
Column-for-column mirrors of alert_authorization_batches /
alert_authorization_jwts, including the partial unique index on
(user_id, day) WHERE status = 'unused'. day is a UTC calendar day. Both
batch tables carry nullable notify_hour_min_utc and timezone because one store
implementation writes both; only the SMS side ever fills them in. timezone
carries its purpose as a SQL comment, which SQLite preserves verbatim in
sqlite_master, so .schema shows the reason beside the column.
sms_alert_search_cursors
user_id PK, endorser_after_id, partner_after_at, created_at,
updated_at. Same shape as alert_search_cursors.
This table is not optional decoration. alert_search_cursors holds one row per
DID. Two independent daily runs against one cursor row means the first run
advances past the delta and the second sees an empty result — the FCM digest and
the SMS digest would disagree at random depending on which timer fired first.
Separate JWT inventories force separate cursors.
Daily send path
Per-channel plumbing
Introduce AlertSearchChannel = "fcm" | "sms", defaulting to "fcm"
everywhere, so no existing call site or test changes.
src/alertSearch/cursors.ts:loadAlertSearchCursors(userId, channel = "fcm")andadvanceAlertSearchCursors(userId, result, channel = "fcm")pick the table from a{ fcm: "alert_search_cursors", sms: "sms_alert_search_cursors" }map.src/alertSearch/cycle.ts: threadschannelthrough to the cursor calls.src/alertSearch/daily.ts:runDailyAlertSearch(userId, now, cycleInput, channel = "fcm")selects the JWT inventory —alertAuthorizationDborsmsAlertAuthorizationDb— from the same channel value. Selection, retrieval, digest building, and consume-only-when-both-sources-complete are otherwise untouched.
src/alertSearch/smsNotify.ts
deliverAlertSearchSms(result) mirrors deliverAlertSearchNotification:
- Eligibility is the identical predicate:
consumed && digest.completed && digest.hasUpdates && digest.totalCount > 0. Consumption of the day's SMS JWT is what makes later ticks on the same local day no-ops, so no separate "already texted today" flag is needed. - Recipients:
sms_registrationsrows for the DID withverified = 1, deduplicated by number. - Body:
Gift Economies: you have N new updates. https://giftopia.meplusReply STOP to end.Kept inside 160 GSM-7 characters — a second segment is a second charge for the privilege of a longer sentence. The link ishttps://giftopia.me;https://giftopia.techis the claim@contextnamespace and never appears in a message. - Each send writes
alert-sentoralert-send-failedtosms_phone_logwith the provider message id. Send failures are logged and do not roll back cursor advancement or JWT consumption, matching the FCM path.
src/alertSearch/smsScheduler.ts
startSmsAlertSearchScheduler(), started from src/index.ts alongside the
other two, with its own interval and its own process-local in-flight guard.
- Selects users with one query per pass,
listPendingForDay({ day, hourMinute }): everyone holding an unused JWT for the current UTC day, each flagged by whether theirnotify_hour_min_utchas arrived. A user who has already run that day holds no unused JWT for it and does not appear (§Scheduler selection). - Calls
runDailyAlertSearch(userId, now, {}, "sms")on the due ones, at mostALERT_SEARCH_USER_CONCURRENCYin flight, thendeliverAlertSearchSms. - Interval
SMS_ALERT_SEARCH_INTERVAL_MS, default 5 minutes, with a 150-second initial offset so the SMS pass and the FCM pass do not hit Endorser in the same instant. - Logs
[SmsAlertSearchScheduler] Pass started/Pass completed in …, matching the existing prefix convention.
Scheduler selection
Both passes choose their users in SQL rather than by asking about each one.
listPendingForDay joins the newest batch per user — picked with a window
function, because a batch holding a consumed JWT survives the next upload, so
"the user's notify time" is not a plain join — against that day's unused JWTs,
and returns a due flag per row. Two properties fall out: a user who has
already run today is absent rather than queried and discarded, and deferred
stays countable because held users still appear.
The comparison is text, which is chronological only because HH:MM is
zero-padded. A stored value that is not HH:MM would sort above every real one
and defer that user forever, so the query guards it with GLOB and treats an
unreadable value as due — the same fail-open the column's own documentation
promises.
Due users are worked ALERT_SEARCH_USER_CONCURRENCY at a time. The per-user
work is two external round trips against independent cursors, so a serial loop
spends the pass waiting rather than working; measured at 50ms of latency, a
2000-user pass falls from ~106s to ~13s, which is the difference between fitting
in a five-minute tick and being skipped by the in-flight guard. The bound stays
low on purpose: the ceiling is latency, and Endorser and Partner are shared
infrastructure that a wide fan-out would only move the queue into.
Notification hour
A batch's notify_hour_min_utc is the hour the user asked to hear from the
service, sent as two UTC integers and stored zero-padded as one HH:MM value.
The SMS scheduler holds that user's whole daily pass until the clock reaches it.
A batch cannot omit the hour, so this gate applies to every user rather than to the subset who expressed a preference. That is deliberate: the alternative default is "the first tick after midnight UTC", which is one tick for everybody.
The gate sits ahead of the search, not ahead of the text. Running the search
consumes that UTC day's JWT, and isAlertSearchSmsEligible fires only for the
run that consumed it, so a run at the top of the day would spend the credential
and reach the chosen hour with nothing left to send.
The hour lives on the batch rather than inside the delegated JWTs because those
are the alertSearch credential: their nbf/exp bound a whole UTC day, and
narrowing them to an hour would narrow when the search may run against Endorser,
not when the user hears about it.
The stored value is an instant of the UTC day, not a wall clock, so it does not
follow the user through a daylight-saving change: a Denver user wanting 18:00
local sends 00:30 UTC in summer, and when that region returns to -07:00 the
same UTC instant reads 17:00 locally. The correction available today is a fresh
batch carrying the new UTC hour, which a client uploads roughly every 100 days
anyway.
The batch's optional timezone is recorded against a mechanism that would close
that gap sooner by re-deriving notify_hour_min_utc from the zone's rules. None
runs, and nothing reads the column. Two candidates are open, and §Rejected
records what is known about each: a job that sweeps zones whose rules changed,
or a stored next-firing instant that each send recomputes, which needs no
scheduled job. Measured cost is the reason neither is urgent — a formatter
cached per zone resolves a user's local wall clock in about 1.4µs, half the cost
of the per-user SQLite lookup the pass already performs.
There is no upper bound within the day. A service that was down at the chosen
hour and comes back six hours later still runs that UTC day; a silent day is the
worse failure. The text lands on the first tick at or after the hour, so within
one SMS_ALERT_SEARCH_INTERVAL_MS in the ordinary case. An hour late in the UTC
day leaves a correspondingly short window before the day key rolls.
Every batch that arrives through the route carries an hour, so the ungated path
is a floor rather than a setting: a row whose notify_hour_min_utc is NULL or
unreadable is treated as due, because a value nobody can read must not silence a
channel the user asked for. Only a write that bypasses the route can produce
one.
Cost of two inventories
A user on both channels produces two Endorser and two Partner queries per day and needs 200 minted JWTs. Both channels report the same underlying updates from their own cursor, so the counts can differ transiently when one channel's retrieval fails and the other's succeeds. This is inherent to separate inventories; §Rejected records the shared-inventory alternative that avoids it.
Provider
src/services/smsService.ts defines the port:
export type SmsSendResult =
| { status: "sent"; messageId: string }
| { status: "failed"; error: string };
export type SmsSender = (to: string, body: string) => Promise<SmsSendResult>;
Default implementation is Twilio over plain fetch against
https://api.twilio.com/2010-04-01/Accounts/{SID}/Messages.json with HTTP Basic
auth, form-encoded body, To / From (or MessagingServiceSid) / Body. No
twilio SDK dependency: the repo already talks to Endorser and Partner with
fetch, and the one call needed here is a form POST.
Configuration absent or incomplete: sends return
{ status: "failed", error: "SMS_NOT_CONFIGURED" } and log once at startup. The
process boots and FCM keeps working. A texting outage should not take
push down with it.
NODE_ENV=test-local with no Twilio credentials uses a console adapter that
prints the message instead of sending it.
Configuration
New variables, added to .env.example and the README environment table:
| Variable | Required | Default |
|---|---|---|
SMS_ENABLED |
no | false — routes return 503 SMS_DISABLED, scheduler does not start |
SMS_CODE_SECRET |
yes when enabled | none — HMAC key for code and phone hashes; absent is a startup failure |
TWILIO_ACCOUNT_SID |
yes when enabled | none |
TWILIO_AUTH_TOKEN |
yes when enabled | none |
TWILIO_FROM_NUMBER or TWILIO_MESSAGING_SERVICE_SID |
one of the two | none |
SMS_CODE_TTL_SEC |
no | 600 |
SMS_CODE_MAX_ATTEMPTS |
no | 5 |
SMS_ACTION_JWT_MAX_AGE_SEC |
no | 300 |
SMS_MAX_DIDS_PER_PHONE |
no | 5 |
SMS_ALERT_SEARCH_INTERVAL_MS |
no | 300000 |
SMS_REQUIRE_ACTION_CLAIM |
no | false until phase 12, true after |
SMS_DEV_ECHO_CODE |
no | false; honored only under NODE_ENV=test-local |
Rotating SMS_CODE_SECRET invalidates every pending code and orphans every
stored phone_hash. Rotate between deploys, not casually.
Files
Placement follows the layer each file belongs to, not the feature it serves.
src/ is organized by role — routes, middleware, db, services, util,
models, vc — and src/alertSearch is the one pipeline folder, which the SMS
delivery path joins rather than duplicates. No src/sms directory.
New:
| File | Why here |
|---|---|
src/routes/notifySms.ts |
HTTP surface, beside notifications.ts |
src/middleware/smsActionJwt.ts |
Request-chain stage, beside auth.ts |
src/db/smsRegistrationsSqlite.ts |
Table access, beside fcmTokensSqlite.ts |
src/db/smsPhoneLogSqlite.ts |
Table access |
src/db/smsAlertAuthorizationSqlite.ts |
Table access, mirrors alertAuthorizationSqlite.ts |
src/services/smsService.ts |
Outbound transport with credentials, the SmsSender port and its Twilio adapter — the SMS counterpart of firebase.ts plus pushService.ts |
src/util/smsPhoneNumber.ts |
Pure string work: E.164 normalization, masking. Same shape as maskToken.ts |
src/util/smsVerificationCode.ts |
Pure crypto: mint, HMAC, timing-safe compare. Takes the secret as an argument so it stays testable; the caller reads SMS_CODE_SECRET from env.ts |
src/models/smsRegistration.ts |
Row-shape interfaces, beside device.ts |
src/alertSearch/smsNotify.ts |
Digest-to-message orchestration, the exact twin of alertSearch/notify.ts |
src/alertSearch/smsScheduler.ts |
Interval pass, the exact twin of alertSearch/scheduler.ts |
Tests for this work sit in test/, mirroring the src/ layout
(test/routes/notifySms.test.ts for src/routes/notifySms.ts). The existing
src/alertSearch/*.test.ts files stay where they are.
The split between services/smsService.ts and alertSearch/smsNotify.ts copies
the split the FCM path already uses: services/pushService.ts knows how to send
one message and nothing about why, while alertSearch/notify.ts knows which
users deserve a message and what it should say. Twilio credentials never appear
above the services layer, and digest logic never appears below it.
Modified:
src/db/sqlite.ts— five tables, their indexes, andnotify_hour_min_utcin place oftimezoneon both alert-authorization batch tablessrc/db/alertAuthorizationSqlite.ts—notify_hour_min_utcthrough the shared store,timezonedropped, plusdeleteAllForUsersrc/services/alertAuthorization.ts— notify-hour and UTC-day JWT validation, and the clock helpers (formatHourMinuteUtc,parseHourMinuteUtc,storedNotifyLabel,utcCalendarDay,utcDayStartSeconds,utcHourMinute)src/alertSearch/daily.ts— selects by UTC day and reportsutcDay;InvalidAlertAuthorizationTimezoneErrorremovedsrc/alertSearch/scheduler.ts— set-based selection, bounded concurrency, and the notify hour on the push channelsrc/util/concurrency.ts— the bounded pool both passes run users throughsrc/routes/notifications.ts— the FCM twin of the revoke routesrc/middleware/auth.ts— carry the decoded payload onreq.authsrc/types/express.d.ts— the widenedauthtypesrc/env.ts— the SMS variables, read the same wayENDORSER_URLissrc/index.ts— mount/notify-sms, addDELETEto CORS, start the SMS schedulersrc/alertSearch/cursors.ts,cycle.ts,daily.ts— channel parameterpackage.json— test globREADME.md,.env.example,CHANGELOG.md
Phases
- 1. Schema and db modules. Five tables in
SCHEMA_SQL;smsRegistrationsSqlite,smsPhoneLogSqlite,smsAlertAuthorizationSqlite. Tests: insert/read round trips, the(user_id, phone_e164)uniqueness, the partial unique index on unused SMS JWTs. - 2. Phone and code utilities.
util/smsPhoneNumber.tsandutil/smsVerificationCode.ts: E.164 normalization, masking, code mint, HMAC, timing-safe compare. Tests: normalization table including the rejections, code is always six digits, compare does not short-circuit. - 3. Stop discarding the decoded payload.
requireAuthsetsreq.auth.payloadfrom the valuedecodeAndVerifyJwtalready returns;src/types/express.d.tsupdated. No behavior change to existing routes. - 4. Twilio sender.
SmsSenderport, Twilio adapter, console adapter, unconfigured path. Tests use an injected sender; no test touches the network. - 5. Phone routes. GET / POST / PUT / DELETE wired to stages 1–2,
mounted in
index.ts,DELETEadded to CORS. EnforceSMS_MAX_DIDS_PER_PHONEon both POST and PUT, counting verified rows. Tests: happy path, wrong code, expired code, attempt exhaustion, re-POST on a verified number sends nothing, DELETE nullsphone_e164in the log but keepsphone_hash, the sixth DID is refused at PUT even when its POST was accepted under the limit, and unverified rows from other DIDs do not count toward it. Disclosure tests carry their own weight: a POST rejected at the limit returns nodids, a PUT rejected at the limit returns them, a GET with?phoneNumber=returns them only to a DID verified on that number, and the code is consumed either way so a second PUT with the same code cannot re-ask. - 6. SMS alert-authorization route. POST plus the PUT alias, reusing
validateAlertAuthorizationBatch, requiring a verified phone, storing to the SMS tables. Tests: batch of 100 stored, second batch replaces unused rows and leaves consumed ones,409with no verified phone. - 7. Per-channel cursors and daily run. Channel parameter through
cursors.ts,cycle.ts,daily.ts, defaulting to"fcm". Tests: an SMS run advances onlysms_alert_search_cursors; the FCM suite passes unchanged. - 8. Delivery and scheduler.
alertSearch/smsNotify.tsandalertSearch/smsScheduler.ts, started inindex.ts. Tests: eligibility predicate, verified-only recipients, one send per number, failures logged without rolling back consumption, second tick the same day sends nothing. - 9. Opt-out and caps.
POST /notify-sms/inboundfor Twilio's webhook:STOP/UNSUBSCRIBEmarks every registration for that numberverified = 0and logsopt-out;STARTrequires a fresh POST + code;HELPreturns a fixed reply. Enforce the per-DID daily send cap. The webhook authenticates by Twilio'sX-Twilio-Signature, not by JWT — it is Twilio calling, not a user. - 10. Docs. README sections for the endpoints, the action claim, the SMS
tables, and the environment table;
.env.example;CHANGELOG.md. - 11.
requireSmsActionJwt. The action-authorization stage, built last. Every check in the stage-3 table, each with its own code, plus the replay insert and thesms_action_jwt_usetable. Add the middleware to all five routes. Tests: one per failure mode, plus the same token rejected on second use. - 12. Wrap-up.
- Prefix every line this service prints with an ISO-8601 UTC timestamp.
A
src/util/log.tswrapper (log.info/log.error) that prependsnew Date().toISOString()and forwards toconsole, applied to every existingconsole.log/console.errorcall site insrc/— routes, middleware, services, db modules, and all three schedulers. A log line without a timestamp cannot answer "when", which is the only question anyone asks a log at 3am. - Flip
SMS_REQUIRE_ACTION_CLAIMto defaulttrueand record the flip inCHANGELOG.md. Phase 11 is what makes that default safe. - Turn
SMS_ENABLEDon in the deployment environment, with Twilio credentials,SMS_CODE_SECRET, and completed A2P 10DLC registration all in place. - End-to-end pass against a real handset: register, verify, authorize a
batch, receive one daily digest text,
STOP, confirm no further sends. - Confirm
sms_phone_logholds one row per action taken during that pass, and that DELETE nulledphone_e164while leavingphone_hash. - Confirm
SMS_DEV_ECHO_CODEis unset (orfalse) andNODE_ENVis nottest-localin the deployed environment. - Prune-job follow-up filed for
sms_phone_logretention andsms_action_jwt_userows.
- Prefix every line this service prints with an ISO-8601 UTC timestamp.
A
- 13. Notification hour and revocation.
notify_hour_min_utcon both batch tables and throughvalidateAlertAuthorizationBatch;isNotifyTimeReachedgating the SMS pass ahead of the search;deleteAllForUseron the shared store behindDELETE /notify-sms/alert-authorization(actionrevoke-alert-search) andDELETE /notifications/alert-authorization. Tests: the hour/minute validation and its ranges; a pass that defers before the hour, runs after it, and closes again when the UTC day rolls; a batch with no hour running unchanged; a failing due-check counted as failed rather than deferred; the round trip of a stored hour; deletion of consumed and unused rows together while the other channel and the other DIDs are untouched; the verified phone surviving a revocation; and the wrong action claim refused. - 14. Drop the batch timezone.
timezoneout of the request body, both batch tables, the store record, and the response.daybecomes a UTC calendar day, each JWT must cover the whole of the day it names, andrunDailyAlertSearchselects by UTC day and reportsutcDay. Tests: the UTC-day selector including the rollover at midnight, a real-date check that refuses2026-02-30rather than rolling it into March, and a partial-window JWT refused at upload.- End-to-end pass of the hour against a real handset: authorize with a
UTC hour an hour out, confirm no text before it and one after, then
DELETEand confirm the next day is silent. - Confirm the app mints its 100 JWTs on UTC-day boundaries. The batch is refused otherwise, so this is the one client change this phase forces.
- End-to-end pass of the hour against a real handset: authorize with a
UTC hour an hour out, confirm no text before it and one after, then
- 15. Set-based selection, bounded concurrency, and the hour on FCM.
listPendingForDayon the shared store replaces per-usergetLatestBatchpolling in both passes;forEachWithConcurrencybounds the per-user work; the notify hour andtimezoneare accepted on the FCM route and both schedulers gate on the stored hour; the hour is required on both channels, so every batch states its own.isNotifyTimeReachedis deleted rather than left beside the SQL that now decides the same question. Tests: the query's day filter, spent-user exclusion, hour boundary, newest-batch selection, channel isolation, and the unreadable-value fail-open; the pool's limit, ordering at a limit of one, and error propagation; both schedulers deferring and then running a user as the hour passes.
Tests
package.json runs tsx --test src/alertSearch/*.test.ts. That glob will not
pick up a single SMS test. Change it in phase 1, before there is anything to
miss, to cover both trees:
tsx --test "test/**/*.test.ts" "src/**/*.test.ts".
tsconfig.json excludes src/**/*.test.ts, so no test is type-checked by the
build. tsconfig.test.json covers src and test together with noEmit, and
pnpm run typecheck runs it. The build config keeps rootDir: "src", so no
test code reaches dist.
Route tests inject the SmsSender and the db modules the way
deliverAlertSearchNotification already accepts listTokens / send. No test
sends a real message.
Operational notes
- US A2P 10DLC registration is required before Twilio will carry
application-to-person traffic on a long code. Brand and campaign registration
take days and carry per-campaign fees. Unregistered traffic gets filtered by
carriers, silently, with a
sentstatus from the API. Aside: the message reports success, the carrier drops it, and everyone is happy except the person who never got the text — the telecom industry's version of a participation trophy. STOPhandling is mandatory, not a nicety. Twilio auto-replies and blocks the number at its end; phase 10 keeps this service's own state in agreement so it stops burning sends on a blocked number.- Every send costs money. The per-day cap is enforced by JWT consumption,
the same mechanism the FCM path uses, plus an explicit per-DID daily counter
read from
sms_phone_logas a backstop. sms_phone_logholds phone numbers, which the FCM tables never did. It is the first table in this database with directly identifying personal data. Retention:phone_e164is nulled on delete, and a prune job for log rows older than 400 days belongs in a follow-up.- Single replica. The SMS scheduler's in-flight guard is process-local, exactly like the other two. A second replica double-texts.
Rejected
One shared JWT inventory across channels. /notify-sms/alert-authorization
would write into alert_authorization_*, one daily retrieval per user would
build one digest, and delivery would fan out to FCM tokens and verified phones.
Half the Endorser traffic, half the minted JWTs, one cursor, and both channels
always agreeing. Rejected in favor of separate SMS tables so the SMS channel's
credentials, cursor, and failures are fully independent of the push channel's —
at the cost of the duplicate retrieval documented in §Cost of two inventories.
Separate SMS tables sharing alert_search_cursors. Cheapest to build and
silently wrong: two independent daily runs against one per-DID cursor row means
whichever fires first consumes the delta and the other reports nothing.
Storing verification codes in plaintext. A database file that leaks would hand over live codes for every pending registration. HMAC costs one function call.
Trusting req.did alone in stage 3. The DID says who is calling, not what
they authorized. Without the claim check, any valid identity JWT for that DID —
including one minted for an unrelated purpose and captured — deletes a phone
registration.
The twilio SDK. One form POST does not justify a dependency tree.
Gating the notification hour at the send instead of the search. The natural
reading of "send at 18:00" is a check inside deliverAlertSearchSms. It would
never fire: the daily run consumes that local day's JWT, and eligibility depends
on the run that consumed it, so the midnight tick would spend the day's
credential and 18:00 would find digest: null. Holding the whole pass costs one
indexed batch lookup per user per tick.
Carrying the hour inside the delegated JWTs. They are the alertSearch
credential; nbf/exp bound a whole local day. Narrowing them to an hour would
narrow when the search may run against Endorser, not when the user hears about
it, and would make the hour unchangeable without re-minting all 100.
A window rather than a floor. "Send between 18:00 and 19:00" would leave a user silent for the day whenever the service was down across that hour, which is the failure the daily digest exists to avoid. The hour is a floor, and a late text beats none.
An ISO-8601 notifyTime carrying its own ±HH:MM offset. It was
self-describing — "18:00-06:00" cannot be misread as a wall clock — which is
the property a bare pair of integers lacks. Rejected once the fields were named
notifyHourUtc and notifyMinuteUtc: a name that states the frame carries it
as surely as an offset does, and unlike an offset it cannot disagree with the
value beside it. Two integers also validate by range rather than by a 16-line
regex that had to accept basic and extended offsets, optional seconds, optional
fractions, and an optional leading date that changed nothing.
Deleting phone registrations along with the authorization. Turning alerts off would then cost a fresh 6-digit round trip to turn them back on. The two are separate routes because they are separate decisions.
Scheduling from an IANA timezone instead of a stored UTC instant. The zone
is the only value that tracks a local hour across a daylight-saving transition,
so scheduling from it would make the notification hour survive one. Rejected for
now on the strength of the drift being corrected by the next batch upload, which
the 100-day inventory forces anyway. The zone is accepted and stored regardless,
unread, so that whichever mechanism wins has the value it needs.
Not a reason: cost. Resolving a user's local wall clock with an
Intl.DateTimeFormat cached per zone measures ~1.4µs, against ~2.8µs for the
per-user SQLite lookup the pass already performs — the zone math is half the
price of a query the scheduler pays today, and 100k users cost ~137ms once per
tick. The 34µs/user figure that makes it look expensive comes from constructing
the formatter inside the loop; a Map keyed by zone removes it, and a
deployment sees a handful of distinct zones rather than the full IANA set.
A weekend job that shifts stored UTC times for zones whose DST changed. It
would keep the indexed WHERE notify_hour_min_utc <= ? shape while staying correct
across transitions. Weighed against a stored next-firing instant that each
send recomputes from the zone: the job has to run forever or users drift
silently for up to six months; it consults Intl per zone anyway, so it is the
same computation merely batched and delayed; and it has to be driven by the
rules rather than a calendar, because Lord Howe shifts 30 minutes, the southern
hemisphere runs opposite, and legislatures abolish DST on short notice (Mexico,
2022). The next-firing-instant design does its maintenance at the moment each
row is used, which is when the answer is needed and when it is cheapest to be
sure of. Neither is built; the column that either would need is.
Keeping local calendar days with the timezone removed. There would be no
frame left to resolve them in: day would be a label the server could not
check, and the daily run would have no way to know which label meant today. UTC
days give both the check and the selector one frame, at the cost of a client
that must mint its windows on UTC midnights.