362 lines
18 KiB
Markdown
362 lines
18 KiB
Markdown
# Plan: New Activity Screen on alertSearch
|
||
|
||
## Status
|
||
|
||
Built in the app, the notification service and endorser-ch. The last change,
|
||
leaving a person's own gives and offers out of their
|
||
`personalPlanContributions`, is in endorser-ch's Unreleased CHANGELOG section
|
||
and ships with its next release; Playwright spec 50 depends on it.
|
||
|
||
## Goal
|
||
|
||
The New Activity screen (`/new-activity`, `src/views/NewActivityView.vue`)
|
||
shows the same total the daily push or text reported, broken down by kind of
|
||
activity. Each kind opens a page with the exact list and count for that kind.
|
||
When a page's count differs from the number the person was told, the page
|
||
says why.
|
||
|
||
## The two endpoints
|
||
|
||
alertSearch has two halves. Each returns **lists**, not counts; a count is a
|
||
list's length.
|
||
|
||
| Bucket | Server | What it holds | Needs |
|
||
|---|---|---|---|
|
||
| `claims` | endorser `/api/v2/report/alertSearch` | Claims by others with you as `toEntity` or `fromEntity` (offers, gives, every other claim type) | — |
|
||
| `personalPlanContributions` | endorser | Gives and offers by others to or from projects you issued or are agent of | — |
|
||
| `trackedPlanUpdates` | endorser | Edits to starred projects | `planHandleIds` |
|
||
| `trackedPlanClaims` | endorser | Gives and offers to or from starred projects | `planHandleIds` |
|
||
| `plansNearby` | endorser | Projects created or edited in the search area | `location` |
|
||
| `profilesNearby` | partner `/api/partner/alertSearch` | Profiles created or edited in the search area | `location` |
|
||
| `profilesSimilar` | partner | Profiles close in meaning to your profile, best first, at most 50, not paged | a profile with text |
|
||
| `plansSimilar` | partner | Projects close in meaning to your profile, best first, at most 50, not paged | a profile with text |
|
||
|
||
Paging facts (from the endorser-ch handlers):
|
||
|
||
- Each server has one cursor covering all of its buckets. Endorser cursors
|
||
look like `1234567`; partner cursors look like `45678-1234567` (partner
|
||
position, then main position). Clients treat both as opaque text, compared
|
||
only for equality.
|
||
- `afterCursor` returns changes after it, oldest first; `beforeCursor` returns
|
||
changes before it, newest first. Both together bound a window.
|
||
- A page holds at most 50 rows per list, all cut at the same change point;
|
||
`hasMore` is one flag for the whole page.
|
||
- A row's `changeSeq` is not a cursor, so no cursor points at a single item.
|
||
|
||
## How the notification service counts
|
||
|
||
`notification-wakeup-service` runs one daily alertSearch per channel and
|
||
identity:
|
||
|
||
- `src/alertSearch/retrieve.ts` calls both endpoints with the channel's stored
|
||
`afterCursor`, `planHandleIds` and `location`, pages forward while `hasMore`
|
||
(at most 20 pages), and merges the pages.
|
||
- `src/alertSearch/digest.ts` totals all eight buckets.
|
||
- Cursors are stored per channel (`fcm` and `sms` in separate tables) and
|
||
advance after every run in which both sources complete, sent or not. Each
|
||
daily count therefore covers only changes after the previous run, whether or
|
||
not the person opened the app in between, so no item is counted in two
|
||
notifications. Rejected: subtracting the previous day's count, which would
|
||
subtract it twice.
|
||
- An upload carrying a cursor for a server replaces that server's stored
|
||
cursor, and the first run counts from it. An upload without one leaves the
|
||
stored cursor alone.
|
||
- A run with no stored cursor for a server only stores that server's cursor
|
||
and counts nothing from it. The app avoids this by sending cursors with
|
||
every upload that turns a channel on (see "Starting from the latest
|
||
change").
|
||
- The SMS body is `Gift Economies: you have N new updates.
|
||
https://giftopia.me/deep-link/new-activity?c=<start cursors> Reply STOP to
|
||
end.` The FCM payload carries title, body, `type: "alert_search"` and `c`.
|
||
|
||
## Read state
|
||
|
||
Stored as JSON in one settings column, `newActivityCursors`
|
||
(`NewActivityCursorState` in `src/libs/newActivity.ts`):
|
||
|
||
- **Visit window**, one cursor per server. The summary counts every bucket from
|
||
its start, the same window the notification service counts from once the
|
||
visit is uploaded to it, except a bucket whose list was opened after that
|
||
start was recorded: it counts from the end of that list (`summaryStart` in
|
||
`src/libs/newActivity.ts`, `countSummaryActivity` in
|
||
`src/services/newActivity.ts`).
|
||
- **Bucket window**, one per bucket, on that bucket's server. A detail page
|
||
lists everything since the end of its last load, so opening it marks the
|
||
bucket read; reopening it shows nothing new until more arrives, and "Show
|
||
earlier" lists what it showed before. A bucket never opened starts where the
|
||
summary does, so its first detail count equals the summary's.
|
||
- **Uploaded**, the cursors last sent to each notification channel.
|
||
|
||
A window records where a load counted from (`from`) and when that cursor was
|
||
recorded (`fromSeenAt`), where it ended (`to`) and when (`seenAt`), and when
|
||
the visit began (`at`). A summary load less than 30 minutes after `at` is the
|
||
same visit and counts from the same `from`, so reloading the summary keeps the
|
||
counts of unopened lists; a later load starts a new visit from the last
|
||
`to`. "Mark unread from this item" does not exist, since no cursor points at an
|
||
item.
|
||
|
||
The first load ever has no cursor. It only records where to start and shows
|
||
nothing as new:
|
||
|
||
1. The app sends each server a request with no cursor and no ID. The server
|
||
answers with its most recent rows and its latest cursors
|
||
(`nextAfterCursor`).
|
||
2. The app ignores the rows: every count is 0 and every list is empty
|
||
(`countServer` and `loadBucket` in `src/services/newActivity.ts` count
|
||
nothing without a start).
|
||
3. It stores a visit ending at those cursors (`seedIfNeeded`, `seededState`),
|
||
with a start time long past, so the next load counts from them.
|
||
|
||
Home and the detail pages seed before counting; the summary's own first load
|
||
does the same, since the end of a load with no start becomes the start of
|
||
the next visit. Turning on notifications seeds too, with the cursors it
|
||
uploads. Seeding needs both servers to answer; otherwise nothing is stored,
|
||
nothing shows as new, and the next load tries again. A detail page's "Show
|
||
earlier" still lists what came before the seed.
|
||
|
||
Arriving through a notification link with no visit yet counts from the
|
||
link's `c` instead.
|
||
|
||
Rejected: starting endorser after the oldest `lastAcked*JwtId` (`afterId`) so
|
||
items unread on the earlier screen stay unread; the app would then depend on
|
||
the ID path that endorser-ch plans to retire.
|
||
|
||
Opening a detail page also moves the `lastAcked*JwtId` read markers
|
||
(`claims` → offers to you, `personalPlanContributions` → offers to your
|
||
projects, `trackedPlanUpdates` → starred changes), which the Recent Offers
|
||
pages use to mark the last-read offer.
|
||
|
||
## Summary screen
|
||
|
||
- One alertSearch call per server from the visit start, paging forward while
|
||
`hasMore` up to 4 pages; past that a count shows "+".
|
||
- Same inputs as the notification service: `planHandleIds` from
|
||
`settings.starredPlanHandleIds`, `location` from the first
|
||
`settings.searchBoxes` entry converted from `BoundingBox` to
|
||
`minLocLat/maxLocLat/minLocLon/maxLocLon`.
|
||
- The total at the top is the sum of all eight buckets, as the service totals
|
||
them, so with no activity since the run it equals the notified number.
|
||
- With no search box, the location rows link to `/search-area`; with no
|
||
starred projects, the tracked rows show a hint.
|
||
- A server that fails shows "?" on its buckets and the error; the other
|
||
server's counts appear, and the visit does not move.
|
||
|
||
## Home screen
|
||
|
||
The Home screen shows one tile per bucket with something unseen, in the
|
||
summary's order, from the same eight alertSearch buckets
|
||
(`countUnseenActivity` in `src/services/newActivity.ts`).
|
||
|
||
- Each tile shows the count (with "+" past the page limit) and the bucket's
|
||
label, and opens that bucket's detail page with `?expected=` set to the
|
||
count. "View All New Activity For You" opens the summary.
|
||
- A bucket counts from the end of whichever load the person saw last: the
|
||
summary's visit or that bucket's own list (`unseenStart` in
|
||
`src/libs/newActivity.ts`, ordered by each window's `seenAt`). It never
|
||
uses a visit's start, so a summary just loaded shows no tiles, even inside
|
||
the 30-minute visit in which the summary keeps showing its counts.
|
||
- Buckets on one server with the same start share one request, so with no
|
||
list opened since the summary, Home makes one request per server.
|
||
- Home writes the cursor state only to seed it on the first load ever; it
|
||
never starts a visit otherwise, and never uploads to the notification
|
||
service.
|
||
- Tile test IDs are `newActivityNumber-<bucket>`.
|
||
|
||
## Detail pages
|
||
|
||
One view, `/new-activity/:bucket` (`NewActivityBucketView.vue`), calls the
|
||
same endpoint from the bucket start and shows only that bucket, so its count
|
||
is exact. A per-bucket legacy endpoint cannot match it: `/report/offers`
|
||
holds only offers, while `claims` holds every claim type that names you.
|
||
|
||
- The heading shows its own count. The summary and the Home tiles pass their
|
||
count as `?expected=`; when they differ, a note says how many came before
|
||
the tapped count started, or that the list has fewer than it showed.
|
||
Rejected: handing Home's rows to the detail page, which saves one request
|
||
while this query runs anyway after a deep link, a relaunch, a capped count
|
||
or a different bucket start.
|
||
- "Show more" continues forward; "Show earlier" pages back with
|
||
`beforeCursor` from the bucket start, listing items already seen.
|
||
- Rows (`NewActivityRow.vue`) link to `/claim/:id`, `/project/:handleId`,
|
||
`/user-profile/:rowId`; similarity matches show their match percentage.
|
||
- `trackedPlanUpdates` keeps the before/after diff (`PlanChangeDiff.vue`),
|
||
fed by `plansLastUpdatedBetween` for the page's handle IDs with `afterId`
|
||
just before the oldest edit, because alertSearch plan rows carry no earlier
|
||
claim.
|
||
- The `claims` and `personalPlanContributions` pages link to
|
||
`RecentOffersToUserView` and `RecentOffersToUserProjectsView`.
|
||
- endorser-ch returns `claim` on `claims` rows, so the page can say what was
|
||
offered or given.
|
||
|
||
## Re-registering notifications on visit
|
||
|
||
When the summary (`/new-activity`) opens a new visit and the push or text
|
||
channel is on, the app re-uploads that channel's alert authorization in the
|
||
background, after the counts are on screen: a fresh
|
||
JWT batch replacing the unused one, with the visit's end cursors, the starred
|
||
handle IDs and the search area. The next daily count then starts from this
|
||
visit and uses the person's current starred projects and area.
|
||
|
||
- It reuses `mintAlertAuthorizationBatch` and the POST
|
||
`/notifications/alert-authorization` and POST
|
||
`/notify-sms/alert-authorization` routes
|
||
(`src/services/notifications/newActivityRefresh.ts`).
|
||
- It is skipped inside the same visit, and when the channel already counts
|
||
from this visit's start and nothing is new.
|
||
- Detail pages and the Home screen never upload. A detail page's window is
|
||
narrower than the service's single cursor per server, so uploading it would
|
||
drop the other buckets' unseen items from the next notification.
|
||
- Changing the text time re-uploads without cursors, so nothing waiting for
|
||
the next text is dropped.
|
||
|
||
Notification service changes this needs:
|
||
|
||
1. `AlertAuthorizationRequestBody` accepts optional `endorserAfterCursor`,
|
||
`partnerAfterCursor`, `planHandleIds` and `location`; the routes write all
|
||
four into that channel's cursor table.
|
||
2. The daily run sends the stored `planHandleIds` and `location` to
|
||
alertSearch.
|
||
3. `replaceUnusedBatch` skips days that already hold a consumed JWT.
|
||
Without that, a re-upload after today's run installs an unused JWT for
|
||
today, and the scheduler runs and pushes a second time the same day (SMS
|
||
has a per-phone daily cap; FCM has none).
|
||
4. `profilesSimilar` and `plansSimilar` join the total on both sides:
|
||
`mergePartner` keeps them (deduplicated by profile `rowId` and plan
|
||
`handleId`) and `ALERT_SEARCH_DIGEST_BUCKETS` lists them.
|
||
5. Cursors advance only when both sources complete, so a half-failed run
|
||
repeats the whole window on its retry.
|
||
6. A run with no stored cursor for a source seeds that cursor and counts
|
||
nothing from it.
|
||
|
||
## Starting from the latest change
|
||
|
||
Turning on push or text notifications (the toggles in `AccountViewView.vue`)
|
||
sends the latest cursors with the first upload, so the first notification
|
||
counts only what happens after the person turned it on. The app gets those
|
||
cursors from one alertSearch call per server, made just before the upload,
|
||
and stores them as its visit when it has none. Each server stands alone: one
|
||
that does not answer gets the end of the last visit instead, and with neither
|
||
its cursor is left out of the upload. Service change 6 covers only that last
|
||
case.
|
||
|
||
## Notification links carry the window
|
||
|
||
Each run holds the cursors it counted from (`startCursors` on the daily
|
||
result). The text and the push carry them, so the app can tell whether it is
|
||
looking at the same window the person was told about.
|
||
|
||
- SMS URL: one parameter joining both servers' start cursors with `.`:
|
||
`https://giftopia.me/deep-link/new-activity?c=1234567.45678-1234567`.
|
||
That adds about 25 characters; the body stays inside one 160-character
|
||
segment for counts under 10,000. A first run counts nothing and sends no
|
||
text.
|
||
- FCM: the same `c` in the data payload, and a tap routes to
|
||
`/new-activity?c=…`.
|
||
- App: `deepLinkQuerySchemas["new-activity"]` accepts optional `c`. When `c`
|
||
differs from the visit start, the summary also counts from `c` and compares
|
||
totals: fewer since the visit means "you may have already seen some of what
|
||
it counted"; more means "this includes N from before that notification".
|
||
- Optional, for an exact replay: also send the run's end cursors and offer a
|
||
"Show what the notification counted" view using `afterCursor` = start and
|
||
`beforeCursor` = end. That doubles the URL addition and can push the text
|
||
past one segment.
|
||
|
||
## Steps
|
||
|
||
App (`crowd-funder-for-time-pwa`):
|
||
|
||
- [x] HTTP client and paging in `src/libs/alertSearch.ts`.
|
||
- [x] Bucket registry and cursor state in `src/libs/newActivity.ts`; loads in
|
||
`src/services/newActivity.ts`.
|
||
- [x] Settings column `newActivityCursors`, migration
|
||
`012_add_newActivityCursors_to_settings`.
|
||
- [x] `NewActivityView.vue` as the summary, with the `c` comparison.
|
||
- [x] `NewActivityBucketView.vue`, `NewActivityRow.vue`, `PlanChangeDiff.vue`,
|
||
route `/new-activity/:bucket`.
|
||
- [x] Re-upload on visit; latest cursors from the enable toggles.
|
||
- [x] `c` in the deep-link query schema; FCM taps open New Activity with `c`.
|
||
- [x] First load seeds cursors and shows nothing; no `afterId` sent.
|
||
- [x] Home tiles for all eight buckets, each opening its detail page, counted
|
||
from `unseenStart`; the legacy `/report/offers`,
|
||
`offersToPlansOwnedByMe` and starred-changes counts removed from Home.
|
||
- [x] Jest tests (`src/test/newActivity.test.ts`); Playwright specs 50 and 55
|
||
updated.
|
||
- [x] Playwright specs 50 and 55 pass against local servers in Chromium and
|
||
Firefox.
|
||
- [x] CHANGELOG entry (1.4.10).
|
||
- [x] Test and start-up fixes found while running the specs (Home records
|
||
where New Activity starts in parallel with the feed, Affirm Delivery waits
|
||
for the offer's project, a WebAssembly explanation on the start-up error
|
||
screen, test warm-up, retry and reporter).
|
||
- [x] Release the app after endorser-ch is deployed, since detail rows read
|
||
`claim` from `alertSearch`.
|
||
|
||
Notification service (`notification-wakeup-service`):
|
||
|
||
- [x] Service changes 1–6, `c` in the SMS link and FCM data, tests, README,
|
||
CHANGELOG.
|
||
- [x] Deploy to test and production.
|
||
|
||
endorser-ch:
|
||
|
||
- [x] `claim` on `claims` rows of `/api/v2/report/alertSearch`, CHANGELOG.
|
||
- [x] An Offer that names its plan in `itemOffered.isPartOf` (as the app
|
||
sends) gets `toEntity`, so it reaches the owner's
|
||
`personalPlanContributions` and `trackedPlanClaims`; migration V23 fills
|
||
existing offers without moving `changeSeq`; test, CHANGELOG.
|
||
- [x] Commit the `isPartOf` change: `CHANGELOG.md`, `sql/README.md`,
|
||
`sql/V23__offer_is_part_of_to_entity.sqlite3`, `claim.service.js`,
|
||
`endorser.db.service.js`, `test/controller-endorser-6-plans-totals.js`.
|
||
- [x] Deploy to test, then production, running Flyway on the main DB so V23
|
||
applies.
|
||
- [x] Leave a person's own gives and offers out of their
|
||
`personalPlanContributions`:
|
||
- endorser-ch: `AND j.issuer != ?`, bound to the requesting DID, on both
|
||
halves of the `UNION` in `jwtsForUserPlanContributions`
|
||
(`endorser.db.service.js`), as `jwtsWithDidAfterId` does.
|
||
- endorser-ch test in `controller-endorser-6-plans-totals.js`: user 1's own
|
||
offer to user 1's plan is absent from user 1's
|
||
`personalPlanContributions`, and user 2's offer is present.
|
||
- App spec 50 "Record an offer": User 0 seeds New Activity on Home, User 1
|
||
records the offer on a project User 0 owns, then User 0 checks the
|
||
`personalPlanContributions` tile and list. "Affirm delivery" gets User 1
|
||
as giver and the project as recipient.
|
||
- endorser-ch CHANGELOG entry (Unreleased); the notification service needs
|
||
no change, since its counts come from the same query.
|
||
|
||
## Out of scope
|
||
|
||
- The native background fetcher (`TimeSafariNativeFetcher`) keeps its own
|
||
read state.
|
||
|
||
## History
|
||
|
||
Closed findings from reading the notification service before these changes
|
||
(2026-10-15):
|
||
|
||
- The schedulers called `runDailyAlertSearch(userId, now, {})`, and the upload
|
||
body had no field for `location` or `planHandleIds`, so `trackedPlanUpdates`,
|
||
`trackedPlanClaims`, `plansNearby` and `profilesNearby` were always empty: the
|
||
reported total was `claims` + `personalPlanContributions`.
|
||
- `profilesSimilar` and `plansSimilar` were dropped when partner pages merged.
|
||
- Each source's cursor advanced on its own, so when one source completed and
|
||
the other failed, the retry started past the completed source's rows and
|
||
never counted them.
|
||
- A first run counted the server's newest page, up to 50 rows per bucket of old
|
||
activity.
|
||
|
||
Closed decisions:
|
||
|
||
- 2026-10-16: the first load seeds cursors and shows nothing, replacing the
|
||
`afterId` handoff from the oldest `lastAcked*JwtId`, so the app sends
|
||
`alertSearch` only cursors and endorser-ch can retire the ID path. A person
|
||
upgrading with unread offers sees them under "Show earlier" and on the
|
||
Recent Offers pages, not as new.
|
||
- 2026-10-16: detail pages query `alertSearch` themselves and do not reuse
|
||
Home's rows; an in-memory hand-off would be a second path that every
|
||
entry other than a fresh, uncapped Home tile skips.
|
||
- 2026-10-24: `personalPlanContributions` leaves out the requester's own
|
||
gives and offers, matching `claims`, which already left out the requester's
|
||
own claims. Before this, a person who offered to their own project saw it
|
||
counted as new activity. The plan was closed and moved to history.
|