# Starred Projects Polling Implementation **Author**: Matthew Raymer **Version**: 1.0.0 **Created**: 2025-10-06 06:23:11 UTC **Based on**: `loadNewStarredProjectChanges` from crowd-funder-for-time ## Overview This document adapts the sophisticated `loadNewStarredProjectChanges` implementation from the crowd-funder-for-time project to the Daily Notification Plugin's polling system. This provides a user-curated, change-focused notification system that monitors only projects the user has explicitly starred. ## Implementation Specifications ### API & Data Contracts #### `/api/v2/report/plansLastUpdatedBetween` Endpoint Specification **URL**: `POST {apiServer}/api/v2/report/plansLastUpdatedBetween` **Request Headers**: ``` Content-Type: application/json Authorization: Bearer {JWT_TOKEN} User-Agent: TimeSafari-DailyNotificationPlugin/1.0.0 ``` **Request Body**: ```json { "planIds": ["plan_handle_1", "plan_handle_2", "..."], "afterId": "jwt_id_cursor", "beforeId": "jwt_id_cursor_optional", "limit": 100 } ``` **Pagination Strategy**: - **Cursor-based**: Uses `afterId` (required) and `beforeId` (optional) for pagination - **Max Page Size**: 100 records per request (configurable via `limit` parameter) - **Ordering**: Results ordered by `jwtId` in ascending order (monotonic, lexicographic) - **JWT ID Format**: `{timestamp}_{random}_{hash}` - lexicographically sortable - **Rate Limits**: 100 requests per minute per DID, 429 status with `Retry-After` header **Response Format** (Canonical Schema): ```json { "data": [ { "planSummary": { "jwtId": "1704067200_abc123_def45678", "handleId": "project_handle", "name": "Project Name", "description": "Project description", "issuerDid": "did:key:issuer_did", "agentDid": "did:key:agent_did", "startTime": "2025-01-01T00:00:00Z", "endTime": "2025-01-31T23:59:59Z", "locLat": 40.7128, "locLon": -74.0060, "url": "https://project-url.com", "version": "1.0.0" }, "previousClaim": { "jwtId": "1703980800_xyz789_0badf00d", "claimType": "project_update", "claimData": { "status": "in_progress", "progress": 0.75, "lastModified": "2025-01-01T12:00:00Z" }, "metadata": { "createdAt": "2025-01-01T10:00:00Z", "updatedAt": "2025-01-01T12:00:00Z" } } } ], "hitLimit": false, "pagination": { "hasMore": true, "nextAfterId": "1704153600_mno345_0badf00d" } } ``` **Note**: All responses include the `pagination` block with nullable fields. Empty responses have `pagination.hasMore: false` and `pagination.nextAfterId: null`. **Error Codes & Retry Semantics**: **400 Bad Request** - Invalid request format: ```json { "error": "invalid_request", "message": "Invalid planIds format: expected array of strings", "details": { "field": "planIds", "expected": "string[]", "received": "string" }, "requestId": "req_abc123" } ``` - **Retry**: No retry, fix request format **401 Unauthorized** - Invalid/expired JWT: ```json { "error": "unauthorized", "message": "JWT token expired", "details": { "expiredAt": "2025-01-01T12:00:00Z", "currentTime": "2025-01-01T12:05:00Z" }, "requestId": "req_def45678" } ``` - **Retry**: Refresh token and retry once **403 Forbidden** - Insufficient permissions: ```json { "error": "forbidden", "message": "Insufficient permissions to access starred projects", "details": { "requiredScope": "notifications:read", "userScope": "notifications:write" }, "requestId": "req_ghi789" } ``` - **Retry**: No retry, requires user action **429 Too Many Requests** - Rate limited: ```json { "error": "Rate limit exceeded", "code": "RATE_LIMIT_EXCEEDED", "message": "Rate limit exceeded for DID", "details": { "limit": 100, "window": "1m", "resetAt": "2025-01-01T12:01:00Z" }, "retryAfter": 60, "requestId": "req_jkl012" } ``` - **Retry**: Retry after `retryAfter` seconds **500 Internal Server Error** - Server error: ```json { "error": "internal_server_error", "message": "Database connection timeout", "details": { "component": "database", "operation": "query_starred_projects" }, "requestId": "req_mno345" } ``` - **Retry**: Exponential backoff (1s, 2s, 4s, 8s) **503 Service Unavailable** - Temporary outage: ```json { "error": "service_unavailable", "message": "Service temporarily unavailable for maintenance", "details": { "maintenanceWindow": "2025-01-01T12:00:00Z to 2025-01-01T13:00:00Z" }, "retryAfter": 3600, "requestId": "req_pqr67890" } ``` - **Retry**: Exponential backoff with jitter **Request Limits**: - **Max `planIds` per request**: 1000 project IDs - **Max request body size**: 1MB (JSON) - **Max response size**: 10MB (JSON) - **Request timeout**: 30 seconds **Strong Consistency Guarantees**: - **Immutable Results**: Results for a given `(afterId, beforeId)` are immutable once returned - **No Late Writes**: No reshuffling of results after initial response - **Cursor Stability**: `nextAfterId` is guaranteed to be strictly greater than the last item's `jwtId`. Clients **must not** reuse the last item's `jwtId` as `afterId`; always use `pagination.nextAfterId`. - **Eventual Consistency**: Maximum 5-second delay between change creation and visibility in report endpoint #### JWT ID Ordering & Uniqueness Guarantees **JWT ID Format Specification**: ``` jwtId = {timestamp}_{random}_{hash} ``` **Components**: - **timestamp**: Unix timestamp in seconds (10 digits, zero-padded) - **random**: Cryptographically secure random string (6 characters, alphanumeric, fixed-width) - **hash**: SHA-256 hash of `{timestamp}_{random}_{content}` (8 characters, hex, fixed-width, zero-padded) **Example**: `1704067200_abc123_def45678` **Fixed-Width Rule**: All components are fixed-width with zero-padding where applicable to ensure lexicographic ordering works correctly. **Uniqueness Guarantees**: - **Globally Unique**: JWT IDs are globally unique across all projects and users - **Strictly Increasing**: For any given project, JWT IDs are strictly increasing over time - **Tie-Break Rules**: If timestamps collide, random component ensures uniqueness - **Collision Resistance**: The truncated 8-hex (32-bit) suffix provides ~2^32 collision resistance; the underlying SHA-256 is larger but we only keep 32 bits **Ordering Proof**: ```typescript function compareJwtIds(a: string, b: string): number { // Lexicographic comparison works because: // 1. Timestamp is fixed-width (10 digits) // 2. Random component is fixed-width (6 chars) // 3. Hash component is fixed-width (8 chars) return a.localeCompare(b); } // Example: "1704067200_abc123_def45678" < "1704153600_xyz789_0badf00d" ``` **Eventual Consistency Bounds**: - **Maximum Delay**: 5 seconds between change creation and API visibility - **Typical Delay**: 1-2 seconds under normal load - **Consistency Model**: Read-after-write consistency within 5 seconds - **Partition Tolerance**: System remains available during network partitions **Related Endpoints**: - `GET /api/v2/plans/{planId}/metadata` - Batch plan metadata lookup - `POST /api/v2/plans/acknowledge` - Mark changes as acknowledged - `GET /api/v2/plans/batch` - Batch plan summary retrieval #### `POST /api/v2/plans/acknowledge` Endpoint Specification **URL**: `POST {apiServer}/api/v2/plans/acknowledge` **Request Headers**: ``` Content-Type: application/json Authorization: Bearer {JWT_TOKEN} X-Idempotency-Key: {uuid} ``` **Request Body**: ```json { "acknowledgedJwtIds": ["1704067200_abc123_def45678", "1704153600_mno345_0badf00d"], "acknowledgedAt": "2025-01-01T12:00:00Z", "clientVersion": "TimeSafari-DailyNotificationPlugin/1.0.0" } ``` **Response Format**: ```json { "acknowledged": 2, "failed": 0, "alreadyAcknowledged": 0, "acknowledgmentId": "ack_xyz789", "timestamp": "2025-01-01T12:00:00Z" } ``` **Acknowledgment Semantics**: - **Per-JWT-ID**: Each `jwtId` must be acknowledged individually - **Idempotency**: Duplicate acknowledgments are ignored (not errors) - **Rate Limits**: 1000 acknowledgments per minute per DID - **Atomicity**: All acknowledgments in single request succeed or fail together - **Recovery**: Failed acknowledgments can be retried with same `X-Idempotency-Key` ### Storage & Schema #### Database Schemas **`active_identity` Table**: ```sql CREATE TABLE active_identity ( id INTEGER PRIMARY KEY AUTOINCREMENT, activeDid TEXT NOT NULL UNIQUE, lastUpdated DATETIME DEFAULT CURRENT_TIMESTAMP, created_at DATETIME DEFAULT CURRENT_TIMESTAMP ); CREATE INDEX idx_active_identity_did ON active_identity(activeDid); CREATE INDEX idx_active_identity_updated ON active_identity(lastUpdated); ``` **`settings` Table**: ```sql CREATE TABLE settings ( id INTEGER PRIMARY KEY AUTOINCREMENT, accountDid TEXT NOT NULL UNIQUE, apiServer TEXT NOT NULL DEFAULT 'https://api.endorser.ch', starredPlanHandleIds TEXT DEFAULT '[]', -- JSON array of strings lastAckedStarredPlanChangesJwtId TEXT NULL, lastAckedTimestamp DATETIME NULL, notificationPreferences TEXT DEFAULT '{}', -- JSON object created_at DATETIME DEFAULT CURRENT_TIMESTAMP, updated_at DATETIME DEFAULT CURRENT_TIMESTAMP, FOREIGN KEY (accountDid) REFERENCES active_identity(activeDid) ); CREATE INDEX idx_settings_account ON settings(accountDid); CREATE INDEX idx_settings_updated ON settings(updated_at); ``` **Note**: `accountDid` has `UNIQUE` constraint to prevent multiple rows per account. **Migration Steps**: **Android (Room)**: ```kotlin // Migration from version 1 to 2 val MIGRATION_1_2 = object : Migration(1, 2) { override fun migrate(database: SupportSQLiteDatabase) { database.execSQL(""" CREATE TABLE IF NOT EXISTS settings ( id INTEGER PRIMARY KEY AUTOINCREMENT, accountDid TEXT NOT NULL UNIQUE, apiServer TEXT NOT NULL DEFAULT 'https://api.endorser.ch', starredPlanHandleIds TEXT DEFAULT '[]', lastAckedStarredPlanChangesJwtId TEXT NULL, lastAckedTimestamp DATETIME NULL, notificationPreferences TEXT DEFAULT '{}', created_at DATETIME DEFAULT CURRENT_TIMESTAMP, updated_at DATETIME DEFAULT CURRENT_TIMESTAMP ) """) database.execSQL("CREATE INDEX IF NOT EXISTS idx_settings_account ON settings(accountDid)") } } ``` **iOS (Core Data)**: ```swift // Core Data Model Version 2 class Settings: NSManagedObject { @NSManaged var accountDid: String @NSManaged var apiServer: String @NSManaged var starredPlanHandleIds: String // JSON string @NSManaged var lastAckedStarredPlanChangesJwtId: String? @NSManaged var lastAckedTimestamp: Date? @NSManaged var notificationPreferences: String // JSON string @NSManaged var createdAt: Date @NSManaged var updatedAt: Date } ``` **Web (IndexedDB)**: ```typescript // IndexedDB Schema Version 2 const dbSchema = { stores: { settings: { keyPath: 'accountDid', // Use accountDid as primary key (matches SQL UNIQUE constraint) indexes: { updatedAt: { keyPath: 'updatedAt', unique: false } } } } }; // Data structure matches SQL schema interface SettingsRecord { accountDid: string; // Primary key apiServer: string; starredPlanHandleIds: string; // JSON string lastAckedStarredPlanChangesJwtId?: string; lastAckedTimestamp?: Date; notificationPreferences: string; // JSON string createdAt: Date; updatedAt: Date; } ``` **Serialization Rules**: - `starredPlanHandleIds`: JSON array of strings, empty array `[]` as default - `lastAckedStarredPlanChangesJwtId`: Nullable string, reset to `null` on DID change - `notificationPreferences`: JSON object with boolean flags for notification types ### Auth & Security #### `DailyNotificationJWTManager` Contract **Token Generation**: ```typescript interface JWTClaims { iss: string; // activeDid (issuer) aud: string; // "endorser-api" exp: number; // expiration timestamp iat: number; // issued at timestamp scope: string; // "notifications" jti: string; // unique token ID } interface JWTConfig { secret: string; // JWT signing secret expirationMinutes: number; // Token lifetime (default: 60) refreshThresholdMinutes: number; // Refresh threshold (default: 10) clockSkewSeconds: number; // Clock skew tolerance (default: 30) } ``` **Token Lifecycle**: - **Generation**: On `setActiveDid()` or token expiration - **Rotation**: Automatic refresh when `exp - iat < refreshThresholdMinutes` - **Expiry**: Hard expiration at `exp` timestamp - **Clock Skew**: Accept tokens within `clockSkewSeconds` of current time - **Refresh Policy**: Exponential backoff on refresh failures (1s, 2s, 4s, 8s, max 30s) **Required Headers**: ``` Authorization: Bearer {JWT_TOKEN} X-Request-ID: {uuid} X-Client-Version: TimeSafari-DailyNotificationPlugin/1.0.0 ``` **CORS & Response Headers**: ``` Access-Control-Expose-Headers: Retry-After, X-Request-ID, X-Rate-Limit-Remaining Access-Control-Allow-Origin: https://timesafari.com Access-Control-Allow-Methods: POST, OPTIONS Access-Control-Allow-Headers: Authorization, Content-Type, X-Request-ID, X-Client-Version ``` **Note**: API sets `Access-Control-Expose-Headers` for `Retry-After` and other response headers clients must read. **Security Constraints**: - JWT must be signed with shared secret between plugin and Endorser.ch - `activeDid` in JWT must match authenticated user's DID - Tokens are single-use for sensitive operations (acknowledgment) - Rate limiting applied per DID, not per token #### JWT Secret Storage & Key Management **Android (Android Keystore)**: ```kotlin class SecureJWTStorage(private val context: Context) { private val keyStore = KeyStore.getInstance("AndroidKeyStore") private val keyAlias = "timesafari_jwt_secret" fun storeJWTSecret(secret: String) { val keyGenerator = KeyGenerator.getInstance(KeyProperties.KEY_ALGORITHM_AES, "AndroidKeyStore") val keyGenParameterSpec = KeyGenParameterSpec.Builder( keyAlias, KeyProperties.PURPOSE_ENCRYPT or KeyProperties.PURPOSE_DECRYPT ) .setBlockModes(KeyProperties.BLOCK_MODE_GCM) .setEncryptionPaddings(KeyProperties.ENCRYPTION_PADDING_NONE) .build() keyGenerator.init(keyGenParameterSpec) keyGenerator.generateKey() // Encrypt and store secret with IV persistence val cipher = Cipher.getInstance("AES/GCM/NoPadding") keyStore.load(null) // Load keystore before accessing keys cipher.init(Cipher.ENCRYPT_MODE, keyStore.getKey(keyAlias, null) as SecretKey) val encryptedSecret = cipher.doFinal(secret.toByteArray()) val iv = cipher.iv // Store IV + ciphertext as Base64 (IV persistence critical for decryption) val combined = Base64.encodeToString(iv + encryptedSecret, Base64.DEFAULT) val prefs = context.getSharedPreferences("secure_prefs", Context.MODE_PRIVATE) prefs.edit().putString("jwt_secret", combined).apply() } fun getJWTSecret(): String? { val prefs = context.getSharedPreferences("secure_prefs", Context.MODE_PRIVATE) val combined = prefs.getString("jwt_secret", null) ?: return null val decoded = Base64.decode(combined, Base64.DEFAULT) val iv = decoded.sliceArray(0..11) // First 12 bytes are IV val ciphertext = decoded.sliceArray(12..decoded.lastIndex) val cipher = Cipher.getInstance("AES/GCM/NoPadding") keyStore.load(null) // Load keystore before accessing keys cipher.init(Cipher.DECRYPT_MODE, keyStore.getKey(keyAlias, null) as SecretKey, GCMParameterSpec(128, iv)) return String(cipher.doFinal(ciphertext)) } } ``` **iOS (iOS Keychain)**: ```swift class SecureJWTStorage { private let keychain = Keychain(service: "com.timesafari.dailynotification") func storeJWTSecret(_ secret: String) throws { let data = secret.data(using: .utf8)! try keychain.set(data, key: "jwt_secret") } func getJWTSecret() throws -> String? { guard let data = try keychain.getData("jwt_secret") else { return nil } return String(data: data, encoding: .utf8) } } ``` **Web (Encrypted Storage)**: ```typescript class SecureJWTStorage { private static readonly STORAGE_KEY = 'timesafari_jwt_secret'; private static readonly KEY_PAIR_NAME = 'timesafari_keypair'; static async storeJWTSecret(secret: string): Promise { // Generate or retrieve RSA key pair for wrapping let keyPair = await this.getOrGenerateKeyPair(); // Generate ephemeral AES key const aesKey = await crypto.subtle.generateKey( { name: 'AES-GCM', length: 256 }, true, ['encrypt', 'decrypt'] ); // Encrypt secret with AES key const iv = crypto.getRandomValues(new Uint8Array(12)); const encrypted = await crypto.subtle.encrypt( { name: 'AES-GCM', iv }, aesKey, new TextEncoder().encode(secret) ); // Wrap AES key with RSA public key const wrappedKey = await crypto.subtle.wrapKey( 'raw', aesKey, keyPair.publicKey, { name: 'RSA-OAEP', hash: 'SHA-256' } ); // Store wrapped key + IV + ciphertext in IndexedDB (RSA keypair export persisted) await navigator.storage?.persist(); const db = await this.getEncryptedDB(); await db.put('secrets', { id: this.STORAGE_KEY, wrappedKey, iv, ciphertext: encrypted }); } static async getJWTSecret(): Promise { const db = await this.getEncryptedDB(); const record = await db.get('secrets', this.STORAGE_KEY); if (!record) return null; // Unwrap AES key with RSA private key const keyPair = await this.getOrGenerateKeyPair(); const aesKey = await crypto.subtle.unwrapKey( 'raw', record.wrappedKey, keyPair.privateKey, { name: 'RSA-OAEP', hash: 'SHA-256' }, { name: 'AES-GCM', length: 256 }, false, ['decrypt'] ); // Decrypt secret const decrypted = await crypto.subtle.decrypt( { name: 'AES-GCM', iv: record.iv }, aesKey, record.ciphertext ); return new TextDecoder().decode(decrypted); } private static async getOrGenerateKeyPair(): Promise { // Try to retrieve existing key pair try { const publicKey = await crypto.subtle.importKey( 'spki', await this.getStoredKey('public'), { name: 'RSA-OAEP', hash: 'SHA-256' }, false, ['wrapKey'] ); const privateKey = await crypto.subtle.importKey( 'pkcs8', await this.getStoredKey('private'), { name: 'RSA-OAEP', hash: 'SHA-256' }, false, ['unwrapKey'] ); return { publicKey, privateKey }; } catch { // Generate new key pair const keyPair = await crypto.subtle.generateKey( { name: 'RSA-OAEP', modulusLength: 2048, publicExponent: new Uint8Array([1, 0, 1]) }, true, ['wrapKey', 'unwrapKey'] ); // Store key pair await this.storeKey('public', await crypto.subtle.exportKey('spki', keyPair.publicKey)); await this.storeKey('private', await crypto.subtle.exportKey('pkcs8', keyPair.privateKey)); return keyPair; } } } ``` **Note**: Web implementation uses RSA-OAEP to wrap ephemeral AES keys, storing the wrapped key + IV + ciphertext in IndexedDB. The RSA key pair is generated once and persisted for future use. **Key Rotation Plan**: - **Rotation Frequency**: Every 90 days - **Grace Window**: 7 days overlap for old/new keys - **Rotation Process**: 1. Generate new key 2. Update server-side key registry 3. Distribute new key to clients 4. Wait 7 days grace period 5. Revoke old key - **Emergency Rotation**: Immediate revocation with 1-hour grace window **Required Claims Verification Checklist**: ```typescript interface JWTVerificationChecklist { // Required claims iss: string; // Must match activeDid aud: string; // Must be "endorser-api" exp: number; // Must be in future iat: number; // Must be in past scope: string; // Must include "notifications" jti: string; // Must be unique (replay protection) // Verification rules clockSkewTolerance: number; // ±30 seconds maxTokenAge: number; // 1 hour replayWindow: number; // 5 minutes } ``` ### Background Execution & Scheduling #### Platform Constraints & Strategies **Android (WorkManager)**: ```kotlin // WorkManager Configuration - PeriodicWorkRequest is authoritative for polling val constraints = Constraints.Builder() .setRequiredNetworkType(NetworkType.CONNECTED) .setRequiresBatteryNotLow(false) .setRequiresCharging(false) .setRequiresDeviceIdle(false) .setRequiresStorageNotLow(false) .build() // Periodic polling (authoritative approach) val periodicWorkRequest = PeriodicWorkRequestBuilder( 15, TimeUnit.MINUTES, // Minimum interval 5, TimeUnit.MINUTES // Flex interval ) .setConstraints(constraints) .setBackoffCriteria(BackoffPolicy.EXPONENTIAL, 1, TimeUnit.MINUTES) .setInputData(workDataOf( "activeDid" to activeDid, "pollingConfig" to pollingConfigJson )) .addTag("starred_projects_polling") .build() // Enqueue periodic work WorkManager.getInstance(context) .enqueueUniquePeriodicWork( "starred_projects_polling_${activeDid}", ExistingPeriodicWorkPolicy.REPLACE, periodicWorkRequest ) // One-time work for immediate catch-up (e.g., after app open) fun scheduleImmediateCatchUp() { val oneTimeRequest = OneTimeWorkRequestBuilder() .setConstraints(constraints) .setInputData(workDataOf( "activeDid" to activeDid, "immediate" to true )) .addTag("starred_projects_catchup") .build() WorkManager.getInstance(context) .enqueueUniqueWork( "starred_projects_catchup_${activeDid}", ExistingWorkPolicy.REPLACE, oneTimeRequest ) } ``` **iOS (BGTaskScheduler)**: ```swift // BGTaskScheduler Configuration let taskIdentifier = "com.timesafari.dailynotification.starred-projects-polling" // Register background task BGTaskScheduler.shared.register( forTaskWithIdentifier: taskIdentifier, using: nil ) { task in self.handleStarredProjectsPolling(task: task as! BGAppRefreshTask) } // Schedule background task let request = BGAppRefreshTaskRequest(identifier: taskIdentifier) request.earliestBeginDate = Date(timeIntervalSinceNow: 15 * 60) // 15 minutes try BGTaskScheduler.shared.submit(request) // Required Info.plist entries /* BGTaskSchedulerPermittedIdentifiers com.timesafari.dailynotification.starred-projects-polling UIBackgroundModes background-app-refresh */ // Background task handler with proper completion func handleStarredProjectsPolling(task: BGAppRefreshTask) { // Set expiration handler immediately to avoid budget penalties task.expirationHandler = { task.setTaskCompleted(success: false) } Task { do { let result = try await pollStarredProjectChanges() // Explicit completion inside async flow task.setTaskCompleted(success: true) // Schedule next task scheduleNextBackgroundTask() } catch { // Explicit completion on error task.setTaskCompleted(success: false) } } } ``` **Web (Service Worker + Alarms)**: ```typescript // Service Worker Background Sync self.addEventListener('sync', event => { if (event.tag === 'starred-projects-polling') { event.waitUntil(pollStarredProjects()); } }); // Chrome Alarms API for scheduling // Chrome Extension only chrome.alarms.create('starred-projects-polling', { delayInMinutes: 15, periodInMinutes: 60 }); ``` **Cron Expression Parser**: - **Format**: Standard cron with 5 fields: `minute hour day month weekday` - **Timezone**: All times in UTC, convert to local timezone for display - **Examples**: - `"0 9,17 * * *"` - 9 AM and 5 PM daily - `"0 10 * * 1-5"` - 10 AM weekdays only - `"*/15 * * * *"` - Every 15 minutes #### Background Constraints & Edge Cases **Android Doze/Idle Allowances**: - **Doze Mode**: WorkManager jobs deferred until maintenance window (typically 1-2 hours) - **App Standby**: Apps unused for 3+ days get reduced background execution - **Battery Optimization**: Users can whitelist apps for unrestricted background execution - **Maximum Tolerated Poll Latency**: 4 hours (SLO: 95% of polls within 2 hours) - **Fallback Strategy**: Show "stale data" banner if last successful poll > 6 hours ago **iOS BGAppRefresh Budget Expectations**: - **Daily Budget**: ~30 seconds of background execution per day - **Task Deferral**: iOS may defer tasks for hours during low battery/usage - **Fallback UX**: Show "Last updated X hours ago" with manual refresh button - **Background Modes**: Require "Background App Refresh" capability - **Task Expiration**: 30-second timeout for background tasks **Web Background Execution Strategies**: - **Chrome Alarms**: Extension-only, not available for web apps - **Service Worker**: Limited to 5-minute execution window - **Background Sync**: Requires user interaction to register - **Push Notifications**: Use push to "wake" app for polling - **Periodic Background Sync**: Experimental, limited browser support - **Fallback Strategy**: Poll on app focus/visibility change **Web Implementation**: ```typescript // Service Worker Background Sync self.addEventListener('sync', event => { if (event.tag === 'starred-projects-polling') { event.waitUntil(pollStarredProjects()); } }); // Push-based wake-up self.addEventListener('push', event => { if (event.data?.json()?.type === 'poll-trigger') { event.waitUntil(pollStarredProjects()); } }); // Visibility-based polling document.addEventListener('visibilitychange', () => { if (!document.hidden) { // App became visible, check if we need to poll const lastPoll = localStorage.getItem('lastPollTimestamp'); const now = Date.now(); if (now - parseInt(lastPoll) > 3600000) { // 1 hour pollStarredProjects().then(() => { localStorage.setItem('lastPollTimestamp', now.toString()); }); } } }); ``` ### Polling Logic & State Machine #### State Diagram ``` [Start] → [Validate Config] → [Check Starred Projects] → [Check Last Ack ID] ↓ ↓ ↓ ↓ [Error] [Skip Poll] [Skip Poll] [Skip Poll] ↓ ↓ ↓ ↓ [End] [End] [End] [End] ↓ [Make API Call] ← [Valid Config] ← [Has Starred Projects] ← [Has Last Ack ID] ↓ ↓ ↓ ↓ [Network Error] [Parse Response] [Process Results] [Generate Notifications] ↓ ↓ ↓ ↓ [Retry Logic] [Schedule Delivery] [Success] [Acknowledge Delivery] ↓ ↓ ↓ ↓ [Exponential Backoff] [Acknowledge Delivery] [End] [Update Watermark] ↓ ↓ ↓ ↓ [Max Retries] [Update Watermark] [End] [Commit State] ↓ ↓ ↓ ↓ [End] [Commit State] [End] [End] ↓ [End] ``` **State Transitions**: 1. **Validate Config**: Check `activeDid`, `apiServer`, authentication 2. **Check Starred Projects**: Verify `starredPlanHandleIds` is non-empty 3. **Check Last Ack ID**: If `lastAckedStarredPlanChangesJwtId` is null, run Bootstrap Watermark; then continue 4. **Make API Call**: Execute authenticated POST request 5. **Process Results**: Parse response and extract change count 6. **Generate Notifications**: Create user notifications for changes 7. **Update Watermark**: Advance watermark only after successful delivery AND acknowledgment #### Watermark Bootstrap Path **Bootstrap Implementation**: ```typescript async function bootstrapWatermark(activeDid: string, starredPlanHandleIds: string[]): Promise { try { // Fetch most recent jwtId with limit:1 const bootstrapResponse = await makeAuthenticatedPostRequest( `${apiServer}/api/v2/report/plansLastUpdatedBetween`, { planIds: starredPlanHandleIds, limit: 1 } ); if (bootstrapResponse.data && bootstrapResponse.data.length > 0) { const mostRecentJwtId = bootstrapResponse.data[0].planSummary.jwtId; // Set watermark to most recent jwtId await db.query( 'UPDATE settings SET lastAckedStarredPlanChangesJwtId = ? WHERE accountDid = ?', [mostRecentJwtId, activeDid] ); console.log(`Bootstrap watermark set to: ${mostRecentJwtId}`); return mostRecentJwtId; } else { // No existing data, watermark remains null for first poll console.log('No existing data found, watermark remains null'); return null; } } catch (error) { console.error('Bootstrap watermark failed:', error); throw error; } } ``` **Bootstrap Integration**: ```typescript // In main polling flow if (!config.lastAckedStarredPlanChangesJwtId) { console.log('No watermark found, bootstrapping...'); await bootstrapWatermark(config.activeDid, config.starredPlanHandleIds); // Re-fetch config to get updated watermark config = await getUserConfiguration(); } ``` #### Transactional Outbox Pattern **Classic Outbox Implementation**: ```sql -- Outbox table for reliable delivery (watermark advances only after delivery + acknowledgment) CREATE TABLE notification_outbox ( id INTEGER PRIMARY KEY AUTOINCREMENT, jwt_id TEXT NOT NULL, content TEXT NOT NULL, created_at DATETIME DEFAULT CURRENT_TIMESTAMP, delivered_at DATETIME NULL, retry_count INTEGER DEFAULT 0, max_retries INTEGER DEFAULT 3 ); CREATE INDEX idx_outbox_undelivered ON notification_outbox(delivered_at) WHERE delivered_at IS NULL; ``` **Atomic Transaction Pattern**: ```sql -- Phase 1: Atomic commit of outbox only (watermark stays unchanged) BEGIN TRANSACTION; INSERT INTO notification_outbox (jwt_id, content) VALUES (?, ?); -- Do NOT update watermark here - it advances only after delivery + acknowledgment COMMIT; ``` **Separate Dispatcher Process**: ```typescript class NotificationDispatcher { async processOutbox(): Promise { const undelivered = await db.query(` SELECT * FROM notification_outbox WHERE delivered_at IS NULL AND retry_count < max_retries ORDER BY created_at ASC LIMIT 10 `); for (const notification of undelivered) { try { await this.deliverNotification(notification); // Mark as delivered in separate transaction await db.query(` UPDATE notification_outbox SET delivered_at = datetime('now') WHERE id = ? `, [notification.id]); } catch (error) { // Increment retry count await db.query(` UPDATE notification_outbox SET retry_count = retry_count + 1 WHERE id = ? `, [notification.id]); } } } } ``` **Recovery Rules**: - **Crash After Commit**: Outbox contains pending notifications, dispatcher processes them - **Crash Before Commit**: Safe to retry - no state change occurred - **Dispatcher Failure**: Retry count prevents infinite loops, max retries = 3 - **Cleanup**: Delete delivered notifications older than 7 days #### Acknowledgment Semantics vs Watermark **Watermark Advancement Policy**: ```typescript interface WatermarkPolicy { // Watermark advances ONLY after successful notification delivery AND acknowledgment advanceAfterDelivery: true; advanceAfterAck: true; // Acknowledgment is REQUIRED, not advisory ackRequired: true; // Watermark remains unchanged until both conditions met atomicAdvancement: true; } ``` **Implementation Flow**: ```typescript async function processPollingResults(results: PollingResult[]): Promise { for (const result of results) { // 1. Insert into outbox (not yet delivered) await db.query(` INSERT INTO notification_outbox (jwt_id, content) VALUES (?, ?) `, [result.jwtId, JSON.stringify(result.content)]); // 2. Watermark remains unchanged until delivery + ack // (No watermark update here) } // 3. Dispatcher delivers notifications const { deliveredJwtIds, latestJwtId } = await notificationDispatcher.processOutbox(); // 4. After successful delivery, call acknowledgment endpoint await acknowledgeDeliveredNotifications(deliveredJwtIds); // 5. ONLY NOW advance watermark await advanceWatermark(latestJwtId); } ``` **Prevents Replays and Gaps**: - **No Replays**: Watermark only advances after successful delivery + ack - **No Gaps**: Failed deliveries remain in outbox for retry - **Atomicity**: Watermark advancement is atomic with acknowledgment #### Transactional Outbox Pattern (Correct Approach) **Outbox-First Pattern** (watermark advances only after delivery + acknowledgment): **Phase 1 - Commit Outbox Only**: ```sql BEGIN TRANSACTION; INSERT INTO notification_outbox (accountDid, jwtId, notificationData, created_at) VALUES (?, ?, ?, datetime('now')); -- Do NOT update watermark yet COMMIT; ``` **Phase 2 - After Delivery + Acknowledgment**: ```sql -- Only after successful notification delivery AND acknowledgment UPDATE settings SET lastAckedStarredPlanChangesJwtId = ? WHERE accountDid = ?; UPDATE notification_outbox SET delivered_at = datetime('now'), acknowledged_at = datetime('now') WHERE jwtId = ?; ``` **Recovery Rules**: - **Crash After Watermark Update**: On restart, check `notification_outbox` table for uncommitted notifications - **Crash Before Watermark Update**: Safe to retry - no state change occurred - **Partial Notification Failure**: Rollback watermark update, retry entire transaction - **Acknowledgment Endpoint**: Call `POST /api/v2/plans/acknowledge` after successful notification delivery **Recovery Implementation**: ```typescript async function recoverPendingNotifications(): Promise { const pending = await db.query('SELECT * FROM notification_outbox WHERE created_at < ?', [Date.now() - 300000]); // 5 minutes ago for (const notification of pending) { try { await scheduleNotification(notification.content); await db.query('DELETE FROM notification_outbox WHERE id = ?', [notification.id]); } catch (error) { // Log error, will retry on next recovery cycle console.error('Failed to recover notification:', error); } } } ``` ### Notifications UX & Content #### Copy Templates **Single Project Update**: ``` Title: "Project Update" Body: "{projectName} has been updated" ``` **Multiple Project Updates**: ``` Title: "Project Updates" Body: "You have {count} new updates in your starred projects" ``` **Localization Keys**: ```json { "notifications.starred_projects.single_update": "{projectName} has been updated", "notifications.starred_projects.multiple_updates": "You have {count} new updates in your starred projects", "notifications.starred_projects.action_view": "View Updates", "notifications.starred_projects.action_dismiss": "Dismiss" } ``` **Per-Platform Capability Matrix**: | Feature | Android | iOS | Web | |---------|---------|-----|-----| | Sound | ✅ | ✅ | ✅ | | Vibration | ✅ | ✅ | ❌ | | Action Buttons | ✅ | ✅ | ✅ | | Grouping | ✅ | ✅ | ✅ | | Deep Links | ✅ | ✅ | ✅ | | Rich Media | ✅ | ✅ | ❌ | | Persistent | ✅ | ✅ | ❌ | **Notification Collapsing Rules**: - **Single Update**: Show individual project notification - **Multiple Updates (2-5)**: Show grouped notification with count - **Many Updates (6+)**: Show summary notification with "View All" action - **Time Window**: Collapse updates within 5-minute window #### Notifications UX Contract **Deep Link Routes**: ``` timesafari://projects/updates?jwtIds=1704067200_abc123_def45678,1704153600_mno345_0badf00d timesafari://projects/{projectId}/details?jwtId=1704067200_abc123_def45678 timesafari://notifications/starred-projects timesafari://projects/updates?shortlink=abc123def456789 ``` **Argument Validation Rules**: ```typescript interface DeepLinkValidation { jwtIds: string[]; // Max 10 JWT IDs per request projectId: string; // Must match /^[a-zA-Z0-9_-]+$/ jwtId: string; // Must match /^[0-9]{10}_[a-zA-Z0-9]{6}_[a-f0-9]{8}$/ shortlink: string; // Server-generated shortlink for "View All" } // Validation implementation function validateDeepLinkParams(params: any): DeepLinkValidation { if (params.jwtIds && Array.isArray(params.jwtIds)) { if (params.jwtIds.length > 10) { throw new Error('Too many JWT IDs, use shortlink instead'); } params.jwtIds.forEach(id => { if (!/^[0-9]{10}_[a-zA-Z0-9]{6}_[a-f0-9]{8}$/.test(id)) { throw new Error(`Invalid JWT ID format: ${id}`); } }); } return params; } ``` **"View All" Shortlink Example**: ```typescript // Server generates shortlink for large result sets const shortlink = await generateShortlink({ jwtIds: ['1704067200_abc123_def45678', '1704153600_mno345_0badf00d', /* ... 50 more */], expiresAt: Date.now() + 86400000 // 24 hours }); // Deep link uses shortlink instead of long query string const deepLink = `timesafari://projects/updates?shortlink=${shortlink}`; ``` **Action Button IDs**: ```typescript interface NotificationActions { viewUpdates: 'view_updates'; viewProject: 'view_project'; dismiss: 'dismiss'; snooze: 'snooze_1h'; markRead: 'mark_read'; } ``` **Expected Callbacks**: ```typescript interface NotificationCallbacks { onViewUpdates: (jwtIds: string[]) => void; onViewProject: (projectId: string, jwtId: string) => void; onDismiss: (notificationId: string) => void; onSnooze: (notificationId: string, duration: number) => void; onMarkRead: (jwtIds: string[]) => void; } ``` **Grouping Keys Per Platform**: - **Android**: `GROUP_KEY_STARRED_PROJECTS` with summary notification - **iOS**: `threadIdentifier: "starred-projects"` with collapse identifier - **Web**: `tag: "starred-projects"` with replace behavior **i18n Catalog Ownership**: - **Plugin Responsibility**: Core notification templates and action labels - **Host App Responsibility**: Project names, user-specific content, locale preferences - **Pluralization Rules**: Use ICU MessageFormat for proper pluralization **Staleness UX**: **Copy & i18n Keys**: ```json { "staleness.banner.title": "Data may be outdated", "staleness.banner.message": "Last updated {hours} hours ago. Tap to refresh.", "staleness.banner.action_refresh": "Refresh Now", "staleness.banner.action_settings": "Settings", "staleness.banner.dismiss": "Dismiss" } ``` **Per-Platform Behavior**: **Android**: ```kotlin // Show banner in notification area fun showStalenessBanner(hoursSinceUpdate: Int) { val notification = NotificationCompat.Builder(context, CHANNEL_ID) .setSmallIcon(R.drawable.ic_warning) .setContentTitle(context.getString(R.string.staleness_banner_title)) .setContentText(context.getString(R.string.staleness_banner_message, hoursSinceUpdate)) .setPriority(NotificationCompat.PRIORITY_LOW) .addAction(R.drawable.ic_refresh, "Refresh", refreshPendingIntent) .addAction(R.drawable.ic_settings, "Settings", settingsPendingIntent) .setAutoCancel(true) .build() notificationManager.notify(STALENESS_NOTIFICATION_ID, notification) } ``` **iOS**: ```swift // Show banner in app func showStalenessBanner(hoursSinceUpdate: Int) { let banner = BannerView() banner.title = NSLocalizedString("staleness.banner.title", comment: "") banner.message = String(format: NSLocalizedString("staleness.banner.message", comment: ""), hoursSinceUpdate) banner.addAction(title: NSLocalizedString("staleness.banner.action_refresh", comment: "")) { self.refreshData() } banner.addAction(title: NSLocalizedString("staleness.banner.action_settings", comment: "")) { self.openSettings() } banner.show() } ``` **Web**: ```typescript // Show toast notification function showStalenessBanner(hoursSinceUpdate: number): void { const toast = document.createElement('div'); toast.className = 'staleness-banner'; toast.innerHTML = ` `; document.body.appendChild(toast); // Auto-dismiss after 10 seconds setTimeout(() => toast.remove(), 10000); } ``` ### Error Handling & Telemetry #### Retry Matrix | Error Type | Max Attempts | Backoff Strategy | Jitter | |------------|--------------|------------------|--------| | Network Timeout | 3 | Exponential (1s, 2s, 4s) | ±25% | | HTTP 429 | 5 | Linear (Retry-After) | ±10% | | HTTP 5xx | 3 | Exponential (2s, 4s, 8s) | ±50% | | JSON Parse | 1 | None | None | | Auth Failure | 2 | Linear (30s) | None | **Logging Schema**: ```typescript interface PollingLogEntry { timestamp: string; level: 'DEBUG' | 'INFO' | 'WARN' | 'ERROR'; event: string; activeDid: string; projectCount: number; changeCount: number; duration: number; error?: string; metadata?: Record; } ``` **Metrics Schema**: ```typescript interface PollingMetrics { pollsTotal: number; pollsSuccessful: number; pollsFailed: number; changesFound: number; notificationsGenerated: number; throttlesHit: number; averageResponseTime: number; lastPollTimestamp: string; } ``` **PII Policy**: - **Logs**: No PII in logs, use hashed identifiers - **Metrics**: Aggregate data only, no individual user tracking - **Storage**: Encrypt sensitive data at rest - **Transmission**: Use HTTPS for all API calls #### Telemetry, Privacy & Retention **Final Metric Names & Cardinality Limits**: ```typescript interface TelemetryMetrics { // Polling metrics 'starred_projects.poll.attempts': number; // Cardinality: 1 'starred_projects.poll.success': number; // Cardinality: 1 'starred_projects.poll.failure': number; // Cardinality: 1 'starred_projects.poll.duration_ms': number; // Cardinality: 1 'starred_projects.poll.changes_found': number; // Cardinality: 1 'starred_projects.poll.notifications_generated': number; // Cardinality: 1 // Error metrics 'starred_projects.error.network': number; // Cardinality: 1 'starred_projects.error.auth': number; // Cardinality: 1 'starred_projects.error.rate_limit': number; // Cardinality: 1 'starred_projects.error.parse': number; // Cardinality: 1 // Performance metrics 'starred_projects.api.latency_p95_ms': number; // Cardinality: 1 'starred_projects.api.latency_p99_ms': number; // Cardinality: 1 'starred_projects.api.throughput_rps': number; // Cardinality: 1 } ``` **Log Redaction List**: ```typescript const PII_REDACTION_PATTERNS = [ /did:key:[a-zA-Z0-9]+/g, // DID identifiers /\b\d{10}_[A-Za-z0-9]{6}_[a-f0-9]{8}\b/g, // JWT IDs (timestamp_random_hash) /Bearer [a-zA-Z0-9._-]+/g, // JWT tokens /activeDid":\s*"[^"]+"/g, // Active DID in JSON /accountDid":\s*"[^"]+"/g, // Account DID in JSON /issuerDid":\s*"[^"]+"/g, // Issuer DID in JSON /agentDid":\s*"[^"]+"/g, // Agent DID in JSON ]; ``` **Retention Policy**: - **Settings Data**: Retain for 2 years after last activity - **Watermarks**: Retain for 1 year after last poll - **Notification History**: Retain for 30 days - **Error Logs**: Retain for 90 days - **Performance Metrics**: Retain for 1 year - **User Preferences**: Retain until user deletion **Per-Platform "No-PII at Rest" Checklist**: **Android**: - [ ] Encrypt `settings` table with SQLCipher - [ ] Store JWT secrets in Android Keystore - [ ] Hash DIDs before logging - [ ] Use encrypted SharedPreferences for sensitive data - [ ] Implement secure deletion of expired data **iOS**: - [ ] Encrypt Core Data with NSFileProtectionComplete - [ ] Store JWT secrets in iOS Keychain - [ ] Hash DIDs before logging - [ ] Use Keychain for sensitive user data - [ ] Implement secure deletion of expired data **Web**: - [ ] Encrypt IndexedDB with Web Crypto API - [ ] Store JWT secrets in encrypted storage (not localStorage) - [ ] Hash DIDs before logging - [ ] Use secure HTTP-only cookies for session data - [ ] Implement secure deletion of expired data ### Testing Artifacts #### Testing Fixtures & SLAs **Golden JSON Fixtures**: **Empty Response**: ```json { "data": [], "hitLimit": false, "pagination": { "hasMore": false, "nextAfterId": null } } ``` **Small Response (3 items)**: ```json { "data": [ { "planSummary": { "jwtId": "1704067200_abc123_def45678", "handleId": "test_project_1", "name": "Test Project 1", "description": "First test project", "issuerDid": "did:key:test_issuer_1", "agentDid": "did:key:test_agent_1", "locLat": 40.7128, "locLon": -74.0060 }, "previousClaim": { "jwtId": "1703980800_xyz789_0badf00d", "claimType": "project_update" } }, { "planSummary": { "jwtId": "1704153600_mno345_0badf00d", "handleId": "test_project_2", "name": "Test Project 2", "description": "Second test project", "issuerDid": "did:key:test_issuer_2", "agentDid": "did:key:test_agent_2", "locLat": null, "locLon": null }, "previousClaim": { "jwtId": "1704067200_stu901_1cafebad", "claimType": "project_update" } }, { "planSummary": { "jwtId": "1704240000_new123_0badf00d", "handleId": "test_project_3", "name": "Test Project 3", "description": "Third test project", "issuerDid": "did:key:test_issuer_3", "agentDid": "did:key:test_agent_3", "locLat": 37.7749, "locLon": -122.4194 }, "previousClaim": { "jwtId": "1704153600_old456_0badf00d", "claimType": "project_update" } } ], "hitLimit": false, "pagination": { "hasMore": false, "nextAfterId": null } } ``` **Paginated Response (100 items)**: ```json { "data": [ // ... 100 items with jwtIds from "1704067200_abc123_def45678" to "1704153600_xyz789_0badf00d" ], "hitLimit": true, "pagination": { "hasMore": true, "nextAfterId": "1704240000_new123_1cafebad" } } ``` **Rate Limited Response**: ```json { "error": "Rate limit exceeded", "code": "RATE_LIMIT_EXCEEDED", "message": "Rate limit exceeded for DID", "details": { "limit": 100, "window": "1m", "resetAt": "2025-01-01T12:01:00Z" }, "retryAfter": 60, "requestId": "req_jkl012" } ``` **Malformed Response**: ```json { "data": [ { "planSummary": { "jwtId": "invalid_jwt_id", "handleId": null, "name": "", "description": null, "locLat": null, "locLon": null }, "previousClaim": { "jwtId": "1703980800_xyz789_0badf00d", "claimType": "project_update" } } ], "hitLimit": false, "pagination": { "hasMore": false, "nextAfterId": null } } ``` **Mixed Order Response (should never happen)**: ```json { "data": [ { "planSummary": { "jwtId": "1704153600_mno345_0badf00d", "handleId": "test_project_2", "locLat": null, "locLon": null } }, { "planSummary": { "jwtId": "1704067200_abc123_def45678", "handleId": "test_project_1", "locLat": 40.7128, "locLon": -74.0060 } } ], "hitLimit": false, "pagination": { "hasMore": false, "nextAfterId": null } } ``` **Target SLAs**: - **P95 Latency**: 500ms for `/api/v2/report/plansLastUpdatedBetween` - **P99 Latency**: 2s for `/api/v2/report/plansLastUpdatedBetween` - **Availability**: 99.9% uptime - **Missed Poll Window**: 4 hours before "stale" banner - **Recovery Time**: 30 seconds after network restoration - **Data Freshness**: 5 seconds maximum delay ### Cross-References to Code #### Key Implementation Files **EnhancedDailyNotificationFetcher**: - **Location**: `src/android/EnhancedDailyNotificationFetcher.java` - **Key Methods**: `fetchAllTimeSafariData()`, `fetchProjectsLastUpdated()` - **Integration**: Extends existing `DailyNotificationFetcher` with TimeSafari-specific endpoints **DailyNotificationStorage**: - **Location**: `src/android/DailyNotificationStorage.java` - **Key Methods**: `getString()`, `putString()`, `shouldFetchNewContent()` - **Integration**: Stores polling configuration and watermark state **DailyNotificationDatabase**: - **Location**: `src/android/DailyNotificationDatabase.java` - **Key Methods**: `settingsDao()`, `activeIdentityDao()` - **Integration**: Provides SQLite access for user settings and identity **scheduleUserNotification**: - **Location**: `src/android/DailyNotificationScheduler.java` - **Parameters**: `NotificationContent`, `scheduleTime`, `priority` - **Side Effects**: Creates `PendingIntent`, schedules with `AlarmManager` **Original loadNewStarredProjectChanges**: - **Location**: `crowd-funder-for-time/src/views/HomeView.vue` (lines 876-901) - **Key Logic**: API call to `/api/v2/report/plansLastUpdatedBetween` - **Parity Check**: Same endpoint, same request/response format, same error handling #### Implementation Checklist - [ ] **API Integration**: Implement `/api/v2/report/plansLastUpdatedBetween` endpoint - [ ] **Database Schema**: Create `settings` table with proper indices - [ ] **JWT Management**: Integrate with `DailyNotificationJWTManager` - [ ] **Background Tasks**: Add polling to existing WorkManager/BGTaskScheduler - [ ] **State Management**: Implement watermark persistence and atomic updates - [ ] **Error Handling**: Add retry logic and exponential backoff - [ ] **Notifications**: Generate contextual notifications for project changes - [ ] **Testing**: Create mock fixtures and contract tests - [ ] **Telemetry**: Add logging and metrics collection - [ ] **Documentation**: Update API docs and integration guides ### Migration & Multi-Identity Edges #### ActiveDid Change Behavior **When `activeDid` Changes**: ```typescript interface ActiveDidChangePolicy { // Reset watermark and starred list for security resetWatermark: true; resetStarredProjects: true; // Clear all cached data clearCache: true; // Invalidate all pending notifications cancelPendingNotifications: true; // Reset authentication state clearJWT: true; } ``` **Migration Implementation**: ```typescript async function handleActiveDidChange(newDid: string, oldDid: string): Promise { // 1. Cancel all pending notifications await cancelAllPendingNotifications(); // 2. Clear cached data await clearPollingCache(); // 3. Reset watermark await db.query('UPDATE settings SET lastAckedStarredPlanChangesJwtId = NULL WHERE accountDid = ?', [newDid]); // 4. Reset starred projects (user must re-star) await db.query('UPDATE settings SET starredPlanHandleIds = ? WHERE accountDid = ?', ['[]', newDid]); // 5. Clear JWT authentication await jwtManager.clearAuthentication(); // 6. Log the change await logActiveDidChange(newDid, oldDid); } ``` **Migration Story for Pre-Existing Users**: **Phase 1 - Detection**: ```sql -- Detect users with existing notification settings but no starred projects polling SELECT accountDid FROM settings WHERE notificationPreferences IS NOT NULL AND (starredPlanHandleIds IS NULL OR starredPlanHandleIds = '[]'); ``` **Phase 2 - Migration**: ```typescript async function migrateExistingUsers(): Promise { const existingUsers = await db.query(` SELECT accountDid, notificationPreferences FROM settings WHERE notificationPreferences IS NOT NULL AND (starredPlanHandleIds IS NULL OR starredPlanHandleIds = '[]') `); for (const user of existingUsers) { // Enable starred projects polling with default settings await db.query(` UPDATE settings SET starredPlanHandleIds = '[]', lastAckedStarredPlanChangesJwtId = NULL, updated_at = datetime('now') WHERE accountDid = ? `, [user.accountDid]); // Log migration await logUserMigration(user.accountDid); } } ``` **Phase 3 - User Notification**: ```typescript async function notifyUsersOfNewFeature(): Promise { const migratedUsers = await getMigratedUsers(); for (const user of migratedUsers) { // Send in-app notification about new starred projects feature await sendFeatureNotification(user.accountDid, { title: 'New Feature: Starred Projects', body: 'You can now get notifications for projects you star. Tap to learn more.', actionUrl: 'timesafari://features/starred-projects' }); } } ``` **Safe Carry-Over Strategy**: - **No Automatic Carry-Over**: For security, never automatically carry over starred projects between DIDs - **User Choice**: Provide UI for users to manually re-star projects after DID change - **Migration Wizard**: Guide users through re-configuring their starred projects - **Data Retention**: Keep old starred projects data for 30 days for user reference **Edge Case Handling**: - **Concurrent DID Changes**: Use database locks to prevent race conditions - **Partial Migration**: Rollback mechanism for failed migrations - **User Cancellation**: Allow users to opt-out of starred projects polling - **Data Cleanup**: Automatic cleanup of orphaned data after 30 days ## Key Advantages Over Generic "New Offers" - **User-Controlled**: Only monitors projects user has explicitly starred - **Change-Focused**: Shows actual updates to projects, not just new offers - **Rich Context**: Returns both project summary and previous claim data - **Efficient**: Only checks projects user cares about - **Targeted**: More relevant than generic offer notifications ## Implementation Architecture ### 1. **Data Requirements** The plugin needs access to TimeSafari's user data: ```typescript interface StarredProjectsPollingConfig { // From active_identity table activeDid: string; // From settings table apiServer: string; starredPlanHandleIds: string[]; // JSON array of project IDs lastAckedStarredPlanChangesJwtId?: string; // Last acknowledged change // Authentication jwtSecret: string; tokenExpirationMinutes: number; } ``` ### 2. **Database Queries** ```sql -- Get active DID SELECT activeDid FROM active_identity WHERE id = 1; -- Get user settings SELECT apiServer, starredPlanHandleIds, lastAckedStarredPlanChangesJwtId FROM settings WHERE accountDid = ?; ``` ### 3. **API Integration** **Endpoint**: `POST /api/v2/report/plansLastUpdatedBetween` **Request**: ```json { "planIds": ["project1", "project2", "..."], "afterId": "last_acknowledged_jwt_id" } ``` **Response**: ```json { "data": [ { "planSummary": { "jwtId": "project_jwt_id", "handleId": "project_handle", "name": "Project Name", "description": "Project description", "issuerDid": "issuer_did", "agentDid": "agent_did", "startTime": "2025-01-01T00:00:00Z", "endTime": "2025-01-31T23:59:59Z", "locLat": 40.7128, "locLon": -74.0060, "url": "https://project-url.com" }, "previousClaim": { // Previous claim data for comparison } } ], "hitLimit": false, "pagination": { "hasMore": false, "nextAfterId": null } } ``` ## Platform-Specific Implementation ### Android Implementation **File**: `src/android/StarredProjectsPollingManager.java` ```java package com.timesafari.dailynotification; import android.content.Context; import android.util.Log; import com.google.gson.Gson; import com.google.gson.reflect.TypeToken; import java.util.*; import java.util.concurrent.CompletableFuture; /** * Manages polling for starred project changes * Adapts crowd-funder-for-time loadNewStarredProjectChanges pattern */ public class StarredProjectsPollingManager { private static final String TAG = "StarredProjectsPollingManager"; private final Context context; private final DailyNotificationJWTManager jwtManager; private final DailyNotificationStorage storage; private final Gson gson; public StarredProjectsPollingManager(Context context, DailyNotificationJWTManager jwtManager, DailyNotificationStorage storage) { this.context = context; this.jwtManager = jwtManager; this.storage = storage; this.gson = new Gson(); } /** * Main polling method - adapts loadNewStarredProjectChanges */ public CompletableFuture pollStarredProjectChanges() { return CompletableFuture.supplyAsync(() -> { try { // 1. Get user configuration StarredProjectsConfig config = getUserConfiguration(); if (config == null || !isValidConfiguration(config)) { Log.d(TAG, "Invalid configuration, skipping poll"); return new StarredProjectsPollingResult(0, false, "Invalid configuration"); } // 2. Check if we have starred projects if (config.starredPlanHandleIds == null || config.starredPlanHandleIds.isEmpty()) { Log.d(TAG, "No starred projects, skipping poll"); return new StarredProjectsPollingResult(0, false, "No starred projects"); } // 3. Check if we have last acknowledged ID, bootstrap if missing if (config.lastAckedStarredPlanChangesJwtId == null || config.lastAckedStarredPlanChangesJwtId.isEmpty()) { Log.d(TAG, "No last acknowledged ID, running bootstrap watermark"); String bootstrapWatermark = bootstrapWatermark(config.activeDid, config.starredPlanHandleIds); if (bootstrapWatermark != null) { // Update config with bootstrap watermark and reload storage.putString("lastAckedStarredPlanChangesJwtId", bootstrapWatermark); config = loadStarredProjectsConfig(config.activeDid); Log.d(TAG, "Bootstrap watermark set: " + bootstrapWatermark); } else { Log.w(TAG, "Bootstrap watermark failed, skipping poll"); return new StarredProjectsPollingResult(0, false, "Bootstrap watermark failed"); } } // 4. Make API call Log.d(TAG, "Polling " + config.starredPlanHandleIds.size() + " starred projects"); PlansLastUpdatedResponse response = fetchStarredProjectsWithChanges(config); // 5. Process results int changeCount = response.data != null ? response.data.size() : 0; Log.d(TAG, "Found " + changeCount + " project changes"); return new StarredProjectsPollingResult(changeCount, response.hitLimit, null); } catch (Exception e) { Log.e(TAG, "Error polling starred project changes: " + e.getMessage(), e); return new StarredProjectsPollingResult(0, false, e.getMessage()); } }); } /** * Fetch starred projects with changes from Endorser.ch API */ private PlansLastUpdatedResponse fetchStarredProjectsWithChanges(StarredProjectsConfig config) { try { String url = config.apiServer + "/api/v2/report/plansLastUpdatedBetween"; // Prepare request body Map requestBody = new HashMap<>(); requestBody.put("planIds", config.starredPlanHandleIds); requestBody.put("afterId", config.lastAckedStarredPlanChangesJwtId); // Make authenticated POST request return makeAuthenticatedPostRequest(url, requestBody, PlansLastUpdatedResponse.class); } catch (Exception e) { Log.e(TAG, "Error fetching starred projects: " + e.getMessage(), e); return new PlansLastUpdatedResponse(); // Return empty response } } /** * Get user configuration from storage */ private StarredProjectsConfig getUserConfiguration() { try { // Get active DID String activeDid = storage.getString("activeDid", null); if (activeDid == null) { Log.w(TAG, "No active DID found"); return null; } // Get settings String apiServer = storage.getString("apiServer", null); String starredPlanHandleIdsJson = storage.getString("starredPlanHandleIds", "[]"); String lastAckedId = storage.getString("lastAckedStarredPlanChangesJwtId", null); // Parse starred project IDs Type listType = new TypeToken>(){}.getType(); List starredPlanHandleIds = gson.fromJson(starredPlanHandleIdsJson, listType); return new StarredProjectsConfig(activeDid, apiServer, starredPlanHandleIds, lastAckedId); } catch (Exception e) { Log.e(TAG, "Error getting user configuration: " + e.getMessage(), e); return null; } } /** * Validate configuration */ private boolean isValidConfiguration(StarredProjectsConfig config) { return config.activeDid != null && !config.activeDid.isEmpty() && config.apiServer != null && !config.apiServer.isEmpty(); } /** * Make authenticated POST request */ private T makeAuthenticatedPostRequest(String url, Map requestBody, Class responseClass) { // Implementation similar to EnhancedDailyNotificationFetcher // Uses jwtManager for authentication // Returns parsed response return null; // Placeholder } // Data classes public static class StarredProjectsConfig { public final String activeDid; public final String apiServer; public final List starredPlanHandleIds; public final String lastAckedStarredPlanChangesJwtId; public StarredProjectsConfig(String activeDid, String apiServer, List starredPlanHandleIds, String lastAckedStarredPlanChangesJwtId) { this.activeDid = activeDid; this.apiServer = apiServer; this.starredPlanHandleIds = starredPlanHandleIds; this.lastAckedStarredPlanChangesJwtId = lastAckedStarredPlanChangesJwtId; } } public static class StarredProjectsPollingResult { public final int changeCount; public final boolean hitLimit; public final String error; public StarredProjectsPollingResult(int changeCount, boolean hitLimit, String error) { this.changeCount = changeCount; this.hitLimit = hitLimit; this.error = error; } } public static class PlansLastUpdatedResponse { public List data = new ArrayList<>(); public boolean hitLimit; } public static class PlanSummaryAndPreviousClaim { public PlanSummary planSummary; public Map previousClaim; } public static class PlanSummary { public String jwtId; public String handleId; public String name; public String description; public String issuerDid; public String agentDid; public String startTime; public String endTime; public Double locLat; public Double locLon; public String url; } } ``` ### iOS Implementation **File**: `ios/Plugin/StarredProjectsPollingManager.swift` ```swift import Foundation import UserNotifications /** * iOS implementation of starred projects polling * Adapts crowd-funder-for-time loadNewStarredProjectChanges pattern */ class StarredProjectsPollingManager { private let TAG = "StarredProjectsPollingManager" private let database: DailyNotificationDatabase private let jwtManager: DailyNotificationJWTManager init(database: DailyNotificationDatabase, jwtManager: DailyNotificationJWTManager) { self.database = database self.jwtManager = jwtManager } /** * Main polling method - adapts loadNewStarredProjectChanges */ func pollStarredProjectChanges() async throws -> StarredProjectsPollingResult { do { // 1. Get user configuration guard let config = try await getUserConfiguration() else { print("\(TAG): Invalid configuration, skipping poll") return StarredProjectsPollingResult(changeCount: 0, hitLimit: false, error: "Invalid configuration") } // 2. Check if we have starred projects guard let starredPlanHandleIds = config.starredPlanHandleIds, !starredPlanHandleIds.isEmpty else { print("\(TAG): No starred projects, skipping poll") return StarredProjectsPollingResult(changeCount: 0, hitLimit: false, error: "No starred projects") } // 3. Check if we have last acknowledged ID, bootstrap if missing guard let lastAckedId = config.lastAckedStarredPlanChangesJwtId, !lastAckedId.isEmpty else { print("\(TAG): No last acknowledged ID, running bootstrap watermark") let bootstrapWatermark = try await bootstrapWatermark(activeDid: config.activeDid, starredPlanHandleIds: starredPlanHandleIds) if let bootstrapWatermark = bootstrapWatermark { // Update config with bootstrap watermark and reload try await updateLastAckedId(bootstrapWatermark, for: config.activeDid) let updatedConfig = try await getUserConfiguration() if let updatedConfig = updatedConfig { print("\(TAG): Bootstrap watermark set: \(bootstrapWatermark)") // Continue with updated config return try await pollStarredProjectChangesWithConfig(updatedConfig) } } else { print("\(TAG): Bootstrap watermark failed, skipping poll") return StarredProjectsPollingResult(changeCount: 0, hitLimit: false, error: "Bootstrap watermark failed") } } // 4. Make API call print("\(TAG): Polling \(starredPlanHandleIds.count) starred projects") let response = try await fetchStarredProjectsWithChanges(config: config) // 5. Process results let changeCount = response.data?.count ?? 0 print("\(TAG): Found \(changeCount) project changes") return StarredProjectsPollingResult(changeCount: changeCount, hitLimit: response.hitLimit, error: nil) } catch { print("\(TAG): Error polling starred project changes: \(error)") return StarredProjectsPollingResult(changeCount: 0, hitLimit: false, error: error.localizedDescription) } } /** * Fetch starred projects with changes from Endorser.ch API */ private func fetchStarredProjectsWithChanges(config: StarredProjectsConfig) async throws -> PlansLastUpdatedResponse { let url = URL(string: "\(config.apiServer)/api/v2/report/plansLastUpdatedBetween")! // Prepare request body let requestBody: [String: Any] = [ "planIds": config.starredPlanHandleIds!, "afterId": config.lastAckedStarredPlanChangesJwtId! ] // Make authenticated POST request return try await makeAuthenticatedPostRequest(url: url, requestBody: requestBody) } /** * Get user configuration from database */ private func getUserConfiguration() async throws -> StarredProjectsConfig? { // Get active DID guard let activeDid = try await database.getActiveDid() else { print("\(TAG): No active DID found") return nil } // Get settings let settings = try await database.getSettings(accountDid: activeDid) // Parse starred project IDs let starredPlanHandleIds = try JSONDecoder().decode([String].self, from: (settings.starredPlanHandleIds ?? "[]").data(using: .utf8)!) return StarredProjectsConfig( activeDid: activeDid, apiServer: settings.apiServer, starredPlanHandleIds: starredPlanHandleIds, lastAckedStarredPlanChangesJwtId: settings.lastAckedStarredPlanChangesJwtId ) } /** * Make authenticated POST request */ private func makeAuthenticatedPostRequest(url: URL, requestBody: [String: Any]) async throws -> PlansLastUpdatedResponse { var request = URLRequest(url: url) request.httpMethod = "POST" request.setValue("application/json", forHTTPHeaderField: "Content-Type") // Add JWT authentication if let jwtToken = jwtManager.getCurrentJWTToken() { request.setValue("Bearer \(jwtToken)", forHTTPHeaderField: "Authorization") } // Add request body request.httpBody = try JSONSerialization.data(withJSONObject: requestBody) // Execute request let (data, response) = try await URLSession.shared.data(for: request) guard let httpResponse = response as? HTTPURLResponse, httpResponse.statusCode == 200 else { throw NSError(domain: "StarredProjectsPollingManager", code: -1, userInfo: [NSLocalizedDescriptionKey: "HTTP request failed"]) } return try JSONDecoder().decode(PlansLastUpdatedResponse.self, from: data) } // Data structures struct StarredProjectsConfig { let activeDid: String let apiServer: String? let starredPlanHandleIds: [String]? let lastAckedStarredPlanChangesJwtId: String? } struct StarredProjectsPollingResult { let changeCount: Int let hitLimit: Bool let error: String? } struct PlansLastUpdatedResponse: Codable { let data: [PlanSummaryAndPreviousClaim]? let hitLimit: Bool } struct PlanSummaryAndPreviousClaim: Codable { let planSummary: PlanSummary let previousClaim: [String: Any]? } struct PlanSummary: Codable { let jwtId: String let handleId: String let name: String let description: String let issuerDid: String let agentDid: String let startTime: String let endTime: String let locLat: Double? let locLon: Double? let url: String? } } ``` ### Web Implementation **File**: `src/web/StarredProjectsPollingManager.ts` ```typescript /** * Web implementation of starred projects polling * Adapts crowd-funder-for-time loadNewStarredProjectChanges pattern */ export class StarredProjectsPollingManager { private config: StarredProjectsConfig | null = null; private jwtManager: any; // JWT manager instance constructor(jwtManager: any) { this.jwtManager = jwtManager; } /** * Main polling method - adapts loadNewStarredProjectChanges */ async pollStarredProjectChanges(): Promise { try { // 1. Get user configuration const config = await this.getUserConfiguration(); if (!config || !this.isValidConfiguration(config)) { console.log('StarredProjectsPollingManager: Invalid configuration, skipping poll'); return { changeCount: 0, hitLimit: false, error: 'Invalid configuration' }; } // 2. Check if we have starred projects if (!config.starredPlanHandleIds || config.starredPlanHandleIds.length === 0) { console.log('StarredProjectsPollingManager: No starred projects, skipping poll'); return { changeCount: 0, hitLimit: false, error: 'No starred projects' }; } // 3. Check if we have last acknowledged ID, bootstrap if missing if (!config.lastAckedStarredPlanChangesJwtId) { console.log('StarredProjectsPollingManager: No last acknowledged ID, running bootstrap watermark'); const bootstrapWatermark = await this.bootstrapWatermark(config.activeDid, config.starredPlanHandleIds); if (bootstrapWatermark) { // Update config with bootstrap watermark and reload await this.updateLastAckedId(bootstrapWatermark, config.activeDid); const updatedConfig = await this.getUserConfiguration(); if (updatedConfig) { console.log(`StarredProjectsPollingManager: Bootstrap watermark set: ${bootstrapWatermark}`); // Continue with updated config return await this.pollStarredProjectChangesWithConfig(updatedConfig); } } else { console.log('StarredProjectsPollingManager: Bootstrap watermark failed, skipping poll'); return { changeCount: 0, hitLimit: false, error: 'Bootstrap watermark failed' }; } } // 4. Make API call console.log(`StarredProjectsPollingManager: Polling ${config.starredPlanHandleIds.length} starred projects`); const response = await this.fetchStarredProjectsWithChanges(config); // 5. Process results const changeCount = response.data?.length || 0; console.log(`StarredProjectsPollingManager: Found ${changeCount} project changes`); return { changeCount, hitLimit: response.hitLimit, error: null }; } catch (error) { console.error('StarredProjectsPollingManager: Error polling starred project changes:', error); return { changeCount: 0, hitLimit: false, error: String(error) }; } } /** * Fetch starred projects with changes from Endorser.ch API */ private async fetchStarredProjectsWithChanges(config: StarredProjectsConfig): Promise { const url = `${config.apiServer}/api/v2/report/plansLastUpdatedBetween`; // Prepare request body const requestBody = { planIds: config.starredPlanHandleIds, afterId: config.lastAckedStarredPlanChangesJwtId }; // Make authenticated POST request return await this.makeAuthenticatedPostRequest(url, requestBody); } /** * Get user configuration from storage */ private async getUserConfiguration(): Promise { try { // Get active DID from localStorage or IndexedDB const activeDid = localStorage.getItem('activeDid'); if (!activeDid) { console.warn('StarredProjectsPollingManager: No active DID found'); return null; } // Get settings from localStorage or IndexedDB const apiServer = localStorage.getItem('apiServer'); const starredPlanHandleIdsJson = localStorage.getItem('starredPlanHandleIds') || '[]'; const lastAckedId = localStorage.getItem('lastAckedStarredPlanChangesJwtId'); // Parse starred project IDs const starredPlanHandleIds = JSON.parse(starredPlanHandleIdsJson); return { activeDid, apiServer: apiServer || '', starredPlanHandleIds, lastAckedStarredPlanChangesJwtId: lastAckedId || undefined }; } catch (error) { console.error('StarredProjectsPollingManager: Error getting user configuration:', error); return null; } } /** * Validate configuration */ private isValidConfiguration(config: StarredProjectsConfig): boolean { return !!(config.activeDid && config.apiServer); } /** * Make authenticated POST request */ private async makeAuthenticatedPostRequest(url: string, requestBody: any): Promise { const headers: Record = { 'Content-Type': 'application/json' }; // Add JWT authentication const jwtToken = await this.jwtManager.getCurrentJWTToken(); if (jwtToken) { headers['Authorization'] = `Bearer ${jwtToken}`; } const response = await fetch(url, { method: 'POST', headers, body: JSON.stringify(requestBody) }); if (!response.ok) { throw new Error(`HTTP ${response.status}: ${response.statusText}`); } return await response.json(); } } // Type definitions interface StarredProjectsConfig { activeDid: string; apiServer: string; starredPlanHandleIds: string[]; lastAckedStarredPlanChangesJwtId?: string; } interface StarredProjectsPollingResult { changeCount: number; hitLimit: boolean; error: string | null; } interface PlansLastUpdatedResponse { data?: PlanSummaryAndPreviousClaim[]; hitLimit: boolean; } interface PlanSummaryAndPreviousClaim { planSummary: PlanSummary; previousClaim?: Record; } interface PlanSummary { jwtId: string; handleId: string; name: string; description: string; issuerDid: string; agentDid: string; startTime: string; endTime: string; locLat?: number; locLon?: number; url?: string; } ``` ## Integration with Existing Plugin ### 1. **Extend ContentFetchConfig** Add starred projects polling to the existing configuration: ```typescript export interface ContentFetchConfig { // ... existing fields ... // Starred Projects Polling starredProjectsPolling?: { enabled: boolean; schedule: string; // Cron expression maxConcurrentPolls?: number; pollTimeoutMs?: number; retryAttempts?: number; }; } ``` ### 2. **Update Background Workers** **Android**: Extend `DailyNotificationFetchWorker.java` ```java // Add to doWork() method if (config.starredProjectsPolling != null && config.starredProjectsPolling.enabled) { StarredProjectsPollingManager pollingManager = new StarredProjectsPollingManager( getApplicationContext(), jwtManager, storage); CompletableFuture pollingResult = pollingManager.pollStarredProjectChanges(); // Process polling results StarredProjectsPollingResult result = pollingResult.get(); if (result.changeCount > 0) { // Generate notifications for project changes generateProjectChangeNotifications(result); } } ``` **iOS**: Extend `DailyNotificationBackgroundTasks.swift` ```swift // Add to handleBackgroundFetch if let starredProjectsConfig = config.starredProjectsPolling, starredProjectsConfig.enabled { let pollingManager = StarredProjectsPollingManager(database: database, jwtManager: jwtManager) do { let result = try await pollingManager.pollStarredProjectChanges() if result.changeCount > 0 { // Generate notifications for project changes try await generateProjectChangeNotifications(result: result) } } catch { print("Error polling starred projects: \(error)") } } ``` ### 3. **Notification Generation** Create notifications based on polling results: ```typescript function generateProjectChangeNotifications(result: StarredProjectsPollingResult): void { if (result.changeCount > 0) { const notification = { id: `starred_projects_${Date.now()}`, title: 'Project Updates', body: `You have ${result.changeCount} new updates in your starred projects`, priority: 'normal', sound: true, vibration: true }; // Schedule notification scheduleUserNotification(notification); } } ``` ## Configuration Example ```typescript const config: ConfigureOptions = { // ... existing configuration ... contentFetch: { enabled: true, schedule: "0 9,17 * * *", // 9 AM and 5 PM daily callbacks: { onSuccess: async (data) => { console.log('Content fetch successful:', data); } }, // Starred Projects Polling starredProjectsPolling: { enabled: true, schedule: "0 10,16 * * *", // 10 AM and 4 PM daily maxConcurrentPolls: 3, pollTimeoutMs: 15000, retryAttempts: 2 } } }; ``` ## Benefits of This Implementation 1. **User-Curated**: Only monitors projects user explicitly cares about 2. **Change-Focused**: Shows actual project updates, not just new offers 3. **Rich Context**: Provides both current and previous project state 4. **Efficient**: Reduces API calls by focusing on relevant projects 5. **Scalable**: Handles large numbers of starred projects gracefully 6. **Reliable**: Includes proper error handling and retry logic 7. **Cross-Platform**: Consistent implementation across Android, iOS, and Web ## Testing Strategy 1. **Unit Tests**: Test polling logic with mock API responses 2. **Integration Tests**: Test with real Endorser.ch API endpoints 3. **Performance Tests**: Verify efficient handling of large project lists 4. **Error Handling Tests**: Test network failures and API errors 5. **User Experience Tests**: Verify notification relevance and timing This implementation provides a sophisticated, user-focused notification system that adapts the proven `loadNewStarredProjectChanges` pattern to the Daily Notification Plugin's architecture.