daily-notification-plugin/doc/BACKGROUND_DATA_FETCHING_PLAN.md

Background Data Fetching Implementation Plan

Author: Matthew Raymer
Version: 1.0.0
Created: 2025-10-02 07:47:04 UTC
Last Updated: 2025-10-02 12:45:00 UTC

Overview

This document outlines the enhancement plan for background data fetching in the Daily Notification Plugin to support TimeSafari integration with Option A architecture. This plan builds upon our existing implementation and adds:

• Option A Enhancement: ActiveDid-aware authentication on existing infrastructure
• TimeSafari Integration: PlatformServiceMixin coordination with current plugin
• Authentication Layer: JWT/DID support over existing HTTP callback system
• Database Enhancement: Extend current storage with activeDid management
• Event & Change Management: Identity change detection on existing notification system
• API Integration: Endorser.ch endpoints through current ContentFetchConfig

This document serves as the enhancement roadmap for adding TimeSafari capabilities to our existing, working plugin.

Current Implementation Baseline

✅ Already Implemented & Working

• Android Storage: SharedPreferences + SQLite with migration (DailyNotificationStorage.java, DailyNotificationDatabase.java)
• Web Storage: IndexedDB with comprehensive service worker (sw.ts, IndexedDBManager)
• Callback System: HTTP/local callbacks with circuit breaker (callback-registry.ts)
• Configuration: Database path, TTL, retention settings (ConfigureOptions)
• ETag Support: Conditional HTTP requests (DailyNotificationETagManager.java)
• Dual Scheduling: Content fetch + user notification separation
• Cross-Platform API: Unified TypeScript interface (DailyNotificationPlugin)

⚠️ Enhancement Required for TimeSafari

• ActiveDid Integration: Add activeDid-awareness to existing authentication
• JWT Generation: Enhance HTTP layer with DID-based tokens
• Identity Change Detection: Add event listeners to existing callback system
• Endorser.ch APIs: Extend ContentFetchConfig with TimeSafari endpoints
• Platform Auth: Add Android Keystore/iOS Keychain to existing storage

Consolidated Architecture: Option A Platform Overview

TimeSafari Host App
        ↓ (provides activeDid)
Daily Notification Plugin → Native Background Executor
                          ↓
                    HTTP Client + Auth
                          ↓
                   API Server Response
                          ↓
                   Parse & Cache Data (plugin storage)
                          ↓
                   Trigger Notifications

Option A: Host Always Provides activeDid

Core Principle: Host application queries its own database and provides activeDid to plugin.

Why Option A Is Superior:

1. Clear Separation: Host owns identity management, plugin owns notifications
2. No Database Conflicts: Zero shared database access between host and plugin
3. Security Isolation: Plugin data physically separated from user data
4. Platform Independence: Works consistently regardless of host's database technology
5. Simplified Implementation: Fewer moving parts, clearer debugging

Android Implementation Strategy

A. Background Execution Framework

• Use WorkManager for reliable background HTTP requests
• Enhance existing Native HTTP clients (already implemented):
  • Extend DailyNotificationETagManager.java with JWT headers
  • Add JWT authentication to DailyNotificationFetcher.java
• Handle Android-specific constraints: Doze mode, app standby, battery optimization

B. Authentication Enhancement - Extend Current Infrastructure

Goal: Enhance existing DailyNotificationETagManager.java and DailyNotificationFetcher.java with JWT authentication

// Enhance existing Android infrastructure with JWT authentication
class DailyNotificationJWTManager {
    private val storage: DailyNotificationStorage
    private val currentActiveDid: String? = null
    
    // Add JWT generation to existing fetcher
    fun generateJWTForActiveDid(activeDid: String, expiresInSeconds: Int): String {
        val payload = mapOf(
            "exp" to (System.currentTimeMillis() / 1000 + expiresInSeconds),
            "iat" to (System.currentTimeMillis() / 1000),
            "iss" to activeDid,
            "aud" to "timesafari.notifications",
            "sub" to activeDid
        )
        return signWithDID(payload, activeDid)
    }
    
    // Enhance existing HTTP client with JWT headers
    fun enhanceHttpClientWithJWT(connection: HttpURLConnection, activeDid: String) {
        val jwt = generateJWTForActiveDid(activeDid, 60)
        connection.setRequestProperty("Authorization", "Bearer $jwt")
        connection.setRequestProperty("Content-Type", "application/json")
    }
}

### C. HTTP Request Enhancement - Extend Existing Fetcher

**Goal**: Enhance existing `DailyNotificationFetcher.java` with Endorser.ch API support

```kotlin
// Enhance existing DailyNotificationFetcher.java with TimeSafari APIs
class EnhancedDailyNotificationFetcher : DailyNotificationFetcher {
    private val jwtManager: DailyNotificationJWTManager
    
    suspend fun fetchEndorserOffers(activeDid: String, afterId: String?): Result {
        val connection = HttpURLConnection("$apiServer/api/v2/report/offers")
        
        // Add JWT authentication to existing connection
        jwtManager.enhanceHttpClientWithJWT(connection, activeDid)
        
        // Add Endorser.ch specific parameters
        connection.setRequestProperty("recipientDid", activeDid)
        if (afterId != null) {
            connection.setRequestProperty("afterId", afterId)
        }
        
        // Use existing storeAndScheduleNotification method
        return fetchAndStoreContent(connection)
    }
}

iOS Implementation Strategy

A. Background Execution Framework

• Use BGTaskScheduler for background HTTP requests
• Replace axios with native iOS HTTP clients:
  • URLSession for HTTP requests
  • Combine framework for async/await patterns

B. Authentication Implementation

// JWT Generation in iOS
class JWTHelper {
    func generateJWT(userDid: String, expiresInSeconds: Int) -> String {
        let payload: [String: Any] = [
            "exp": Int(Date().timeIntervalSince1970) + expiresInSeconds,
            "iat": Int(Date().timeIntervalSince1970),
            "iss": userDid
        ]
        return signWithDID(payload, userDid)
    }
}

C. HTTP Request Implementation

// Background HTTP Task
class DataFetchTask {
    func fetchData() async {
        let jwt = generateJWT(userDid: activeDid, expiresInSeconds: 60)
        var request = URLRequest(url: apiURL)
        request.setValue("Bearer \(jwt)", forHTTPHeaderField: "Authorization")
        request.setValue("application/json", forHTTPHeaderField: "Content-Type")
        
        do {
            let (data, _) = try await URLSession.shared.data(for: request)
            let offersResponse = try JSONDecoder().decode(OffersResponse.self, from: data)
            await scheduleNotification(with: offersResponse.data)
        } catch {
            // Handle errors
        }
    }
}

Data Models & Type Safety

Shared TypeScript Interfaces

// Definitions for native bridge
interface OffersResponse {
    data: OfferSummaryRecord[];
    hitLimit: boolean;
}

interface OfferSummaryRecord {
    jwtId: string;
    handleId: string;
    issuedAt: string;
    offeredByDid: string;
    recipientDid: string;
    unit: string;
    amount: number;
    // ... other fields
}

Native Implementations

• Kotlin sealed classes for type-safe responses
• Swift Codable structs for JSON parsing
• Shared error handling patterns

Configuration Management

Plugin Configuration

interface PluginConfig {
    apiServer: string;
    jwtExpirationSeconds: number;
    requestTimeoutMs: number;
    retryAttempts: number;
    activeDid: string;  // Simplified to single active DID
    lastKnownOfferId?: string;
    lastKnownPlanId?: string;
}

Platform-Specific Settings

• Android: Manage API keys in AndroidManifest.xml, use SharedPreferences for runtime config
• iOS: Use Info.plist for static config, UserDefaults for runtime settings

Error Handling & Resilience

Network Error Handling

• Connectivity checks before making requests
• Exponential backoff for retry scenarios
• Circuit breaker pattern for API failures
• Graceful degradation when offline

Authentication Error Handling

• Token refresh mechanisms
• Fallback to anonymous requests when authentication fails
• Secure credential storage using platform keychains

Cache & State Management

Data Persistence

Platform-Specific Storage Architecture

Android/Electron Platforms:

• @capacitor-community/sqlite plugin integration for native SQLite access
• Shared plugin database - Plugin manages its own SQLite database instance
• Direct SQL execution via plugin's dbExec() methods for complex queries
• Background worker integration for database operations during content fetch

Web Platform:

• absurd-sql for SQLite support in browser (managed by host application)
• Plugin delegation pattern - Plugin provides SQL queries, host executes them
• IndexedDB fallback for basic caching when SQLite unavailable

iOS Platform:

• Core Data integration via native Swift implementation
• Background task compatibility with iOS background refresh constraints

State Synchronization

• JavaScript → Native configuration updates
• Native → JavaScript status reporting
• Cross-platform state consistency
• Background ↔ Foreground state synchronization
• Database logging for audit trails and debugging

Enhanced Caching Strategy

Based on TimeSafari's optimization patterns:

• Batch-oriented processing for API requests to reduce overhead
• Intelligent batching with configurable timing (max 100ms wait, max 10 items)
• Memory-optimized caching with automatic cleanup (keep last 1000 log entries)
• Request deduplication to prevent redundant API calls
• Performance monitoring with operation timing and metrics collection

Performance Optimizations

Request Optimization

• Deduplication of identical requests
• Batch requests when possible
• Intelligent polling based on user activity

Memory Management

• Background memory limits enforcement
• Cache cleanup on memory pressure
• Resource lifecycle management

Critical Requirement: Plugin Must Know When activeDid Changes

Security Implications of Missing ActiveDid Change Detection

Without immediate activeDid change detection, the plugin faces severe risks:

• Cross-User Data Exposure: Plugin fetches notifications for wrong user after identity switch
• Unauthorized API Access: JWT tokens valid for incorrect user context
• Background Task Errors: Content fetching operates with wrong identity

Event-Based Solution

Host Application Responsibility: Dispatch activeDidChanged event

// TimeSafari PlatformServiceMixin modification
async $updateActiveDid(newDid: string | null): Promise<void> {
  // Update TimeSafari's active_identity table (existing logic)
  await this.$dbExec(
    "UPDATE active_identity SET activeDid = ?, lastUpdated = datetime('now') WHERE id = 1",
    [newDid || ""]
  );
  
  // CRITICAL: Notify plugin of change IMMEDIATELY
  document.dispatchEvent(new CustomEvent('activeDidChanged', {
    detail: { activeDid: newDid }
  }));
}

Plugin Responsibility: Listen and respond to changes

// Plugin service layer implementation
plugin.onActiveDidChange(async (newActiveDid) => {
  // 1. Clear all cached content for previous identity
  await plugin.clearCacheForNewIdentity();
  
  // 2. Refresh authentication tokens with new activeDid
  await plugin.refreshAuthenticationForNewIdentity(newActiveDid);
  
  // 3. Restart background tasks with correct identity
  await plugin.updateBackgroundTaskIdentity(newActiveDid);
  
  logger.info(`[DailyNotificationService] ActiveDid updated to: ${newActiveDid}`);
});

TimeSafari Integration Patterns

ActiveDid Management Analysis

TimeSafari's Database Architecture:

• Table: active_identity (single row with id = 1)
• Content: activeDid TEXT, lastUpdated DATETIME
• Purpose: Single source of truth for active user identity

Access via PlatformServiceMixin:

// Retrieving activeDid in TimeSafari components
const activeIdentity = await this.$getActiveIdentity();
const activeDid = activeIdentity.activeDid;

// Updating activeDid via PlatformServiceMixin
async $updateActiveDid(newDid: string | null): Promise<void> {
  await this.$dbExec(
    "UPDATE active_identity SET activeDid = ?, lastUpdated = datetime('now') WHERE id = 1",
    [newDid || ""]
  );
}

Plugin Hosting Strategy

Existing Plugin Usage: TimeSafari already uses Capacitor plugins extensively via CapacitorPlatformService.ts

Recommended Integration Architecture:

• Plugin integrates as standard Capacitor plugin
• Host provides activeDid via event-driven pattern
• Plugin manages own isolated storage
• Clear separation of responsibilities maintained

Integration Points

Enhanced Plugin Interface for Host Application Integration

Database Integration Patterns

Android/Electron: Host-Provided activeDid Approach (Option A)

// Host queries its own database and provides activeDid
const activeIdentity = await this.$getActiveIdentity(); // Uses host's CapacitorSQLite
await plugin.setActiveDidFromHost(activeIdentity.activeDid);

// Plugin configures its own isolated database
await plugin.configureDatabase({
  platform: 'android',
  storageType: 'plugin-managed'  // Plugin owns its storage
});

// Set up activeDid change listener for future changes
plugin.onActiveDidChange(async (newActiveDid) => {
  await plugin.clearCacheForNewIdentity();
  await plugin.refreshAuthenticationForNewIdentity(newActiveDid);
  logger.info(`[TimeSafari] ActiveDid changed to: ${newActiveDid}`);
});

Web: Host-Provided activeDid Approach (Option A)

// Host queries its absurd-sql database and provides activeDid
const activeIdentity = await this.$getActiveIdentity(); // Uses host's absurd-sql
await plugin.setActiveDidFromHost(activeIdentity.activeDid);

// Plugin uses host-delegated storage for its own data
await plugin.configureDatabase({
  platform: 'web',
  storageType: 'host-managed'  // Plugin delegates to host for storage
});

// Plugin operates independently with provided activeDid
const results = await plugin.executeContentFetch(contentConfig);

iOS: Host-Provided activeDid Approach (Option A)

// Host queries its CapacitorSQLite database and provides activeDid
const activeIdentity = await this.$getActiveIdentity(); // Uses host's CapacitorSQLite
await plugin.setActiveDidFromHost(activeIdentity.activeDid);

// Plugin configures its own Core Data storage
await plugin.configureDatabase({
  platform: 'ios',
  storageType: 'plugin-managed'  // Plugin owns Core Data storage
});

// Plugin operates with provided activeDid, no database sharing needed
const results = await plugin.executeBackgroundContentFetch();

Enhancement Required: Extend Current Plugin Interface

Current Interface (already implemented):

interface DailyNotificationPlugin {
  configure(options: ConfigureOptions): Promise<void>;
  scheduleContentFetch(config: ContentFetchConfig): Promise<void>;
  scheduleUserNotification(config: UserNotificationConfig): Promise<void>;
  // ... existing methods (see definitions.ts)
}

Enhancement Required (add to existing interface):

interface EnhancedDailyNotificationPlugin extends DailyNotificationPlugin {
  // Enhanced configuration with activeDid support
  configure(options: ConfigureOptions & {
    activeDidIntegration?: {
      platform: 'android' | 'ios' | 'web' | 'electron';
      storageType: 'plugin-managed' | 'host-managed';
    };
  }): Promise<void>;
  
  // Host-provided activeDid Management (Option A Implementation)
  setActiveDidFromHost(activeDid: string): Promise<void>;
  
  // Critical: ActiveDid Change Handling
  onActiveDidChange(callback: (newActiveDid: string) => Promise<void>): void;
  refreshAuthenticationForNewIdentity(activeDid: string): Promise<void>;
}

Background Scheduling with Hybrid activeDid Management

• Integrate with existing WorkManager/BGTaskScheduler
• Coordinate API fetch timing with notification schedules
• Handle app lifecycle events (background/foreground)
• Implement host-provided activeDid access (Option A):
  • Always: Host provides activeDid via setActiveDidFromHost()
  • No Database Sharing: Plugin never accesses TimeSafari's active_identity table
• Critical: Plugin MUST know when activeDid changes for:
  • Event-Based Notification: Host dispatches activeDidChanged events
  • Cache Invalidation: Clear cached content when user switches identity
  • Token Refresh: Generate new JWT tokens with updated active Did
  • Background Task Coordination: Restart tasks with new identity context
• Maintain clear separation: Host owns identity management, plugin owns notifications

Migration & Testing Strategy

Gradual Migration

1. Phase 1: Implement basic HTTP + JWT authentication
2. Phase 2: Add caching and state management
3. Phase 3: Integrate with notification scheduling
4. Phase 4: Add passkey authentication support

Testing Approach with Host-Provided activeDid Management

• Unit tests for JWT generation and HTTP clients with activeDid
• Integration tests for API endpoint interactions using TimeSafari active_identity patterns
• Host-provided activeDid testing:
  • Test setActiveDidFromHost() with TimeSafari PlatformServiceMixin
  • Test host event dispatch and plugin event listening
  • Critical: Test onActiveDidChange() listener with identity switches
  • Test cache invalidation and token refresh during activeDid changes
  • Verify database isolation between host and plugin
• Background testing on real devices (doze mode, app backgrounding)
• Authentication testing with actual DID credentials from TimeSafari active_identity table
• Cross-platform testing for Android/Electron (SQLite access) vs Web (host delegation) patterns

API Endpoints to Support

Offers to User Endpoint

GET {apiServer}/api/v2/report/offers?recipientDid={userDid}&afterId={jwtId}&beforeId={jwtId}

Response Structure:

{
  "data": Array<OfferSummaryRecord>,
  "hitLimit": boolean
}

Offers to User Projects Endpoint

GET {apiServer}/api/v2/report/offersToPlansOwnedByMe?afterId={jwtId}&beforeId={jwtId}

Response Structure:

{
  "data": Array<OfferToPlanSummaryRecord>,
  "hitLimit": boolean
}

Authentication Implementation Strategy

Option 1: Simple DID Authentication (Basic)

• Generate traditional JWT using DID signing
• Short-lived tokens (60 seconds)
• Suitable for basic notification data fetching
• Use did-jwt library for token generation and verification
• Based on TimeSafari's existing JWT implementation patterns

Option 2: Enhanced Passkey Authentication (Advanced)

• Leverage device biometrics/security keys
• Longer-lived tokens with automatic refresh
• Support for cached authentication state
• Better user experience for frequent polling
• Integrate with SimpleWebAuthn for cross-platform biometric support
• Support JWANT tokens (JWT + WebAuthn) for enhanced security

Platform-Specific Considerations

Android Considerations

• Use OkHttp or native Android HTTP clients
• Handle certificate pinning if required
• Support Android Keystore for secure key storage
• Handle biometric prompt integration for passkeys

iOS Considerations

• Use URLSession for HTTP requests
• Support iOS Keychain for authentication tokens
• Handle Face ID/Touch ID integration for passkeys
• Support certificate pinning if required
• Use BGTaskScheduler for reliable background execution
• Handle iOS-specific background refresh restrictions
• Support Core Data for notification metadata persistence

Data Flow Integration Points

Token Generation

• Accept activeDid as input
• Generate JWT authentication token using DID signing
• Include activeDid as both issuer (iss) and subject (sub)
• Return token for immediate use or caching

Request Execution

• Construct full API URLs with query parameters
• Apply authentication headers
• Execute HTTP requests with proper error handling
• Return structured response data

Caching Strategy

• Support token caching with expiration management
• Implement request deduplication for same endpoints
• Support cache invalidation for authentication failures

Implementation Phases

Phase 1: Extend Core Infrastructure (Building on Existing)

• Extend existing Android DailyNotificationFetcher.java with JWT authentication
• Enhance existing iOS implementation (when added) with authentication layer
• Add JWT generation to existing DailyNotificationETagManager.java
• Enhance current ConfigureOptions with activeDid integration
• Build on existing error handling (circuit breaker already implemented)

Phase 2: ActiveDid Integration & TimeSafari API Enhancement

• Add host-provided activeDid management to existing plugin interface
• Extend existing configure() method with activeDid options
• Enhance existing ContentFetchConfig with Endorser.ch API endpoints
• Add setActiveDidFromHost() and onActiveDidChange() to existing interface
• Integrate existing callback-registry.ts with activeDid-aware callbacks
• Enhance existing platform storage:
  • Android: Extend existing SQLite with activeDid-aware JWT storage
  • Web: Enhance existing IndexedDB with activeDid support (no host delegation needed initially)

Phase 3: Background Enhancement & TimeSafari Coordination

• Enhance existing WorkManager integration with activeDid-aware workers
• Coordinate existing notification scheduling with TimeSafari PlatformServiceMixin
• Extend existing app lifecycle handling with activeDid change detection
• Enhance existing state synchronization with identity management
• Critical: Enhance retry policies for activeDid changes:
  • Android: Modify DailyNotificationFetchWorker.java retry logic to detect activeDid changes during retry sequence
  • Web: Enhance callback-registry.ts to refresh authentication before retry attempts
  • Unify: Standardize retry delays across platforms (Android 1min→1hour vs Web 1sec→1min)
• Integrate activeDid change detection into existing circuit breaker and error handling systems

Phase 4: TimeSafari Integration & Advanced Features

• Integrate with TimeSafari's existing PlatformServiceMixin patterns
• Add Endorser.ch API endpoint support to existing ContentFetchConfig
• Implement DID-based authentication alongside existing callback system
• Enhance existing testing with TimeSafari-specific scenarios

Current Scheduled Event Update Policies

✅ Existing Consistent Policies

• Retry Logic: Exponential backoff with platform-specific limits (Android: 5 retries, Web: 5 retries)
• Circuit Breaker: Opens after 5 consecutive failures
• Fallback Content: Uses cached/emergency content when all retries fail
• ETag Updates: Conditional requests with 304 Not Modified handling
• Error Classification: Network/Storage errors retryable, Permission/Config errors not retryable

⚠️ Enhancement Required for TimeSafari Integration

• ActiveDid Change Detection: Handle identity switches during scheduled events
• Authentication Refresh: Update JWT tokens for ongoing retry attempts
• Cache Invalidation: Clear cached content when activeDid changes
• Platform Policy Unification: Standardize retry delays and fallback mechanisms

TimeSafari-Aware Update Policy

interface TimeSafariUpdatePolicy extends ContentFetchConfig {
  activeDidAwareRetry?: {
    maxRetriesDuringActiveDidChange: number;     // More retries during identity change
    authenticationRefreshDelay: number;          // Time to refresh auth before retry
    cacheInvalidationOnChange: boolean;          // Clear cache when activeDid changes
  };
}

Success Criteria

• Functional Requirements: API data fetching works reliably in background with activeDid awareness
• Performance Requirements: Requests complete within 30 seconds, including authentication refresh
• Security Requirements: ActiveDid-based authentication with token refresh during retries
• Reliability Requirements: Enhanced retry policies that handle activeDid changes gracefully
• Integration Requirements: Seamless integration with existing plugin APIs + TimeSafari patterns
• Testing Requirements: Comprehensive test coverage including activeDid change scenarios
• Authentication Requirements: DID-based JWT with automatic refresh during scheduled events
• Optimization Requirements: Intelligent retry policies based on error type and activeDid state
• Logging Requirements: Structured logging with activeDid context and retry state tracking
• Cross-Platform Requirements: Unified activeDid-aware retry and fallback mechanisms

Risks & Mitigation

Technical Risks

• Background execution limits: Mitigated by using platform-specific background task systems
• Authentication complexity: Mitigated by implementing gradual migration path
• Cross-platform consistency: Mitigated by shared interfaces and careful API design

Timeline Risks

• Platform-specific complexity: Mitigated by prioritizing Android first, then iOS
• Testing complexity: Mitigated by automated testing and CI/CD integration
• Integration challenges: Mitigated by maintaining backward compatibility

---

Status: Enhancement plan for existing implementation (Option A) - Ready for implementation
Next Steps: Begin Phase 1 - enhance existing Android HTTP infrastructure with JWT authentication
Dependencies: Existing plugin infrastructure, Android Studio, Capacitor CLI, TimeSafari PlatformServiceMixin
Enhancement Approach:

• Build on Existing: Leverage current SQLite, IndexedDB, callback system, and dual scheduling
• Option A Integration: Add activeDid management to existing configuration and HTTP layers
• TimeSafari Enhancement: Extend current ContentFetchConfig with Endorser.ch API endpoints
• Authentication Layer: Add JWT/DID authentication over existing HTTP infrastructure
• Event Integration: Enhance existing callback system with activeDid change detection