# Background Data Fetching Implementation Plan

**Author**: Matthew Raymer  
**Version**: 1.0.0  
**Created**: 2025-10-02 07:47:04 UTC  
**Last Updated**: 2025-10-02 12:45:00 UTC  

## Overview

This document outlines the **enhancement plan** for background data fetching in the Daily Notification Plugin to support TimeSafari integration with Option A architecture. This plan builds upon our existing implementation and adds:

- **Option A Enhancement**: ActiveDid-aware authentication on existing infrastructure
- **TimeSafari Integration**: PlatformServiceMixin coordination with current plugin
- **Authentication Layer**: JWT/DID support over existing HTTP callback system
- **Database Enhancement**: Extend current storage with activeDid management
- **Event & Change Management**: Identity change detection on existing notification system
- **API Integration**: Endorser.ch endpoints through current ContentFetchConfig

This document serves as the enhancement roadmap for adding TimeSafari capabilities to our existing, working plugin.

## Current Implementation Baseline

### ✅ Already Implemented & Working
- **Android Storage**: SharedPreferences + SQLite with migration (`DailyNotificationStorage.java`, `DailyNotificationDatabase.java`)
- **Web Storage**: IndexedDB with comprehensive service worker (`sw.ts`, `IndexedDBManager`)
- **Callback System**: HTTP/local callbacks with circuit breaker (`callback-registry.ts`)
- **Configuration**: Database path, TTL, retention settings (`ConfigureOptions`)
- **ETag Support**: Conditional HTTP requests (`DailyNotificationETagManager.java`)
- **Dual Scheduling**: Content fetch + user notification separation
- **Cross-Platform API**: Unified TypeScript interface (`DailyNotificationPlugin`)

### ⚠️ Enhancement Required for TimeSafari
- **ActiveDid Integration**: Add activeDid-awareness to existing authentication
- **JWT Generation**: Enhance HTTP layer with DID-based tokens
- **Identity Change Detection**: Add event listeners to existing callback system
- **Endorser.ch APIs**: Extend `ContentFetchConfig` with TimeSafari endpoints
- **Platform Auth**: Add Android Keystore/iOS Keychain to existing storage

## Consolidated Architecture: Option A Platform Overview

```
TimeSafari Host App
        ↓ (provides activeDid)
Daily Notification Plugin → Native Background Executor
                          ↓
                    HTTP Client + Auth
                          ↓
                   API Server Response
                          ↓
                   Parse & Cache Data (plugin storage)
                          ↓
                   Trigger Notifications
```

### **Option A: Host Always Provides activeDid**

**Core Principle**: Host application queries its own database and provides activeDid to plugin.

### **Why Option A Is Superior:**
1. **Clear Separation**: Host owns identity management, plugin owns notifications
2. **No Database Conflicts**: Zero shared database access between host and plugin
3. **Security Isolation**: Plugin data physically separated from user data
4. **Platform Independence**: Works consistently regardless of host's database technology
5. **Simplified Implementation**: Fewer moving parts, clearer debugging

## Android Implementation Strategy

### A. Background Execution Framework

- **Use WorkManager** for reliable background HTTP requests
- **Enhance** existing Native HTTP clients (already implemented):
  - Extend `DailyNotificationETagManager.java` with JWT headers
  - Add JWT authentication to `DailyNotificationFetcher.java`
- **Handle Android-specific constraints**: Doze mode, app standby, battery optimization

### B. Authentication Enhancement - Extend Current Infrastructure

**Goal**: Enhance existing `DailyNotificationETagManager.java` and `DailyNotificationFetcher.java` with JWT authentication

```kotlin
// Enhance existing Android infrastructure with JWT authentication
class DailyNotificationJWTManager {
    private val storage: DailyNotificationStorage
    private val currentActiveDid: String? = null
    
    // Add JWT generation to existing fetcher
    fun generateJWTForActiveDid(activeDid: String, expiresInSeconds: Int): String {
        val payload = mapOf(
            "exp" to (System.currentTimeMillis() / 1000 + expiresInSeconds),
            "iat" to (System.currentTimeMillis() / 1000),
            "iss" to activeDid,
            "aud" to "timesafari.notifications",
            "sub" to activeDid
        )
        return signWithDID(payload, activeDid)
    }
    
    // Enhance existing HTTP client with JWT headers
    fun enhanceHttpClientWithJWT(connection: HttpURLConnection, activeDid: String) {
        val jwt = generateJWTForActiveDid(activeDid, 60)
        connection.setRequestProperty("Authorization", "Bearer $jwt")
        connection.setRequestProperty("Content-Type", "application/json")
    }
}

### C. HTTP Request Enhancement - Extend Existing Fetcher

**Goal**: Enhance existing `DailyNotificationFetcher.java` with Endorser.ch API support

```kotlin
// Enhance existing DailyNotificationFetcher.java with TimeSafari APIs
class EnhancedDailyNotificationFetcher : DailyNotificationFetcher {
    private val jwtManager: DailyNotificationJWTManager
    
    suspend fun fetchEndorserOffers(activeDid: String, afterId: String?): Result {
        val connection = HttpURLConnection("$apiServer/api/v2/report/offers")
        
        // Add JWT authentication to existing connection
        jwtManager.enhanceHttpClientWithJWT(connection, activeDid)
        
        // Add Endorser.ch specific parameters
        connection.setRequestProperty("recipientDid", activeDid)
        if (afterId != null) {
            connection.setRequestProperty("afterId", afterId)
        }
        
        // Use existing storeAndScheduleNotification method
        return fetchAndStoreContent(connection)
    }
}
```

## iOS Implementation Strategy

### A. Background Execution Framework

- **Use BGTaskScheduler** for background HTTP requests
- **Replace axios** with native iOS HTTP clients:
  - URLSession for HTTP requests
  - Combine framework for async/await patterns

### B. Authentication Implementation

```swift
// JWT Generation in iOS
class JWTHelper {
    func generateJWT(userDid: String, expiresInSeconds: Int) -> String {
        let payload: [String: Any] = [
            "exp": Int(Date().timeIntervalSince1970) + expiresInSeconds,
            "iat": Int(Date().timeIntervalSince1970),
            "iss": userDid
        ]
        return signWithDID(payload, userDid)
    }
}
```

### C. HTTP Request Implementation

```swift
// Background HTTP Task
class DataFetchTask {
    func fetchData() async {
        let jwt = generateJWT(userDid: activeDid, expiresInSeconds: 60)
        var request = URLRequest(url: apiURL)
        request.setValue("Bearer \(jwt)", forHTTPHeaderField: "Authorization")
        request.setValue("application/json", forHTTPHeaderField: "Content-Type")
        
        do {
            let (data, _) = try await URLSession.shared.data(for: request)
            let offersResponse = try JSONDecoder().decode(OffersResponse.self, from: data)
            await scheduleNotification(with: offersResponse.data)
        } catch {
            // Handle errors
        }
    }
}
```

## Data Models & Type Safety

### Shared TypeScript Interfaces

```typescript
// Definitions for native bridge
interface OffersResponse {
    data: OfferSummaryRecord[];
    hitLimit: boolean;
}

interface OfferSummaryRecord {
    jwtId: string;
    handleId: string;
    issuedAt: string;
    offeredByDid: string;
    recipientDid: string;
    unit: string;
    amount: number;
    // ... other fields
}
```

### Native Implementations

- **Kotlin sealed classes** for type-safe responses
- **Swift Codable structs** for JSON parsing
- **Shared error handling** patterns

## Configuration Management

### Plugin Configuration

```typescript
interface PluginConfig {
    apiServer: string;
    jwtExpirationSeconds: number;
    requestTimeoutMs: number;
    retryAttempts: number;
    activeDid: string;  // Simplified to single active DID
    lastKnownOfferId?: string;
    lastKnownPlanId?: string;
}
```

### Platform-Specific Settings

- **Android**: Manage API keys in `AndroidManifest.xml`, use SharedPreferences for runtime config
- **iOS**: Use `Info.plist` for static config, UserDefaults for runtime settings

## Error Handling & Resilience

### Network Error Handling

- **Connectivity checks** before making requests
- **Exponential backoff** for retry scenarios
- **Circuit breaker pattern** for API failures
- **Graceful degradation** when offline

### Authentication Error Handling

- **Token refresh** mechanisms
- **Fallback to anonymous** requests when authentication fails
- **Secure credential storage** using platform keychains

## Cache & State Management

### Data Persistence

#### **Platform-Specific Storage Architecture**

**Android/Electron Platforms:**
- **@capacitor-community/sqlite** plugin integration for native SQLite access
- **Shared plugin database** - Plugin manages its own SQLite database instance
- **Direct SQL execution** via plugin's `dbExec()` methods for complex queries
- **Background worker integration** for database operations during content fetch

**Web Platform:**
- **absurd-sql** for SQLite support in browser (managed by host application)
- **Plugin delegation pattern** - Plugin provides SQL queries, host executes them
- **IndexedDB fallback** for basic caching when SQLite unavailable

**iOS Platform:**
- **Core Data integration** via native Swift implementation
- **Background task compatibility** with iOS background refresh constraints

### State Synchronization

- **JavaScript → Native** configuration updates
- **Native → JavaScript** status reporting
- **Cross-platform state consistency**
- **Background ↔ Foreground** state synchronization
- **Database logging** for audit trails and debugging

### Enhanced Caching Strategy

Based on TimeSafari's optimization patterns:

- **Batch-oriented processing** for API requests to reduce overhead
- **Intelligent batching** with configurable timing (max 100ms wait, max 10 items)
- **Memory-optimized caching** with automatic cleanup (keep last 1000 log entries)
- **Request deduplication** to prevent redundant API calls
- **Performance monitoring** with operation timing and metrics collection

## Performance Optimizations

### Request Optimization

- **Deduplication** of identical requests
- **Batch requests** when possible
- **Intelligent polling** based on user activity

### Memory Management

- **Background memory limits** enforcement
- **Cache cleanup** on memory pressure
- **Resource lifecycle** management

## Critical Requirement: Plugin Must Know When activeDid Changes

### **Security Implications of Missing ActiveDid Change Detection**

**Without immediate activeDid change detection, the plugin faces severe risks:**
- **Cross-User Data Exposure**: Plugin fetches notifications for wrong user after identity switch
- **Unauthorized API Access**: JWT tokens valid for incorrect user context  
- **Background Task Errors**: Content fetching operates with wrong identity

### **Event-Based Solution**

**Host Application Responsibility**: Dispatch `activeDidChanged` event
```typescript
// TimeSafari PlatformServiceMixin modification
async $updateActiveDid(newDid: string | null): Promise<void> {
  // Update TimeSafari's active_identity table (existing logic)
  await this.$dbExec(
    "UPDATE active_identity SET activeDid = ?, lastUpdated = datetime('now') WHERE id = 1",
    [newDid || ""]
  );
  
  // CRITICAL: Notify plugin of change IMMEDIATELY
  document.dispatchEvent(new CustomEvent('activeDidChanged', {
    detail: { activeDid: newDid }
  }));
}
```

**Plugin Responsibility**: Listen and respond to changes
```typescript
// Plugin service layer implementation
plugin.onActiveDidChange(async (newActiveDid) => {
  // 1. Clear all cached content for previous identity
  await plugin.clearCacheForNewIdentity();
  
  // 2. Refresh authentication tokens with new activeDid
  await plugin.refreshAuthenticationForNewIdentity(newActiveDid);
  
  // 3. Restart background tasks with correct identity
  await plugin.updateBackgroundTaskIdentity(newActiveDid);
  
  logger.info(`[DailyNotificationService] ActiveDid updated to: ${newActiveDid}`);
});
```

## TimeSafari Integration Patterns

### **ActiveDid Management Analysis**

**TimeSafari's Database Architecture:**
- **Table**: `active_identity` (single row with `id = 1`)
- **Content**: `activeDid TEXT`, `lastUpdated DATETIME`
- **Purpose**: Single source of truth for active user identity

**Access via PlatformServiceMixin:**
```typescript
// Retrieving activeDid in TimeSafari components
const activeIdentity = await this.$getActiveIdentity();
const activeDid = activeIdentity.activeDid;

// Updating activeDid via PlatformServiceMixin
async $updateActiveDid(newDid: string | null): Promise<void> {
  await this.$dbExec(
    "UPDATE active_identity SET activeDid = ?, lastUpdated = datetime('now') WHERE id = 1",
    [newDid || ""]
  );
}
```

### **Plugin Hosting Strategy**

**Existing Plugin Usage**: TimeSafari already uses Capacitor plugins extensively via `CapacitorPlatformService.ts`

**Recommended Integration Architecture:**
- Plugin integrates as standard Capacitor plugin
- Host provides activeDid via event-driven pattern
- Plugin manages own isolated storage
- Clear separation of responsibilities maintained

## Integration Points

### Enhanced Plugin Interface for Host Application Integration

#### **Database Integration Patterns**

**Android/Electron: Host-Provided activeDid Approach (Option A)**
```typescript
// Host queries its own database and provides activeDid
const activeIdentity = await this.$getActiveIdentity(); // Uses host's CapacitorSQLite
await plugin.setActiveDidFromHost(activeIdentity.activeDid);

// Plugin configures its own isolated database
await plugin.configureDatabase({
  platform: 'android',
  storageType: 'plugin-managed'  // Plugin owns its storage
});

// Set up activeDid change listener for future changes
plugin.onActiveDidChange(async (newActiveDid) => {
  await plugin.clearCacheForNewIdentity();
  await plugin.refreshAuthenticationForNewIdentity(newActiveDid);
  logger.info(`[TimeSafari] ActiveDid changed to: ${newActiveDid}`);
});
```

**Web: Host-Provided activeDid Approach (Option A)**
```typescript
// Host queries its absurd-sql database and provides activeDid
const activeIdentity = await this.$getActiveIdentity(); // Uses host's absurd-sql
await plugin.setActiveDidFromHost(activeIdentity.activeDid);

// Plugin uses host-delegated storage for its own data
await plugin.configureDatabase({
  platform: 'web',
  storageType: 'host-managed'  // Plugin delegates to host for storage
});

// Plugin operates independently with provided activeDid
const results = await plugin.executeContentFetch(contentConfig);
```

**iOS: Host-Provided activeDid Approach (Option A)**
```typescript
// Host queries its CapacitorSQLite database and provides activeDid
const activeIdentity = await this.$getActiveIdentity(); // Uses host's CapacitorSQLite
await plugin.setActiveDidFromHost(activeIdentity.activeDid);

// Plugin configures its own Core Data storage
await plugin.configureDatabase({
  platform: 'ios',
  storageType: 'plugin-managed'  // Plugin owns Core Data storage
});

// Plugin operates with provided activeDid, no database sharing needed
const results = await plugin.executeBackgroundContentFetch();
```

#### **Enhancement Required: Extend Current Plugin Interface**

**Current Interface** (already implemented):
```typescript
interface DailyNotificationPlugin {
  configure(options: ConfigureOptions): Promise<void>;
  scheduleContentFetch(config: ContentFetchConfig): Promise<void>;
  scheduleUserNotification(config: UserNotificationConfig): Promise<void>;
  // ... existing methods (see definitions.ts)
}
```

**Enhancement Required** (add to existing interface):
```typescript
interface EnhancedDailyNotificationPlugin extends DailyNotificationPlugin {
  // Enhanced configuration with activeDid support
  configure(options: ConfigureOptions & {
    activeDidIntegration?: {
      platform: 'android' | 'ios' | 'web' | 'electron';
      storageType: 'plugin-managed' | 'host-managed';
    };
  }): Promise<void>;
  
  // Host-provided activeDid Management (Option A Implementation)
  setActiveDidFromHost(activeDid: string): Promise<void>;
  
  // Critical: ActiveDid Change Handling
  onActiveDidChange(callback: (newActiveDid: string) => Promise<void>): void;
  refreshAuthenticationForNewIdentity(activeDid: string): Promise<void>;
}
```

### Background Scheduling with Hybrid activeDid Management

- **Integrate** with existing WorkManager/BGTaskScheduler
- **Coordinate** API fetch timing with notification schedules
- **Handle** app lifecycle events (background/foreground)
- **Implement** host-provided activeDid access (Option A):
  - **Always**: Host provides activeDid via `setActiveDidFromHost()`
  - **No Database Sharing**: Plugin never accesses TimeSafari's active_identity table
- **Critical**: Plugin **MUST** know when activeDid changes for:
  - **Event-Based Notification**: Host dispatches `activeDidChanged` events
  - **Cache Invalidation**: Clear cached content when user switches identity
  - **Token Refresh**: Generate new JWT tokens with updated active Did
  - **Background Task Coordination**: Restart tasks with new identity context
- **Maintain** clear separation: Host owns identity management, plugin owns notifications

## Migration & Testing Strategy

### Gradual Migration

1. **Phase 1**: Implement basic HTTP + JWT authentication
2. **Phase 2**: Add caching and state management
3. **Phase 3**: Integrate with notification scheduling
4. **Phase 4**: Add passkey authentication support

### Testing Approach with Host-Provided activeDid Management

- **Unit tests** for JWT generation and HTTP clients with activeDid
- **Integration tests** for API endpoint interactions using TimeSafari active_identity patterns
- **Host-provided activeDid testing**:
  - Test `setActiveDidFromHost()` with TimeSafari PlatformServiceMixin
  - Test host event dispatch and plugin event listening
  - **Critical**: Test `onActiveDidChange()` listener with identity switches
  - Test cache invalidation and token refresh during activeDid changes
  - Verify database isolation between host and plugin
- **Background testing** on real devices (doze mode, app backgrounding)
- **Authentication testing** with actual DID credentials from TimeSafari active_identity table
- **Cross-platform testing** for Android/Electron (SQLite access) vs Web (host delegation) patterns

## API Endpoints to Support

### Offers to User Endpoint

```
GET {apiServer}/api/v2/report/offers?recipientDid={userDid}&afterId={jwtId}&beforeId={jwtId}
```

**Response Structure:**
```json
{
  "data": Array<OfferSummaryRecord>,
  "hitLimit": boolean
}
```

### Offers to User Projects Endpoint

```
GET {apiServer}/api/v2/report/offersToPlansOwnedByMe?afterId={jwtId}&beforeId={jwtId}
```

**Response Structure:**
```json
{
  "data": Array<OfferToPlanSummaryRecord>,
  "hitLimit": boolean
}
```

## Authentication Implementation Strategy

### Option 1: Simple DID Authentication (Basic)

- Generate traditional JWT using DID signing
- Short-lived tokens (60 seconds)
- Suitable for basic notification data fetching
- Use `did-jwt` library for token generation and verification
- Based on TimeSafari's existing JWT implementation patterns

### Option 2: Enhanced Passkey Authentication (Advanced)

- Leverage device biometrics/security keys
- Longer-lived tokens with automatic refresh
- Support for cached authentication state
- Better user experience for frequent polling
- Integrate with SimpleWebAuthn for cross-platform biometric support
- Support JWANT tokens (JWT + WebAuthn) for enhanced security

## Platform-Specific Considerations

### Android Considerations

- Use OkHttp or native Android HTTP clients
- Handle certificate pinning if required
- Support Android Keystore for secure key storage
- Handle biometric prompt integration for passkeys

### iOS Considerations

- Use URLSession for HTTP requests
- Support iOS Keychain for authentication tokens
- Handle Face ID/Touch ID integration for passkeys
- Support certificate pinning if required
- Use BGTaskScheduler for reliable background execution
- Handle iOS-specific background refresh restrictions
- Support Core Data for notification metadata persistence

## Data Flow Integration Points

### Token Generation

- Accept activeDid as input
- Generate JWT authentication token using DID signing
- Include activeDid as both issuer (`iss`) and subject (`sub`)
- Return token for immediate use or caching

### Request Execution

- Construct full API URLs with query parameters
- Apply authentication headers
- Execute HTTP requests with proper error handling
- Return structured response data

### Caching Strategy

- Support token caching with expiration management
- Implement request deduplication for same endpoints
- Support cache invalidation for authentication failures

## Implementation Phases

### Phase 1: Extend Core Infrastructure (Building on Existing)

- **Extend existing** Android `DailyNotificationFetcher.java` with JWT authentication
- **Enhance existing** iOS implementation (when added) with authentication layer
- **Add JWT generation** to existing `DailyNotificationETagManager.java`
- **Enhance current** `ConfigureOptions` with activeDid integration
- **Build on existing** error handling (circuit breaker already implemented)

### Phase 2: ActiveDid Integration & TimeSafari API Enhancement

- **Add** host-provided activeDid management to existing plugin interface
- **Extend** existing `configure()` method with activeDid options
- **Enhance** existing `ContentFetchConfig` with Endorser.ch API endpoints
- **Add** `setActiveDidFromHost()` and `onActiveDidChange()` to existing interface
- **Integrate** existing `callback-registry.ts` with activeDid-aware callbacks
- **Enhance** existing platform storage:
  - **Android**: Extend existing SQLite with activeDid-aware JWT storage
  - **Web**: Enhance existing IndexedDB with activeDid support (no host delegation needed initially)

### Phase 3: Background Enhancement & TimeSafari Coordination

- **Enhance** existing WorkManager integration with activeDid-aware workers
- **Coordinate** existing notification scheduling with TimeSafari PlatformServiceMixin
- **Extend** existing app lifecycle handling with activeDid change detection
- **Enhance** existing state synchronization with identity management

### Phase 4: TimeSafari Integration & Advanced Features

- **Integrate** with TimeSafari's existing PlatformServiceMixin patterns
- **Add** Endorser.ch API endpoint support to existing `ContentFetchConfig`
- **Implement** DID-based authentication alongside existing callback system
- **Enhance** existing testing with TimeSafari-specific scenarios

## Success Criteria

- [ ] **Functional Requirements**: API data fetching works reliably in background
- [ ] **Performance Requirements**: Requests complete within 30 seconds
- [ ] **Security Requirements**: Secure credential storage and token management
- [ ] **Reliability Requirements**: Handles network failures and offline scenarios
- [ ] **Integration Requirements**: Seamless integration with existing plugin APIs
- [ ] **Testing Requirements**: Comprehensive test coverage for all platforms
- [ ] **Authentication Requirements**: Support both DID-based JWT and Passkey JWANT tokens
- [ ] **Optimization Requirements**: Implement batch processing with sub-100ms delays
- [ ] **Logging Requirements**: Structured logging with database persistence for debugging
- [ ] **Cross-Platform Requirements**: Unified SQLite/IndexedDB storage across platforms

## Risks & Mitigation

### Technical Risks

- **Background execution limits**: Mitigated by using platform-specific background task systems
- **Authentication complexity**: Mitigated by implementing gradual migration path
- **Cross-platform consistency**: Mitigated by shared interfaces and careful API design

### Timeline Risks

- **Platform-specific complexity**: Mitigated by prioritizing Android first, then iOS
- **Testing complexity**: Mitigated by automated testing and CI/CD integration
- **Integration challenges**: Mitigated by maintaining backward compatibility

---

**Status**: Enhancement plan for existing implementation (Option A) - Ready for implementation  
**Next Steps**: Begin Phase 1 - enhance existing Android HTTP infrastructure with JWT authentication  
**Dependencies**: Existing plugin infrastructure, Android Studio, Capacitor CLI, TimeSafari PlatformServiceMixin  
**Enhancement Approach**: 
- **Build on Existing**: Leverage current SQLite, IndexedDB, callback system, and dual scheduling
- **Option A Integration**: Add activeDid management to existing configuration and HTTP layers
- **TimeSafari Enhancement**: Extend current ContentFetchConfig with Endorser.ch API endpoints
- **Authentication Layer**: Add JWT/DID authentication over existing HTTP infrastructure
- **Event Integration**: Enhance existing callback system with activeDid change detection