Files
daily-notification-plugin/doc/test-app-ios/IOS_PREFETCH_TESTING.md
Matthew Raymer 6d25cdd033 docs(ios): add comprehensive testing guide and refine iOS parity directive
Add iOS prefetch testing guide with detailed procedures, log checklists,
and behavior classification. Enhance iOS test app requirements with
security constraints, sign-off checklists, and changelog structure.
Update main directive with testing strategy and method behavior mapping.

Changes:
- Add IOS_PREFETCH_TESTING.md with simulator/device test plans, log
  diagnostics, telemetry expectations, and test run templates
- Add DailyNotificationBackgroundTaskTestHarness.swift as reference
  implementation for BGTaskScheduler testing
- Enhance IOS_TEST_APP_REQUIREMENTS.md with security/privacy constraints,
  review checklists, CI hints, and glossary cross-links
- Update 0003-iOS-Android-Parity-Directive.md with testing strategy
  section, method behavior classification, and validation matrix updates

All documents now include changelog stubs, cross-references, and
completion criteria for Phase 1 implementation and testing.
2025-11-15 02:41:28 +00:00

24 KiB
Raw Blame History

iOS Prefetch Testing Guide

Purpose: How to test background prefetch for DailyNotificationPlugin on iOS (simulator + device)

Plugin Target: DailyNotificationPlugin v3.x (iOS)
Phase: Phase 1 Prefetch MVP
Last Updated: 2025-11-15
Status: 🎯 ACTIVE - Testing guide for Phase 1+ implementation

Note: This guide assumes the iOS test app is implemented as described in doc/test-app-ios/IOS_TEST_APP_REQUIREMENTS.md.

Android parity: Behavior is aligned with test-apps/android-test-app where platform constraints allow. Timing and BGTask heuristics will differ from Android's exact alarms:

  • Android: Exact alarms via AlarmManager / WorkManager
  • iOS: Heuristic BGTaskScheduler; no hard guarantee of 5-min prefetch

Non-Goals (Phase 1)

Out of scope for Phase 1 testing:

  • No testing of silent push notifications
  • No testing of advanced rolling-window policies beyond "schedule one daily notification"
  • No guarantees of exact N-minute timing (iOS heuristics only)
  • No testing of multi-day rolling windows (deferred to Phase 2)

Table of Contents

  1. Goals
  2. Assumptions
  3. Simulator Test Plan (Logic-Correctness)
  4. Real Device Test Plan (Timing & Reliability)
  5. Log Checklist
  6. Behavior Classification
  7. Test Harness Reference

Goals

Verify that the iOS prefetch path actually runs:

  • BGTask is registered
  • BGTask is scheduled
  • Fetch executes
  • Data is persisted & used by the notification

Separate concerns:

  • "Does the logic work?" → Test in Simulator
  • "Does timing behave?" → Test on Real Device

Time & T-Lead Rules

Critical: These rules prevent bugs when tweaking prefetch timing.

Time Storage

  • notificationTime is stored in UTC (not local time)
  • All internal calculations use UTC timestamps
  • Display to users may convert to local time, but storage remains UTC

Prefetch Lead Calculation

  • Prefetch lead: prefetchTime = notificationTime - leadMinutes
  • Default lead: 5 minutes (300 seconds)
  • Minimum lead: 1 minute (60 seconds) - iOS requires at least 1 minute in future
  • Maximum lead: No hard limit, but iOS heuristics favor shorter windows

Edge Cases

Negative/Too-Small Lead Times:

  • If notificationTime - leadMinutes < now + 60 seconds:
    • Cap prefetch at now + 60 seconds (minimum iOS requirement)
    • Log: [DNP-FETCH] Prefetch lead too small, capping at minimum (60s)
    • Or disable prefetch if notification is too soon

DST (Daylight Saving Time) Changes:

  • UTC storage prevents DST issues
  • Test case: Schedule notification at 2:00 AM on DST change day
  • Verify: Prefetch and notification times remain correct (UTC-based)

Timezone Changes:

  • Test case: User changes timezone between schedule and fire
  • Verify: Notification fires at correct UTC time (not local time)
  • Log: [DNP-SCHEDULER] Timezone changed, but UTC schedule unchanged

Test Cases

  1. DST Change:

    • Schedule notification for 2:00 AM on DST transition day
    • Verify prefetch and notification use UTC, not local time
    • Check logs show UTC timestamps
  2. Timezone Change:

    • Schedule notification
    • Change device timezone
    • Verify notification fires at correct UTC time
    • Check logs confirm UTC-based scheduling
  3. Too-Small Lead:

    • Schedule notification 30 seconds in future
    • Verify prefetch is capped at 60 seconds minimum
    • Check logs show capping behavior

If You Only Have 30 Minutes

Quick validation checklist (each step links to log verification):

  1. Build test app (per IOS_TEST_APP_REQUIREMENTS.md)
  2. Run on iOS Simulator
  3. Schedule notification 5 minutes in future → See Log Checklist → Notification Scheduled
  4. Background app (Home button / Cmd+Shift+H)
  5. Xcode → Debug → Simulate Background Fetch → See Log Checklist → BGTask Fired
  6. Confirm logs show: Registration → Scheduling → BGTask handler → Fetch success → Task completed
  7. Wait for notification time or manually trigger → See Log Checklist → Notification Delivery
  8. Verify notification uses cached content (check logs for [DNP-FETCH] Found cached content) → See Log Checklist → Notification Delivery

Assumptions

  • Using BGTaskScheduler (BGAppRefreshTask) for prefetch
  • App has background modes enabled:
    • Background fetch
    • Background processing (if using BGProcessingTask)
  • Info.plist has:
    • BGTaskSchedulerPermittedIdentifiers array with task identifier: com.timesafari.dailynotification.fetch
  • Plugin exposes:
    • scheduleDailyNotification() method that schedules both prefetch and notification

Architecture at a Glance

Flow Overview:

JS/HTML Test App → Capacitor Bridge → DailyNotificationPlugin (iOS)
                                                      ↓
                                    BGTaskScheduler (BGAppRefreshTask)
                                                      ↓
                                    HTTP API (JWT, ETag, etc.)
                                                      ↓
                                    Cache/DB (UserDefaults/CoreData)
                                                      ↓
                                    Notification Delivery
                                    (uses cached content if valid,
                                     falls back to live fetch)

Key Components:

  • Test App: Provides UI to trigger plugin methods (see IOS_TEST_APP_REQUIREMENTS.md)
  • Plugin: Handles scheduling, fetching, caching, notification delivery
  • BGTaskScheduler: iOS background execution for prefetch
  • Storage: Persists schedules and cached content
  • Notification System: Delivers notifications using prefetched or live content

Verification Points:

  • BGTask registered at app startup
  • BGTask scheduled 5 minutes before notification time
  • BGTask executes and fetches content
  • Content persisted to cache/DB
  • Notification delivery uses cached content

Simulator Test Plan (Logic-Correctness)

Objective: Confirm that when a background task fires, your prefetch code runs end-to-end.

1. Harden Logging (One-Time Prep)

Add structured logs at key points:

On app startup:

[DNP-FETCH] Registering BGTaskScheduler task (id=com.timesafari.dailynotification.fetch)

When scheduling:

[DNP-FETCH] Scheduling BGAppRefreshTask (earliestBeginDate=2025-11-14T05:48:00Z)
[DNP-FETCH] Prefetch scheduled 5 minutes before notification (notificationTime=2025-11-14T05:53:00Z)

When BGTask handler fires:

[DNP-FETCH] BGTask handler invoked (task.identifier=com.timesafari.dailynotification.fetch)

Inside prefetch logic:

[DNP-FETCH] Starting fetch (notificationTime=2025-11-14T05:53:00Z, scheduleId=...)
[DNP-FETCH] Fetch success: status=200, items=1, ttl=86400
[DNP-FETCH] Fetch failed: error=..., willRetry=false

When integrating with notifications:

[DNP-FETCH] Using cached content for notification at 2025-11-14T05:53:00Z
[DNP-FETCH] No cached content, falling back to on-demand fetch

2. Start App in Simulator

  1. Clean build, run on iPhone 15 simulator (or similar)
  2. Verify console logs:
    • [DNP-FETCH] Registering BGTaskScheduler task... at startup

3. Schedule a Notification with Prefetch

  1. Call plugin's scheduleDailyNotification() from JS layer
  2. Check logs:
    • [DNP-FETCH] Scheduling BGAppRefreshTask... with sensible earliestBeginDate
    • [DNP-SCHEDULER] Scheduling notification for 2025-11-14T05:53:00Z
    • Any DB/state writes for the schedule

Note: For a full happy-path log example, see IOS_TEST_APP_REQUIREMENTS.md Testing Scenarios → Basic Functionality.

4. Simulate Background Fetch

With app running:

  1. Background the app (Home button / Cmd+Shift+H)
  2. In Xcode menu:
    • Debug → Simulate Background Fetch or
    • Debug → Simulate Background Refresh

Expected logs (in order):

[DNP-FETCH] BGTask handler invoked (task.identifier=...)
[DNP-FETCH] Starting fetch (notificationTime=..., scheduleId=...)
[DNP-FETCH] API call: GET /api/notifications (JWT present, ETag=...)
[DNP-FETCH] Fetch success: status=200, bytes=1234, ttl=86400
[DNP-FETCH] Cached content for scheduleId=...
[DNP-FETCH] Task completed (success=true)

See Sample Prefetch Response & Mapping below for expected API response structure.

5. Trigger or Wait for Notification

Option A: Wait for scheduled time

  • Wait until scheduled time
  • Look for:
    • [DNP-FETCH] Using cached content for notification at ...
    • [DNP-SCHEDULER] Notification delivered (id=..., title=...)

Option B: Manual trigger

  • Use test app UI to trigger notification test path
  • Confirm notification shows content derived from prefetch, not fallback

6. Sample Prefetch Response & Mapping

Example API Response:

When prefetch succeeds, the API returns JSON like:

{
  "id": "notif-2025-11-15",
  "title": "Your daily update",
  "body": "Here is today's summary…",
  "ttl": 86400,
  "scheduled_for": "2025-11-15T05:53:00Z"
}

Response Mapping:

  • id → Used for notification identification and deduplication
  • title → Notification title (displayed to user)
  • body → Notification body (displayed to user)
  • ttl → Cache validity check at delivery time (seconds)
  • scheduled_for → Cross-check with notificationTime to ensure alignment

Log Verification:

When you see [DNP-FETCH] Fetch success (status=200, bytes=1234, ttl=86400), the ttl=86400 should match the ttl field in the JSON response. The scheduled_for timestamp should match the notificationTime used in scheduling.

7. Negative-Path Tests

Network failure:

  1. Turn off network in Simulator (Settings → Network)
  2. Trigger background fetch
  3. Expect:
    • [DNP-FETCH] Fetch failed: error=NetworkError, willRetry=false
    • Then either:
      • Cached content used if available, or
      • Fallback path logged: [DNP-FETCH] No cached content, falling back to on-demand fetch

Permission denied:

  1. Deny notification permissions
  2. Attempt to schedule
  3. Expect:
    • [DNP-PLUGIN] notifications_denied for scheduleDailyNotification
    • Error returned: { error: "notifications_denied", ... }

Known OS Limitations

Critical: These are iOS system limitations, not plugin bugs. If these conditions occur, it is not a plugin bug; confirm logs and document behavior.

iOS may NOT run BGTasks if:

  • App has been force-quit by the user (iOS won't run BGTask for force-quit apps)
  • Background App Refresh is disabled in Settings → [Your App]
  • Device is in Low Power Mode and idle
  • Device battery is critically low

iOS timing heuristics:

  • iOS may delay or batch tasks; prefetch might run much later than earliestBeginDate (up to 15+ minutes)
  • iOS adapts BGTask frequency based on user behavior, app usage, and energy constraints
  • Notification delivery can drift by ±180 seconds (this is expected, not a bug)

What to do:

  • Check logs to confirm plugin logic executed correctly
  • Document the OS-imposed delay in test results
  • Verify fallback mechanisms work when prefetch is delayed

Real Device Test Plan (Timing & Reliability)

Objective: Confirm that prefetch happens near the intended time (e.g., 5 minutes before) in realistic conditions.

1. Install Dev Build on iPhone

  • Same BGTask + background modes config as simulator
  • Confirm logs visible via:
    • Xcode → Devices & Simulators → device → open console, or
    • os_log aggregator / remote logging

2. Baseline Run

  1. Open app
  2. Schedule a notification for 3060 minutes in the future with prefetch
  3. Lock the phone and leave it idle, plugged in (best-case for iOS)

3. Monitor Logs

Look for:

  • BGTask scheduled with appropriate earliestBeginDate
  • Some time before the notification:
    • BGTask handler invoked
    • Prefetch fetch executes
  • At notification fire:
    • Notification uses prefetched data

4. Variations

Test different conditions:

  • Device on battery vs plugged in
  • App force-quit vs just backgrounded
  • Multiple days in a row: iOS will adapt its heuristics; see if prefetch becomes more or less reliable
  • Low Power Mode: May delay/disable background tasks
  • Background App Refresh disabled: Should fail gracefully

5. Success Criteria

In "good" conditions (plugged in, WiFi, not force-quit):

  • Prefetch runs at least once before the notification in the majority of tests
  • Failure modes are logged clearly:
    • So you know when timing failed because of iOS, not your code
  • For at least one full test cycle, logs and telemetry counts confirm that the sequence: scheduled → executed → success → used is coherent

Acceptable outcomes:

  • Prefetch runs within 15 minutes of earliestBeginDate (iOS heuristic window)
  • If prefetch misses, fallback to on-demand fetch works
  • All failures are logged with clear reasons

Test Run Result Template

Use this template when recording test runs (especially on real devices):

Date:
Device model:
iOS version:
App build / commit:
Lead time (T-Lead - see glossary):
Scenario ID(s) tested:
Conditions: (plugged vs battery, Wi-Fi vs LTE, Low Power Mode on/off, Background App Refresh on/off)
Outcome summary:
  - Prefetch scheduled at: 
  - BGTask executed at: 
  - Notification fired at:
  - Cached content used: yes/no
Failures observed (if any) and key log lines:
Notes / follow-ups:

Log Checklist

When everything is wired correctly, one full cycle should produce:

1. App Launch

Expected logs:

[DNP-FETCH] Registering BGTaskScheduler task (id=com.timesafari.dailynotification.fetch)
[DNP-PLUGIN] Startup complete (hasPendingSchedules=true|false)

If you see registration but not startup:

  • Check plugin initialization in AppDelegate
  • Verify Capacitor plugin registration
  • Check for initialization errors in logs

If you don't see registration:

  • Check Info.plist has BGTaskSchedulerPermittedIdentifiers
  • Verify task registered in AppDelegate before app finishes launching
  • Check for registration errors in logs

2. Notification Scheduled (from JS / plugin)

Expected logs:

[DNP-FETCH] Scheduling prefetch for notification at 2025-11-14T05:53:00Z
[DNP-FETCH] BGAppRefreshTask scheduled (earliestBeginDate=2025-11-14T05:48:00Z)
[DNP-SCHEDULER] Scheduling notification for 2025-11-14T05:53:00Z
[DNP-STORAGE] Persisted schedule to DB (id=..., type=DAILY, ...)

Note: For a full happy-path log example, see IOS_TEST_APP_REQUIREMENTS.md Testing Scenarios → Basic Functionality.

If you see scheduling but not BGTask scheduled:

  • Check BGTaskScheduler submission succeeded
  • Verify earliestBeginDate is at least 1 minute in future
  • Check for BGTaskScheduler errors in logs

If you see BGTask scheduled but not persisted:

  • Check database write operations
  • Verify storage layer is initialized
  • Check for storage errors in logs

3. BGTask Fired (simulator or device)

Expected logs:

[DNP-FETCH] BGTask handler invoked (id=com.timesafari.dailynotification.fetch)
[DNP-FETCH] Resolved next notification needing content (time=..., scheduleId=...)
[DNP-FETCH] Starting fetch from <URL> (notificationTime=..., jwtPresent=true)
[DNP-FETCH] Fetch success (status=200, bytes=1234, ttl=86400)
[DNP-FETCH] Cached content for scheduleId=...
[DNP-FETCH] Task completed (success=true)

If you see "Registering BGTaskScheduler task" but never see "BGTask handler invoked" on device:

  • Check Background App Refresh enabled in Settings → [Your App]
  • Verify app not force-quit (iOS won't run BGTask for force-quit apps)
  • Check Info.plist identifiers match code exactly (case-sensitive)
  • Verify task was actually scheduled (check earliestBeginDate in logs)
  • On Simulator: Use Xcode → Debug → Simulate Background Fetch

If you see handler invoked but not fetch starting:

  • Check notification resolution logic
  • Verify schedule exists in database
  • Check for resolution errors in logs

If you see fetch starting but not success:

  • Check network connectivity
  • Verify API endpoint is accessible
  • Check JWT/authentication
  • Look for HTTP error codes in logs

4. Notification Delivery

Expected logs:

[DNP-SCHEDULER] Preparing content for notification at 2025-11-14T05:53:00Z
[DNP-FETCH] Found cached content (age=300s, source=prefetch)
[DNP-SCHEDULER] Notification built (title=..., body=..., id=...)
[DNP-SCHEDULER] Notification scheduled/delivered

If you see "Preparing content" but not "Found cached content":

  • Check if prefetch actually ran (see BGTask logs)
  • Verify cache/DB read operations
  • Check TTL validation (content may have expired)
  • Look for fallback to on-demand fetch

If you see "Found cached content" but not "Notification built":

  • Check notification building logic
  • Verify notification permissions granted
  • Check for notification construction errors

5. Failure Cases

Expected logs:

[DNP-FETCH] Fetch failed (error=NetworkError, httpStatus=0, willRetry=false)
[DNP-FETCH] No cached content available, falling back to on-demand fetch
[DNP-FETCH] Task completed (success=false, reason=NETWORK)

If you see fetch failures repeatedly:

  • Check network connectivity
  • Verify API endpoint is correct
  • Check authentication (JWT) is valid
  • Look for specific error reasons in logs (NETWORK, AUTH, SYSTEM)

If you see "No cached content" but prefetch should have run:

  • Check if BGTask actually executed (see BGTask logs)
  • Verify cache/DB write operations succeeded
  • Check for cache expiration (TTL may have passed)
  • Verify fallback to on-demand fetch works

If you can walk through a log and trace all steps like this, you're in good shape.


Behavior Classification

Testable matrix of deterministic vs heuristic behavior:

Bucket Component / Method Deterministic? Test on Notes
A BGTaskScheduler.shared.register Yes Sim + Dev Registration must always log & succeed/fail deterministically
A configure(), getLastNotification(), cancelAllNotifications(), getNotificationStatus(), updateSettings() Yes Sim + Dev Logic & I/O-only, no timing dependencies
A getBatteryStatus(), getPowerState(), getRollingWindowStats() Yes Sim + Dev State reading, deterministic
A testJWTGeneration(), testEndorserAPI() Yes Sim + Dev API call logic, deterministic
A Fetch function logic (HTTP calls, DB writes, JSON parsing) Yes Sim + Dev Code path is deterministic
A Error handling, retry logic, fallbacks Yes Sim + Dev Logic is deterministic
A Log emission / structured logging Yes Sim + Dev Logging is deterministic
A Notification building logic (given data) Yes Sim + Dev Title/body/payload construction is deterministic
A TTL validation at delivery time Yes Sim + Dev Validation logic is deterministic
B scheduleDailyNotification() Logic yes, timing no Sim + Dev Use logs to verify scheduling but not run time
B maintainRollingWindow() Logic yes, timing no Sim + Dev Logic deterministic, but when iOS allows execution is heuristic
B BGTaskScheduler scheduling (earliestBeginDate) Logic yes, timing no Sim + Dev earliestBeginDate is a hint, not a guarantee
B Notification trigger + prefetch relationship Logic yes, timing no Sim + Dev "If cached content exists, use it" is deterministic; whether prefetch ran in time is heuristic
C Time between earliestBeginDate and task execution No Device Purely heuristic, iOS controls when tasks run
C Background task execution timing (BGTaskScheduler) No Device iOS heuristics control execution timing
C Notification delivery timing (UNUserNotificationCenter) No Device iOS controls delivery timing (±180s tolerance)
C Reboot recovery detection (uptime comparison) No Device May vary based on system state

Bucket Summary:

  • 🟢 Bucket A (Deterministic): Test in Simulator and Device - Logic correctness
  • 🟡 Bucket B (Partially Deterministic): Test flow in Simulator, timing on Device
  • 🔴 Bucket C (Heuristic): Test on Real Device only - Timing and reliability

Test Harness Reference

File: ios/Plugin/DailyNotificationBackgroundTaskTestHarness.swift

Purpose: Use this harness if the plugin code is broken or being refactored; it isolates BGTask behavior from plugin wiring.

What it demonstrates:

  • Task registration
  • Task scheduling
  • Task handler implementation
  • Expiration handling
  • Completion reporting

Usage:

  • Reference implementation when building actual prefetch logic
  • Standalone testing of BGTaskScheduler behavior
  • Debugging BGTask registration/scheduling issues
  • See file for detailed usage examples and testing instructions

Integration with Main Directive

This testing guide supports:

  • Phase 1: Core prefetch functionality testing
  • Phase 2: Advanced features testing (rolling window, TTL)
  • Phase 3: TimeSafari integration testing (JWT, ETag)
  • Validation Matrix: Cross-platform feature validation

Telemetry and Monitoring

Expected Counters (even if not yet implemented):

Define counters you expect the runtime to emit:

  • dnp_prefetch_scheduled_total - Total prefetch tasks scheduled
  • dnp_prefetch_executed_total - Total prefetch tasks executed
  • dnp_prefetch_success_total - Total successful prefetch executions
  • dnp_prefetch_failure_total{reason="NETWORK|AUTH|SYSTEM"} - Total failed prefetch executions by reason
  • dnp_prefetch_used_for_notification_total - Total notifications using prefetched content

Telemetry Pipeline:

These counters MUST be emitted via the same pipeline as Android (e.g., structlog → rsyslog → Prometheus/Influx/Loki). If telemetry is not yet wired on iOS, mark tests that rely on counters as P2 and fall back to log inspection for Phase 1.

Success Criteria:

  • For at least one full test cycle, logs and telemetry counts confirm that the sequence: scheduled → executed → success → used is coherent
  • Counters increment as expected through the prefetch lifecycle
  • Failure counters provide clear reason categorization

Test Campaign Sign-Off Checklist (Phase 1)

Use this checklist to verify Phase 1 prefetch testing is complete:

  • Simulator: one full happy-path cycle verified (Schedule → BGTask → Fetch → Delivery)
  • Device: at least one "best-case" run (plugged in, Wi-Fi, not force-quit) where prefetch runs before notification and cached content is used
  • At least one failure path logged (NETWORK or PERMISSIONS) and handled gracefully
  • Time & T-Lead edge-case tests run (too-small lead, timezone change) or explicitly deferred
  • Telemetry counters either wired and validated, or marked P2 with log-based fallback documented

See also:

  • doc/directives/0003-iOS-Android-Parity-Directive.md - Main implementation directive
  • doc/test-app-ios/IOS_TEST_APP_REQUIREMENTS.md - Test app setup requirements

Glossary

BGTaskScheduler iOS framework for scheduling background tasks (BGAppRefreshTask / BGProcessingTask). Provides heuristic-based background execution, not exact timing guarantees.

UNUserNotificationCenter iOS notification framework for scheduling and delivering user notifications. Handles permission requests and notification delivery.

T-Lead The lead time between prefetch and notification fire, e.g., 5 minutes. Prefetch is scheduled at notificationTime - T-Lead.

Bucket A/B/C Deterministic vs heuristic classification used in Behavior Classification:

  • Bucket A (Deterministic): Test in Simulator and Device - Logic correctness
  • Bucket B (Partially Deterministic): Test flow in Simulator, timing on Device
  • Bucket C (Heuristic): Test on Real Device only - Timing and reliability

UTC Coordinated Universal Time. All internal timestamps are stored in UTC to avoid DST and timezone issues.

earliestBeginDate The earliest time iOS may execute a BGTask. This is a hint, not a guarantee; iOS may run the task later based on heuristics.


Status: 🎯 READY FOR USE
Next Steps: Use this guide when implementing and testing Phase 1+ prefetch functionality


Changelog (high-level)

  • 2025-11-15 — Initial Phase 1 version (prefetch MVP, Android parity)