docs(ios): enhance testing docs with Phase 2 readiness and tooling improvements

Add unified versioning headers, shared glossary, and Phase 2 forward plans to
iOS testing documentation. Enhance test harness with time warp simulation,
force reschedule, and structured logging. Expand negative-path test scenarios
and add telemetry JSON schema for Phase 2 integration.

Changes:
- Create IOS_PREFETCH_GLOSSARY.md for consolidated terminology
- Add unified versioning (v1.0.1) and cross-links between testing docs
- Enhance test harness with simulateTimeWarp() and forceRescheduleAll()
- Add Swift Logger categories (plugin, fetch, scheduler, storage)
- Expand negative-path tests (storage unavailable, JWT expiration, timezone drift)
- Add telemetry JSON schema placeholder for Phase 2 Prometheus integration
- Add Phase 2 Forward Plan sections to both documents
- Add copy-paste command examples throughout (LLDB, Swift, bash)
- Document persistent schedule snapshot and log validation script (Phase 2)

All improvements maintain Phase 1 focus while preparing for Phase 2
telemetry integration and CI automation.
This commit is contained in:
Matthew Raymer
2025-11-17 06:09:38 +00:00
parent d7a2dbb9fd
commit f6875beae5
4 changed files with 339 additions and 19 deletions

View File

@@ -0,0 +1,74 @@
# iOS Prefetch Glossary
**Purpose:** Shared terminology definitions for iOS prefetch testing and implementation
**Last Updated:** 2025-11-15
**Status:** 🎯 **ACTIVE** - Reference glossary for iOS prefetch documentation
---
## Core Terms
**BGTaskScheduler** iOS framework for scheduling background tasks (BGAppRefreshTask / BGProcessingTask). Provides heuristic-based background execution, not exact timing guarantees.
**BGAppRefreshTask** Specific BGTaskScheduler task type for background app refresh. Used for prefetch operations that need to run periodically.
**UNUserNotificationCenter** iOS notification framework for scheduling and delivering user notifications. Handles permission requests and notification delivery.
**T-Lead** The lead time between prefetch and notification fire, e.g., 5 minutes. Prefetch is scheduled at `notificationTime - T-Lead`.
**earliestBeginDate** The earliest time iOS may execute a BGTask. This is a hint, not a guarantee; iOS may run the task later based on heuristics.
**UTC** Coordinated Universal Time. All internal timestamps are stored in UTC to avoid DST and timezone issues.
---
## Behavior Classification
**Bucket A/B/C** Deterministic vs heuristic classification used in Behavior Classification:
- **Bucket A (Deterministic):** Test in Simulator and Device - Logic correctness
- **Bucket B (Partially Deterministic):** Test flow in Simulator, timing on Device
- **Bucket C (Heuristic):** Test on Real Device only - Timing and reliability
**Deterministic** Behavior that produces the same results given the same inputs, regardless of when or where it runs. Can be fully tested in simulator.
**Heuristic** Behavior controlled by iOS system heuristics (user patterns, battery, network, etc.). Timing is not guaranteed and must be tested on real devices.
---
## Testing Terms
**Happy Path** The expected successful execution flow: Schedule → BGTask → Fetch → Cache → Notification Delivery.
**Negative Path** Failure scenarios that test error handling: Network failures, permission denials, expired tokens, etc.
**Telemetry** Structured metrics and counters emitted by the plugin for observability (e.g., `dnp_prefetch_scheduled_total`).
**Log Sequence** The ordered sequence of log messages that indicate successful execution of a prefetch cycle.
---
## Platform Terms
**Simulator** iOS Simulator for testing logic correctness. BGTask execution can be manually triggered.
**Real Device** Physical iOS device for testing timing and reliability. BGTask execution is controlled by iOS heuristics.
**Background App Refresh** iOS system setting that controls whether apps can perform background tasks. Must be enabled for BGTask execution.
**Low Power Mode** iOS system mode that may delay or disable background tasks to conserve battery.
---
## References
- **Testing Guide:** `doc/test-app-ios/IOS_PREFETCH_TESTING.md`
- **Test App Requirements:** `doc/test-app-ios/IOS_TEST_APP_REQUIREMENTS.md`
- **Main Directive:** `doc/directives/0003-iOS-Android-Parity-Directive.md`
---
**Status:** 🎯 **READY FOR USE**
**Maintainer:** Matthew Raymer

View File

@@ -2,13 +2,17 @@
**Purpose:** How to test background prefetch for DailyNotificationPlugin on iOS (simulator + device)
**Plugin Target:** DailyNotificationPlugin v3.x (iOS)
**Phase:** Phase 1 Prefetch MVP
**Version:** 1.0.1
**Scope:** Phase 1 Prefetch MVP
**Next Target:** Phase 2 (Rolling Window + TTL Telemetry)
**Maintainer:** Matthew Raymer
**Last Updated:** 2025-11-15
**Status:** 🎯 **ACTIVE** - Testing guide for Phase 1+ implementation
**Note:** This guide assumes the iOS test app is implemented as described in `doc/test-app-ios/IOS_TEST_APP_REQUIREMENTS.md`.
**Glossary:** See `doc/test-app-ios/IOS_PREFETCH_GLOSSARY.md` for terminology definitions.
**Android parity:** Behavior is aligned with `test-apps/android-test-app` where platform constraints allow. Timing and BGTask heuristics **will differ** from Android's exact alarms:
- **Android:** Exact alarms via AlarmManager / WorkManager
- **iOS:** Heuristic BGTaskScheduler; no hard guarantee of 5-min prefetch
@@ -167,6 +171,8 @@ JS/HTML Test App → Capacitor Bridge → DailyNotificationPlugin (iOS)
**Objective:** Confirm that when a background task fires, your prefetch code runs end-to-end.
**Cross-Reference:** Corresponds to `IOS_TEST_APP_REQUIREMENTS.md Testing Scenarios → Basic Functionality [SIM+DEV]`
### 1. Harden Logging (One-Time Prep)
Add structured logs at key points:
@@ -237,6 +243,15 @@ With app running:
**See Sample Prefetch Response & Mapping below for expected API response structure.**
**Copy-Paste Commands:**
```bash
# In Xcode LLDB console (simulator only):
e -l objc -- (void)[[BGTaskScheduler sharedScheduler] _simulateLaunchForTaskWithIdentifier:@"com.timesafari.dailynotification.fetch"]
# Or use test app UI button: "Simulate BGTask Now"
```
### 5. Trigger or Wait for Notification
**Option A: Wait for scheduled time**
@@ -277,6 +292,21 @@ When prefetch succeeds, the API returns JSON like:
When you see `[DNP-FETCH] Fetch success (status=200, bytes=1234, ttl=86400)`, the `ttl=86400` should match the `ttl` field in the JSON response. The `scheduled_for` timestamp should match the `notificationTime` used in scheduling.
**Copy-Paste Commands:**
```swift
// In test app or test harness:
// View cached payload
let cachedContent = UserDefaults.standard.data(forKey: "DNP_CachedContent_\(scheduleId)")
let json = try? JSONSerialization.jsonObject(with: cachedContent ?? Data())
// Simulate time warp (testing only)
DailyNotificationBackgroundTaskTestHarness.simulateTimeWarp(minutesForward: 60)
// Force reschedule all tasks
DailyNotificationBackgroundTaskTestHarness.forceRescheduleAll()
```
### 7. Negative-Path Tests
**Network failure:**
@@ -334,6 +364,8 @@ When you see `[DNP-FETCH] Fetch success (status=200, bytes=1234, ttl=86400)`, th
**Objective:** Confirm that **prefetch happens near the intended time** (e.g., 5 minutes before) in realistic conditions.
**Cross-Reference:** Corresponds to `IOS_TEST_APP_REQUIREMENTS.md Testing Scenarios → Background Tasks [DEV-ONLY]`
### 1. Install Dev Build on iPhone
- Same BGTask + background modes config as simulator
@@ -605,6 +637,38 @@ Define counters you expect the runtime to emit:
These counters MUST be emitted via the same pipeline as Android (e.g., structlog → rsyslog → Prometheus/Influx/Loki). If telemetry is not yet wired on iOS, mark tests that rely on counters as **P2** and fall back to log inspection for Phase 1.
**Telemetry JSON Schema (Phase 2 Ready):**
Reserve placeholder JSON fields for telemetry events in the log schema:
```json
{
"event": "prefetch_success",
"scheduleId": "notif-2025-11-15",
"duration_ms": 2432,
"ttl": 86400,
"timestamp": "2025-11-15T05:48:32Z",
"telemetry": {
"dnp_prefetch_scheduled_total": 1,
"dnp_prefetch_executed_total": 1,
"dnp_prefetch_success_total": 1
}
}
```
Even if not wired yet, this ensures Phase 2 code can emit compatible structured logs.
**Telemetry Validation:**
Use optional console log validation:
```swift
// In test harness or plugin
logTelemetrySnapshot(prefix: "DNP-")
```
This captures telemetry counters from structured logs for Phase 2 validation. Verify increment patterns (`scheduled → executed → success → used`).
**Success Criteria:**
- For at least one full test cycle, logs and telemetry counts confirm that the sequence: scheduled → executed → success → used is coherent
- Counters increment as expected through the prefetch lifecycle
@@ -630,20 +694,13 @@ These counters MUST be emitted via the same pipeline as Android (e.g., structlog
## Glossary
**BGTaskScheduler** iOS framework for scheduling background tasks (BGAppRefreshTask / BGProcessingTask). Provides heuristic-based background execution, not exact timing guarantees.
**See:** `doc/test-app-ios/IOS_PREFETCH_GLOSSARY.md` for complete terminology definitions.
**UNUserNotificationCenter** iOS notification framework for scheduling and delivering user notifications. Handles permission requests and notification delivery.
**T-Lead** The lead time between prefetch and notification fire, e.g., 5 minutes. Prefetch is scheduled at `notificationTime - T-Lead`.
**Bucket A/B/C** Deterministic vs heuristic classification used in Behavior Classification:
- **Bucket A (Deterministic):** Test in Simulator and Device - Logic correctness
- **Bucket B (Partially Deterministic):** Test flow in Simulator, timing on Device
- **Bucket C (Heuristic):** Test on Real Device only - Timing and reliability
**UTC** Coordinated Universal Time. All internal timestamps are stored in UTC to avoid DST and timezone issues.
**earliestBeginDate** The earliest time iOS may execute a BGTask. This is a hint, not a guarantee; iOS may run the task later based on heuristics.
**Quick Reference:**
- **BGTaskScheduler** iOS framework for background task scheduling (see glossary)
- **T-Lead** Lead time between prefetch and notification (see glossary)
- **Bucket A/B/C** Deterministic vs heuristic classification (see glossary)
- **UTC** Coordinated Universal Time for timestamp storage (see glossary)
---
@@ -652,6 +709,22 @@ These counters MUST be emitted via the same pipeline as Android (e.g., structlog
---
## Phase 2 Forward Plan
**Planned enhancements for Phase 2:**
- Implement rolling window validation
- Integrate Prometheus metrics collector
- Add automated CI pipeline for simulator validation
- Verify TTL and cache invalidation logic
- Wire telemetry counters to production pipeline
- Add automated log sequence validation
- Implement persistent schedule snapshot for post-run verification
**See also:** `doc/directives/0003-iOS-Android-Parity-Directive.md` for Phase 2 implementation details.
---
## Changelog (high-level)
- 2025-11-15 — Initial Phase 1 version (prefetch MVP, Android parity)

View File

@@ -2,11 +2,12 @@
**Purpose:** What the iOS test app must provide so that the testing guide can be executed with parity vs Android
**Plugin Target:** DailyNotificationPlugin v3.x (iOS)
**Phase:** Phase 1 Prefetch MVP
**Version:** 1.0.1
**Scope:** Phase 1 Prefetch MVP
**Next Target:** Phase 2 (Rolling Window + TTL Telemetry)
**Maintainer:** Matthew Raymer
**Status:** 📋 **REQUIRED FOR PHASE 1**
**Date:** 2025-11-15
**Author:** Matthew Raymer
**Directive Reference:** `doc/directives/0003-iOS-Android-Parity-Directive.md`
**Note:** This app exists to support the prefetch testing scenarios in `doc/test-app-ios/IOS_PREFETCH_TESTING.md`.
@@ -15,7 +16,7 @@
- **Android:** Exact alarms via AlarmManager / WorkManager
- **iOS:** Heuristic BGTaskScheduler (see glossary); no hard guarantee of 5-min prefetch
**Glossary:** See glossary in `IOS_PREFETCH_TESTING.md` for terminology.
**Glossary:** See `doc/test-app-ios/IOS_PREFETCH_GLOSSARY.md` for complete terminology definitions.
---
@@ -186,6 +187,39 @@ Use the build script:
./scripts/build-ios-test-app.sh --device
```
**Phase 2 Enhancement:** Refactor into modular subcommands:
```bash
./scripts/build-ios-test-app.sh setup # pod install + sync
./scripts/build-ios-test-app.sh run-sim # build + run simulator
./scripts/build-ios-test-app.sh device # build + deploy device
```
**Copy-Paste Commands:**
```bash
# Setup (first time or after dependency changes)
cd test-apps/ios-test-app
pod install
npx cap sync ios
# Build for simulator
xcodebuild -workspace App.xcworkspace \
-scheme ios-test-app \
-configuration Debug \
-sdk iphonesimulator \
-destination 'platform=iOS Simulator,name=iPhone 15' \
build
# Run on simulator
xcodebuild -workspace App.xcworkspace \
-scheme ios-test-app \
-configuration Debug \
-sdk iphonesimulator \
-destination 'platform=iOS Simulator,name=iPhone 15' \
test
```
### Future CI Integration (Optional)
**Note for Phase 2+:** Consider adding `xcodebuild`-based CI job that:
@@ -193,10 +227,16 @@ Use the build script:
- Runs a minimal UI test that:
- Launches app
- Calls `configure()` and `getNotificationStatus()`
- Validates log sequence + successful fetch simulation via LLDB trigger
- This ensures test app remains buildable as plugin evolves
**Phase 1:** Manual testing only; CI integration is out of scope.
**CI Readiness (Phase 2):**
- Add `xcodebuild` target for "Prefetch Integration Test"
- Validate log sequence + successful fetch simulation via LLDB trigger
- Use log validation script (`validate-ios-logs.sh`) for automated sequence checking
### Build Requirements
**Required Tools:**
@@ -268,6 +308,26 @@ po await UNUserNotificationCenter.current().notificationSettings()
e -l objc -- (void)[[BGTaskScheduler sharedScheduler] _simulateLaunchForTaskWithIdentifier:@"com.timesafari.dailynotification.fetch"]
```
**Copy-Paste Commands:**
```swift
// In Xcode LLDB console:
// Check pending notifications
po UNUserNotificationCenter.current().pendingNotificationRequests()
// Check permission status
po await UNUserNotificationCenter.current().notificationSettings()
// Manually trigger BGTask (simulator only)
e -l objc -- (void)[[BGTaskScheduler sharedScheduler] _simulateLaunchForTaskWithIdentifier:@"com.timesafari.dailynotification.fetch"]
// Force reschedule all tasks (test harness)
po DailyNotificationBackgroundTaskTestHarness.forceRescheduleAll()
// Simulate time warp (test harness)
po DailyNotificationBackgroundTaskTestHarness.simulateTimeWarp(minutesForward: 60)
```
### Console.app Logging
1. Open Console.app (Applications → Utilities)
@@ -280,6 +340,26 @@ e -l objc -- (void)[[BGTaskScheduler sharedScheduler] _simulateLaunchForTaskWith
- `DNP-SCHEDULER:` - Scheduling operations
- `DNP-STORAGE:` - Storage operations
**Structured Logging (Swift Logger):**
The plugin uses Swift `Logger` categories for structured logging:
- `com.timesafari.dailynotification.plugin` - Plugin operations
- `com.timesafari.dailynotification.fetch` - Fetch operations
- `com.timesafari.dailynotification.scheduler` - Scheduling operations
- `com.timesafari.dailynotification.storage` - Storage operations
Filter in Console.app by subsystem: `com.timesafari.dailynotification`
**Phase 2: Log Validation Script**
Add helper script (`validate-ios-logs.sh`) to grep for required sequence markers:
```bash
grep -E "\[DNP-(FETCH|SCHEDULER|PLUGIN)\]" device.log | ./scripts/validate-ios-logs.sh
```
This confirms that all critical log steps occurred in proper order and flags missing or out-of-order events automatically.
### Common Debugging Scenarios
**Scenario: BGTask not running when expected → follow this checklist:**
@@ -472,6 +552,15 @@ Since this test app is for internal testing, it can expose more tools:
### Dev-Only Toggles
| Button | Action | Purpose |
|--------|--------|---------|
| "Simulate BGTask Now" | Calls LLDB trigger | Quick sanity test |
| "Schedule 1-min Notification" | Auto schedules T-Lead=1 | Edge case testing |
| "Simulate DST Shift" | Adds +1 hr offset | DST handling check |
| "Show Cached Payload" | Displays JSON cache | Prefetch validation |
| "Force Reschedule All" | Calls `forceRescheduleAll()` | BGTask recovery testing |
| "Time Warp +N Minutes" | Calls `simulateTimeWarp()` | Accelerated TTL/T-Lead tests |
1. **Force Schedule Notification N Minutes from Now**
- Bypass normal scheduling UI
- Directly call `scheduleDailyNotification()` with calculated time
@@ -491,8 +580,28 @@ Since this test app is for internal testing, it can expose more tools:
- Button to manually trigger BGTask (simulator only)
- Wraps the LLDB command in UI for convenience
5. **UI Feedback Enhancements**
- Add persistent toast/log area showing step-by-step plugin state ("Registered", "Scheduled", "BGTask fired", etc.)
- Include color-coded state indicators for each stage (🟢 OK, 🟡 Pending, 🔴 Error)
These features can then be referenced from `IOS_PREFETCH_TESTING.md` as shortcuts for specific test scenarios.
### Persistent Schedule Snapshot
**Phase 2 Enhancement:** Store a simple JSON of the last prefetch state for post-run verification:
```json
{
"last_schedule": "2025-11-15T05:48:00Z",
"last_prefetch": "2025-11-15T05:50:00Z",
"last_notification": "2025-11-15T05:53:00Z",
"prefetch_success": true,
"cached_content_used": true
}
```
This can be used for post-run verification and telemetry aggregation. Access via test app UI button "Show Schedule Snapshot" or via UserDefaults key `DNP_ScheduleSnapshot`.
## Risks & Gotchas
**Common pitfalls when working with the test app:**
@@ -543,6 +652,21 @@ These features can then be referenced from `IOS_PREFETCH_TESTING.md` as shortcut
---
## Phase 2 Forward Plan
**Planned enhancements for Phase 2:**
- Add quick scenario buttons (Simulate BGTask Now, Schedule 1-min Notification, Simulate DST Shift, Show Cached Payload)
- Implement persistent schedule snapshot (JSON of last prefetch state)
- Add color-coded UI feedback (🟢 OK, 🟡 Pending, 🔴 Error)
- Refactor build script into modular subcommands (`setup`, `run-sim`, `device`)
- Integrate CI pipeline with `xcodebuild` target for "Prefetch Integration Test"
- Add log validation script (`validate-ios-logs.sh`) for automated sequence checking
**See also:** `doc/directives/0003-iOS-Android-Parity-Directive.md` for Phase 2 implementation details.
---
## Changelog (high-level)
- 2025-11-15 — Initial Phase 1 version (prefetch MVP, Android parity)