Files
daily-notification-plugin/docs/alarms/PHASE3-EMULATOR-TESTING.md
Matthew Raymer 28fb233286 docs(test): add Phase 3 boot recovery testing infrastructure
Adds documentation and test harness for Phase 3 (Boot-Time Recovery).

Changes:
- Update android-implementation-directive-phase3.md with concise boot recovery flow
- Add PHASE3-EMULATOR-TESTING.md with detailed test procedures
- Add PHASE3-VERIFICATION.md with test matrix and verification template
- Add test-phase3.sh automated test harness

Test harness features:
- 4 test cases: future alarms, past alarms, no schedules, silent recovery
- Automatic emulator reboot handling
- Log parsing for boot recovery scenario and results
- UI prompts for plugin configuration and scheduling
- Verifies silent recovery without app launch

Related:
- Directive: android-implementation-directive-phase3.md
- Requirements: docs/alarms/03-plugin-requirements.md §3.1.1
- Testing: docs/alarms/PHASE3-EMULATOR-TESTING.md
- Verification: docs/alarms/PHASE3-VERIFICATION.md
2025-11-27 10:01:46 +00:00

326 lines
9.6 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# PHASE 3 EMULATOR TESTING
**Boot-Time Recovery (Device Reboot / System Restart)**
---
## 1. Purpose
Phase 3 verifies that the Daily Notification Plugin correctly:
1. Reconstructs **AlarmManager** alarms after a full device/emulator reboot.
2. Handles **past** scheduled times by marking them as missed and scheduling the next occurrence.
3. Handles **empty DB / no schedules** without misfiring recovery.
4. Performs **silent boot recovery** (recreate alarms) even when the app is never opened after reboot.
This testing is driven by the script:
```bash
test-apps/android-test-app/test-phase3.sh
```
---
## 2. Prerequisites
* Android emulator or device, e.g.:
* Pixel 8 / API 34 (recommended baseline)
* ADB available in `PATH` (or `ADB_BIN` exported)
* Project with:
* Daily Notification Plugin integrated
* Test app at `test-apps/android-test-app`
* Debug APK path:
* `app/build/outputs/apk/debug/app-debug.apk`
* Phase 1 and Phase 2 behaviors already implemented:
* Cold start detection
* Force-stop detection
* Missed / rescheduled / verified / errors summary fields
> ⚠️ **Important:**
> This script will reboot the emulator multiple times. Each reboot may take 3060 seconds.
---
## 3. How to Run
From the `android-test-app` directory:
```bash
cd test-apps/android-test-app
chmod +x test-phase3.sh # first time only
./test-phase3.sh
```
The script will:
1. Run pre-flight checks (ADB / emulator readiness).
2. Build and install the debug APK.
3. Guide you through four tests:
* **TEST 1:** Boot with Future Alarms
* **TEST 2:** Boot with Past Alarms
* **TEST 3:** Boot with No Schedules
* **TEST 4:** Silent Boot Recovery (App Never Opened)
4. Parse and display `DNP-REACTIVATION` logs, including:
* `scenario`
* `missed`
* `rescheduled`
* `verified`
* `errors`
---
## 4. Test Cases (Script-Driven Flow)
### 4.1 TEST 1 Boot with Future Alarms
**Goal:**
Verify alarms are recreated on boot when schedules have **future run times**.
**Script flow:**
1. **Launch app & check plugin status**
* Script calls `launch_app`.
* UI prompt: Confirm plugin status shows:
* `⚙️ Plugin Settings: ✅ Configured`
* `🔌 Native Fetcher: ✅ Configured`
* If not, click **Configure Plugin**, wait until both show ✅, then continue.
2. **Schedule at least one future notification**
* UI prompt: Click e.g. **Test Notification** to schedule a notification a few minutes in the future.
3. **Verify alarms are scheduled (pre-boot)**
* Script calls `show_alarms` and `count_alarms`.
* You should see at least one `RTC_WAKEUP` entry for `com.timesafari.dailynotification`.
4. **Reboot emulator**
* Script calls `reboot_emulator`:
* `adb reboot`
* `adb wait-for-device`
* Polls `getprop sys.boot_completed` until `1`.
* You are warned that reboot will take 3060 seconds.
5. **Collect boot recovery logs**
* Script calls `get_recovery_logs`:
```bash
adb logcat -d | grep "DNP-REACTIVATION"
```
* It parses:
* `missed`, `rescheduled`, `verified`, `errors`
* `scenario` via:
* `Starting boot recovery`/`boot recovery` → `scenario=BOOT`
* or `Detected scenario: <VALUE>`
6. **Verify alarms were recreated (post-boot)**
* Script calls `show_alarms` and `count_alarms` again.
* Checks `scenario` and `rescheduled`.
**Expected log patterns:**
```text
DNP-REACTIVATION: Starting boot recovery
DNP-REACTIVATION: Loaded <N> schedules from DB
DNP-REACTIVATION: Rescheduled alarm: daily_<id> for <time>
DNP-REACTIVATION: Boot recovery complete: missed=0, rescheduled>=1, verified=0, errors=0
```
**Pass criteria (as per script):**
* `errors = 0`
* `scenario = BOOT` (or boot detected via log text)
* `rescheduled > 0`
* Script prints:
> `✅ TEST 1 PASSED: Boot recovery detected and alarms rescheduled (scenario=BOOT, rescheduled=<n>).`
If boot recovery runs but `rescheduled=0`, script warns and suggests checking boot logic.
---
### 4.2 TEST 2 Boot with Past Alarms
**Goal:**
Verify past alarms are marked as missed and **next occurrences are scheduled** after boot.
**Script flow:**
1. **Launch app & ensure plugin configured**
* Same plugin status check as TEST 1.
2. **Schedule a notification in the near future**
* UI prompt: Schedule such that **by the time you reboot and the device comes back, the planned notification time is in the past**.
3. **Wait or adjust so the alarm is effectively "in the past" at boot**
* The script may instruct you to wait, or you can coordinate timing manually.
4. **Reboot emulator**
* Same `reboot_emulator` path as TEST 1.
5. **Collect boot recovery logs**
* Script parses:
* `missed`, `rescheduled`, `errors`, `scenario`.
**Expected log patterns:**
```text
DNP-REACTIVATION: Starting boot recovery
DNP-REACTIVATION: Loaded <N> schedules from DB
DNP-REACTIVATION: Marked missed notification: daily_<id>
DNP-REACTIVATION: Rescheduled alarm: daily_<id> for <next_time>
DNP-REACTIVATION: Boot recovery complete: missed>=1, rescheduled>=1, errors=0
```
**Pass criteria:**
* `errors = 0`
* `missed >= 1`
* `rescheduled >= 1`
* Script prints:
> `✅ TEST 2 PASSED: Past alarms detected and next occurrence scheduled (missed=<m>, rescheduled=<r>).`
If `missed >= 1` but `rescheduled = 0`, script warns that reschedule logic may be incomplete.
---
### 4.3 TEST 3 Boot with No Schedules
**Goal:**
Verify boot recovery handles an **empty DB / no schedules** safely and does **not** schedule anything.
**Script flow:**
1. **Uninstall app to clear DB/state**
* Script calls:
```bash
adb uninstall com.timesafari.dailynotification
```
2. **Reinstall APK**
* Script reinstalls `app-debug.apk`.
3. **Launch app WITHOUT scheduling anything**
* Script launches app; you do not configure or schedule.
4. **Collect boot/logs**
* Script reads `DNP-REACTIVATION` logs and checks:
* if there are no logs, or
* if there's a "No schedules found / present" message, or
* if `scenario=NONE` and `rescheduled=0`.
**Expected patterns:**
* *Ideal simple case:* **No** `DNP-REACTIVATION` logs at all, or:
* Explicit message in logs:
```text
DNP-REACTIVATION: ... No schedules found ...
```
**Pass criteria (as per script):**
* If **no logs**:
* Pass: `TEST 3 PASSED: No recovery logs when there are no schedules (safe behavior).`
* If logs exist:
* Contains `No schedules found` / `No schedules present` **and** `rescheduled=0`, or
* `scenario = NONE` and `rescheduled = 0`.
Any case where `rescheduled > 0` with an empty DB is flagged as a warning (boot recovery misfiring).
---
### 4.4 TEST 4 Silent Boot Recovery (App Never Opened)
**Goal:**
Verify that boot recovery **occurs silently**, recreating alarms **without opening the app** after reboot.
**Script flow:**
1. **Launch app and configure plugin**
* Same plugin status flow:
* Ensure both plugin checks are ✅.
* Schedule a future notification via UI.
2. **Verify alarms are scheduled**
* Script shows alarms and counts (`before_count`).
3. **Reboot emulator**
* Script runs `reboot_emulator` and explicitly warns:
* Do **not** open the app after reboot.
* After emulator returns, script instructs you to **not touch the app UI**.
4. **Collect boot recovery logs**
* Script gathers and parses `DNP-REACTIVATION` lines.
5. **Verify alarms were recreated without app launch**
* Script calls `show_alarms` and `count_alarms` again.
* Uses `rescheduled` + alarm count to decide.
**Pass criteria (as per script):**
* `rescheduled > 0` after boot, and
* Alarm count after boot is > 0, and
* App was **never** launched by the user after reboot.
Script prints one of:
```text
✅ TEST 4 PASSED: Boot recovery occurred silently and alarms were recreated (rescheduled=<n>) without app launch.
✅ TEST 4 PASSED: Boot recovery occurred silently (rescheduled=<n>), but alarm count check unclear.
```
If boot recovery logs are present but no alarms appear, script warns; if no boot-recovery logs are found at all, script suggests verifying the boot receiver and BOOT_COMPLETED permission.
---
## 5. Overall Summary Section (from Script)
At the end, the script prints:
```text
TEST 1: Boot with Future Alarms
- Check logs for boot recovery and rescheduled>0
TEST 2: Boot with Past Alarms
- Check logs for missed>=1 and rescheduled>=1
TEST 3: Boot with No Schedules
- Check that no recovery runs or that an explicit 'No schedules found' is logged without rescheduling
TEST 4: Silent Boot Recovery
- Check that boot recovery occurred and alarms were recreated without app launch
```
Use this as a quick checklist after a run.
---
## 6. Troubleshooting Notes
* If **no boot recovery logs** ever appear:
* Check that `BootReceiver` is declared and `RECEIVE_BOOT_COMPLETED` permission is set.
* Ensure the app is installed in internal storage (not moved to SD).
* If **errors > 0** in summary:
* Inspect the full `DNP-REACTIVATION` logs printed by the script.
* If **alarming duplication** is observed:
* Review `runBootRecovery` and dedupe logic around re-scheduling.
---
## 7. Related Documentation
- [Phase 3 Directive](../android-implementation-directive-phase3.md) - Implementation details
- [Phase 3 Verification](./PHASE3-VERIFICATION.md) - Verification report
- [Phase 1 Testing Guide](./PHASE1-EMULATOR-TESTING.md) - Prerequisite testing
- [Phase 2 Testing Guide](./PHASE2-EMULATOR-TESTING.md) - Prerequisite testing
- [Activation Guide](./ACTIVATION-GUIDE.md) - How to use directives
- [Plugin Requirements](./03-plugin-requirements.md) - Requirements Phase 3 implements
---
**Status**: Ready for testing (Phase 3 implementation pending)
**Last Updated**: November 2025