trent_larson/daily-notification-plugin
4
0
Fork 0
Code Issues Pull Requests 2 Projects Releases Wiki Activity

docs(test): add Phase 3 boot recovery testing infrastructure

Browse Source 
Adds documentation and test harness for Phase 3 (Boot-Time Recovery).

Changes:
- Update android-implementation-directive-phase3.md with concise boot recovery flow
- Add PHASE3-EMULATOR-TESTING.md with detailed test procedures
- Add PHASE3-VERIFICATION.md with test matrix and verification template
- Add test-phase3.sh automated test harness

Test harness features:
- 4 test cases: future alarms, past alarms, no schedules, silent recovery
- Automatic emulator reboot handling
- Log parsing for boot recovery scenario and results
- UI prompts for plugin configuration and scheduling
- Verifies silent recovery without app launch

Related:
- Directive: android-implementation-directive-phase3.md
- Requirements: docs/alarms/03-plugin-requirements.md §3.1.1
- Testing: docs/alarms/PHASE3-EMULATOR-TESTING.md
- Verification: docs/alarms/PHASE3-VERIFICATION.md
This commit is contained in:
Matthew Raymer
2025-11-27 10:01:46 +00:00
parent c8a3906449
commit 28fb233286
4 changed files with 1258 additions and 560 deletions
Download Patch File Download Diff File Expand all files Collapse all files

325
docs/alarms/PHASE3-EMULATOR-TESTING.md Normal file
View File

@@ -0,0 +1,325 @@
# PHASE 3 – EMULATOR TESTING
**Boot-Time Recovery (Device Reboot / System Restart)**
---
## 1. Purpose
Phase 3 verifies that the Daily Notification Plugin correctly:
1. Reconstructs **AlarmManager** alarms after a full device/emulator reboot.
2. Handles **past** scheduled times by marking them as missed and scheduling the next occurrence.
3. Handles **empty DB / no schedules** without misfiring recovery.
4. Performs **silent boot recovery** (recreate alarms) even when the app is never opened after reboot.
This testing is driven by the script:
```bash
test-apps/android-test-app/test-phase3.sh
```
---
## 2. Prerequisites
* Android emulator or device, e.g.:
* Pixel 8 / API 34 (recommended baseline)
* ADB available in `PATH` (or `ADB_BIN` exported)
* Project with:
* Daily Notification Plugin integrated
* Test app at `test-apps/android-test-app`
* Debug APK path:
* `app/build/outputs/apk/debug/app-debug.apk`
* Phase 1 and Phase 2 behaviors already implemented:
* Cold start detection
* Force-stop detection
* Missed / rescheduled / verified / errors summary fields
> ⚠️ **Important:**
> This script will reboot the emulator multiple times. Each reboot may take 30–60 seconds.
---
## 3. How to Run
From the `android-test-app` directory:
```bash
cd test-apps/android-test-app
chmod +x test-phase3.sh # first time only
./test-phase3.sh
```
The script will:
1. Run pre-flight checks (ADB / emulator readiness).
2. Build and install the debug APK.
3. Guide you through four tests:
* **TEST 1:** Boot with Future Alarms
* **TEST 2:** Boot with Past Alarms
* **TEST 3:** Boot with No Schedules
* **TEST 4:** Silent Boot Recovery (App Never Opened)
4. Parse and display `DNP-REACTIVATION` logs, including:
* `scenario`
* `missed`
* `rescheduled`
* `verified`
* `errors`
---
## 4. Test Cases (Script-Driven Flow)
### 4.1 TEST 1 – Boot with Future Alarms
**Goal:**
Verify alarms are recreated on boot when schedules have **future run times**.
**Script flow:**
1. **Launch app & check plugin status**
* Script calls `launch_app`.
* UI prompt: Confirm plugin status shows:
* `⚙️ Plugin Settings: ✅ Configured`
* `🔌 Native Fetcher: ✅ Configured`
* If not, click **Configure Plugin**, wait until both show ✅, then continue.
2. **Schedule at least one future notification**
* UI prompt: Click e.g. **Test Notification** to schedule a notification a few minutes in the future.
3. **Verify alarms are scheduled (pre-boot)**
* Script calls `show_alarms` and `count_alarms`.
* You should see at least one `RTC_WAKEUP` entry for `com.timesafari.dailynotification`.
4. **Reboot emulator**
* Script calls `reboot_emulator`:
* `adb reboot`
* `adb wait-for-device`
* Polls `getprop sys.boot_completed` until `1`.
* You are warned that reboot will take 30–60 seconds.
5. **Collect boot recovery logs**
* Script calls `get_recovery_logs`:
```bash
adb logcat -d | grep "DNP-REACTIVATION"
```
* It parses:
* `missed`, `rescheduled`, `verified`, `errors`
* `scenario` via:
* `Starting boot recovery`/`boot recovery` → `scenario=BOOT`
* or `Detected scenario: <VALUE>`
6. **Verify alarms were recreated (post-boot)**
* Script calls `show_alarms` and `count_alarms` again.
* Checks `scenario` and `rescheduled`.
**Expected log patterns:**
```text
DNP-REACTIVATION: Starting boot recovery
DNP-REACTIVATION: Loaded <N> schedules from DB
DNP-REACTIVATION: Rescheduled alarm: daily_<id> for <time>
DNP-REACTIVATION: Boot recovery complete: missed=0, rescheduled>=1, verified=0, errors=0
```
**Pass criteria (as per script):**
* `errors = 0`
* `scenario = BOOT` (or boot detected via log text)
* `rescheduled > 0`
* Script prints:
> `✅ TEST 1 PASSED: Boot recovery detected and alarms rescheduled (scenario=BOOT, rescheduled=<n>).`
If boot recovery runs but `rescheduled=0`, script warns and suggests checking boot logic.
---
### 4.2 TEST 2 – Boot with Past Alarms
**Goal:**
Verify past alarms are marked as missed and **next occurrences are scheduled** after boot.
**Script flow:**
1. **Launch app & ensure plugin configured**
* Same plugin status check as TEST 1.
2. **Schedule a notification in the near future**
* UI prompt: Schedule such that **by the time you reboot and the device comes back, the planned notification time is in the past**.
3. **Wait or adjust so the alarm is effectively "in the past" at boot**
* The script may instruct you to wait, or you can coordinate timing manually.
4. **Reboot emulator**
* Same `reboot_emulator` path as TEST 1.
5. **Collect boot recovery logs**
* Script parses:
* `missed`, `rescheduled`, `errors`, `scenario`.
**Expected log patterns:**
```text
DNP-REACTIVATION: Starting boot recovery
DNP-REACTIVATION: Loaded <N> schedules from DB
DNP-REACTIVATION: Marked missed notification: daily_<id>
DNP-REACTIVATION: Rescheduled alarm: daily_<id> for <next_time>
DNP-REACTIVATION: Boot recovery complete: missed>=1, rescheduled>=1, errors=0
```
**Pass criteria:**
* `errors = 0`
* `missed >= 1`
* `rescheduled >= 1`
* Script prints:
> `✅ TEST 2 PASSED: Past alarms detected and next occurrence scheduled (missed=<m>, rescheduled=<r>).`
If `missed >= 1` but `rescheduled = 0`, script warns that reschedule logic may be incomplete.
---
### 4.3 TEST 3 – Boot with No Schedules
**Goal:**
Verify boot recovery handles an **empty DB / no schedules** safely and does **not** schedule anything.
**Script flow:**
1. **Uninstall app to clear DB/state**
* Script calls:
```bash
adb uninstall com.timesafari.dailynotification
```
2. **Reinstall APK**
* Script reinstalls `app-debug.apk`.
3. **Launch app WITHOUT scheduling anything**
* Script launches app; you do not configure or schedule.
4. **Collect boot/logs**
* Script reads `DNP-REACTIVATION` logs and checks:
* if there are no logs, or
* if there's a "No schedules found / present" message, or
* if `scenario=NONE` and `rescheduled=0`.
**Expected patterns:**
* *Ideal simple case:* **No** `DNP-REACTIVATION` logs at all, or:
* Explicit message in logs:
```text
DNP-REACTIVATION: ... No schedules found ...
```
**Pass criteria (as per script):**
* If **no logs**:
* Pass: `TEST 3 PASSED: No recovery logs when there are no schedules (safe behavior).`
* If logs exist:
* Contains `No schedules found` / `No schedules present` **and** `rescheduled=0`, or
* `scenario = NONE` and `rescheduled = 0`.
Any case where `rescheduled > 0` with an empty DB is flagged as a warning (boot recovery misfiring).
---
### 4.4 TEST 4 – Silent Boot Recovery (App Never Opened)
**Goal:**
Verify that boot recovery **occurs silently**, recreating alarms **without opening the app** after reboot.
**Script flow:**
1. **Launch app and configure plugin**
* Same plugin status flow:
* Ensure both plugin checks are ✅.
* Schedule a future notification via UI.
2. **Verify alarms are scheduled**
* Script shows alarms and counts (`before_count`).
3. **Reboot emulator**
* Script runs `reboot_emulator` and explicitly warns:
* Do **not** open the app after reboot.
* After emulator returns, script instructs you to **not touch the app UI**.
4. **Collect boot recovery logs**
* Script gathers and parses `DNP-REACTIVATION` lines.
5. **Verify alarms were recreated without app launch**
* Script calls `show_alarms` and `count_alarms` again.
* Uses `rescheduled` + alarm count to decide.
**Pass criteria (as per script):**
* `rescheduled > 0` after boot, and
* Alarm count after boot is > 0, and
* App was **never** launched by the user after reboot.
Script prints one of:
```text
✅ TEST 4 PASSED: Boot recovery occurred silently and alarms were recreated (rescheduled=<n>) without app launch.
✅ TEST 4 PASSED: Boot recovery occurred silently (rescheduled=<n>), but alarm count check unclear.
```
If boot recovery logs are present but no alarms appear, script warns; if no boot-recovery logs are found at all, script suggests verifying the boot receiver and BOOT_COMPLETED permission.
---
## 5. Overall Summary Section (from Script)
At the end, the script prints:
```text
TEST 1: Boot with Future Alarms
- Check logs for boot recovery and rescheduled>0
TEST 2: Boot with Past Alarms
- Check logs for missed>=1 and rescheduled>=1
TEST 3: Boot with No Schedules
- Check that no recovery runs or that an explicit 'No schedules found' is logged without rescheduling
TEST 4: Silent Boot Recovery
- Check that boot recovery occurred and alarms were recreated without app launch
```
Use this as a quick checklist after a run.
---
## 6. Troubleshooting Notes
* If **no boot recovery logs** ever appear:
* Check that `BootReceiver` is declared and `RECEIVE_BOOT_COMPLETED` permission is set.
* Ensure the app is installed in internal storage (not moved to SD).
* If **errors > 0** in summary:
* Inspect the full `DNP-REACTIVATION` logs printed by the script.
* If **alarming duplication** is observed:
* Review `runBootRecovery` and dedupe logic around re-scheduling.
---
## 7. Related Documentation
- [Phase 3 Directive](../android-implementation-directive-phase3.md) - Implementation details
- [Phase 3 Verification](./PHASE3-VERIFICATION.md) - Verification report
- [Phase 1 Testing Guide](./PHASE1-EMULATOR-TESTING.md) - Prerequisite testing
- [Phase 2 Testing Guide](./PHASE2-EMULATOR-TESTING.md) - Prerequisite testing
- [Activation Guide](./ACTIVATION-GUIDE.md) - How to use directives
- [Plugin Requirements](./03-plugin-requirements.md) - Requirements Phase 3 implements
---
**Status**: Ready for testing (Phase 3 implementation pending)
**Last Updated**: November 2025