trent_larson/daily-notification-plugin
4
0
Fork 0
Code Issues Pull Requests 2 Projects Releases Wiki Activity

docs(test): add Phase 3 boot recovery testing infrastructure

Browse Source 
Adds documentation and test harness for Phase 3 (Boot-Time Recovery).

Changes:
- Update android-implementation-directive-phase3.md with concise boot recovery flow
- Add PHASE3-EMULATOR-TESTING.md with detailed test procedures
- Add PHASE3-VERIFICATION.md with test matrix and verification template
- Add test-phase3.sh automated test harness

Test harness features:
- 4 test cases: future alarms, past alarms, no schedules, silent recovery
- Automatic emulator reboot handling
- Log parsing for boot recovery scenario and results
- UI prompts for plugin configuration and scheduling
- Verifies silent recovery without app launch

Related:
- Directive: android-implementation-directive-phase3.md
- Requirements: docs/alarms/03-plugin-requirements.md §3.1.1
- Testing: docs/alarms/PHASE3-EMULATOR-TESTING.md
- Verification: docs/alarms/PHASE3-VERIFICATION.md
This commit is contained in:
Matthew Raymer
2025-11-27 10:01:46 +00:00
parent c8a3906449
commit 28fb233286
4 changed files with 1258 additions and 560 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/alarms/PHASE3-EMULATOR-TESTING.md
+325
View File
@@ -0,0 +1,325 @@
# PHASE 3 – EMULATOR TESTING
**Boot-Time Recovery (Device Reboot / System Restart)**
---
## 1. Purpose
Phase 3 verifies that the Daily Notification Plugin correctly:
1. Reconstructs **AlarmManager** alarms after a full device/emulator reboot.
2. Handles **past** scheduled times by marking them as missed and scheduling the next occurrence.
3. Handles **empty DB / no schedules** without misfiring recovery.
4. Performs **silent boot recovery** (recreate alarms) even when the app is never opened after reboot.
This testing is driven by the script:
```bash
test-apps/android-test-app/test-phase3.sh
```
---
## 2. Prerequisites
* Android emulator or device, e.g.:
* Pixel 8 / API 34 (recommended baseline)
* ADB available in `PATH` (or `ADB_BIN` exported)
* Project with:
* Daily Notification Plugin integrated
* Test app at `test-apps/android-test-app`
* Debug APK path:
* `app/build/outputs/apk/debug/app-debug.apk`
* Phase 1 and Phase 2 behaviors already implemented:
* Cold start detection
* Force-stop detection
* Missed / rescheduled / verified / errors summary fields
> ⚠️ **Important:**
> This script will reboot the emulator multiple times. Each reboot may take 30–60 seconds.
---
## 3. How to Run
From the `android-test-app` directory:
```bash
cd test-apps/android-test-app
chmod +x test-phase3.sh # first time only
./test-phase3.sh
```
The script will:
1. Run pre-flight checks (ADB / emulator readiness).
2. Build and install the debug APK.
3. Guide you through four tests:
* **TEST 1:** Boot with Future Alarms
* **TEST 2:** Boot with Past Alarms
* **TEST 3:** Boot with No Schedules
* **TEST 4:** Silent Boot Recovery (App Never Opened)
4. Parse and display `DNP-REACTIVATION` logs, including:
* `scenario`
* `missed`
* `rescheduled`
* `verified`
* `errors`
---
## 4. Test Cases (Script-Driven Flow)
### 4.1 TEST 1 – Boot with Future Alarms
**Goal:**
Verify alarms are recreated on boot when schedules have **future run times**.
**Script flow:**
1. **Launch app & check plugin status**
* Script calls `launch_app`.
* UI prompt: Confirm plugin status shows:
* `⚙️ Plugin Settings: ✅ Configured`
* `🔌 Native Fetcher: ✅ Configured`
* If not, click **Configure Plugin**, wait until both show ✅, then continue.
2. **Schedule at least one future notification**
* UI prompt: Click e.g. **Test Notification** to schedule a notification a few minutes in the future.
3. **Verify alarms are scheduled (pre-boot)**
* Script calls `show_alarms` and `count_alarms`.
* You should see at least one `RTC_WAKEUP` entry for `com.timesafari.dailynotification`.
4. **Reboot emulator**
* Script calls `reboot_emulator`:
* `adb reboot`
* `adb wait-for-device`
* Polls `getprop sys.boot_completed` until `1`.
* You are warned that reboot will take 30–60 seconds.
5. **Collect boot recovery logs**
* Script calls `get_recovery_logs`:
```bash
adb logcat -d | grep "DNP-REACTIVATION"
```
* It parses:
* `missed`, `rescheduled`, `verified`, `errors`
* `scenario` via:
* `Starting boot recovery`/`boot recovery` → `scenario=BOOT`
* or `Detected scenario: <VALUE>`
6. **Verify alarms were recreated (post-boot)**
* Script calls `show_alarms` and `count_alarms` again.
* Checks `scenario` and `rescheduled`.
**Expected log patterns:**
```text
DNP-REACTIVATION: Starting boot recovery
DNP-REACTIVATION: Loaded <N> schedules from DB
DNP-REACTIVATION: Rescheduled alarm: daily_<id> for <time>
DNP-REACTIVATION: Boot recovery complete: missed=0, rescheduled>=1, verified=0, errors=0
```
**Pass criteria (as per script):**
* `errors = 0`
* `scenario = BOOT` (or boot detected via log text)
* `rescheduled > 0`
* Script prints:
> `✅ TEST 1 PASSED: Boot recovery detected and alarms rescheduled (scenario=BOOT, rescheduled=<n>).`
If boot recovery runs but `rescheduled=0`, script warns and suggests checking boot logic.
---
### 4.2 TEST 2 – Boot with Past Alarms
**Goal:**
Verify past alarms are marked as missed and **next occurrences are scheduled** after boot.
**Script flow:**
1. **Launch app & ensure plugin configured**
* Same plugin status check as TEST 1.
2. **Schedule a notification in the near future**
* UI prompt: Schedule such that **by the time you reboot and the device comes back, the planned notification time is in the past**.
3. **Wait or adjust so the alarm is effectively "in the past" at boot**
* The script may instruct you to wait, or you can coordinate timing manually.
4. **Reboot emulator**
* Same `reboot_emulator` path as TEST 1.
5. **Collect boot recovery logs**
* Script parses:
* `missed`, `rescheduled`, `errors`, `scenario`.
**Expected log patterns:**
```text
DNP-REACTIVATION: Starting boot recovery
DNP-REACTIVATION: Loaded <N> schedules from DB
DNP-REACTIVATION: Marked missed notification: daily_<id>
DNP-REACTIVATION: Rescheduled alarm: daily_<id> for <next_time>
DNP-REACTIVATION: Boot recovery complete: missed>=1, rescheduled>=1, errors=0
```
**Pass criteria:**
* `errors = 0`
* `missed >= 1`
* `rescheduled >= 1`
* Script prints:
> `✅ TEST 2 PASSED: Past alarms detected and next occurrence scheduled (missed=<m>, rescheduled=<r>).`
If `missed >= 1` but `rescheduled = 0`, script warns that reschedule logic may be incomplete.
---
### 4.3 TEST 3 – Boot with No Schedules
**Goal:**
Verify boot recovery handles an **empty DB / no schedules** safely and does **not** schedule anything.
**Script flow:**
1. **Uninstall app to clear DB/state**
* Script calls:
```bash
adb uninstall com.timesafari.dailynotification
```
2. **Reinstall APK**
* Script reinstalls `app-debug.apk`.
3. **Launch app WITHOUT scheduling anything**
* Script launches app; you do not configure or schedule.
4. **Collect boot/logs**
* Script reads `DNP-REACTIVATION` logs and checks:
* if there are no logs, or
* if there's a "No schedules found / present" message, or
* if `scenario=NONE` and `rescheduled=0`.
**Expected patterns:**
* *Ideal simple case:* **No** `DNP-REACTIVATION` logs at all, or:
* Explicit message in logs:
```text
DNP-REACTIVATION: ... No schedules found ...
```
**Pass criteria (as per script):**
* If **no logs**:
* Pass: `TEST 3 PASSED: No recovery logs when there are no schedules (safe behavior).`
* If logs exist:
* Contains `No schedules found` / `No schedules present` **and** `rescheduled=0`, or
* `scenario = NONE` and `rescheduled = 0`.
Any case where `rescheduled > 0` with an empty DB is flagged as a warning (boot recovery misfiring).
---
### 4.4 TEST 4 – Silent Boot Recovery (App Never Opened)
**Goal:**
Verify that boot recovery **occurs silently**, recreating alarms **without opening the app** after reboot.
**Script flow:**
1. **Launch app and configure plugin**
* Same plugin status flow:
* Ensure both plugin checks are ✅.
* Schedule a future notification via UI.
2. **Verify alarms are scheduled**
* Script shows alarms and counts (`before_count`).
3. **Reboot emulator**
* Script runs `reboot_emulator` and explicitly warns:
* Do **not** open the app after reboot.
* After emulator returns, script instructs you to **not touch the app UI**.
4. **Collect boot recovery logs**
* Script gathers and parses `DNP-REACTIVATION` lines.
5. **Verify alarms were recreated without app launch**
* Script calls `show_alarms` and `count_alarms` again.
* Uses `rescheduled` + alarm count to decide.
**Pass criteria (as per script):**
* `rescheduled > 0` after boot, and
* Alarm count after boot is > 0, and
* App was **never** launched by the user after reboot.
Script prints one of:
```text
✅ TEST 4 PASSED: Boot recovery occurred silently and alarms were recreated (rescheduled=<n>) without app launch.
✅ TEST 4 PASSED: Boot recovery occurred silently (rescheduled=<n>), but alarm count check unclear.
```
If boot recovery logs are present but no alarms appear, script warns; if no boot-recovery logs are found at all, script suggests verifying the boot receiver and BOOT_COMPLETED permission.
---
## 5. Overall Summary Section (from Script)
At the end, the script prints:
```text
TEST 1: Boot with Future Alarms
- Check logs for boot recovery and rescheduled>0
TEST 2: Boot with Past Alarms
- Check logs for missed>=1 and rescheduled>=1
TEST 3: Boot with No Schedules
- Check that no recovery runs or that an explicit 'No schedules found' is logged without rescheduling
TEST 4: Silent Boot Recovery
- Check that boot recovery occurred and alarms were recreated without app launch
```
Use this as a quick checklist after a run.
---
## 6. Troubleshooting Notes
* If **no boot recovery logs** ever appear:
* Check that `BootReceiver` is declared and `RECEIVE_BOOT_COMPLETED` permission is set.
* Ensure the app is installed in internal storage (not moved to SD).
* If **errors > 0** in summary:
* Inspect the full `DNP-REACTIVATION` logs printed by the script.
* If **alarming duplication** is observed:
* Review `runBootRecovery` and dedupe logic around re-scheduling.
---
## 7. Related Documentation
- [Phase 3 Directive](../android-implementation-directive-phase3.md) - Implementation details
- [Phase 3 Verification](./PHASE3-VERIFICATION.md) - Verification report
- [Phase 1 Testing Guide](./PHASE1-EMULATOR-TESTING.md) - Prerequisite testing
- [Phase 2 Testing Guide](./PHASE2-EMULATOR-TESTING.md) - Prerequisite testing
- [Activation Guide](./ACTIVATION-GUIDE.md) - How to use directives
- [Plugin Requirements](./03-plugin-requirements.md) - Requirements Phase 3 implements
---
**Status**: Ready for testing (Phase 3 implementation pending)
**Last Updated**: November 2025
docs/alarms/PHASE3-VERIFICATION.md
+201
View File
@@ -0,0 +1,201 @@
# Phase 3 – Boot-Time Recovery Verification
**Plugin:** Daily Notification Plugin
**Scope:** Boot-Time Recovery (Recreate Alarms After Reboot)
**Related Docs:**
- `android-implementation-directive-phase3.md`
- `PHASE3-EMULATOR-TESTING.md`
- `test-phase3.sh`
- `000-UNIFIED-ALARM-DIRECTIVE.md`
---
## 1. Objective
Phase 3 confirms that the Daily Notification Plugin:
1. Reconstructs all daily notification alarms after a **full device reboot**.
2. Correctly handles **past** vs **future** schedules:
- Past: mark as missed, schedule next occurrence
- Future: simply recreate alarms
3. Handles **empty DB / no schedules** without misfiring recovery.
4. Performs **silent boot recovery** (no app launch required) when schedules exist.
5. Logs a consistent, machine-readable summary:
- `scenario`
- `missed`
- `rescheduled`
- `verified`
- `errors`
Verification is performed via the emulator harness:
```bash
cd test-apps/android-test-app
./test-phase3.sh
```
---
## 2. Test Matrix (From Script)
| ID | Scenario | Script Test | Expected Behavior | Result | Notes |
| --- | --------------------------------------- | ------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------ | ------ | ----- |
| 3.1 | Boot with Future Alarms | TEST 1 – Boot with Future Alarms | `scenario=BOOT`, `rescheduled>0`, `errors=0`; alarms present after boot | ☐ | |
| 3.2 | Boot with Past Alarms | TEST 2 – Boot with Past Alarms | `missed>=1` and `rescheduled>=1`, `errors=0`; past schedules detected and next occurrences scheduled | ☐ | |
| 3.3 | Boot with No Schedules (Empty DB) | TEST 3 – Boot with No Schedules | Either no recovery logs **or** explicit "No schedules found/present" or `scenario=NONE` with `rescheduled=0`, `errors=0` | ☐ | |
| 3.4 | Silent Boot Recovery (App Never Opened) | TEST 4 – Silent Boot Recovery (App Never Opened) | `rescheduled>0`, alarms present after boot, and no user launch required; `errors=0` | ☐ | |
Fill **Result** and **Notes** after running `test-phase3.sh` on your baseline emulator/device.
---
## 3. Expected Log Patterns
The script filters logs with:
```bash
adb logcat -d | grep "DNP-REACTIVATION"
```
### 3.1 Boot with Future Alarms (3.1 / TEST 1)
* Typical logs:
```text
DNP-REACTIVATION: Starting boot recovery
DNP-REACTIVATION: Loaded <N> schedules from DB
DNP-REACTIVATION: Rescheduled alarm: daily_<id> for <time>
DNP-REACTIVATION: Boot recovery complete: missed=0, rescheduled=<r>, verified=0, errors=0
```
* The script interprets this as:
* `scenario = BOOT` (via "Starting boot recovery" or "boot recovery" text or `Detected scenario: BOOT`)
* `rescheduled > 0`
* `errors = 0`
### 3.2 Boot with Past Alarms (3.2 / TEST 2)
* Typical logs:
```text
DNP-REACTIVATION: Starting boot recovery
DNP-REACTIVATION: Loaded <N> schedules from DB
DNP-REACTIVATION: Marked missed notification: daily_<id>
DNP-REACTIVATION: Rescheduled alarm: daily_<id> for <next_time>
DNP-REACTIVATION: Boot recovery complete: missed=<m>, rescheduled=<r>, verified=0, errors=0
```
* The script parses `missed` and `rescheduled` and passes when:
* `missed >= 1`
* `rescheduled >= 1`
* `errors = 0`
### 3.3 Boot with No Schedules (3.3 / TEST 3)
Two acceptable patterns:
1. **No `DNP-REACTIVATION` logs at all** → safe behavior
2. Explicit "no schedules" logs:
```text
DNP-REACTIVATION: ... No schedules found ...
```
or a neutral scenario:
```text
DNP-REACTIVATION: ... scenario=NONE ...
DNP-REACTIVATION: Boot recovery complete: missed=0, rescheduled=0, verified=0, errors=0
```
The script passes when:
* Either `logs` are empty, or
* Logs contain "No schedules found / present" with `rescheduled=0`, or
* `scenario=NONE` and `rescheduled=0`.
Any `rescheduled>0` in this state is flagged as a potential boot-recovery misfire.
### 3.4 Silent Boot Recovery (3.4 / TEST 4)
* Expected:
```text
DNP-REACTIVATION: Starting boot recovery
DNP-REACTIVATION: Loaded <N> schedules from DB
DNP-REACTIVATION: Rescheduled alarm: daily_<id> for <time>
DNP-REACTIVATION: Boot recovery complete: missed=0, rescheduled=<r>, verified=0, errors=0
```
* After reboot:
* `count_alarms` > 0
* User **did not** relaunch the app manually
Script passes if:
* `rescheduled>0`, and
* Alarm count after boot is > 0, and
* Boot recovery is detected from logs (via "Starting boot recovery"/"boot recovery" or scenario).
---
## 4. Latest Known Good Run (Template)
> Fill this in after your first clean emulator run.
**Environment**
* Device: Pixel 8 API 34 (Android 14)
* App ID: `com.timesafari.dailynotification`
* Build: Debug `app-debug.apk` from commit `<GIT_HASH>`
* Script: `./test-phase3.sh`
* Date: 2025-11-XX
**Observed Results**
* ☐ **3.1 – Boot with Future Alarms**
* `scenario=BOOT`
* `missed=0, rescheduled=<r>, errors=0`
* ☐ **3.2 – Boot with Past Alarms**
* `missed=<m>=1`, `rescheduled=<r>≥1`, `errors=0`
* ☐ **3.3 – Boot with No Schedules**
* Either no logs, or explicit "No schedules found" with `rescheduled=0`
* ☐ **3.4 – Silent Boot Recovery**
* `rescheduled>0`, alarms present after boot, app not opened
**Conclusion:**
> Phase 3 **Boot-Time Recovery** is successfully verified on emulator using `test-phase3.sh`. This is the canonical baseline for future regression testing and refactors to `ReactivationManager` and `BootReceiver`.
---
## 5. Overall Status
> Update once the first emulator run is complete.
* **Implementation Status:** ☐ Pending / ✅ Implemented (Boot receiver + `runBootRecovery`)
* **Test Harness:** ✅ `test-phase3.sh` in `test-apps/android-test-app`
* **Emulator Verification:** ☐ Pending / ✅ Completed
Once all test cases pass:
> **Overall Status:** ✅ **VERIFIED** – Phase 3 boot-time recovery is implemented and emulator-tested, aligned with `android-implementation-directive-phase3.md` and the unified alarm directive.
---
## 6. Related Documentation
- [Phase 3 Directive](../android-implementation-directive-phase3.md) - Implementation details
- [Phase 3 Emulator Testing](./PHASE3-EMULATOR-TESTING.md) - Test procedures
- [Phase 1 Verification](./PHASE1-VERIFICATION.md) - Prerequisite verification
- [Phase 2 Verification](./PHASE2-VERIFICATION.md) - Prerequisite verification
- [Plugin Requirements](./03-plugin-requirements.md) - Requirements this phase implements
- [Platform Capability Reference](./01-platform-capability-reference.md) - OS-level facts
---
**Status**: ☐ **PENDING** – Phase 3 implementation and testing pending
**Last Updated**: November 2025