docs(test): add Phase 2 force stop recovery testing infrastructure
Adds documentation and test harness for Phase 2 (Force Stop Detection & Recovery). Changes: - Add PHASE2-EMULATOR-TESTING.md with detailed test procedures - Add PHASE2-VERIFICATION.md with test matrix and verification template - Add test-phase2.sh automated test harness Test harness features: - 3 test cases: force stop with cleared alarms, intact alarms, empty DB - Automatic force stop simulation via adb - Log parsing for scenario detection and recovery results - UI prompts for plugin configuration and scheduling Related: - Directive: android-implementation-directive-phase2.md - Requirements: docs/alarms/03-plugin-requirements.md §3.1.4 - Testing: docs/alarms/PHASE2-EMULATOR-TESTING.md - Verification: docs/alarms/PHASE2-VERIFICATION.md
This commit is contained in:
Matthew Raymer
195
docs/alarms/PHASE2-VERIFICATION.md
Normal file
195
docs/alarms/PHASE2-VERIFICATION.md
Normal file
@@ -0,0 +1,195 @@
|
||||
# Phase 2 – Force Stop Recovery Verification
|
||||
|
||||
**Plugin:** Daily Notification Plugin
|
||||
**Scope:** Force stop detection & recovery (App ID: `com.timesafari.dailynotification`)
|
||||
**Related Docs:**
|
||||
|
||||
- `android-implementation-directive-phase2.md`
|
||||
- `03-plugin-requirements.md` (Force Stop & App Termination behavior)
|
||||
- `PHASE2-EMULATOR-TESTING.md`
|
||||
- `000-UNIFIED-ALARM-DIRECTIVE.md`
|
||||
|
||||
---
|
||||
|
||||
## 1. Objective
|
||||
|
||||
Phase 2 verifies that the Daily Notification Plugin:
|
||||
|
||||
1. Correctly detects **force stop** conditions where alarms may have been cleared.
|
||||
2. **Reschedules** future notifications when schedules exist in the database but alarms are missing.
|
||||
3. **Avoids heavy recovery** when alarms are intact (no false positives).
|
||||
4. **Does not misfire** force-stop recovery on first launch / empty database.
|
||||
|
||||
Phase 2 builds on Phase 1, which already covers:
|
||||
|
||||
- Cold start detection
|
||||
- Missed notification marking
|
||||
- Basic alarm verification
|
||||
|
||||
---
|
||||
|
||||
## 2. Test Method
|
||||
|
||||
Verification is performed using the emulator test harness:
|
||||
|
||||
```bash
|
||||
cd test-apps/android-test-app
|
||||
./test-phase2.sh
|
||||
```
|
||||
|
||||
This script:
|
||||
|
||||
* Builds and installs the debug APK (`app/build/outputs/apk/debug/app-debug.apk`).
|
||||
* Guides the tester through UI steps for scheduling notifications and configuring the plugin.
|
||||
* Simulates:
|
||||
* `force-stop` behavior via `adb shell am force-stop ...`
|
||||
* "Soft stop" / process kill via `adb shell am kill ...`
|
||||
* First launch / empty DB via uninstall + reinstall
|
||||
* Collects and parses `DNP-REACTIVATION` log lines, extracting:
|
||||
* `scenario`
|
||||
* `missed`
|
||||
* `rescheduled`
|
||||
* `verified`
|
||||
* `errors`
|
||||
|
||||
Detailed steps and expectations are documented in `PHASE2-EMULATOR-TESTING.md`.
|
||||
|
||||
---
|
||||
|
||||
## 3. Test Matrix
|
||||
|
||||
| ID | Scenario | Method / Script Step | Expected Behavior | Result | Notes |
|
||||
| --- | ---------------------------------------------- | ------------------------------------------------------ | ------------------------------------------------------------------------------------------------------ | ------ | ----- |
|
||||
| 2.1 | Force stop clears alarms | `test-phase2.sh` – TEST 1: Force Stop – Alarms Cleared | `scenario=FORCE_STOP`, `rescheduled>0`, `errors=0` | ☐ | |
|
||||
| 2.2 | Force stop / process stop with alarms intact | `test-phase2.sh` – TEST 2: Soft Stop – Alarms Intact | `scenario != FORCE_STOP`, `rescheduled=0`, `errors=0` | ☐ | |
|
||||
| 2.3 | First launch / empty DB (no schedules present) | `test-phase2.sh` – TEST 3: First Launch / No Schedules | Either no recovery logs **or** `scenario=NONE` (or equivalent) and `rescheduled=0`, `errors=0` | ☐ | |
|
||||
|
||||
> Fill in **Result** and **Notes** after executing the script on your baseline emulator/device.
|
||||
|
||||
---
|
||||
|
||||
## 4. Expected Log Patterns
|
||||
|
||||
### 4.1 Force Stop – Alarms Cleared (Test 2.1)
|
||||
|
||||
Typical expected `DNP-REACTIVATION` log patterns:
|
||||
|
||||
```text
|
||||
DNP-REACTIVATION: Detected scenario: FORCE_STOP
|
||||
DNP-REACTIVATION: Rescheduled alarm: daily_<id> for <time>
|
||||
DNP-REACTIVATION: Rescheduled missing alarm: daily_<id> at <time>
|
||||
DNP-REACTIVATION: Force stop recovery completed: missed=1, rescheduled=1, verified=0, errors=0
|
||||
```
|
||||
|
||||
The **script** will report:
|
||||
|
||||
```text
|
||||
✅ TEST 1 PASSED: Force stop detected and alarms rescheduled (scenario=FORCE_STOP, rescheduled=1).
|
||||
```
|
||||
|
||||
### 4.2 Soft Stop – Alarms Intact (Test 2.2)
|
||||
|
||||
Typical expected patterns:
|
||||
|
||||
```text
|
||||
DNP-REACTIVATION: Detected scenario: COLD_START
|
||||
DNP-REACTIVATION: Cold start recovery completed: missed=0, rescheduled=0, verified>=0, errors=0
|
||||
```
|
||||
|
||||
The script should **not** treat this as a force-stop recovery:
|
||||
|
||||
```text
|
||||
✅ TEST 2 PASSED: No heavy force-stop recovery when alarms are intact (scenario=COLD_START, rescheduled=0).
|
||||
```
|
||||
|
||||
(Adjust `scenario` name to match your actual implementation.)
|
||||
|
||||
### 4.3 First Launch / Empty DB (Test 2.3)
|
||||
|
||||
Two acceptable patterns:
|
||||
|
||||
1. **No recovery logs at all** (`DNP-REACTIVATION` absent), or
|
||||
2. Explicit "no schedules" scenario, e.g.:
|
||||
|
||||
```text
|
||||
DNP-REACTIVATION: No schedules present — skipping recovery (first launch)
|
||||
```
|
||||
|
||||
or
|
||||
|
||||
```text
|
||||
DNP-REACTIVATION: Detected scenario: NONE
|
||||
```
|
||||
|
||||
Script-level success message might be:
|
||||
|
||||
```text
|
||||
✅ TEST 3 PASSED: NONE scenario detected with no rescheduling.
|
||||
```
|
||||
|
||||
or:
|
||||
|
||||
```text
|
||||
✅ TEST 3 PASSED: No recovery logs when there are no schedules (safe behavior).
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Latest Known Good Run (Emulator) – Placeholder
|
||||
|
||||
> Update this section after your first successful run.
|
||||
|
||||
**Environment**
|
||||
|
||||
* Device: Pixel 8 API 34 (Android 14)
|
||||
* App ID: `com.timesafari.dailynotification`
|
||||
* Build: Debug `app-debug.apk` from commit `<GIT_HASH>`
|
||||
* Script: `./test-phase2.sh`
|
||||
* Date: 2025-11-XX
|
||||
|
||||
**Observed Results**
|
||||
|
||||
* ☐ **2.1 – Force Stop / Alarms Cleared**
|
||||
* `scenario=FORCE_STOP`
|
||||
* `missed=1, rescheduled=1, verified=0, errors=0`
|
||||
|
||||
* ☐ **2.2 – Soft Stop / Alarms Intact**
|
||||
* `scenario=COLD_START` (or equivalent non-force-stop scenario)
|
||||
* `rescheduled=0, errors=0`
|
||||
|
||||
* ☐ **2.3 – First Launch / Empty DB**
|
||||
* `scenario=NONE` (or no logs)
|
||||
* `rescheduled=0, errors=0`
|
||||
|
||||
**Conclusion:**
|
||||
|
||||
> To be filled after first successful emulator run.
|
||||
|
||||
---
|
||||
|
||||
## 6. Overall Status
|
||||
|
||||
> To be updated once the first emulator pass is complete.
|
||||
|
||||
* **Implementation Status:** ☐ Pending / ✅ Implemented in `ReactivationManager` (Android plugin)
|
||||
* **Test Harness:** ✅ `test-phase2.sh` in `test-apps/android-test-app`
|
||||
* **Emulator Verification:** ☐ Pending / ✅ Completed (update when done)
|
||||
|
||||
Once all boxes are checked:
|
||||
|
||||
> **Overall Status:** ✅ **VERIFIED** – Phase 2 behavior is implemented, emulator-tested, and aligned with `03-plugin-requirements.md` and `android-implementation-directive-phase2.md`.
|
||||
|
||||
---
|
||||
|
||||
## 7. Related Documentation
|
||||
|
||||
- [Phase 2 Directive](../android-implementation-directive-phase2.md) - Implementation details
|
||||
- [Phase 2 Emulator Testing](./PHASE2-EMULATOR-TESTING.md) - Test procedures
|
||||
- [Phase 1 Verification](./PHASE1-VERIFICATION.md) - Prerequisite verification
|
||||
- [Plugin Requirements](./03-plugin-requirements.md) - Requirements this phase implements
|
||||
- [Platform Capability Reference](./01-platform-capability-reference.md) - OS-level facts
|
||||
|
||||
---
|
||||
|
||||
**Status**: ☐ **PENDING** – Phase 2 implementation and testing pending
|
||||
**Last Updated**: November 2025
|
||||
Reference in New Issue
Block a user