trent_larson/daily-notification-plugin
4
0
Fork 0
Code Issues Pull Requests 2 Projects Releases Wiki Activity

docs(test): add Phase 2 force stop recovery testing infrastructure

Browse Source 
Adds documentation and test harness for Phase 2 (Force Stop Detection & Recovery).

Changes:
- Add PHASE2-EMULATOR-TESTING.md with detailed test procedures
- Add PHASE2-VERIFICATION.md with test matrix and verification template
- Add test-phase2.sh automated test harness

Test harness features:
- 3 test cases: force stop with cleared alarms, intact alarms, empty DB
- Automatic force stop simulation via adb
- Log parsing for scenario detection and recovery results
- UI prompts for plugin configuration and scheduling

Related:
- Directive: android-implementation-directive-phase2.md
- Requirements: docs/alarms/03-plugin-requirements.md §3.1.4
- Testing: docs/alarms/PHASE2-EMULATOR-TESTING.md
- Verification: docs/alarms/PHASE2-VERIFICATION.md
This commit is contained in:
Matthew Raymer
2025-11-27 10:01:40 +00:00
parent 3151a1cc31
commit c8a3906449
3 changed files with 1083 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

358
docs/alarms/PHASE2-EMULATOR-TESTING.md Normal file
View File

@@ -0,0 +1,358 @@
# PHASE 2 – EMULATOR TESTING
**Force Stop Detection & Recovery**
---
## 1. Purpose
Phase 2 verifies that the Daily Notification Plugin correctly:
1. Detects **force stop** scenarios (where alarms may be cleared by the OS).
2. **Reschedules** future notifications when alarms are missing but schedules remain in the database.
3. **Avoids heavy recovery** when alarms are still intact.
4. **Does not misfire** force-stop recovery on first launch / empty database.
This document defines the emulator test procedure for Phase 2 using the script:
```bash
test-apps/android-test-app/test-phase2.sh
```
---
## 2. Prerequisites
* Android emulator or device, e.g.:
* Pixel 8 / API 34 (recommended baseline)
* ADB available in `PATH` (or `ADB_BIN` exported)
* Project built with:
* Daily Notification Plugin integrated
* Test app at: `test-apps/android-test-app`
* Debug APK path:
* `app/build/outputs/apk/debug/app-debug.apk`
* Phase 1 behavior already implemented and verified:
* Cold start detection
* Missed notification marking
> **Note:** Some OS/device combinations do not clear alarms on `am force-stop`. In such cases, TEST 1 may partially skip or only verify scenario logging.
---
## 3. How to Run
From the `android-test-app` directory:
```bash
cd test-apps/android-test-app
chmod +x test-phase2.sh # first time only
./test-phase2.sh
```
The script will:
1. Perform pre-flight checks (ADB/emulator).
2. Build and install the debug APK.
3. Guide you through three tests:
* TEST 1: Force stop – alarms cleared
* TEST 2: Force stop / process stop – alarms intact
* TEST 3: First launch / empty DB safeguard
4. Print parsed recovery summaries from `DNP-REACTIVATION` logs.
You will be prompted for **UI actions** at each step (e.g., configuring the plugin, pressing "Test Notification").
---
## 4. Test Cases
### 4.1 TEST 1 – Force Stop with Cleared Alarms
**Goal:**
Verify that when a force stop clears alarms, the plugin:
* Detects the **FORCE_STOP** scenario.
* Reschedules future notifications.
* Completes recovery with `errors=0`.
**Steps (Script Flow):**
1. **Launch & configure plugin**
* Script launches the app.
* In the app UI, confirm:
* `⚙️ Plugin Settings: ✅ Configured`
* `🔌 Native Fetcher: ✅ Configured`
* If not, press **Configure Plugin** and wait until both are ✅.
2. **Schedule a future notification**
* Click **Test Notification** (or equivalent) to schedule a notification a few minutes in the future.
3. **Verify alarms are scheduled**
* Script runs:
```bash
adb shell dumpsys alarm | grep com.timesafari.dailynotification
```
* Confirm at least one `RTC_WAKEUP` alarm for `com.timesafari.dailynotification`.
4. **Force stop the app**
* Script executes:
```bash
adb shell am force-stop com.timesafari.dailynotification
```
5. **Confirm alarms after force stop**
* Script re-runs `dumpsys alarm`.
* Ideal test case: **0** alarms for `com.timesafari.dailynotification` (alarms cleared).
6. **Trigger recovery**
* Script clears logcat and launches the app.
* Wait ~5 seconds for recovery.
7. **Collect and inspect logs**
* Script collects `DNP-REACTIVATION` logs and parses:
* `scenario=<value>`
* `missed=<n>`
* `rescheduled=<n>`
* `verified=<n>`
* `errors=<n>`
**Expected Logs (Ideal Case):**
* Scenario:
```text
DNP-REACTIVATION: Detected scenario: FORCE_STOP
```
* Alarm handling:
```text
DNP-REACTIVATION: Rescheduled alarm: daily_<id> for <time>
DNP-REACTIVATION: Rescheduled missing alarm: daily_<id> at <time>
```
* Summary:
```text
DNP-REACTIVATION: Force stop recovery completed: missed=1, rescheduled=1, verified=0, errors=0
```
**Pass Criteria:**
* `scenario=FORCE_STOP`
* `rescheduled > 0`
* `errors = 0`
* Script prints:
> `✅ TEST 1 PASSED: Force stop detected and alarms rescheduled (scenario=FORCE_STOP, rescheduled=1).`
**Partial / Edge Cases:**
* If alarms remain after `force-stop` on this device:
* Script warns that FORCE_STOP may not fully trigger.
* Mark this as **environment-limited** rather than a plugin failure.
---
### 4.2 TEST 2 – Force Stop / Process Stop with Intact Alarms
**Goal:**
Ensure we **do not run heavy force-stop recovery** when alarms are still intact.
**Steps (Script Flow):**
1. **Launch & configure plugin**
* Same as TEST 1 (ensure both plugin statuses are ✅).
2. **Schedule another future notification**
* Click **Test Notification** again to create a second schedule.
3. **Verify alarms are scheduled**
* Confirm multiple alarms for `com.timesafari.dailynotification` via `dumpsys alarm`.
4. **Simulate a "soft stop"**
* Script runs:
```bash
adb shell am kill com.timesafari.dailynotification
```
* Intent: stop the process but **not** clear alarms (actual behavior may vary by OS).
5. **Check alarms after soft stop**
* Ensure alarms are still present in `dumpsys alarm`.
6. **Trigger recovery**
* Script clears logcat and relaunches the app.
* Wait ~5 seconds.
7. **Collect logs**
* Script parses `DNP-REACTIVATION` logs for:
* `scenario`
* `rescheduled`
* `errors`
**Expected Behavior:**
* `scenario` should **not** be `FORCE_STOP` (e.g., `COLD_START` or another non-force-stop scenario, depending on your implementation).
* `rescheduled = 0`.
* `errors = 0`.
**Pass Criteria:**
* Alarms still present after soft stop.
* Recovery summary indicates **no force-stop-level rescheduling** when alarms are intact:
* `scenario != FORCE_STOP`
* `rescheduled = 0`
* Script prints:
> `✅ TEST 2 PASSED: No heavy force-stop recovery when alarms intact (scenario=<value>, rescheduled=0).`
If `scenario=FORCE_STOP` and `rescheduled>0` here, treat this as a **warning** and review scenario detection logic.
---
### 4.3 TEST 3 – First Launch / Empty DB Safeguard
**Goal:**
Ensure **force-stop recovery is not mis-triggered** when the app is freshly installed with **no schedules**.
**Steps (Script Flow):**
1. **Clear state**
* Script uninstalls the app to clear DB/state:
```bash
adb uninstall com.timesafari.dailynotification
```
2. **Reinstall APK**
* Script reinstalls `app-debug.apk`.
3. **Launch app without scheduling anything**
* Script launches the app.
* Do **not** schedule notifications or configure plugin beyond initial display.
4. **Collect logs**
* Script grabs `DNP-REACTIVATION` logs and parses:
* `scenario`
* `rescheduled`
**Expected Behavior:**
* Either:
* No `DNP-REACTIVATION` logs at all (no recovery run), **or**
* A specific "no schedules" scenario, e.g.:
```text
DNP-REACTIVATION: No schedules present — skipping recovery (first launch)
```
or
```text
DNP-REACTIVATION: Detected scenario: NONE
```
* In both cases:
* `rescheduled = 0`.
**Pass Criteria:**
* If **no logs**:
* Pass: recovery correctly doesn't run at all on empty DB.
* If logs present:
* `scenario=NONE` (or equivalent) **and** `rescheduled=0`.
Script will report success when:
* `scenario == NONE_SCENARIO_VALUE` and `rescheduled=0`, or
* No recovery logs are found.
---
## 5. Latest Known Good Run (Template)
Fill this in after your first successful emulator run.
```markdown
---
## Latest Known Good Run (Emulator)
**Environment**
- Device: Pixel 8 API 34 (Android 14)
- App ID: `com.timesafari.dailynotification`
- Build: Debug APK (`app-debug.apk`) from commit `<GIT_HASH>`
- Script: `./test-phase2.sh`
- Date: 2025-11-XX
**Results**
- ✅ TEST 1: Force Stop – Alarms Cleared
- `scenario=FORCE_STOP`
- `missed=1, rescheduled=1, verified=0, errors=0`
- ✅ TEST 2: Force Stop / Process Stop – Alarms Intact
- `scenario=COLD_START` (or equivalent non-force-stop scenario)
- `rescheduled=0, errors=0`
- ✅ TEST 3: First Launch / No Schedules
- `scenario=NONE` (or no logs)
- `rescheduled=0`
**Conclusion:**
Phase 2 **Force Stop Detection & Recovery** is verified on the emulator using `test-phase2.sh`. This run is the canonical reference for future regression testing.
```
---
## 6. Troubleshooting
### Alarms Not Cleared on Force Stop
**Symptom**: `am force-stop` doesn't clear alarms in AlarmManager
**Cause**: Some Android versions/emulators don't clear alarms on force stop
**Solution**:
- This is expected behavior on some systems
- TEST 1 will run as Phase 1 (cold start) recovery
- For full force stop validation, test on a device/OS that clears alarms
- Script will report this as an environment limitation, not a failure
### Scenario Not Detected as FORCE_STOP
**Symptom**: Logs show `COLD_START` even when alarms were cleared
**Possible Causes**:
1. Scenario detection logic not implemented (Phase 2 not complete)
2. Alarm count check failing (`alarmsExist()` returning true when it shouldn't)
3. Database query timing issue
**Solution**:
- Verify Phase 2 implementation is complete
- Check `ReactivationManager.detectScenario()` implementation
- Review logs for alarm existence checks
- Verify `alarmsExist()` uses PendingIntent check (not `nextAlarmClock`)
### Recovery Doesn't Reschedule Alarms
**Symptom**: `rescheduled=0` even when alarms were cleared
**Possible Causes**:
1. Schedules not in database
2. Reschedule logic failing
3. Alarm scheduling permissions missing
**Solution**:
- Verify schedules exist in database
- Check logs for reschedule errors
- Verify exact alarm permission is granted
- Review `performForceStopRecovery()` implementation
---
## 7. Related Documentation
- [Phase 2 Directive](../android-implementation-directive-phase2.md) - Implementation details
- [Phase 2 Verification](./PHASE2-VERIFICATION.md) - Verification report
- [Phase 1 Testing Guide](./PHASE1-EMULATOR-TESTING.md) - Prerequisite testing
- [Activation Guide](./ACTIVATION-GUIDE.md) - How to use directives
- [Plugin Requirements](./03-plugin-requirements.md) - Requirements Phase 2 implements
---
**Status**: Ready for testing (Phase 2 implementation pending)
**Last Updated**: November 2025

195
docs/alarms/PHASE2-VERIFICATION.md Normal file
View File

@@ -0,0 +1,195 @@
# Phase 2 – Force Stop Recovery Verification
**Plugin:** Daily Notification Plugin
**Scope:** Force stop detection & recovery (App ID: `com.timesafari.dailynotification`)
**Related Docs:**
- `android-implementation-directive-phase2.md`
- `03-plugin-requirements.md` (Force Stop & App Termination behavior)
- `PHASE2-EMULATOR-TESTING.md`
- `000-UNIFIED-ALARM-DIRECTIVE.md`
---
## 1. Objective
Phase 2 verifies that the Daily Notification Plugin:
1. Correctly detects **force stop** conditions where alarms may have been cleared.
2. **Reschedules** future notifications when schedules exist in the database but alarms are missing.
3. **Avoids heavy recovery** when alarms are intact (no false positives).
4. **Does not misfire** force-stop recovery on first launch / empty database.
Phase 2 builds on Phase 1, which already covers:
- Cold start detection
- Missed notification marking
- Basic alarm verification
---
## 2. Test Method
Verification is performed using the emulator test harness:
```bash
cd test-apps/android-test-app
./test-phase2.sh
```
This script:
* Builds and installs the debug APK (`app/build/outputs/apk/debug/app-debug.apk`).
* Guides the tester through UI steps for scheduling notifications and configuring the plugin.
* Simulates:
* `force-stop` behavior via `adb shell am force-stop ...`
* "Soft stop" / process kill via `adb shell am kill ...`
* First launch / empty DB via uninstall + reinstall
* Collects and parses `DNP-REACTIVATION` log lines, extracting:
* `scenario`
* `missed`
* `rescheduled`
* `verified`
* `errors`
Detailed steps and expectations are documented in `PHASE2-EMULATOR-TESTING.md`.
---
## 3. Test Matrix
| ID | Scenario | Method / Script Step | Expected Behavior | Result | Notes |
| --- | ---------------------------------------------- | ------------------------------------------------------ | ------------------------------------------------------------------------------------------------------ | ------ | ----- |
| 2.1 | Force stop clears alarms | `test-phase2.sh` – TEST 1: Force Stop – Alarms Cleared | `scenario=FORCE_STOP`, `rescheduled>0`, `errors=0` | ☐ | |
| 2.2 | Force stop / process stop with alarms intact | `test-phase2.sh` – TEST 2: Soft Stop – Alarms Intact | `scenario != FORCE_STOP`, `rescheduled=0`, `errors=0` | ☐ | |
| 2.3 | First launch / empty DB (no schedules present) | `test-phase2.sh` – TEST 3: First Launch / No Schedules | Either no recovery logs **or** `scenario=NONE` (or equivalent) and `rescheduled=0`, `errors=0` | ☐ | |
> Fill in **Result** and **Notes** after executing the script on your baseline emulator/device.
---
## 4. Expected Log Patterns
### 4.1 Force Stop – Alarms Cleared (Test 2.1)
Typical expected `DNP-REACTIVATION` log patterns:
```text
DNP-REACTIVATION: Detected scenario: FORCE_STOP
DNP-REACTIVATION: Rescheduled alarm: daily_<id> for <time>
DNP-REACTIVATION: Rescheduled missing alarm: daily_<id> at <time>
DNP-REACTIVATION: Force stop recovery completed: missed=1, rescheduled=1, verified=0, errors=0
```
The **script** will report:
```text
✅ TEST 1 PASSED: Force stop detected and alarms rescheduled (scenario=FORCE_STOP, rescheduled=1).
```
### 4.2 Soft Stop – Alarms Intact (Test 2.2)
Typical expected patterns:
```text
DNP-REACTIVATION: Detected scenario: COLD_START
DNP-REACTIVATION: Cold start recovery completed: missed=0, rescheduled=0, verified>=0, errors=0
```
The script should **not** treat this as a force-stop recovery:
```text
✅ TEST 2 PASSED: No heavy force-stop recovery when alarms are intact (scenario=COLD_START, rescheduled=0).
```
(Adjust `scenario` name to match your actual implementation.)
### 4.3 First Launch / Empty DB (Test 2.3)
Two acceptable patterns:
1. **No recovery logs at all** (`DNP-REACTIVATION` absent), or
2. Explicit "no schedules" scenario, e.g.:
```text
DNP-REACTIVATION: No schedules present — skipping recovery (first launch)
```
or
```text
DNP-REACTIVATION: Detected scenario: NONE
```
Script-level success message might be:
```text
✅ TEST 3 PASSED: NONE scenario detected with no rescheduling.
```
or:
```text
✅ TEST 3 PASSED: No recovery logs when there are no schedules (safe behavior).
```
---
## 5. Latest Known Good Run (Emulator) – Placeholder
> Update this section after your first successful run.
**Environment**
* Device: Pixel 8 API 34 (Android 14)
* App ID: `com.timesafari.dailynotification`
* Build: Debug `app-debug.apk` from commit `<GIT_HASH>`
* Script: `./test-phase2.sh`
* Date: 2025-11-XX
**Observed Results**
* ☐ **2.1 – Force Stop / Alarms Cleared**
* `scenario=FORCE_STOP`
* `missed=1, rescheduled=1, verified=0, errors=0`
* ☐ **2.2 – Soft Stop / Alarms Intact**
* `scenario=COLD_START` (or equivalent non-force-stop scenario)
* `rescheduled=0, errors=0`
* ☐ **2.3 – First Launch / Empty DB**
* `scenario=NONE` (or no logs)
* `rescheduled=0, errors=0`
**Conclusion:**
> To be filled after first successful emulator run.
---
## 6. Overall Status
> To be updated once the first emulator pass is complete.
* **Implementation Status:** ☐ Pending / ✅ Implemented in `ReactivationManager` (Android plugin)
* **Test Harness:** ✅ `test-phase2.sh` in `test-apps/android-test-app`
* **Emulator Verification:** ☐ Pending / ✅ Completed (update when done)
Once all boxes are checked:
> **Overall Status:** ✅ **VERIFIED** – Phase 2 behavior is implemented, emulator-tested, and aligned with `03-plugin-requirements.md` and `android-implementation-directive-phase2.md`.
---
## 7. Related Documentation
- [Phase 2 Directive](../android-implementation-directive-phase2.md) - Implementation details
- [Phase 2 Emulator Testing](./PHASE2-EMULATOR-TESTING.md) - Test procedures
- [Phase 1 Verification](./PHASE1-VERIFICATION.md) - Prerequisite verification
- [Plugin Requirements](./03-plugin-requirements.md) - Requirements this phase implements
- [Platform Capability Reference](./01-platform-capability-reference.md) - OS-level facts
---
**Status**: ☐ **PENDING** – Phase 2 implementation and testing pending
**Last Updated**: November 2025