test(android): add auto-reset for TEST 1 and create golden run documentation
Add automatic app state reset for TEST 1 to ensure clean starting state when lingering alarms from TEST 0 are detected. Create PHASE1_TEST1_GOLDEN.md with actual values from successful run. TEST 1 Auto-Reset: - Detect lingering plugin alarms before TEST 1 starts - Automatically uninstall/reinstall app to clear alarms - Verify clean state (0 alarms) before proceeding - Gracefully skip TEST 1 if clean state cannot be achieved - Take failure screenshots when reset fails - Wrap all TEST 1 steps in conditional to skip on reset failure Documentation: - Create PHASE1_TEST1_GOLDEN.md with actual values from passing run - Document auto-reset behavior in golden run steps - Add cross-references between TEST 0 and TEST 1 golden docs - Include actual timestamps, scheduleIds, and recovery metrics This ensures TEST 1 always starts from a known clean state, making test results reliable and reproducible. The golden doc serves as a baseline for comparing future TEST 1 runs.
This commit is contained in:
Matthew Raymer
327
test-apps/android-test-app/docs/PHASE1_TEST1_GOLDEN.md
Normal file
327
test-apps/android-test-app/docs/PHASE1_TEST1_GOLDEN.md
Normal file
@@ -0,0 +1,327 @@
|
||||
# Phase 1 — TEST 1 Golden Run (Force-Stop Recovery - Database Restoration)
|
||||
|
||||
**Related Docs:**
|
||||
- [PHASE1_TEST0_GOLDEN.md](./PHASE1_TEST0_GOLDEN.md)
|
||||
- [PHASE1_TEST1_GOLDEN.md](./PHASE1_TEST1_GOLDEN.md)
|
||||
|
||||
---
|
||||
|
||||
## 1. Test Overview
|
||||
|
||||
**Test Name:** TEST 1 — Force-Stop Recovery: Database Restoration
|
||||
**Purpose:** Verify that after an Android **force-stop** (which clears all scheduled alarms), the plugin:
|
||||
|
||||
1. **Persists** the alarm schedule in its database.
|
||||
2. **Detects** that alarms are missing on app launch.
|
||||
3. **Rebuilds** the missing alarm(s) from the database.
|
||||
4. Restores the **one-notification-per-day** contract:
|
||||
- Before force-stop: 1 alarm
|
||||
- After force-stop: 0 alarms
|
||||
- After recovery: 1 alarm (same trigger time as before force-stop)
|
||||
|
||||
This golden run documents a **known-good execution** on 2025-12-04.
|
||||
|
||||
---
|
||||
|
||||
## 2. Environment & Build Info
|
||||
|
||||
- **Date/Time of Run:** 2025-12-04 (around 09:51–09:55 UTC)
|
||||
- **Command Used:**
|
||||
|
||||
```bash
|
||||
./test-phase1.sh
|
||||
```
|
||||
|
||||
* **Project:** `daily-notification-plugin` — `test-apps/android-test-app`
|
||||
|
||||
* **Gradle:**
|
||||
|
||||
* Build invoked via `./gradlew assembleDebug` (through the script)
|
||||
* Gradle output included:
|
||||
|
||||
```text
|
||||
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
|
||||
```
|
||||
|
||||
* **APK Built:**
|
||||
|
||||
```text
|
||||
./app/build/outputs/apk/debug/app-debug.apk
|
||||
```
|
||||
|
||||
* **App ID:** `com.timesafari.dailynotification`
|
||||
|
||||
> **Note:** Device/emulator model & API level can be added here later if desired.
|
||||
|
||||
---
|
||||
|
||||
## 3. Step-by-Step Execution (Golden Run)
|
||||
|
||||
This section captures the **actual** sequence for the golden run.
|
||||
|
||||
1. Ran `./test-phase1.sh`.
|
||||
2. Pre-flight checks:
|
||||
* ✅ ADB device connected
|
||||
* ✅ Emulator ready
|
||||
3. Build phase:
|
||||
* ✅ Debug APK built successfully.
|
||||
* ✅ APK path: `./app/build/outputs/apk/debug/app-debug.apk`
|
||||
4. Install phase:
|
||||
* ✅ Previous app uninstall succeeded
|
||||
* ✅ APK installed successfully
|
||||
* ✅ App verified in package list
|
||||
* ✅ Logcat cleared (twice)
|
||||
5. **TEST 0** executed:
|
||||
* Configured plugin in UI (if needed).
|
||||
* Scheduled daily notification.
|
||||
* Verified:
|
||||
* Before scheduling: 0 plugin alarms.
|
||||
* After scheduling: 1 plugin alarm for **today**.
|
||||
* After fire/rollover: 1 plugin alarm for **tomorrow**.
|
||||
* **TEST 0 PASSED.**
|
||||
6. **TEST 1** started:
|
||||
* Step 1: Detected 1 lingering plugin alarm (tomorrow's alarm from TEST 0).
|
||||
* Auto-reset:
|
||||
* Uninstalled app
|
||||
* Reinstalled APK
|
||||
* Cleared logcat
|
||||
* Verified plugin alarms = 0
|
||||
* Confirmed **clean starting state** for TEST 1.
|
||||
7. Step 2: Launched app and confirmed plugin configured.
|
||||
8. In UI: tapped **"Test Notification"** button (schedules a notification ~4 minutes in the future).
|
||||
9. Step 3 (pre-FS verify):
|
||||
* Verified **1** plugin alarm exists in AlarmManager.
|
||||
* Confirmed alarm details (time, tag, type).
|
||||
* Confirmed scheduling logs with `source=TEST_NOTIFICATION`.
|
||||
10. Step 4: Performed **force-stop** via `adb shell am force-stop com.timesafari.dailynotification`.
|
||||
11. Step 5 (post-FS verify):
|
||||
* Verified plugin alarms = **0** (force-stop cleared alarms).
|
||||
12. Step 6: Relaunched app (cold start).
|
||||
13. Step 7 (recovery verify):
|
||||
* Verified plugin alarms = **1** after recovery.
|
||||
* Confirmed recovery logs under `DNP-REACTIVATION`:
|
||||
* App launch recovery
|
||||
* Boot recovery
|
||||
* `rescheduled=1`
|
||||
* Confirmed rescheduled alarm uses the **same scheduleId and triggerTime** as pre-force-stop.
|
||||
14. Fire verification was **skipped** (`VERIFY_FIRE=false`).
|
||||
15. TEST 1 summary:
|
||||
* ✅ Before FS: 1 alarm
|
||||
* ✅ After FS: 0 alarms
|
||||
* ✅ After recovery: 1 alarm
|
||||
* ✅ `rescheduled=1`, `errors=0`
|
||||
* ✅ TEST 1 PASSED.
|
||||
|
||||
---
|
||||
|
||||
## 4. Expected Script Output (Key Excerpts)
|
||||
|
||||
These are the **critical excerpts** from the test harness output for a passing TEST 1.
|
||||
|
||||
### 4.1 Clean Start & Auto-Reset
|
||||
|
||||
```text
|
||||
→ Step 1: Clean start - checking for lingering alarms...
|
||||
ℹ️ Current plugin notification alarms: 1
|
||||
ℹ️ System/other alarms: 17 (for context)
|
||||
⚠️ Found 1 lingering plugin alarm(s) - these will interfere with TEST 1.
|
||||
ℹ️ TEST 1 needs a clean state (no existing plugin alarms).
|
||||
ℹ️ Resetting app state via uninstall + reinstall...
|
||||
ℹ️ Uninstalling existing app...
|
||||
✅ App uninstall succeeded (clean slate).
|
||||
ℹ️ Reinstalling APK...
|
||||
✅ App reinstall succeeded.
|
||||
ℹ️ Clearing logcat buffer after reinstall...
|
||||
ℹ️ Rechecking plugin alarms after reset...
|
||||
ℹ️ Plugin alarms after reset: 0 (expected: 0)
|
||||
✅ App state reset complete. TEST 1 starting from clean state.
|
||||
```
|
||||
|
||||
### 4.2 Alarm Scheduled Before Force-Stop
|
||||
|
||||
```text
|
||||
→ Step 3: Verifying alarm exists in AlarmManager (BEFORE force-stop)...
|
||||
ℹ️ Plugin alarms: 1 (expected: 1)
|
||||
ℹ️ System/other alarms: 19 (for context)
|
||||
✅ ✅ Single plugin alarm confirmed in AlarmManager (one per day)
|
||||
ℹ️ Alarm details:
|
||||
RTC_WAKEUP #4: Alarm{161cd2b type 0 origWhen 1764842100000 whenElapsed 13009441 com.timesafari.dailynotification}
|
||||
tag=*walarm*:com.timesafari.daily.NOTIFICATION
|
||||
type=RTC_WAKEUP origWhen=2025-12-04 09:55:00.000 window=0 exactAllowReason=policy_permission repeatInterval=0 count=0 flags=0x3
|
||||
policyWhenElapsed: requester=+3m4s281ms app_standby=-6s566ms device_idle=-- battery_saver=--
|
||||
--
|
||||
ℹ️ Alarm scheduled for: Thu Dec 4 09:55:00 AM UTC 2025 (1764842100000 ms)
|
||||
ℹ️ Checking logs for scheduling confirmation...
|
||||
12-04 09:51:49.150 6803 6867 W DNP-SCHEDULE: Cancelling existing alarm before rescheduling: requestCode=3454, scheduleId=daily_1764841909137, source=TEST_NOTIFICATION
|
||||
12-04 09:51:49.151 6803 6867 I DNP-NOTIFY: Scheduling alarm: triggerTime=2025-12-04 09:55:00, delayMs=190849, requestCode=3454, scheduleId=daily_1764841909137
|
||||
12-04 09:51:49.152 6803 6867 I DNP-SCHEDULE: Scheduling OS alarm: variant=ALARM_CLOCK, action=com.timesafari.daily.NOTIFICATION, triggerTime=1764842100000, requestCode=3454, scheduleId=daily_1764841909137, source=TEST_NOTIFICATION, pendingIntentHash=267839060, showIntentHash=256236029
|
||||
12-04 09:51:49.153 6803 6867 I DNP-NOTIFY: Alarm clock scheduled (setAlarmClock): triggerAt=1764842100000, requestCode=3454
|
||||
```
|
||||
|
||||
### 4.3 Force-Stop & Alarm Clearance
|
||||
|
||||
```text
|
||||
→ Step 4: Force-stopping app (clears all alarms)...
|
||||
⚠️ Force-stop will clear ALL alarms from AlarmManager
|
||||
ℹ️ Executing: adb shell am force-stop com.timesafari.dailynotification
|
||||
ℹ️ Forcing stop of app process...
|
||||
✅ Force stop issued
|
||||
→ Step 5: Verifying alarms are MISSING from AlarmManager (AFTER force-stop)...
|
||||
ℹ️ Plugin alarms after force-stop: 0 (expected: 0)
|
||||
ℹ️ System/other alarms: 17 (for context)
|
||||
✅ ✅ Plugin alarms cleared by force-stop (count: 0)
|
||||
ℹ️ This confirms: Force-stop cleared alarms from AlarmManager
|
||||
```
|
||||
|
||||
### 4.4 Recovery & Rescheduling
|
||||
|
||||
```text
|
||||
→ Step 6: Relaunching app (triggers recovery from database)...
|
||||
ℹ️ Clearing logcat buffer...
|
||||
✅ Logs cleared
|
||||
ℹ️ Launching app...
|
||||
Starting: Intent { cmp=com.timesafari.dailynotification/.MainActivity }
|
||||
✅ App launched
|
||||
→ Step 7: Verifying recovery rebuilt alarms from database...
|
||||
ℹ️ Plugin alarms after recovery: 1 (expected: 1)
|
||||
ℹ️ System/other alarms: 21 (for context)
|
||||
ℹ️ Checking recovery logs...
|
||||
|
||||
12-04 09:52:21.919 6954 7015 I DNP-REACTIVATION: Starting app launch recovery (Phase 1: cold start only)
|
||||
12-04 09:52:21.926 6954 7015 I DNP-REACTIVATION: Cold start recovery: checking for missed notifications
|
||||
12-04 09:52:21.937 6954 7015 I DNP-REACTIVATION: Rescheduled alarm: daily_1764841909137 for 1764842100000
|
||||
12-04 09:52:21.937 6954 7015 I DNP-REACTIVATION: Rescheduled missing alarm: daily_1764841909137 at 1764842100000
|
||||
12-04 09:52:21.952 6954 7015 I DNP-REACTIVATION: Cold start recovery complete: missed=0, rescheduled=1, verified=0, errors=0
|
||||
12-04 09:52:21.952 6954 7015 I DNP-REACTIVATION: App launch recovery completed: missed=0, rescheduled=1, verified=0, errors=0
|
||||
12-04 09:52:22.077 6954 7015 I DNP-REACTIVATION: Starting boot recovery
|
||||
12-04 09:52:22.135 6954 7017 I DNP-REACTIVATION: Loaded 1 schedules from DB
|
||||
12-04 09:52:22.138 6954 7017 I DNP-REACTIVATION: Rescheduled alarm: daily_1764841909137 for 1764842100000
|
||||
12-04 09:52:22.143 6954 7017 I DNP-REACTIVATION: Boot recovery complete: missed=0, rescheduled=1, verified=0, errors=0
|
||||
```
|
||||
|
||||
Final summary:
|
||||
|
||||
```text
|
||||
✅ ✅ Alarms restored in AlarmManager (count: 1)
|
||||
✅ ✅ Recovery logs confirm rescheduling (rescheduled=1)
|
||||
✅ TEST 1 PASSED: Recovery successfully rebuilt alarms from database!
|
||||
ℹ️ Summary:
|
||||
- Before force-stop: 1 alarm(s)
|
||||
- After force-stop: 0 alarm(s) (cleared)
|
||||
- After recovery: 1 alarm(s) (rebuilt)
|
||||
- Rescheduled: 1 alarm(s)
|
||||
- Verified: 0 alarm(s)
|
||||
ℹ️ Skipping fire verification (VERIFY_FIRE=false, set VERIFY_FIRE=true to enable)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Expected `dumpsys alarm` Shapes
|
||||
|
||||
### 5.1 Before Force-Stop (Scheduled)
|
||||
|
||||
* **Plugin alarm count:** 1
|
||||
* Example block (shape, not necessarily exact handle):
|
||||
|
||||
```text
|
||||
RTC_WAKEUP #4: Alarm{161cd2b type 0 origWhen 1764842100000 whenElapsed 13009441 com.timesafari.dailynotification}
|
||||
tag=*walarm*:com.timesafari.daily.NOTIFICATION
|
||||
type=RTC_WAKEUP origWhen=2025-12-04 09:55:00.000 window=0 exactAllowReason=policy_permission repeatInterval=0 count=0 flags=0x3
|
||||
policyWhenElapsed: requester=+3m4s281ms app_standby=-6s566ms device_idle=-- battery_saver=--
|
||||
```
|
||||
|
||||
### 5.2 After Force-Stop
|
||||
|
||||
* **Plugin alarm count:** 0
|
||||
* No `*walarm*:com.timesafari.daily.NOTIFICATION` entries should appear.
|
||||
|
||||
### 5.3 After Recovery
|
||||
|
||||
* **Plugin alarm count:** 1
|
||||
* Alarm should be restored with:
|
||||
* Same `scheduleId`: `daily_1764841909137`
|
||||
* Same `triggerTime`: `1764842100000` → `2025-12-04 09:55:00`
|
||||
|
||||
---
|
||||
|
||||
## 6. Expected `logcat` Patterns
|
||||
|
||||
For a passing run, look for:
|
||||
|
||||
* **Scheduling (before FS):**
|
||||
|
||||
```text
|
||||
DNP-NOTIFY: Scheduling alarm: triggerTime=2025-12-04 09:55:00, delayMs=..., requestCode=3454, scheduleId=daily_1764841909137
|
||||
DNP-SCHEDULE: Scheduling OS alarm: variant=ALARM_CLOCK, action=com.timesafari.daily.NOTIFICATION, triggerTime=1764842100000, requestCode=3454, scheduleId=daily_1764841909137, source=TEST_NOTIFICATION, ...
|
||||
```
|
||||
|
||||
* **Recovery (after FS + relaunch):**
|
||||
|
||||
```text
|
||||
DNP-REACTIVATION: Starting app launch recovery (Phase 1: cold start only)
|
||||
DNP-REACTIVATION: Rescheduled alarm: daily_1764841909137 for 1764842100000
|
||||
DNP-REACTIVATION: Cold start recovery complete: missed=0, rescheduled=1, verified=0, errors=0
|
||||
DNP-REACTIVATION: App launch recovery completed: missed=0, rescheduled=1, verified=0, errors=0
|
||||
|
||||
DNP-REACTIVATION: Starting boot recovery
|
||||
DNP-REACTIVATION: Loaded 1 schedules from DB
|
||||
DNP-REACTIVATION: Rescheduled alarm: daily_1764841909137 for 1764842100000
|
||||
DNP-REACTIVATION: Boot recovery complete: missed=0, rescheduled=1, verified=0, errors=0
|
||||
```
|
||||
|
||||
Key invariants:
|
||||
|
||||
* `rescheduled=1`
|
||||
* `errors=0`
|
||||
* `scheduleId` and `triggerTime` match pre-FS values.
|
||||
|
||||
---
|
||||
|
||||
## 7. Quick Pass/Fail Checklist
|
||||
|
||||
A TEST 1 run is a **PASS** if all of the following are true:
|
||||
|
||||
* **Starting state:**
|
||||
* Script auto-resets if lingering alarms exist from TEST 0.
|
||||
* After reset: plugin alarm count = **0**.
|
||||
|
||||
* **Pre-force-stop:**
|
||||
* Plugin alarm count = **1**.
|
||||
* Alarm is tagged `*walarm*:com.timesafari.daily.NOTIFICATION`.
|
||||
* `triggerTime` and `origWhen` are consistent (e.g. `2025-12-04 09:55:00`, `1764842100000`).
|
||||
* `scheduleId` looks like `daily_<timestamp>` (here: `daily_1764841909137`).
|
||||
* Logs show `source=TEST_NOTIFICATION`.
|
||||
|
||||
* **After force-stop:**
|
||||
* Plugin alarm count = **0**.
|
||||
* No NOTIFICATION alarms remain in `dumpsys`.
|
||||
|
||||
* **After recovery (relaunch):**
|
||||
* Plugin alarm count = **1**.
|
||||
* Logs show `DNP-REACTIVATION` with:
|
||||
* `rescheduled=1`
|
||||
* `errors=0`
|
||||
* Same `scheduleId` and `triggerTime` as pre-FS.
|
||||
|
||||
* **Script summary:**
|
||||
* States:
|
||||
* Before force-stop: 1 alarm
|
||||
* After force-stop: 0 alarms
|
||||
* After recovery: 1 alarm
|
||||
* Ends with: `✅ TEST 1 PASSED: Recovery successfully rebuilt alarms from database!`
|
||||
|
||||
If any of these conditions fail, the run is **NOT GOLDEN** and should not overwrite this reference.
|
||||
|
||||
---
|
||||
|
||||
## 8. Notes / Deviations
|
||||
|
||||
* This golden run **skips fire verification** (`VERIFY_FIRE=false`).
|
||||
Future runs may enable fire verification; if that becomes standard, update this doc.
|
||||
* It is acceptable for both:
|
||||
* **Cold start recovery** and
|
||||
* **Boot recovery**
|
||||
to run on app launch, as long as:
|
||||
* `rescheduled=1`
|
||||
* No duplicate alarms are scheduled.
|
||||
* If OS behavior changes (e.g., force-stop no longer clears alarms on a future Android version), this test's expectations may need to be revised; document any such deviations explicitly here before changing the checklist.
|
||||
Reference in New Issue
Block a user