test(android): fix alarm counting logic and add screenshot capture
Fix alarm counting to correctly parse dumpsys output where app ID and action appear on different lines. Add screenshot capture for test diagnostics and create golden run documentation. Test Harness Improvements: - Fix get_plugin_alarm_count() to track app ID and action separately across alarm block lines (fixes false 0-count bug) - Add show_plugin_alarms_compact() to display complete alarm blocks - Add wait_for_stable_plugin_alarm_count() polling helper to reduce race condition false negatives - Add take_screenshot() and take_failure_screenshot() helpers for automatic test state capture - Integrate screenshots into TEST 0 at key checkpoints - Update TEST 0 messaging to handle race conditions gracefully - Add screenshots/ to .gitignore Documentation: - Create PHASE1_TEST0_GOLDEN.md with actual values from successful run - Document expected script output, UI state, dumpsys shape, and logcat patterns - Include pass/fail checklist for future test runs This fixes the issue where alarm counting always returned 0 because the AWK logic required app ID and action on the same line, but dumpsys output has them on separate lines (header line has app ID, tag line has action).
This commit is contained in:
Matthew Raymer
230
test-apps/android-test-app/docs/PHASE1_TEST0_GOLDEN.md
Normal file
230
test-apps/android-test-app/docs/PHASE1_TEST0_GOLDEN.md
Normal file
@@ -0,0 +1,230 @@
|
||||
# Phase 1 — TEST 0 Golden Run (Daily Rollover Verification)
|
||||
|
||||
**Last Updated:** 2025-12-04
|
||||
**Status:** ✅ PASS (Golden Baseline)
|
||||
|
||||
---
|
||||
|
||||
## 1. Test Overview
|
||||
|
||||
This document captures a **golden baseline** for **Phase 1 – TEST 0: Daily Rollover Verification**.
|
||||
|
||||
**Purpose:** Verify that after a notification fires, the plugin:
|
||||
- Computes **next day's time** (T + 24h)
|
||||
- Schedules **exactly one** `AlarmManager` notification alarm for tomorrow
|
||||
- Does **not** create duplicates
|
||||
- Leaves prefetch in **WorkManager** (not visible in `dumpsys alarm`)
|
||||
|
||||
> **Golden Rule:** If a future run looks like this doc, TEST 0 should be considered a PASS.
|
||||
|
||||
---
|
||||
|
||||
## 2. Environment & Build Info
|
||||
|
||||
### Emulator / Device
|
||||
- **API Level:** 35
|
||||
- **Android Version:** 15
|
||||
|
||||
### Build
|
||||
- **Gradle Version:** 8.13
|
||||
- **Build Warnings:**
|
||||
- `WARNING: Using flatDir should be avoided because it doesn't support any meta-data formats.`
|
||||
- `Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.`
|
||||
|
||||
### Command Used
|
||||
```bash
|
||||
./test-phase1.sh
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
- `ENABLE_SCREENSHOTS=1` (screenshots enabled)
|
||||
- `ADB_BIN=adb` (default)
|
||||
|
||||
---
|
||||
|
||||
## 3. Step-by-Step Execution (Golden Run)
|
||||
|
||||
1. Ran `./test-phase1.sh`.
|
||||
2. Confirmed pre-flight checks (ADB device + emulator ready).
|
||||
3. Allowed script to rebuild and reinstall the app.
|
||||
4. Confirmed plugin status in the UI:
|
||||
- ⚙️ Plugin Settings: ✅ Configured
|
||||
- 🔌 Native Fetcher: ✅ Configured
|
||||
- 🔔 Notifications: ✅ Granted
|
||||
- ⏰ Exact Alarms: ✅ Granted
|
||||
- 📢 Channel: ✅ Enabled (High)
|
||||
5. From the UI, scheduled a **daily notification** for ~1–2 minutes in the future (scheduled for `09:23:00` on 2025-12-04).
|
||||
6. Waited for the notification banner to fire.
|
||||
7. Pressed Enter to continue when prompted.
|
||||
8. Let the script perform the post-rollover alarm check.
|
||||
|
||||
---
|
||||
|
||||
## 4. Expected Script Output (Key Excerpts)
|
||||
|
||||
### 4.1. Pre-Schedule Check
|
||||
```text
|
||||
✅ Found 1 notification alarm (expected: 1) – preliminary check passed.
|
||||
ℹ️ This is preliminary check only; final verdict after rollover.
|
||||
```
|
||||
|
||||
### 4.2. Notification Alarm Details (After Scheduling)
|
||||
```text
|
||||
ℹ️ Notification alarm details:
|
||||
tag=*walarm*:com.timesafari.daily.NOTIFICATION
|
||||
type=RTC_WAKEUP origWhen=2025-12-04 09:23:00.000 window=0 exactAllowReason=policy_permission repeatInterval=0 count=0 flags=0x3
|
||||
policyWhenElapsed: requester=+3m34s315ms app_standby=-10s456ms device_idle=-- battery_saver=--
|
||||
```
|
||||
|
||||
### 4.3. Post-Rollover Check
|
||||
```text
|
||||
ℹ️ Polling for stable alarm count (allowing up to ~10 seconds for Android to settle)...
|
||||
ℹ️ Notification alarms after rollover: 1 (expected: 1)
|
||||
ℹ️ System/other alarms: <N> (for context)
|
||||
ℹ️ Note: Prefetch is scheduled via WorkManager (not AlarmManager), so it won't appear in alarm count
|
||||
```
|
||||
|
||||
### 4.4. Final Verdict
|
||||
```text
|
||||
✅ TEST 0 PASSED: Daily rollover created exactly one NOTIFICATION alarm for tomorrow.
|
||||
Expected state after rollover:
|
||||
✅ 1 notification alarm (AlarmManager) for tomorrow
|
||||
✅ 1 prefetch job (WorkManager) for 2 minutes before tomorrow's notification
|
||||
```
|
||||
|
||||
**Note:** The `origWhen` for tomorrow will be the next day at the same time (e.g., `2025-12-05 09:23:00.000` if scheduled for `2025-12-04 09:23:00.000`).
|
||||
|
||||
---
|
||||
|
||||
## 5. Expected UI State (Screenshots)
|
||||
|
||||
### 5.1 Screenshot Files (Golden Run)
|
||||
|
||||
- `screenshots/phase1_test0_daily_rollover/phase1_test0_daily_rollover_before_scheduling_20251204-091910.png`
|
||||
- **Status:** Active Schedules: **No**; Next Notification: **None scheduled**.
|
||||
|
||||
- `screenshots/phase1_test0_daily_rollover/phase1_test0_daily_rollover_after_scheduling_20251204-091925.png`
|
||||
- **Status:** Active Schedules: **Yes**; Next Notification: **today at 09:23:00 AM**; Pending: **1**.
|
||||
|
||||
- `screenshots/phase1_test0_daily_rollover/phase1_test0_daily_rollover_after_rollover_check_20251204-092307.png`
|
||||
- **Status:** Active Schedules: **Yes**; Next Notification: **tomorrow at 09:23:00 AM** (24 hours later); Pending: **1**.
|
||||
|
||||
---
|
||||
|
||||
## 6. Expected `dumpsys alarm` Shape
|
||||
|
||||
### 6.1. Representative Snippet
|
||||
|
||||
```text
|
||||
RTC_WAKEUP #<N>: Alarm{<handle> type 0 origWhen <timestamp> whenElapsed ... com.timesafari.dailynotification}
|
||||
tag=*walarm*:com.timesafari.daily.NOTIFICATION
|
||||
type=RTC_WAKEUP origWhen=2025-12-05 09:23:00.000 ...
|
||||
...
|
||||
|
||||
Next wake from idle: Alarm{<handle> type 0 origWhen <timestamp> ... com.timesafari.dailynotification}
|
||||
tag=*walarm*:com.timesafari.daily.NOTIFICATION
|
||||
```
|
||||
|
||||
### 6.2. Key Observations
|
||||
|
||||
- There should be **exactly one unique alarm handle** for the plugin (the handle will differ between runs).
|
||||
- It can appear both in the main list and in **"Next wake from idle"**, but counted as **one** alarm (deduplication by alarm handle).
|
||||
- `tag` must be `*walarm*:com.timesafari.daily.NOTIFICATION`.
|
||||
- `type` must be `RTC_WAKEUP`.
|
||||
- `origWhen` should be **tomorrow** at the same time-of-day as the scheduled notification (e.g., `2025-12-05 09:23:00.000` if scheduled for `2025-12-04 09:23:00.000`).
|
||||
|
||||
---
|
||||
|
||||
## 7. Expected `logcat` Patterns
|
||||
|
||||
### 7.1. Scheduling Test Notification
|
||||
|
||||
```text
|
||||
DNP-SCHEDULE: Scheduling next daily alarm: id=daily_..., nextRun=2025-12-04 09:23:00, source=TEST_NOTIFICATION
|
||||
DNP-NOTIFY: Stored notification content in database: id=daily_...
|
||||
DNP-NOTIFY: Scheduling alarm: triggerTime=2025-12-04 09:23:00, ...
|
||||
DNP-SCHEDULE: Scheduling OS alarm: variant=ALARM_CLOCK, action=com.timesafari.daily.NOTIFICATION, ...
|
||||
```
|
||||
|
||||
### 7.2. Rollover on Fire
|
||||
|
||||
```text
|
||||
DNP-SCHEDULE: Scheduling next daily alarm: id=daily_rollover_..., nextRun=2025-12-05 09:23:00, source=ROLLOVER_ON_FIRE
|
||||
DNP-NOTIFY: Stored notification content in database: id=notify_...
|
||||
DNP-NOTIFY: Scheduling alarm: triggerTime=2025-12-05 09:23:00, ...
|
||||
DNP-SCHEDULE: Scheduling OS alarm: variant=ALARM_CLOCK, action=com.timesafari.daily.NOTIFICATION, ...
|
||||
```
|
||||
|
||||
### 7.3. Critical Requirements
|
||||
|
||||
**Both sequences must be present** for a true PASS:
|
||||
- `source=TEST_NOTIFICATION` sequence when scheduling the initial test notification
|
||||
- `source=ROLLOVER_ON_FIRE` sequence when the notification fires and schedules tomorrow's alarm
|
||||
- Times must match: initial schedule time → tomorrow's time (T + 24h)
|
||||
- Example: `2025-12-04 09:23:00` → `2025-12-05 09:23:00`
|
||||
|
||||
---
|
||||
|
||||
## 8. Quick Pass/Fail Checklist
|
||||
|
||||
A run of TEST 0 is a **PASS** if all of the following are true:
|
||||
|
||||
### Script Output
|
||||
- [ ] Shows "Found 1 notification alarm (expected: 1) – preliminary check passed."
|
||||
- [ ] Shows "Notification alarms after rollover: 1 (expected: 1)".
|
||||
- [ ] Ends with "✅ TEST 0 PASSED: Daily rollover created exactly one NOTIFICATION alarm for tomorrow."
|
||||
|
||||
### UI State
|
||||
- [ ] **Before scheduling:** Active Schedules: No; Next Notification: None scheduled.
|
||||
- [ ] **After scheduling:** Active Schedules: Yes; Next Notification: *today* at the chosen time.
|
||||
- [ ] **After rollover:** Active Schedules: Yes; Next Notification: *tomorrow* at the same time.
|
||||
|
||||
### `dumpsys alarm`
|
||||
- [ ] Exactly one `RTC_WAKEUP` alarm with `tag=*walarm*:com.timesafari.daily.NOTIFICATION` for **tomorrow**.
|
||||
- [ ] Same alarm handle may appear under "Next wake from idle", but no second distinct handle.
|
||||
- [ ] `origWhen` timestamp is exactly 24 hours after the initial scheduled time.
|
||||
|
||||
### `logcat`
|
||||
- [ ] Shows both `source=TEST_NOTIFICATION` and `source=ROLLOVER_ON_FIRE` sequences with matching times.
|
||||
- [ ] No duplicate `DNP-SCHEDULE` entries for the same `nextRun` time.
|
||||
- [ ] No errors or warnings related to alarm scheduling.
|
||||
|
||||
---
|
||||
|
||||
## 9. Notes / Deviations
|
||||
|
||||
### Failure Conditions
|
||||
- If there is exactly **0** alarms after rollover, treat as **INCONCLUSIVE** and investigate:
|
||||
- Check `logcat` for `ROLLOVER_ON_FIRE` sequence
|
||||
- Verify `dumpsys alarm` manually
|
||||
- Check for scheduling errors in logs
|
||||
- If there are **>1** alarms after rollover, treat as **FAIL** (duplicate alarm bug):
|
||||
- Check for multiple `DNP-SCHEDULE` entries with same `nextRun`
|
||||
- Verify idempotence checks are working
|
||||
- Check for race conditions between rollover and recovery paths
|
||||
|
||||
### Time-of-Day Variations
|
||||
- Time-of-day may differ in future golden runs; **structure and relationships must remain the same**.
|
||||
- The key is: initial time → tomorrow's time (T + 24h), not the specific hour/minute.
|
||||
|
||||
### Screenshot Timestamps
|
||||
- Screenshot filenames include timestamps (`YYYYMMDD-HHMMSS`), so exact filenames will differ between runs.
|
||||
- Focus on the **content** of screenshots (UI state) rather than exact filenames.
|
||||
|
||||
### Alarm Handle Variations
|
||||
- The alarm handle (e.g., `Alarm{1f00a1b}`) will differ between runs; this is expected.
|
||||
- The important thing is that there is **exactly one unique handle** per scheduled alarm.
|
||||
|
||||
---
|
||||
|
||||
## 10. Updating This Document
|
||||
|
||||
When updating this golden run document:
|
||||
1. Update timestamps and IDs with actual values from your successful run
|
||||
2. Replace placeholder values (marked with "Update...") with real data
|
||||
3. Update screenshot filenames with actual timestamps
|
||||
4. Add any environment-specific notes that might affect future runs
|
||||
5. Document any deviations or edge cases encountered
|
||||
|
||||
**Last Golden Run Date:** 2025-12-04 (09:23:00 scheduled time)
|
||||
|
||||
Reference in New Issue
Block a user