Files
daily-notification-plugin/docs/alarms/PHASE2-EMULATOR-TESTING.md
Matthew Raymer c8a3906449 docs(test): add Phase 2 force stop recovery testing infrastructure
Adds documentation and test harness for Phase 2 (Force Stop Detection & Recovery).

Changes:
- Add PHASE2-EMULATOR-TESTING.md with detailed test procedures
- Add PHASE2-VERIFICATION.md with test matrix and verification template
- Add test-phase2.sh automated test harness

Test harness features:
- 3 test cases: force stop with cleared alarms, intact alarms, empty DB
- Automatic force stop simulation via adb
- Log parsing for scenario detection and recovery results
- UI prompts for plugin configuration and scheduling

Related:
- Directive: android-implementation-directive-phase2.md
- Requirements: docs/alarms/03-plugin-requirements.md §3.1.4
- Testing: docs/alarms/PHASE2-EMULATOR-TESTING.md
- Verification: docs/alarms/PHASE2-VERIFICATION.md
2025-11-27 10:01:40 +00:00

9.9 KiB
Raw Permalink Blame History

PHASE 2 EMULATOR TESTING

Force Stop Detection & Recovery


1. Purpose

Phase 2 verifies that the Daily Notification Plugin correctly:

  1. Detects force stop scenarios (where alarms may be cleared by the OS).
  2. Reschedules future notifications when alarms are missing but schedules remain in the database.
  3. Avoids heavy recovery when alarms are still intact.
  4. Does not misfire force-stop recovery on first launch / empty database.

This document defines the emulator test procedure for Phase 2 using the script:

test-apps/android-test-app/test-phase2.sh

2. Prerequisites

  • Android emulator or device, e.g.:
    • Pixel 8 / API 34 (recommended baseline)
  • ADB available in PATH (or ADB_BIN exported)
  • Project built with:
    • Daily Notification Plugin integrated
    • Test app at: test-apps/android-test-app
  • Debug APK path:
    • app/build/outputs/apk/debug/app-debug.apk
  • Phase 1 behavior already implemented and verified:
    • Cold start detection
    • Missed notification marking

Note: Some OS/device combinations do not clear alarms on am force-stop. In such cases, TEST 1 may partially skip or only verify scenario logging.


3. How to Run

From the android-test-app directory:

cd test-apps/android-test-app
chmod +x test-phase2.sh   # first time only
./test-phase2.sh

The script will:

  1. Perform pre-flight checks (ADB/emulator).
  2. Build and install the debug APK.
  3. Guide you through three tests:
    • TEST 1: Force stop alarms cleared
    • TEST 2: Force stop / process stop alarms intact
    • TEST 3: First launch / empty DB safeguard
  4. Print parsed recovery summaries from DNP-REACTIVATION logs.

You will be prompted for UI actions at each step (e.g., configuring the plugin, pressing "Test Notification").


4. Test Cases

4.1 TEST 1 Force Stop with Cleared Alarms

Goal:

Verify that when a force stop clears alarms, the plugin:

  • Detects the FORCE_STOP scenario.
  • Reschedules future notifications.
  • Completes recovery with errors=0.

Steps (Script Flow):

  1. Launch & configure plugin

    • Script launches the app.
    • In the app UI, confirm:
      • ⚙️ Plugin Settings: ✅ Configured
      • 🔌 Native Fetcher: ✅ Configured
    • If not, press Configure Plugin and wait until both are .
  2. Schedule a future notification

    • Click Test Notification (or equivalent) to schedule a notification a few minutes in the future.
  3. Verify alarms are scheduled

    • Script runs:
      adb shell dumpsys alarm | grep com.timesafari.dailynotification
      
    • Confirm at least one RTC_WAKEUP alarm for com.timesafari.dailynotification.
  4. Force stop the app

    • Script executes:
      adb shell am force-stop com.timesafari.dailynotification
      
  5. Confirm alarms after force stop

    • Script re-runs dumpsys alarm.
    • Ideal test case: 0 alarms for com.timesafari.dailynotification (alarms cleared).
  6. Trigger recovery

    • Script clears logcat and launches the app.
    • Wait ~5 seconds for recovery.
  7. Collect and inspect logs

    • Script collects DNP-REACTIVATION logs and parses:
      • scenario=<value>
      • missed=<n>
      • rescheduled=<n>
      • verified=<n>
      • errors=<n>

Expected Logs (Ideal Case):

  • Scenario:

    DNP-REACTIVATION: Detected scenario: FORCE_STOP
    
  • Alarm handling:

    DNP-REACTIVATION: Rescheduled alarm: daily_<id> for <time>
    DNP-REACTIVATION: Rescheduled missing alarm: daily_<id> at <time>
    
  • Summary:

    DNP-REACTIVATION: Force stop recovery completed: missed=1, rescheduled=1, verified=0, errors=0
    

Pass Criteria:

  • scenario=FORCE_STOP
  • rescheduled > 0
  • errors = 0
  • Script prints:

    ✅ TEST 1 PASSED: Force stop detected and alarms rescheduled (scenario=FORCE_STOP, rescheduled=1).

Partial / Edge Cases:

  • If alarms remain after force-stop on this device:
    • Script warns that FORCE_STOP may not fully trigger.
    • Mark this as environment-limited rather than a plugin failure.

4.2 TEST 2 Force Stop / Process Stop with Intact Alarms

Goal:

Ensure we do not run heavy force-stop recovery when alarms are still intact.

Steps (Script Flow):

  1. Launch & configure plugin

    • Same as TEST 1 (ensure both plugin statuses are ).
  2. Schedule another future notification

    • Click Test Notification again to create a second schedule.
  3. Verify alarms are scheduled

    • Confirm multiple alarms for com.timesafari.dailynotification via dumpsys alarm.
  4. Simulate a "soft stop"

    • Script runs:
      adb shell am kill com.timesafari.dailynotification
      
    • Intent: stop the process but not clear alarms (actual behavior may vary by OS).
  5. Check alarms after soft stop

    • Ensure alarms are still present in dumpsys alarm.
  6. Trigger recovery

    • Script clears logcat and relaunches the app.
    • Wait ~5 seconds.
  7. Collect logs

    • Script parses DNP-REACTIVATION logs for:
      • scenario
      • rescheduled
      • errors

Expected Behavior:

  • scenario should not be FORCE_STOP (e.g., COLD_START or another non-force-stop scenario, depending on your implementation).
  • rescheduled = 0.
  • errors = 0.

Pass Criteria:

  • Alarms still present after soft stop.
  • Recovery summary indicates no force-stop-level rescheduling when alarms are intact:
    • scenario != FORCE_STOP
    • rescheduled = 0
    • Script prints:

      ✅ TEST 2 PASSED: No heavy force-stop recovery when alarms intact (scenario=<value>, rescheduled=0).

If scenario=FORCE_STOP and rescheduled>0 here, treat this as a warning and review scenario detection logic.


4.3 TEST 3 First Launch / Empty DB Safeguard

Goal:

Ensure force-stop recovery is not mis-triggered when the app is freshly installed with no schedules.

Steps (Script Flow):

  1. Clear state

    • Script uninstalls the app to clear DB/state:
      adb uninstall com.timesafari.dailynotification
      
  2. Reinstall APK

    • Script reinstalls app-debug.apk.
  3. Launch app without scheduling anything

    • Script launches the app.
    • Do not schedule notifications or configure plugin beyond initial display.
  4. Collect logs

    • Script grabs DNP-REACTIVATION logs and parses:
      • scenario
      • rescheduled

Expected Behavior:

  • Either:

    • No DNP-REACTIVATION logs at all (no recovery run), or
    • A specific "no schedules" scenario, e.g.:
      DNP-REACTIVATION: No schedules present — skipping recovery (first launch)
      
      or
      DNP-REACTIVATION: Detected scenario: NONE
      
  • In both cases:

    • rescheduled = 0.

Pass Criteria:

  • If no logs:
    • Pass: recovery correctly doesn't run at all on empty DB.
  • If logs present:
    • scenario=NONE (or equivalent) and rescheduled=0.

Script will report success when:

  • scenario == NONE_SCENARIO_VALUE and rescheduled=0, or
  • No recovery logs are found.

5. Latest Known Good Run (Template)

Fill this in after your first successful emulator run.

---
## Latest Known Good Run (Emulator)

**Environment**

- Device: Pixel 8 API 34 (Android 14)
- App ID: `com.timesafari.dailynotification`
- Build: Debug APK (`app-debug.apk`) from commit `<GIT_HASH>`
- Script: `./test-phase2.sh`
- Date: 2025-11-XX

**Results**

- ✅ TEST 1: Force Stop  Alarms Cleared  
  - `scenario=FORCE_STOP`  
  - `missed=1, rescheduled=1, verified=0, errors=0`  

- ✅ TEST 2: Force Stop / Process Stop  Alarms Intact  
  - `scenario=COLD_START` (or equivalent non-force-stop scenario)  
  - `rescheduled=0, errors=0`  

- ✅ TEST 3: First Launch / No Schedules  
  - `scenario=NONE` (or no logs)  
  - `rescheduled=0`  

**Conclusion:**  
Phase 2 **Force Stop Detection & Recovery** is verified on the emulator using `test-phase2.sh`. This run is the canonical reference for future regression testing.

6. Troubleshooting

Alarms Not Cleared on Force Stop

Symptom: am force-stop doesn't clear alarms in AlarmManager

Cause: Some Android versions/emulators don't clear alarms on force stop

Solution:

  • This is expected behavior on some systems
  • TEST 1 will run as Phase 1 (cold start) recovery
  • For full force stop validation, test on a device/OS that clears alarms
  • Script will report this as an environment limitation, not a failure

Scenario Not Detected as FORCE_STOP

Symptom: Logs show COLD_START even when alarms were cleared

Possible Causes:

  1. Scenario detection logic not implemented (Phase 2 not complete)
  2. Alarm count check failing (alarmsExist() returning true when it shouldn't)
  3. Database query timing issue

Solution:

  • Verify Phase 2 implementation is complete
  • Check ReactivationManager.detectScenario() implementation
  • Review logs for alarm existence checks
  • Verify alarmsExist() uses PendingIntent check (not nextAlarmClock)

Recovery Doesn't Reschedule Alarms

Symptom: rescheduled=0 even when alarms were cleared

Possible Causes:

  1. Schedules not in database
  2. Reschedule logic failing
  3. Alarm scheduling permissions missing

Solution:

  • Verify schedules exist in database
  • Check logs for reschedule errors
  • Verify exact alarm permission is granted
  • Review performForceStopRecovery() implementation


Status: Ready for testing (Phase 2 implementation pending)
Last Updated: November 2025