test(android-test-app): unify presentation framework with evidence collection

Browse Source

Implement P0-P5 directives for operator clarity, consistent outcomes, and
easy evidence capture across all test phases.

Changes:
- alarm-test-lib.sh: Add evidence collection (capture_alarms, capture_logcat,
  capture_screenshot), verdict functions (verdict_pass/warn/fail), run directory
  management, and release gating support (RELEASE_GATE_PHASE3)

- test-phase1.sh: Refactor to unified framework with CLI modes (--setup,
  --run, --smoke, --all, --ci), micro-prompts, evidence capture, and verdict
  blocks for all 5 tests

- test-phase2.sh: Add evidence capture, verdict blocks, and STRICTNESS policy
  (soft/hard) for warn vs fail behavior

- test-phase3.sh: Add evidence capture, verdict blocks, release gating
  (--gate-phase3), and fatigue reduction (time estimates, automation hints)

- RUNBOOK-TESTING.md: New comprehensive operator guide (669 lines) covering
  prerequisites, all phases, evidence locations, verdict interpretation,
  common failures, and troubleshooting

All test scripts now use consistent UI helpers (section, substep, info, ok,
warn, error), standardized evidence collection, and clear verdict reporting.
Evidence is saved to timestamped run directories (runs/<RUN_ID>/) with alarms,
logs, and screenshots organized by test phase and scenario.

Tests pass with consistent presentation and reproducible evidence collection.

...

This commit is contained in:

Matthew Raymer

2025-12-24 12:01:16 +00:00

parent 973af9b688

commit ac39255672

5 changed files with 2373 additions and 870 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

1619

test-apps/android-test-app/test-phase1.sh

File diff suppressed because it is too large Load Diff