- Add TestEventsPlugin for receiving ADB broadcast intents - Create operator console UI (console/index.html, console.css, console.js) - Add test plan structure (plan.json) with phases, tests, and steps - Wire all test scripts (phase1, phase2, phase3) with step context and events - Add event emission helpers to alarm-test-lib.sh (step_start, step_pass, etc.) - Update test-phase1.sh with comprehensive prerequisite verification - Register TestEventsPlugin in capacitor.plugins.json - Add console documentation (CONSOLE-USAGE.md, CONSOLE-REMAINING-WORK.md) - Add test implementation alignment tracking (TEST-IMPLEMENTATION-ALIGNMENT.md) This enables real-time test progress tracking via structured events from shell scripts to the operator console UI.
4.7 KiB
4.7 KiB
Test Console Usage Guide
Overview
The Test Console is a Textual-inspired operator interface that provides a structured view of test execution, real-time progress tracking, and evidence management.
Features
Phase A (UI-Only Mode)
- Test Plan Rendering: Visual tree of phases, tests, and steps
- Step Checklists: Track progress with ✓/✗/⚠/→/· status indicators
- Current Step Frame: Detailed instructions for the current step
- Evidence Panel: View captured artifacts (alarms, logs, screenshots)
- Live Feed: Real-time event stream
- State Persistence: Progress saved to localStorage
Phase B (Live Updates)
- ADB Broadcast Integration: Scripts send events via
adb shell am broadcast - Real-Time Updates: Console updates automatically as tests run
- Automatic Status: Step/test status updates from script events
Enabling Live Updates
To enable live event streaming from test scripts:
export DNP_UI_EVENTS=1
./test-phase1.sh
Test Context Setup
In test scripts, set context before executing steps:
# Set phase/test/step context
set_test_context "phase1" "phase1_test0" "p1_t0_s1"
# Emit step events
step_start "p1_t0_s1" "Starting step"
# ... do work ...
step_pass "p1_t0_s1" "Step completed"
Event Types
Step Events
step_start- Step execution beginsstep_pass- Step completed successfullystep_warn- Step completed with warningsstep_fail- Step failed
Operator Events
operator_required- Manual action needed (fromui_prompt())
Evidence Events
artifact- Evidence captured (fromcapture_*()functions)
Verdict Events
test_verdict- Test completed (fromverdict_*()functions)
Step ID Mapping
Step IDs follow the pattern: p{phase}_{test}_s{step}
Examples:
p1_t0_s1= Phase 1, Test 0, Step 1p2_t1_s3= Phase 2, Test 1, Step 3p3_t4_s2= Phase 3, Test 4, Step 2
Console Navigation
- Select Test: Click a test in the left sidebar
- View Steps: Step checklist shows in the middle panel
- Follow Instructions: Current step frame shows what to do
- Mark Progress: Use "Mark Done ✓" or "Mark Fail ✗" buttons
- View Evidence: Evidence panel shows captured artifacts
Manual Step Completion
If scripts aren't driving the console (Phase A only), operators can manually mark steps:
- Select the step in the checklist
- Click "Mark Done ✓" or "Mark Fail ✗"
- Add notes if needed
- Progress is saved automatically
Evidence Management
Evidence is automatically captured when scripts call:
capture_alarms()→alarms/directorycapture_logcat()→logs/directorycapture_screenshot()→screens/directory
Evidence appears in the console's evidence panel when:
- Scripts emit
artifactevents (Phase B) - Operators manually add evidence (Phase A)
Run ID
Each console session generates a unique Run ID:
- Format:
YYYYMMDD_HHMMSS_xxxx - Used for localStorage keys and evidence organization
- Displayed in console header
Troubleshooting
Events Not Appearing
- Check Event Emission: Ensure
DNP_UI_EVENTS=1is set - Check ADB Connection: Verify device is connected
- Check Plugin: Ensure
TestEventsPluginis registered in Capacitor - Check Broadcast: Verify broadcast action matches in plugin
Console Not Loading
- Check Plan JSON: Verify
plan.jsonexists and is valid - Check Browser Console: Look for JavaScript errors
- Check Capacitor: Ensure Capacitor is initialized
Steps Not Updating
- Check Context: Verify
set_test_context()is called - Check Step IDs: Ensure step IDs match
plan.json - Check Events: Verify events are being emitted
Example: Wiring a Test Function
test_example() {
section "TEST: Example Test"
# Set test context
set_test_context "phase1" "phase1_example" ""
# Step 1: Setup
set_test_context "phase1" "phase1_example" "p1_ex_s1"
step_start "p1_ex_s1" "Setting up test"
capture_alarms "example_initial"
step_pass "p1_ex_s1" "Setup complete"
# Step 2: Execute
set_test_context "phase1" "phase1_example" "p1_ex_s2"
step_start "p1_ex_s2" "Executing test"
# ... do work ...
step_pass "p1_ex_s2" "Execution complete"
# Step 3: Verify
set_test_context "phase1" "phase1_example" "p1_ex_s3"
step_start "p1_ex_s3" "Verifying results"
# ... verify ...
step_pass "p1_ex_s3" "Verification passed"
# Verdict
set_test_context "phase1" "phase1_example" "p1_ex_s4"
verdict_pass "example_test" "Test passed"
}
Next Steps
- Wire Remaining Tests: Add
set_test_context()andstep_*()calls to all test functions - Test Console: Build app, enable events, run a test script
- Refine UI: Adjust styling, add features as needed