Files
daily-notification-plugin/test-apps/android-test-app/docs/CONSOLE-USAGE.md
T
pc-sas-publisher f6df9e13fb feat: add operator console and wire test scripts with event emission
- Add TestEventsPlugin for receiving ADB broadcast intents
- Create operator console UI (console/index.html, console.css, console.js)
- Add test plan structure (plan.json) with phases, tests, and steps
- Wire all test scripts (phase1, phase2, phase3) with step context and events
- Add event emission helpers to alarm-test-lib.sh (step_start, step_pass, etc.)
- Update test-phase1.sh with comprehensive prerequisite verification
- Register TestEventsPlugin in capacitor.plugins.json
- Add console documentation (CONSOLE-USAGE.md, CONSOLE-REMAINING-WORK.md)
- Add test implementation alignment tracking (TEST-IMPLEMENTATION-ALIGNMENT.md)

This enables real-time test progress tracking via structured events
from shell scripts to the operator console UI.
2025-12-29 09:37:12 +00:00

4.7 KiB

Test Console Usage Guide

Overview

The Test Console is a Textual-inspired operator interface that provides a structured view of test execution, real-time progress tracking, and evidence management.

Features

Phase A (UI-Only Mode)

  • Test Plan Rendering: Visual tree of phases, tests, and steps
  • Step Checklists: Track progress with ✓/✗/⚠/→/· status indicators
  • Current Step Frame: Detailed instructions for the current step
  • Evidence Panel: View captured artifacts (alarms, logs, screenshots)
  • Live Feed: Real-time event stream
  • State Persistence: Progress saved to localStorage

Phase B (Live Updates)

  • ADB Broadcast Integration: Scripts send events via adb shell am broadcast
  • Real-Time Updates: Console updates automatically as tests run
  • Automatic Status: Step/test status updates from script events

Enabling Live Updates

To enable live event streaming from test scripts:

export DNP_UI_EVENTS=1
./test-phase1.sh

Test Context Setup

In test scripts, set context before executing steps:

# Set phase/test/step context
set_test_context "phase1" "phase1_test0" "p1_t0_s1"

# Emit step events
step_start "p1_t0_s1" "Starting step"
# ... do work ...
step_pass "p1_t0_s1" "Step completed"

Event Types

Step Events

  • step_start - Step execution begins
  • step_pass - Step completed successfully
  • step_warn - Step completed with warnings
  • step_fail - Step failed

Operator Events

  • operator_required - Manual action needed (from ui_prompt())

Evidence Events

  • artifact - Evidence captured (from capture_*() functions)

Verdict Events

  • test_verdict - Test completed (from verdict_*() functions)

Step ID Mapping

Step IDs follow the pattern: p{phase}_{test}_s{step}

Examples:

  • p1_t0_s1 = Phase 1, Test 0, Step 1
  • p2_t1_s3 = Phase 2, Test 1, Step 3
  • p3_t4_s2 = Phase 3, Test 4, Step 2

Console Navigation

  1. Select Test: Click a test in the left sidebar
  2. View Steps: Step checklist shows in the middle panel
  3. Follow Instructions: Current step frame shows what to do
  4. Mark Progress: Use "Mark Done ✓" or "Mark Fail ✗" buttons
  5. View Evidence: Evidence panel shows captured artifacts

Manual Step Completion

If scripts aren't driving the console (Phase A only), operators can manually mark steps:

  1. Select the step in the checklist
  2. Click "Mark Done ✓" or "Mark Fail ✗"
  3. Add notes if needed
  4. Progress is saved automatically

Evidence Management

Evidence is automatically captured when scripts call:

  • capture_alarms() → alarms/ directory
  • capture_logcat() → logs/ directory
  • capture_screenshot() → screens/ directory

Evidence appears in the console's evidence panel when:

  • Scripts emit artifact events (Phase B)
  • Operators manually add evidence (Phase A)

Run ID

Each console session generates a unique Run ID:

  • Format: YYYYMMDD_HHMMSS_xxxx
  • Used for localStorage keys and evidence organization
  • Displayed in console header

Troubleshooting

Events Not Appearing

  1. Check Event Emission: Ensure DNP_UI_EVENTS=1 is set
  2. Check ADB Connection: Verify device is connected
  3. Check Plugin: Ensure TestEventsPlugin is registered in Capacitor
  4. Check Broadcast: Verify broadcast action matches in plugin

Console Not Loading

  1. Check Plan JSON: Verify plan.json exists and is valid
  2. Check Browser Console: Look for JavaScript errors
  3. Check Capacitor: Ensure Capacitor is initialized

Steps Not Updating

  1. Check Context: Verify set_test_context() is called
  2. Check Step IDs: Ensure step IDs match plan.json
  3. Check Events: Verify events are being emitted

Example: Wiring a Test Function

test_example() {
  section "TEST: Example Test"
  
  # Set test context
  set_test_context "phase1" "phase1_example" ""
  
  # Step 1: Setup
  set_test_context "phase1" "phase1_example" "p1_ex_s1"
  step_start "p1_ex_s1" "Setting up test"
  capture_alarms "example_initial"
  step_pass "p1_ex_s1" "Setup complete"
  
  # Step 2: Execute
  set_test_context "phase1" "phase1_example" "p1_ex_s2"
  step_start "p1_ex_s2" "Executing test"
  # ... do work ...
  step_pass "p1_ex_s2" "Execution complete"
  
  # Step 3: Verify
  set_test_context "phase1" "phase1_example" "p1_ex_s3"
  step_start "p1_ex_s3" "Verifying results"
  # ... verify ...
  step_pass "p1_ex_s3" "Verification passed"
  
  # Verdict
  set_test_context "phase1" "phase1_example" "p1_ex_s4"
  verdict_pass "example_test" "Test passed"
}

Next Steps

  1. Wire Remaining Tests: Add set_test_context() and step_*() calls to all test functions
  2. Test Console: Build app, enable events, run a test script
  3. Refine UI: Adjust styling, add features as needed