feat(harbor-pilot): add Playwright test investigation directive

- Create comprehensive MDC rule for systematic Playwright test failure investigation - Integrate rule into harbor_pilot_universal.mdc for team-wide access - Include investigation workflow, diagnostic commands, and evidence-based analysis - Document specific failure patterns (alert stacking, selector conflicts, timing issues) - Provide practical examples from recent test failure investigation - Add investigation commands for error context, trace files, and page snapshots This rule transforms ad-hoc test debugging into systematic investigation process, leveraging Playwright's built-in debugging tools for faster root cause identification.
20 hours ago · 4391cb2881
2 changed files with 358 additions and 1 deletions
--- a/.cursor/rules/harbor_pilot_universal.mdc
+++ b/.cursor/rules/harbor_pilot_universal.mdc
@ -204,3 +204,4 @@ Follow this exact order **after** the Base Contract’s **Objective → Result
 - Prefer clarity over completeness when timeboxed; capture unknowns explicitly.
 - Apply historical comment management rules (see `.cursor/rules/historical_comment_management.mdc`)
 - Apply realistic time estimation rules (see `.cursor/rules/realistic_time_estimation.mdc`)
+- Apply Playwright test investigation rules (see `.cursor/rules/playwright_test_investigation.mdc`)
--- a/.cursor/rules/playwright-test-investigation.mdc
+++ b/.cursor/rules/playwright-test-investigation.mdc
@ -0,0 +1,356 @@
+---
+description: when working with playwright tests either generating them or using them to test code
+alwaysApply: false
+---
+# Playwright Test Investigation — Harbor Pilot Directive
+
+**Author**: Matthew Raymer  
+**Date**: 2025-08-21T14:22Z  
+**Status**: 🎯 **ACTIVE** - Playwright test debugging guidelines
+
+## Objective
+Provide systematic approach for investigating Playwright test failures with focus on UI element conflicts, timing issues, and selector ambiguity.
+
+## Context & Scope
+- **Audience**: Developers debugging Playwright test failures
+- **In scope**: Test failure analysis, selector conflicts, UI state investigation, timing issues
+- **Out of scope**: Test writing best practices, CI/CD configuration
+
+## Artifacts & Links
+- Test results: `test-results/` directory
+- Error context: `error-context.md` files with page snapshots
+- Trace files: `trace.zip` files for failed tests
+- HTML reports: Interactive test reports with screenshots
+
+## Environment & Preconditions
+- OS/Runtime: Linux/Windows/macOS with Node.js
+- Versions: Playwright test framework, browser drivers
+- Services: Local test server (localhost:8080), test data setup
+- Auth mode: None required for test investigation
+
+## Architecture / Process Overview
+Playwright test investigation follows a systematic diagnostic workflow that leverages built-in debugging tools and error context analysis.
+
+```mermaid
+flowchart TD
+    A[Test Failure] --> B[Check Error Context]
+    B --> C[Analyze Page Snapshot]
+    C --> D[Identify UI Conflicts]
+    D --> E[Check Trace Files]
+    E --> F[Verify Selector Uniqueness]
+    F --> G[Test Selector Fixes]
+    G --> H[Document Root Cause]
+    
+    B --> I[Check Test Results Directory]
+    I --> J[Locate Failed Test Results]
+    J --> K[Extract Error Details]
+    
+    D --> L[Multiple Alerts?]
+    L --> M[Button Text Conflicts?]
+    M --> N[Timing Issues?]
+    
+    E --> O[Use Trace Viewer]
+    O --> P[Analyze Action Sequence]
+    P --> Q[Identify Failure Point]
+```
+
+## Interfaces & Contracts
+
+### Test Results Structure
+| Component | Format | Content | Validation |
+|---|---|---|---|
+| Error Context | Markdown | Page snapshot in YAML | Verify DOM state matches test expectations |
+| Trace Files | ZIP archive | Detailed execution trace | Use `npx playwright show-trace` |
+| HTML Reports | Interactive HTML | Screenshots, traces, logs | Check browser for full report |
+| JSON Results | JSON | Machine-readable results | Parse for automated analysis |
+
+### Investigation Commands
+| Step | Command | Expected Output | Notes |
+|---|---|---|---|
+| Locate failed tests | `find test-results -name "*test-name*"` | Test result directories | Use exact test name patterns |
+| Check error context | `cat test-results/*/error-context.md` | Page snapshots | Look for UI state conflicts |
+| View traces | `npx playwright show-trace trace.zip` | Interactive trace viewer | Analyze exact failure sequence |
+
+## Repro: End-to-End Investigation Procedure
+
+### 1. Locate Failed Test Results
+```bash
+# Find all results for a specific test
+find test-results -name "*test-name*" -type d
+
+# Check for error context files
+find test-results -name "error-context.md" | head -5
+```
+
+### 2. Analyze Error Context
+```bash
+# Read error context for specific test
+cat test-results/test-name-test-description-browser/error-context.md
+
+# Look for UI conflicts in page snapshot
+grep -A 10 -B 5 "button.*Yes\|button.*No" test-results/*/error-context.md
+```
+
+### 3. Check Trace Files
+```bash
+# List available trace files
+find test-results -name "*.zip" | grep trace
+
+# View trace in browser
+npx playwright show-trace test-results/test-name/trace.zip
+```
+
+### 4. Investigate Selector Issues
+```typescript
+// Check for multiple elements with same text
+await page.locator('button:has-text("Yes")').count(); // Should be 1
+
+// Use more specific selectors
+await page.locator('div[role="alert"]:has-text("Register") button:has-text("Yes")').click();
+```
+
+## What Works (Evidence)
+- ✅ **Error context files** provide page snapshots showing exact DOM state at failure  
+  - **Time**: 2025-08-21T14:22Z  
+  - **Evidence**: `test-results/60-new-activity-New-offers-for-another-user-chromium/error-context.md` shows both alerts visible  
+  - **Verify at**: Error context files in test results directory
+
+- ✅ **Trace files** capture detailed execution sequence for failed tests  
+  - **Time**: 2025-08-21T14:22Z  
+  - **Evidence**: `trace.zip` files available for all failed tests  
+  - **Verify at**: Use `npx playwright show-trace <filename>`
+
+- ✅ **Page snapshots** reveal UI conflicts like multiple alerts with duplicate button text  
+  - **Time**: 2025-08-21T14:22Z  
+  - **Evidence**: YAML snapshots show registration + export alerts simultaneously  
+  - **Verify at**: Error context markdown files
+
+## What Doesn't (Evidence & Hypotheses)
+- ❌ **Generic selectors** fail with multiple similar elements at `test-playwright/testUtils.ts:161`  
+  - **Time**: 2025-08-21T14:22Z  
+  - **Evidence**: `button:has-text("Yes")` matches both "Yes" and "Yes, Export Data"  
+  - **Hypothesis**: Selector ambiguity due to multiple alerts with conflicting button text  
+  - **Next probe**: Use more specific selectors or dismiss alerts sequentially
+
+- ❌ **Timing-dependent tests** fail due to alert stacking at `src/views/ContactsView.vue:860,1283`  
+  - **Time**: 2025-08-21T14:22Z  
+  - **Evidence**: Both alerts use identical 1000ms delays, ensuring simultaneous display  
+  - **Hypothesis**: Race condition between alert displays creates UI conflicts  
+  - **Next probe**: Implement alert queuing or prevent overlapping alerts
+
+## Risks, Limits, Assumptions
+- **Trace file size**: Large trace files may impact storage and analysis time
+- **Browser compatibility**: Trace viewer requires specific browser support
+- **Test isolation**: Shared state between tests may affect investigation results
+- **Timing sensitivity**: Tests may pass/fail based on system performance
+
+## Next Steps
+| Owner | Task | Exit Criteria | Target Date (UTC) |
+|---|---|---|---|
+| Development Team | Fix test selectors for multiple alerts | All tests pass consistently | 2025-08-22 |
+| Development Team | Implement alert queuing system | No overlapping alerts with conflicting buttons | 2025-08-25 |
+| Development Team | Add test IDs to alert buttons | Unique selectors for all UI elements | 2025-08-28 |
+
+## References
+- [Playwright Trace Viewer Documentation](https://playwright.dev/docs/trace-viewer)
+- [Playwright Test Results](https://playwright.dev/docs/test-reporters)
+- [Test Investigation Workflow](./research_diagnostic.mdc)
+
+## Competence Hooks
+- **Why this works**: Systematic investigation leverages Playwright's built-in debugging tools to identify root causes
+- **Common pitfalls**: Generic selectors fail with multiple similar elements; timing issues create race conditions; alert stacking causes UI conflicts
+- **Next skill unlock**: Implement unique test IDs and handle alert dismissal order in test flows
+- **Teach-back**: "How would you investigate a Playwright test failure using error context, trace files, and page snapshots?"
+
+## Collaboration Hooks
+- **Reviewers**: QA team, test automation engineers
+- **Sign-off checklist**: Error context analyzed, trace files reviewed, root cause identified, fix implemented and tested
+
+## Assumptions & Limits
+- Test results directory structure follows Playwright conventions
+- Trace files are enabled in configuration (`trace: "retain-on-failure"`)
+- Error context files contain valid YAML page snapshots
+- Browser environment supports trace viewer functionality
+
+---
+
+**Status**: Active investigation directive  
+**Priority**: High  
+**Maintainer**: Development team  
+**Next Review**: 2025-09-21
+# Playwright Test Investigation — Harbor Pilot Directive
+
+**Author**: Matthew Raymer  
+**Date**: 2025-08-21T14:22Z  
+**Status**: 🎯 **ACTIVE** - Playwright test debugging guidelines
+
+## Objective
+Provide systematic approach for investigating Playwright test failures with focus on UI element conflicts, timing issues, and selector ambiguity.
+
+## Context & Scope
+- **Audience**: Developers debugging Playwright test failures
+- **In scope**: Test failure analysis, selector conflicts, UI state investigation, timing issues
+- **Out of scope**: Test writing best practices, CI/CD configuration
+
+## Artifacts & Links
+- Test results: `test-results/` directory
+- Error context: `error-context.md` files with page snapshots
+- Trace files: `trace.zip` files for failed tests
+- HTML reports: Interactive test reports with screenshots
+
+## Environment & Preconditions
+- OS/Runtime: Linux/Windows/macOS with Node.js
+- Versions: Playwright test framework, browser drivers
+- Services: Local test server (localhost:8080), test data setup
+- Auth mode: None required for test investigation
+
+## Architecture / Process Overview
+Playwright test investigation follows a systematic diagnostic workflow that leverages built-in debugging tools and error context analysis.
+
+```mermaid
+flowchart TD
+    A[Test Failure] --> B[Check Error Context]
+    B --> C[Analyze Page Snapshot]
+    C --> D[Identify UI Conflicts]
+    D --> E[Check Trace Files]
+    E --> F[Verify Selector Uniqueness]
+    F --> G[Test Selector Fixes]
+    G --> H[Document Root Cause]
+    
+    B --> I[Check Test Results Directory]
+    I --> J[Locate Failed Test Results]
+    J --> K[Extract Error Details]
+    
+    D --> L[Multiple Alerts?]
+    L --> M[Button Text Conflicts?]
+    M --> N[Timing Issues?]
+    
+    E --> O[Use Trace Viewer]
+    O --> P[Analyze Action Sequence]
+    P --> Q[Identify Failure Point]
+```
+
+## Interfaces & Contracts
+
+### Test Results Structure
+| Component | Format | Content | Validation |
+|---|---|---|---|
+| Error Context | Markdown | Page snapshot in YAML | Verify DOM state matches test expectations |
+| Trace Files | ZIP archive | Detailed execution trace | Use `npx playwright show-trace` |
+| HTML Reports | Interactive HTML | Screenshots, traces, logs | Check browser for full report |
+| JSON Results | JSON | Machine-readable results | Parse for automated analysis |
+
+### Investigation Commands
+| Step | Command | Expected Output | Notes |
+|---|---|---|---|
+| Locate failed tests | `find test-results -name "*test-name*"` | Test result directories | Use exact test name patterns |
+| Check error context | `cat test-results/*/error-context.md` | Page snapshots | Look for UI state conflicts |
+| View traces | `npx playwright show-trace trace.zip` | Interactive trace viewer | Analyze exact failure sequence |
+
+## Repro: End-to-End Investigation Procedure
+
+### 1. Locate Failed Test Results
+```bash
+# Find all results for a specific test
+find test-results -name "*test-name*" -type d
+
+# Check for error context files
+find test-results -name "error-context.md" | head -5
+```
+
+### 2. Analyze Error Context
+```bash
+# Read error context for specific test
+cat test-results/test-name-test-description-browser/error-context.md
+
+# Look for UI conflicts in page snapshot
+grep -A 10 -B 5 "button.*Yes\|button.*No" test-results/*/error-context.md
+```
+
+### 3. Check Trace Files
+```bash
+# List available trace files
+find test-results -name "*.zip" | grep trace
+
+# View trace in browser
+npx playwright show-trace test-results/test-name/trace.zip
+```
+
+### 4. Investigate Selector Issues
+```typescript
+// Check for multiple elements with same text
+await page.locator('button:has-text("Yes")').count(); // Should be 1
+
+// Use more specific selectors
+await page.locator('div[role="alert"]:has-text("Register") button:has-text("Yes")').click();
+```
+
+## What Works (Evidence)
+- ✅ **Error context files** provide page snapshots showing exact DOM state at failure  
+  - **Time**: 2025-08-21T14:22Z  
+  - **Evidence**: `test-results/60-new-activity-New-offers-for-another-user-chromium/error-context.md` shows both alerts visible  
+  - **Verify at**: Error context files in test results directory
+
+- ✅ **Trace files** capture detailed execution sequence for failed tests  
+  - **Time**: 2025-08-21T14:22Z  
+  - **Evidence**: `trace.zip` files available for all failed tests  
+  - **Verify at**: Use `npx playwright show-trace <filename>`
+
+- ✅ **Page snapshots** reveal UI conflicts like multiple alerts with duplicate button text  
+  - **Time**: 2025-08-21T14:22Z  
+  - **Evidence**: YAML snapshots show registration + export alerts simultaneously  
+  - **Verify at**: Error context markdown files
+
+## What Doesn't (Evidence & Hypotheses)
+- ❌ **Generic selectors** fail with multiple similar elements at `test-playwright/testUtils.ts:161`  
+  - **Time**: 2025-08-21T14:22Z  
+  - **Evidence**: `button:has-text("Yes")` matches both "Yes" and "Yes, Export Data"  
+  - **Hypothesis**: Selector ambiguity due to multiple alerts with conflicting button text  
+  - **Next probe**: Use more specific selectors or dismiss alerts sequentially
+
+- ❌ **Timing-dependent tests** fail due to alert stacking at `src/views/ContactsView.vue:860,1283`  
+  - **Time**: 2025-08-21T14:22Z  
+  - **Evidence**: Both alerts use identical 1000ms delays, ensuring simultaneous display  
+  - **Hypothesis**: Race condition between alert displays creates UI conflicts  
+  - **Next probe**: Implement alert queuing or prevent overlapping alerts
+
+## Risks, Limits, Assumptions
+- **Trace file size**: Large trace files may impact storage and analysis time
+- **Browser compatibility**: Trace viewer requires specific browser support
+- **Test isolation**: Shared state between tests may affect investigation results
+- **Timing sensitivity**: Tests may pass/fail based on system performance
+
+## Next Steps
+| Owner | Task | Exit Criteria | Target Date (UTC) |
+|---|---|---|---|
+| Development Team | Fix test selectors for multiple alerts | All tests pass consistently | 2025-08-22 |
+| Development Team | Implement alert queuing system | No overlapping alerts with conflicting buttons | 2025-08-25 |
+| Development Team | Add test IDs to alert buttons | Unique selectors for all UI elements | 2025-08-28 |
+
+## References
+- [Playwright Trace Viewer Documentation](https://playwright.dev/docs/trace-viewer)
+- [Playwright Test Results](https://playwright.dev/docs/test-reporters)
+- [Test Investigation Workflow](./research_diagnostic.mdc)
+
+## Competence Hooks
+- **Why this works**: Systematic investigation leverages Playwright's built-in debugging tools to identify root causes
+- **Common pitfalls**: Generic selectors fail with multiple similar elements; timing issues create race conditions; alert stacking causes UI conflicts
+- **Next skill unlock**: Implement unique test IDs and handle alert dismissal order in test flows
+- **Teach-back**: "How would you investigate a Playwright test failure using error context, trace files, and page snapshots?"
+
+## Collaboration Hooks
+- **Reviewers**: QA team, test automation engineers
+- **Sign-off checklist**: Error context analyzed, trace files reviewed, root cause identified, fix implemented and tested
+
+## Assumptions & Limits
+- Test results directory structure follows Playwright conventions
+- Trace files are enabled in configuration (`trace: "retain-on-failure"`)
+- Error context files contain valid YAML page snapshots
+- Browser environment supports trace viewer functionality
+
+---
+
+**Status**: Active investigation directive  
+**Priority**: High  
+**Maintainer**: Development team  
+**Next Review**: 2025-09-21