diff --git a/.cursor/rules/harbor_pilot_universal.mdc b/.cursor/rules/harbor_pilot_universal.mdc index b3714c6c..91d099f7 100644 --- a/.cursor/rules/harbor_pilot_universal.mdc +++ b/.cursor/rules/harbor_pilot_universal.mdc @@ -203,4 +203,5 @@ Follow this exact order **after** the Base Contract’s **Objective β†’ Result - Respect Base *Do-Not* (no filler, no invented facts, no censorship). - Prefer clarity over completeness when timeboxed; capture unknowns explicitly. - Apply historical comment management rules (see `.cursor/rules/historical_comment_management.mdc`) -- Apply realistic time estimation rules (see `.cursor/rules/realistic_time_estimation.mdc`) \ No newline at end of file +- Apply realistic time estimation rules (see `.cursor/rules/realistic_time_estimation.mdc`) +- Apply Playwright test investigation rules (see `.cursor/rules/playwright_test_investigation.mdc`) \ No newline at end of file diff --git a/.cursor/rules/playwright-test-investigation.mdc b/.cursor/rules/playwright-test-investigation.mdc new file mode 100644 index 00000000..c7a97c9d --- /dev/null +++ b/.cursor/rules/playwright-test-investigation.mdc @@ -0,0 +1,356 @@ +--- +description: when working with playwright tests either generating them or using them to test code +alwaysApply: false +--- +# Playwright Test Investigation β€” Harbor Pilot Directive + +**Author**: Matthew Raymer +**Date**: 2025-08-21T14:22Z +**Status**: 🎯 **ACTIVE** - Playwright test debugging guidelines + +## Objective +Provide systematic approach for investigating Playwright test failures with focus on UI element conflicts, timing issues, and selector ambiguity. + +## Context & Scope +- **Audience**: Developers debugging Playwright test failures +- **In scope**: Test failure analysis, selector conflicts, UI state investigation, timing issues +- **Out of scope**: Test writing best practices, CI/CD configuration + +## Artifacts & Links +- Test results: `test-results/` directory +- Error context: `error-context.md` files with page snapshots +- Trace files: `trace.zip` files for failed tests +- HTML reports: Interactive test reports with screenshots + +## Environment & Preconditions +- OS/Runtime: Linux/Windows/macOS with Node.js +- Versions: Playwright test framework, browser drivers +- Services: Local test server (localhost:8080), test data setup +- Auth mode: None required for test investigation + +## Architecture / Process Overview +Playwright test investigation follows a systematic diagnostic workflow that leverages built-in debugging tools and error context analysis. + +```mermaid +flowchart TD + A[Test Failure] --> B[Check Error Context] + B --> C[Analyze Page Snapshot] + C --> D[Identify UI Conflicts] + D --> E[Check Trace Files] + E --> F[Verify Selector Uniqueness] + F --> G[Test Selector Fixes] + G --> H[Document Root Cause] + + B --> I[Check Test Results Directory] + I --> J[Locate Failed Test Results] + J --> K[Extract Error Details] + + D --> L[Multiple Alerts?] + L --> M[Button Text Conflicts?] + M --> N[Timing Issues?] + + E --> O[Use Trace Viewer] + O --> P[Analyze Action Sequence] + P --> Q[Identify Failure Point] +``` + +## Interfaces & Contracts + +### Test Results Structure +| Component | Format | Content | Validation | +|---|---|---|---| +| Error Context | Markdown | Page snapshot in YAML | Verify DOM state matches test expectations | +| Trace Files | ZIP archive | Detailed execution trace | Use `npx playwright show-trace` | +| HTML Reports | Interactive HTML | Screenshots, traces, logs | Check browser for full report | +| JSON Results | JSON | Machine-readable results | Parse for automated analysis | + +### Investigation Commands +| Step | Command | Expected Output | Notes | +|---|---|---|---| +| Locate failed tests | `find test-results -name "*test-name*"` | Test result directories | Use exact test name patterns | +| Check error context | `cat test-results/*/error-context.md` | Page snapshots | Look for UI state conflicts | +| View traces | `npx playwright show-trace trace.zip` | Interactive trace viewer | Analyze exact failure sequence | + +## Repro: End-to-End Investigation Procedure + +### 1. Locate Failed Test Results +```bash +# Find all results for a specific test +find test-results -name "*test-name*" -type d + +# Check for error context files +find test-results -name "error-context.md" | head -5 +``` + +### 2. Analyze Error Context +```bash +# Read error context for specific test +cat test-results/test-name-test-description-browser/error-context.md + +# Look for UI conflicts in page snapshot +grep -A 10 -B 5 "button.*Yes\|button.*No" test-results/*/error-context.md +``` + +### 3. Check Trace Files +```bash +# List available trace files +find test-results -name "*.zip" | grep trace + +# View trace in browser +npx playwright show-trace test-results/test-name/trace.zip +``` + +### 4. Investigate Selector Issues +```typescript +// Check for multiple elements with same text +await page.locator('button:has-text("Yes")').count(); // Should be 1 + +// Use more specific selectors +await page.locator('div[role="alert"]:has-text("Register") button:has-text("Yes")').click(); +``` + +## What Works (Evidence) +- βœ… **Error context files** provide page snapshots showing exact DOM state at failure + - **Time**: 2025-08-21T14:22Z + - **Evidence**: `test-results/60-new-activity-New-offers-for-another-user-chromium/error-context.md` shows both alerts visible + - **Verify at**: Error context files in test results directory + +- βœ… **Trace files** capture detailed execution sequence for failed tests + - **Time**: 2025-08-21T14:22Z + - **Evidence**: `trace.zip` files available for all failed tests + - **Verify at**: Use `npx playwright show-trace ` + +- βœ… **Page snapshots** reveal UI conflicts like multiple alerts with duplicate button text + - **Time**: 2025-08-21T14:22Z + - **Evidence**: YAML snapshots show registration + export alerts simultaneously + - **Verify at**: Error context markdown files + +## What Doesn't (Evidence & Hypotheses) +- ❌ **Generic selectors** fail with multiple similar elements at `test-playwright/testUtils.ts:161` + - **Time**: 2025-08-21T14:22Z + - **Evidence**: `button:has-text("Yes")` matches both "Yes" and "Yes, Export Data" + - **Hypothesis**: Selector ambiguity due to multiple alerts with conflicting button text + - **Next probe**: Use more specific selectors or dismiss alerts sequentially + +- ❌ **Timing-dependent tests** fail due to alert stacking at `src/views/ContactsView.vue:860,1283` + - **Time**: 2025-08-21T14:22Z + - **Evidence**: Both alerts use identical 1000ms delays, ensuring simultaneous display + - **Hypothesis**: Race condition between alert displays creates UI conflicts + - **Next probe**: Implement alert queuing or prevent overlapping alerts + +## Risks, Limits, Assumptions +- **Trace file size**: Large trace files may impact storage and analysis time +- **Browser compatibility**: Trace viewer requires specific browser support +- **Test isolation**: Shared state between tests may affect investigation results +- **Timing sensitivity**: Tests may pass/fail based on system performance + +## Next Steps +| Owner | Task | Exit Criteria | Target Date (UTC) | +|---|---|---|---| +| Development Team | Fix test selectors for multiple alerts | All tests pass consistently | 2025-08-22 | +| Development Team | Implement alert queuing system | No overlapping alerts with conflicting buttons | 2025-08-25 | +| Development Team | Add test IDs to alert buttons | Unique selectors for all UI elements | 2025-08-28 | + +## References +- [Playwright Trace Viewer Documentation](https://playwright.dev/docs/trace-viewer) +- [Playwright Test Results](https://playwright.dev/docs/test-reporters) +- [Test Investigation Workflow](./research_diagnostic.mdc) + +## Competence Hooks +- **Why this works**: Systematic investigation leverages Playwright's built-in debugging tools to identify root causes +- **Common pitfalls**: Generic selectors fail with multiple similar elements; timing issues create race conditions; alert stacking causes UI conflicts +- **Next skill unlock**: Implement unique test IDs and handle alert dismissal order in test flows +- **Teach-back**: "How would you investigate a Playwright test failure using error context, trace files, and page snapshots?" + +## Collaboration Hooks +- **Reviewers**: QA team, test automation engineers +- **Sign-off checklist**: Error context analyzed, trace files reviewed, root cause identified, fix implemented and tested + +## Assumptions & Limits +- Test results directory structure follows Playwright conventions +- Trace files are enabled in configuration (`trace: "retain-on-failure"`) +- Error context files contain valid YAML page snapshots +- Browser environment supports trace viewer functionality + +--- + +**Status**: Active investigation directive +**Priority**: High +**Maintainer**: Development team +**Next Review**: 2025-09-21 +# Playwright Test Investigation β€” Harbor Pilot Directive + +**Author**: Matthew Raymer +**Date**: 2025-08-21T14:22Z +**Status**: 🎯 **ACTIVE** - Playwright test debugging guidelines + +## Objective +Provide systematic approach for investigating Playwright test failures with focus on UI element conflicts, timing issues, and selector ambiguity. + +## Context & Scope +- **Audience**: Developers debugging Playwright test failures +- **In scope**: Test failure analysis, selector conflicts, UI state investigation, timing issues +- **Out of scope**: Test writing best practices, CI/CD configuration + +## Artifacts & Links +- Test results: `test-results/` directory +- Error context: `error-context.md` files with page snapshots +- Trace files: `trace.zip` files for failed tests +- HTML reports: Interactive test reports with screenshots + +## Environment & Preconditions +- OS/Runtime: Linux/Windows/macOS with Node.js +- Versions: Playwright test framework, browser drivers +- Services: Local test server (localhost:8080), test data setup +- Auth mode: None required for test investigation + +## Architecture / Process Overview +Playwright test investigation follows a systematic diagnostic workflow that leverages built-in debugging tools and error context analysis. + +```mermaid +flowchart TD + A[Test Failure] --> B[Check Error Context] + B --> C[Analyze Page Snapshot] + C --> D[Identify UI Conflicts] + D --> E[Check Trace Files] + E --> F[Verify Selector Uniqueness] + F --> G[Test Selector Fixes] + G --> H[Document Root Cause] + + B --> I[Check Test Results Directory] + I --> J[Locate Failed Test Results] + J --> K[Extract Error Details] + + D --> L[Multiple Alerts?] + L --> M[Button Text Conflicts?] + M --> N[Timing Issues?] + + E --> O[Use Trace Viewer] + O --> P[Analyze Action Sequence] + P --> Q[Identify Failure Point] +``` + +## Interfaces & Contracts + +### Test Results Structure +| Component | Format | Content | Validation | +|---|---|---|---| +| Error Context | Markdown | Page snapshot in YAML | Verify DOM state matches test expectations | +| Trace Files | ZIP archive | Detailed execution trace | Use `npx playwright show-trace` | +| HTML Reports | Interactive HTML | Screenshots, traces, logs | Check browser for full report | +| JSON Results | JSON | Machine-readable results | Parse for automated analysis | + +### Investigation Commands +| Step | Command | Expected Output | Notes | +|---|---|---|---| +| Locate failed tests | `find test-results -name "*test-name*"` | Test result directories | Use exact test name patterns | +| Check error context | `cat test-results/*/error-context.md` | Page snapshots | Look for UI state conflicts | +| View traces | `npx playwright show-trace trace.zip` | Interactive trace viewer | Analyze exact failure sequence | + +## Repro: End-to-End Investigation Procedure + +### 1. Locate Failed Test Results +```bash +# Find all results for a specific test +find test-results -name "*test-name*" -type d + +# Check for error context files +find test-results -name "error-context.md" | head -5 +``` + +### 2. Analyze Error Context +```bash +# Read error context for specific test +cat test-results/test-name-test-description-browser/error-context.md + +# Look for UI conflicts in page snapshot +grep -A 10 -B 5 "button.*Yes\|button.*No" test-results/*/error-context.md +``` + +### 3. Check Trace Files +```bash +# List available trace files +find test-results -name "*.zip" | grep trace + +# View trace in browser +npx playwright show-trace test-results/test-name/trace.zip +``` + +### 4. Investigate Selector Issues +```typescript +// Check for multiple elements with same text +await page.locator('button:has-text("Yes")').count(); // Should be 1 + +// Use more specific selectors +await page.locator('div[role="alert"]:has-text("Register") button:has-text("Yes")').click(); +``` + +## What Works (Evidence) +- βœ… **Error context files** provide page snapshots showing exact DOM state at failure + - **Time**: 2025-08-21T14:22Z + - **Evidence**: `test-results/60-new-activity-New-offers-for-another-user-chromium/error-context.md` shows both alerts visible + - **Verify at**: Error context files in test results directory + +- βœ… **Trace files** capture detailed execution sequence for failed tests + - **Time**: 2025-08-21T14:22Z + - **Evidence**: `trace.zip` files available for all failed tests + - **Verify at**: Use `npx playwright show-trace ` + +- βœ… **Page snapshots** reveal UI conflicts like multiple alerts with duplicate button text + - **Time**: 2025-08-21T14:22Z + - **Evidence**: YAML snapshots show registration + export alerts simultaneously + - **Verify at**: Error context markdown files + +## What Doesn't (Evidence & Hypotheses) +- ❌ **Generic selectors** fail with multiple similar elements at `test-playwright/testUtils.ts:161` + - **Time**: 2025-08-21T14:22Z + - **Evidence**: `button:has-text("Yes")` matches both "Yes" and "Yes, Export Data" + - **Hypothesis**: Selector ambiguity due to multiple alerts with conflicting button text + - **Next probe**: Use more specific selectors or dismiss alerts sequentially + +- ❌ **Timing-dependent tests** fail due to alert stacking at `src/views/ContactsView.vue:860,1283` + - **Time**: 2025-08-21T14:22Z + - **Evidence**: Both alerts use identical 1000ms delays, ensuring simultaneous display + - **Hypothesis**: Race condition between alert displays creates UI conflicts + - **Next probe**: Implement alert queuing or prevent overlapping alerts + +## Risks, Limits, Assumptions +- **Trace file size**: Large trace files may impact storage and analysis time +- **Browser compatibility**: Trace viewer requires specific browser support +- **Test isolation**: Shared state between tests may affect investigation results +- **Timing sensitivity**: Tests may pass/fail based on system performance + +## Next Steps +| Owner | Task | Exit Criteria | Target Date (UTC) | +|---|---|---|---| +| Development Team | Fix test selectors for multiple alerts | All tests pass consistently | 2025-08-22 | +| Development Team | Implement alert queuing system | No overlapping alerts with conflicting buttons | 2025-08-25 | +| Development Team | Add test IDs to alert buttons | Unique selectors for all UI elements | 2025-08-28 | + +## References +- [Playwright Trace Viewer Documentation](https://playwright.dev/docs/trace-viewer) +- [Playwright Test Results](https://playwright.dev/docs/test-reporters) +- [Test Investigation Workflow](./research_diagnostic.mdc) + +## Competence Hooks +- **Why this works**: Systematic investigation leverages Playwright's built-in debugging tools to identify root causes +- **Common pitfalls**: Generic selectors fail with multiple similar elements; timing issues create race conditions; alert stacking causes UI conflicts +- **Next skill unlock**: Implement unique test IDs and handle alert dismissal order in test flows +- **Teach-back**: "How would you investigate a Playwright test failure using error context, trace files, and page snapshots?" + +## Collaboration Hooks +- **Reviewers**: QA team, test automation engineers +- **Sign-off checklist**: Error context analyzed, trace files reviewed, root cause identified, fix implemented and tested + +## Assumptions & Limits +- Test results directory structure follows Playwright conventions +- Trace files are enabled in configuration (`trace: "retain-on-failure"`) +- Error context files contain valid YAML page snapshots +- Browser environment supports trace viewer functionality + +--- + +**Status**: Active investigation directive +**Priority**: High +**Maintainer**: Development team +**Next Review**: 2025-09-21