Browse Source

docs: add comprehensive testing infrastructure for P0 features

- Add comprehensive-testing-guide-v2.md with detailed test procedures
- Add testing-quick-reference-v2.md for quick access to test commands
- Add reboot-test-v2.sh for automated reboot testing
- Covers all P0 production-grade features:
  * Channel management (P0 Priority 1)
  * PendingIntent flags & exact alarms (P0 Priority 2)
  * JIT freshness re-check (P0 Priority 3)
  * Recovery coexistence (P0 Priority 4)
- Includes manual and automated testing procedures
- Ready for comprehensive test run after P0 completion
master
Matthew Raymer 1 week ago
parent
commit
6dc714acb2
  1. 1109
      docs/comprehensive-testing-guide-v2.md
  2. 280
      docs/testing-quick-reference-v2.md
  3. 359
      scripts/reboot-test-v2.sh

1109
docs/comprehensive-testing-guide-v2.md

File diff suppressed because it is too large

280
docs/testing-quick-reference-v2.md

@ -0,0 +1,280 @@
# Testing Quick Reference - P0 Production-Grade Features
## Current Version Features
**P0 Priority 1**: Channel Management (ChannelManager)
**P0 Priority 2**: PendingIntent & Exact Alarms (PendingIntentManager)
**P0 Priority 3**: JIT Freshness Re-check (Soft TTL)
**P0 Priority 4**: Recovery Coexistence (RecoveryManager)
## Quick Test Commands
### 1. Basic Functionality Test
```bash
# Launch app
adb shell am start -n com.timesafari.dailynotification/.MainActivity
# Check channel status
adb shell "dumpsys notification | grep -A5 daily_default"
# Check alarms
adb shell "dumpsys alarm | grep timesafari"
# Check exact alarm permission
adb shell "dumpsys alarm | grep SCHEDULE_EXACT_ALARM"
```
### 2. Channel Management Test
```bash
# Check channel exists and is enabled
adb shell "dumpsys notification | grep daily_default"
# Check channel importance (should be 3 = IMPORTANCE_HIGH)
adb shell "dumpsys notification | grep -A5 daily_default | grep importance"
# Open channel settings
adb shell "am start -a android.settings.CHANNEL_NOTIFICATION_SETTINGS -e android.provider.extra.APP_PACKAGE com.timesafari.dailynotification -e android.provider.extra.CHANNEL_ID daily_default"
```
### 3. PendingIntent & Exact Alarms Test
```bash
# Check PendingIntent flags in alarms
adb shell "dumpsys alarm | grep -A10 -B5 timesafari"
# Check exact alarm permission
adb shell "dumpsys alarm | grep SCHEDULE_EXACT_ALARM"
# Open exact alarm settings
adb shell "am start -a android.settings.REQUEST_SCHEDULE_EXACT_ALARM -d package:com.timesafari.dailynotification"
```
### 4. JIT Freshness Re-check Test
```bash
# Clear logs
adb logcat -c
# Schedule notification in app UI, then monitor logs
adb logcat | grep -i "jit\|freshness\|stale"
# Look for:
# - "Content is fresh (age: X minutes), skipping JIT refresh"
# - "Content is stale (age: X minutes), attempting JIT refresh"
# - "JIT refresh succeeded" or "JIT refresh failed, using original content"
```
### 5. Recovery Coexistence Test
```bash
# Clear logs
adb logcat -c
# Launch app multiple times to test cooldown
adb shell am start -n com.timesafari.dailynotification/.MainActivity
sleep 2
adb shell am start -n com.timesafari.dailynotification/.MainActivity
sleep 2
adb shell am start -n com.timesafari.dailynotification/.MainActivity
# Check recovery logs
adb logcat -d | grep -i "recovery.*requested.*app_startup"
adb logcat -d | grep -i "recovery.*performed.*recently.*skipping"
```
### 6. Reboot Recovery Test
```bash
# Schedule notification in app UI first
# Then reboot
adb reboot
# Wait for boot
adb wait-for-device
sleep 30
# Check recovery logs
adb logcat -d | grep -i "recovery.*requested.*boot_completed"
# Check alarms restored
adb shell "dumpsys alarm | grep timesafari"
```
## Automated Test Scripts
### Comprehensive Test Suite
```bash
# Run comprehensive test suite
./scripts/comprehensive-test-v2.sh
```
### Reboot Test Suite
```bash
# Run reboot test suite
./scripts/reboot-test-v2.sh
```
### Python Test Suite
```bash
# Run Python test suite
python3 scripts/daily-notification-test.py
```
## Expected Log Patterns
### Channel Management
```
ChannelManager: Creating new notification channel: daily_default
ChannelManager: Notification channel created with IMPORTANCE_HIGH
```
### PendingIntent & Exact Alarms
```
PendingIntentManager: Scheduled exact alarm with setExactAndAllowWhileIdle
PendingIntentManager: AlarmStatus{exactAlarmsSupported=true, exactAlarmsGranted=true}
```
### JIT Freshness Re-check
```
DailyNotificationReceiver: Content is fresh (age: 5 minutes), skipping JIT refresh
DailyNotificationReceiver: Content is stale (age: 7 hours), attempting JIT refresh
DailyNotificationReceiver: JIT refresh succeeded, using fresh content
```
### Recovery Coexistence
```
RecoveryManager: Recovery requested from: APP_STARTUP
RecoveryManager: Recovery performed recently (2s ago), skipping
RecoveryManager: Recovery requested from: BOOT_COMPLETED
RecoveryManager: Recovery completed from BOOT_COMPLETED: 3/5 recovered, 2 skipped
```
## Success Criteria
### ✅ All Tests Pass When:
1. **Channel Management**:
- Channel exists with ID `daily_default`
- Channel importance is `IMPORTANCE_HIGH` (3)
- Channel blocking detection works
- Settings deep linking works
2. **PendingIntent & Exact Alarms**:
- PendingIntent uses `FLAG_UPDATE_CURRENT | FLAG_IMMUTABLE`
- Exact alarm permission detection works
- Settings deep linking works
- Alarms schedule correctly
3. **JIT Freshness Re-check**:
- Fresh content (< 6 hours) skips refresh
- Stale content (> 6 hours) attempts refresh
- Graceful fallback on refresh failure
- Original content preserved on failure
4. **Recovery Coexistence**:
- App startup recovery works
- Boot recovery works
- 5-minute cooldown prevents duplicate recovery
- Recovery statistics tracked correctly
- State persisted in SharedPreferences
### ❌ Tests Fail When:
- Channel not created or blocked
- PendingIntent flags incorrect
- JIT refresh not working
- Recovery operations conflict
- Boot recovery not triggered
- Alarms not restored after reboot
## Troubleshooting Commands
### Debug Channel Issues
```bash
# Check channel status
adb shell "dumpsys notification | grep -A10 daily_default"
# Check channel importance
adb shell "dumpsys notification | grep -A5 daily_default | grep importance"
```
### Debug Alarm Issues
```bash
# Check scheduled alarms
adb shell "dumpsys alarm | grep timesafari"
# Check exact alarm permission
adb shell "dumpsys alarm | grep SCHEDULE_EXACT_ALARM"
# Check alarm manager state
adb shell "dumpsys alarm | head -20"
```
### Debug Recovery Issues
```bash
# Check recovery logs
adb logcat -d | grep -i "recovery.*requested"
# Check recovery state
adb shell "run-as com.timesafari.dailynotification ls -la /data/data/com.timesafari.dailynotification/shared_prefs/"
# Check boot receiver registration
adb shell "dumpsys package com.timesafari.dailynotification | grep -A5 -B5 receiver"
```
### Debug JIT Freshness Issues
```bash
# Check JIT refresh logs
adb logcat -d | grep -i "jit\|freshness\|stale"
# Check content age calculation
adb logcat -d | grep -i "age.*minutes"
```
## Manual Testing Checklist
### Pre-Test Setup
- [ ] Device/emulator connected via ADB
- [ ] App installed and permissions granted
- [ ] Notification permissions enabled
- [ ] Exact alarm permissions granted (Android 12+)
### Basic Tests
- [ ] App launches successfully
- [ ] Plugin loads without errors
- [ ] Channel created with correct settings
- [ ] Notifications schedule correctly
- [ ] Notifications appear at scheduled time
### P0 Feature Tests
- [ ] Channel management works
- [ ] PendingIntent flags correct
- [ ] Exact alarm permission detection
- [ ] JIT freshness re-check works
- [ ] Recovery coexistence works
- [ ] Boot recovery works
### Edge Case Tests
- [ ] App background notification
- [ ] App closed notification
- [ ] Force stop (expected failure)
- [ ] Reboot recovery
- [ ] Multiple notifications
- [ ] Network connectivity issues
### Performance Tests
- [ ] Memory usage stable
- [ ] CPU usage reasonable
- [ ] No memory leaks
- [ ] Efficient recovery operations
## Quick Status Check
```bash
# One-liner to check all systems
echo "=== Channel Status ===" && adb shell "dumpsys notification | grep -A3 daily_default" && echo "=== Alarm Status ===" && adb shell "dumpsys alarm | grep timesafari" && echo "=== Recovery Status ===" && adb logcat -d | grep -i "recovery.*requested" | tail -3
```
This quick reference covers all the essential testing procedures for the current P0 production-grade version of the DailyNotification plugin.

359
scripts/reboot-test-v2.sh

@ -0,0 +1,359 @@
#!/bin/bash
# reboot-test-v2.sh - Enhanced Reboot Testing with RecoveryManager
set -e
APP_PACKAGE="com.timesafari.dailynotification"
APP_ACTIVITY=".MainActivity"
echo "🔄 DailyNotification Reboot Test Suite v2.0"
echo "=========================================="
echo "Testing RecoveryManager-based reboot recovery"
echo "✅ Boot Receiver with RecoveryManager"
echo "✅ App Startup Recovery Coexistence"
echo "✅ Recovery Statistics Tracking"
echo "✅ Idempotent Recovery Operations"
echo ""
# Test results tracking
declare -A test_results
# Function to run test and track result
run_test() {
local test_name="$1"
local test_function="$2"
echo "🧪 Running: $test_name"
echo "----------------------------------------"
if $test_function; then
echo "✅ PASS: $test_name"
test_results["$test_name"]="PASS"
else
echo "❌ FAIL: $test_name"
test_results["$test_name"]="FAIL"
fi
echo ""
}
# Function to wait for device
wait_for_device() {
echo "⏳ Waiting for device to be ready..."
adb wait-for-device
sleep 15 # Additional wait for boot completion
# Check if device is fully booted
local boot_completed=$(adb shell getprop sys.boot_completed)
if [ "$boot_completed" != "1" ]; then
echo "⏳ Device not fully booted, waiting more..."
sleep 15
fi
}
# Function to schedule notification
schedule_notification() {
echo "📅 Scheduling test notification..."
adb shell am start -n $APP_PACKAGE/$APP_ACTIVITY
sleep 3
echo "⚠️ Manual step: Schedule notification in app"
echo " - Tap 'Test Notification' button"
echo " - Wait for 'Notification scheduled' message"
read -p "Press Enter when notification is scheduled..."
}
# Function to check recovery logs
check_recovery_logs() {
local source="$1"
echo "🔍 Checking recovery logs for: $source"
if adb logcat -d | grep -q "Recovery requested from: $source"; then
echo "✅ Recovery triggered from: $source"
return 0
else
echo "❌ Recovery not triggered from: $source"
return 1
fi
}
# Function to check recovery cooldown
check_recovery_cooldown() {
echo "🔍 Checking recovery cooldown behavior..."
local recovery_count=$(adb logcat -d | grep -c "Recovery requested from: APP_STARTUP")
local cooldown_count=$(adb logcat -d | grep -c "Recovery performed recently.*skipping")
if [ $recovery_count -eq 1 ] && [ $cooldown_count -gt 0 ]; then
echo "✅ Recovery cooldown working correctly"
echo " - Recovery performed: $recovery_count times"
echo " - Cooldown skips: $cooldown_count times"
return 0
else
echo "❌ Recovery cooldown not working"
echo " - Recovery performed: $recovery_count times"
echo " - Cooldown skips: $cooldown_count times"
return 1
fi
}
# Function to check scheduled alarms
check_scheduled_alarms() {
echo "⏰ Checking scheduled alarms..."
if adb shell "dumpsys alarm | grep timesafari" | grep -q "RTC_WAKEUP"; then
echo "✅ Alarms scheduled correctly"
return 0
else
echo "❌ No alarms found"
return 1
fi
}
# Test 1: Boot Recovery with RecoveryManager
test_boot_recovery() {
echo "🔄 Testing Boot Recovery with RecoveryManager..."
# Schedule notification
schedule_notification
# Verify initial scheduling
if ! check_scheduled_alarms; then
echo "❌ Initial scheduling failed"
return 1
fi
# Clear logs before reboot
adb logcat -c
# Reboot device
echo "🔄 Rebooting device..."
adb reboot
wait_for_device
# Check boot recovery logs
if ! check_recovery_logs "BOOT_COMPLETED"; then
echo "❌ Boot recovery not triggered"
return 1
fi
# Check alarms restored
if ! check_scheduled_alarms; then
echo "❌ Alarms not restored after reboot"
return 1
fi
echo "✅ Boot recovery working correctly"
return 0
}
# Test 2: App Startup Recovery Coexistence
test_app_startup_coexistence() {
echo "🔄 Testing App Startup Recovery Coexistence..."
# Clear logs
adb logcat -c
# Launch app multiple times to test cooldown
echo "🚀 Testing app startup recovery cooldown..."
adb shell am start -n $APP_PACKAGE/$APP_ACTIVITY
sleep 2
adb shell am start -n $APP_PACKAGE/$APP_ACTIVITY
sleep 2
adb shell am start -n $APP_PACKAGE/$APP_ACTIVITY
sleep 2
# Check recovery cooldown behavior
if ! check_recovery_cooldown; then
echo "❌ Recovery cooldown not working"
return 1
fi
echo "✅ App startup recovery coexistence working"
return 0
}
# Test 3: Package Replacement Recovery
test_package_replacement() {
echo "🔄 Testing Package Replacement Recovery..."
# Schedule notification
schedule_notification
# Verify initial scheduling
if ! check_scheduled_alarms; then
echo "❌ Initial scheduling failed"
return 1
fi
# Clear logs
adb logcat -c
# Update app (simulate package replacement)
echo "📦 Updating app (simulating package replacement)..."
cd android && ./gradlew assembleDebug
adb install -r app/build/outputs/apk/debug/app-debug.apk
cd ..
# Check package replacement recovery logs
if ! check_recovery_logs "MY_PACKAGE_REPLACED"; then
echo "❌ Package replacement recovery not triggered"
return 1
fi
# Check alarms restored
if ! check_scheduled_alarms; then
echo "❌ Alarms not restored after package replacement"
return 1
fi
echo "✅ Package replacement recovery working"
return 0
}
# Test 4: Recovery Statistics
test_recovery_statistics() {
echo "📊 Testing Recovery Statistics..."
# Clear logs
adb logcat -c
# Perform multiple recovery operations
echo "🔄 Performing multiple recovery operations..."
# Launch app (triggers startup recovery)
adb shell am start -n $APP_PACKAGE/$APP_ACTIVITY
sleep 2
# Launch app again (should be skipped due to cooldown)
adb shell am start -n $APP_PACKAGE/$APP_ACTIVITY
sleep 2
# Check recovery statistics in logs
local recovery_count=$(adb logcat -d | grep -c "Recovery requested from:")
local cooldown_count=$(adb logcat -d | grep -c "Recovery performed recently.*skipping")
if [ $recovery_count -gt 0 ] && [ $cooldown_count -gt 0 ]; then
echo "✅ Recovery statistics tracking working"
echo " - Total recovery requests: $recovery_count"
echo " - Cooldown skips: $cooldown_count"
return 0
else
echo "❌ Recovery statistics not tracking correctly"
return 1
fi
}
# Test 5: Recovery State Persistence
test_recovery_state_persistence() {
echo "💾 Testing Recovery State Persistence..."
# Clear logs
adb logcat -c
# Launch app to trigger recovery
adb shell am start -n $APP_PACKAGE/$APP_ACTIVITY
sleep 2
# Check if recovery state is persisted
local recovery_state=$(adb shell "run-as $APP_PACKAGE ls -la /data/data/$APP_PACKAGE/shared_prefs/ | grep recovery_state")
if [ -n "$recovery_state" ]; then
echo "✅ Recovery state persisted"
echo " - Recovery state file: $recovery_state"
return 0
else
echo "❌ Recovery state not persisted"
return 1
fi
}
# Test 6: Multiple Notifications Recovery
test_multiple_notifications_recovery() {
echo "📱 Testing Multiple Notifications Recovery..."
# Schedule multiple notifications
echo "📅 Scheduling multiple notifications..."
echo "⚠️ Manual step: Schedule 3-4 notifications in app UI"
echo " - Tap 'Test Notification' multiple times"
echo " - Wait for each 'Notification scheduled' message"
read -p "Press Enter when multiple notifications are scheduled..."
# Count initial alarms
local initial_alarms=$(adb shell "dumpsys alarm | grep timesafari | wc -l")
echo "📊 Initial alarms scheduled: $initial_alarms"
if [ $initial_alarms -lt 2 ]; then
echo "❌ Not enough notifications scheduled for test"
return 1
fi
# Clear logs
adb logcat -c
# Reboot device
echo "🔄 Rebooting device..."
adb reboot
wait_for_device
# Check recovery logs
if ! check_recovery_logs "BOOT_COMPLETED"; then
echo "❌ Boot recovery not triggered"
return 1
fi
# Count restored alarms
local restored_alarms=$(adb shell "dumpsys alarm | grep timesafari | wc -l")
echo "📊 Restored alarms: $restored_alarms"
if [ $restored_alarms -eq $initial_alarms ]; then
echo "✅ All notifications recovered correctly"
return 0
else
echo "❌ Not all notifications recovered"
echo " - Initial: $initial_alarms, Restored: $restored_alarms"
return 1
fi
}
# Main test execution
main() {
echo "🚀 Starting Reboot Test Suite v2.0..."
echo ""
# Run all tests
run_test "Boot Recovery with RecoveryManager" test_boot_recovery
run_test "App Startup Recovery Coexistence" test_app_startup_coexistence
run_test "Package Replacement Recovery" test_package_replacement
run_test "Recovery Statistics" test_recovery_statistics
run_test "Recovery State Persistence" test_recovery_state_persistence
run_test "Multiple Notifications Recovery" test_multiple_notifications_recovery
# Print results summary
echo "📊 Reboot Test Results Summary"
echo "============================="
local pass_count=0
local total_count=0
for test_name in "${!test_results[@]}"; do
local result="${test_results[$test_name]}"
echo "$test_name: $result"
if [ "$result" = "PASS" ]; then
((pass_count++))
fi
((total_count++))
done
echo ""
echo "Overall: $pass_count/$total_count tests passed"
if [ $pass_count -eq $total_count ]; then
echo "🎉 All reboot tests passed! RecoveryManager working correctly."
echo "⏰ Wait for scheduled notifications to appear"
exit 0
else
echo "❌ Some reboot tests failed. Check logs for details."
exit 1
fi
}
# Run main function
main
Loading…
Cancel
Save