- Remove isDevelopment environment checks and migrationLog variable
- Replace conditional logging with consistent logger.debug() calls
- Remove development-only validation restrictions
- Maintain all error handling and warning messages
- Let existing logger handle development mode behavior automatically
This simplifies the migration service logging while preserving all
functionality. The existing logger already handles development vs
production mode appropriately.
- Remove explicit transaction wrapping in migration service that caused
"cannot start a transaction within a transaction" errors
- Fix executeSet method call format to include both statement and values
properties as required by Capacitor SQLite plugin
- Update CapacitorPlatformService to properly handle multi-statement SQL
using executeSet for migration SQL blocks
- Ensure migration 004 (active_identity_management) executes atomically
without nested transaction conflicts
- Remove unnecessary try/catch wrapper
Fixes iOS simulator migration failures where:
- Migration 004 would fail with transaction errors
- executeSet would fail with "Must provide a set as Array of {statement,values}"
- Database initialization would fail after migration errors
Tested on iOS simulator with successful migration completion and
active_identity table creation with proper data migration.
- Single SQL source: Define MIG_004_SQL constant to eliminate duplicate SQL definitions
- Atomic execution: Add BEGIN IMMEDIATE/COMMIT/ROLLBACK around migration execution
- Name-only check: Skip migrations already recorded in migrations table
- Guarded operations: Replace table-wide cleanups with conditional UPDATE/DELETE
Changes:
- migration.ts: Extract migration 004 SQL into MIG_004_SQL constant
- migration.ts: Use guarded DELETE/UPDATE to prevent accidental data loss
- migrationService.ts: Wrap migration execution in explicit transactions
- migrationService.ts: Reorder checks to prioritize name-only skipping
Benefits:
- Prevents partial migration failures from corrupting database state
- Eliminates SQL duplication and maintenance overhead
- Maintains existing APIs and logging behavior
- Reduces risk of data loss during migration execution
Test results: All migration tests passing, ID generation working correctly
- Phase 1: Simplify Migration Definition ✅
* Remove duplicate SQL definitions from migration 004
* Eliminate recovery logic that could cause duplicate execution
* Establish single source of truth for migration SQL
- Phase 2: Fix Database Result Handling ✅
* Remove DatabaseResult type assumptions from migration code
* Implement database-agnostic result extraction with extractSingleValue()
* Normalize results from AbsurdSqlDatabaseService and CapacitorPlatformService
- Phase 3: Ensure Atomic Execution ✅
* Remove individual statement execution logic
* Execute migrations as single atomic SQL blocks only
* Add explicit rollback instructions and failure cause logging
* Ensure migration tracking is accurate
- Phase 4: Remove Excessive Debugging ✅
* Move detailed logging to development-only mode
* Preserve essential error logging for production
* Optimize startup performance by reducing logging overhead
* Maintain full debugging capability in development
Migration system now follows single-source, atomic execution principle
with improved performance and comprehensive error handling.
Timestamp: 2025-09-17 05:08:05 UTC
- Fix multi-statement SQL execution issue in Capacitor SQLite
- Add individual statement execution for migration 004_active_identity_management
- Implement automatic recovery for missing active_identity table
- Enhance migration system with better error handling and logging
Problem:
Migration 004 was marked as applied but active_identity table wasn't created
due to multi-statement SQL execution failing silently in Capacitor SQLite.
Solution:
- Extended Migration interface with optional statements array
- Modified migration execution to handle individual statements
- Added bootstrapping hook recovery for missing tables
- Enhanced logging for better debugging
Files changed:
- src/services/migrationService.ts: Enhanced migration execution logic
- src/db-sql/migration.ts: Added recovery mechanism and individual statements
This fix ensures the app automatically recovers from the current broken state
and prevents similar issues in future migrations.
- Consolidate migrations 004, 005, and 006 into single 004_active_identity_management
- Remove redundant migrations 005 (constraint_fix) and 006 (settings_cleanup)
- Implement security-first approach with ON DELETE RESTRICT constraint from start
- Include comprehensive data migration from settings.activeDid to active_identity.activeDid
- Add proper cleanup of orphaned settings records and legacy activeDid values
- Update migrationService.ts validation logic to reflect consolidated structure
- Fix migration name references and enhance validation for hasBackedUpSeed column
- Reduce migration complexity from 3 separate operations to 1 atomic operation
- Maintain data integrity with foreign key constraints and performance indexes
Migration successfully tested on web platform with no data loss or corruption.
Active DID properly migrated: did:ethr:0xCA26A3959D32D2eB5459cE08203DbC4e62e79F5D
Files changed:
- src/db-sql/migration.ts: Consolidated 3 migrations into 1 (-46 lines)
- src/services/migrationService.ts: Updated validation logic (+13 lines)
- Fix syntax error in logger.ts: change 'typeof import' to 'typeof import.meta'
to resolve ESBuild compilation error preventing web build
- Align CapacitorPlatformService.insertNewDidIntoSettings with WebPlatformService:
* Add dynamic constants import to avoid circular dependencies
* Use INSERT OR REPLACE for data integrity
* Set proper default values (finishedOnboarding=false, API servers)
* Remove TODO comment as implementation is now parallel
- Fix Playwright test timing issues in 60-new-activity.spec.ts:
* Replace generic alert selectors with specific alert type targeting
* Change Info alerts from 'Success' to 'Info' filter for proper targeting
* Fix "strict mode violation" errors caused by multiple simultaneous alerts
* Improve test reliability by using established alert handling patterns
- Update migrationService.ts and vite.config.common.mts with related improvements
Test Results: Improved from 2 failed tests to 42/44 passing (95.5% success rate)
Build Status: Web build now compiles successfully without syntax errors
- Update migration 003 to match master deployment (hasBackedUpSeed)
- Rename migration 004 for active_identity table creation
- Update migration service validation for new structure
- Fix TypeScript compatibility issue in migration.ts
- Streamline active identity upgrade plan documentation
- Ensure all migrations are additional per team guidance
Migration structure now follows "additional migrations only" principle:
- 003: hasBackedUpSeed (assumes master deployment)
- 004: active_identity table with data migration
- iOS/Android compatibility confirmed with SQLCipher 4.9.0
Files: migration.ts, migrationService.ts, active-identity-upgrade-plan.md
- Fix $getActiveIdentity() logic flow preventing false "empty table" warnings
- Implement auto-selection of first account when activeDid is null after migration
- Consolidate 003 and 003b migrations into single 003 migration
- Re-introduce foreign key constraint for activeDid referential integrity
- Add comprehensive debug logging for migration troubleshooting
- Remove 003b validation logic and update migration name mapping
Fixes migration from master to active_did_redux branch and ensures system
always has valid activeDid for proper functionality.
- Fix line breaks and indentation for long SQL queries
- Improve readability of error message formatting
- Remove trailing whitespace and standardize spacing
- Apply consistent formatting to active_identity table validation logic
The complex table rewrite approach in migration 003_active_did_separation was
failing on iOS SQLite, causing "no such table: active_identity" errors. The
migration was being marked as applied despite validation failures.
Changes:
- Simplify migration SQL to only create active_identity table and migrate data
- Remove complex table rewrite that was failing on iOS SQLite versions
- Remove foreign key constraint that could cause compatibility issues
- Update validation logic to focus on active_identity table existence only
- Remove validation check for activeDid column removal from settings table
This approach is more reliable across different SQLite versions and platforms
while maintaining the core functionality of separating activeDid into its own
table for better database architecture.
Fixes iOS build database errors and ensures migration completes successfully.
- Reduce migration/platform logging verbosity in development mode (~80-90% less noise)
- Fix AbsurdSQL platform check to support 'development' alongside 'web'
- Add missing WebPlatformService methods (isWorker, initSharedArrayBuffer)
- Fix Electron API endpoint resolution to prevent JSON parsing errors
- Prevent circular database logging that caused [DB-PREVENTED-INFO] spam
Console now shows only essential information while preserving all errors/warnings
- Replace 'any' type assertions with specific types in migration name extraction
* Change '(row as any).name' to '(row as { name: string }).name'
* Add proper null checks and 'in' operator for property access
- Fix database integrity check type safety
* Change '(col: any)' to '(col: unknown)' with type guards
* Use specific type assertion for column name checking
Resolves: @typescript-eslint/no-explicit-any warnings (2 instances)
Impact: Improves type safety without changing functionality
- Add comprehensive IndexedDB to SQLite migration service (1,397 lines)
- Create migration UI with progress tracking and validation (1,492 lines)
- Fix Electron TypeScript compilation and SQLite plugin issues
- Expand migration system with detailed documentation and error handling
- Add development guide and coding standards
Resolves: #electron-startup #database-migration #typescript-errors
Impact: Enables user-friendly database migration with full data verification
- Add detailed file-level documentation with architecture overview and usage examples
- Document all interfaces, classes, and methods with JSDoc comments
- Include migration philosophy, best practices, and schema evolution guidelines
- Add extensive inline documentation for database schema and table purposes
- Document privacy and security considerations in database design
- Provide troubleshooting guidance and logging explanations
- Add template and examples for future migration development
- Include platform-specific documentation for Capacitor SQLite integration
- Document validation and integrity checking processes with detailed steps
The migration system is now thoroughly documented for maintainability and
onboarding of new developers to the codebase.
- Add multi-layered migration validation strategy with schema detection
- Implement database integrity checker that validates all core tables and columns
- Add schema-based migration skipping to prevent re-running applied migrations
- Enhanced error handling for duplicate table/column scenarios with validation
- Add comprehensive logging for migration tracking and database state verification
- Include final validation step to ensure all migrations are properly recorded
The system now properly:
1. Checks if migrations are recorded in the migrations table
2. Validates actual schema exists before attempting to apply migrations
3. Handles edge cases where schema exists but isn't recorded
4. Provides detailed integrity checking of database structure
5. Eliminates SQL errors from duplicate table/column creation attempts
Migration tracking is now working correctly with both migrations properly recorded.
- Root cause: Migration names were not being properly inserted into migrations table
- Fixed parameter binding in Capacitor platform service migration functions
- Added detailed debugging to track SQL execution and parameter passing
- Reverted migrations back to proper form (without IF NOT EXISTS workarounds)
- Enhanced extractMigrationNames to handle Capacitor SQLite result format
The migration system should now properly track applied migrations and avoid
re-running them on subsequent app starts.
- Add comprehensive console logging for Electron with emojis for better visibility
- Use CREATE TABLE IF NOT EXISTS and INSERT OR IGNORE to prevent duplicate creation errors
- Add specialized column existence checking for ALTER TABLE ADD COLUMN operations
- Improve migration tracking with detailed status reporting (applied/skipped counts)
- Add proper error handling for existing schema scenarios
- Enhanced documentation and type safety for migration system
This resolves issues where migrations would fail with 'table already exists' or
'duplicate column' errors when the database schema was already properly set up.
The enhanced logging makes it clear to users when migrations are being skipped
vs. applied, improving the debugging experience in Electron.
- Enhanced migration service error handling and logging
- Added better recovery mechanisms for failed migrations
- Improved database state validation before migration execution
- Added comprehensive error reporting in migration service
- Updated HomeView to handle migration errors gracefully
- Fixed database initialization sequence to prevent constraint violations
This addresses UNIQUE constraint failed errors during app startup
and provides better user feedback during database migration issues.
- Enhanced migration service to handle duplicate column errors gracefully
- Added detection for 'duplicate column' and 'already exists' errors
- Migration service now marks partially applied migrations as complete
- Prevents Electron app crashes due to cross-platform database conflicts
- Improved robustness for database schema migrations
Fixes database initialization issues when switching between platforms
(web, mobile, electron) that may have different migration states.
- Update settings migration to maintain separate master and account records
- Use activeDid/accountDid pattern to differentiate between settings types:
* Master settings: activeDid set, accountDid empty
* Account settings: accountDid set, activeDid empty
- Add detailed logging for settings migration process
- Focus on migrating key fields: firstName, isRegistered, profileImageUrl,
showShortcutBvc, and searchBoxes
- Fix issue where settings were being incorrectly merged into a single record
This change ensures the SQLite database maintains the same settings structure
as Dexie, which is required by the existing codebase.
- Enhanced migration service to filter empty/default records from comparisons
- Updated comparison output to exclude records without meaningful data
- Improved UI display with better visual hierarchy and count indicators
- Added security improvements to prevent sensitive data exposure
- Fixed settings migration to properly handle MASTER_SETTINGS_KEY vs account DIDs
- Updated verification display to show filtered counts and detailed differences
- Improved data formatting for contacts, settings, and accounts sections
- Added proper filtering for missing records to avoid counting empty entries
Changes:
- Filter SQLite records to only include those with actual DIDs/data
- Update comparison counts to reflect meaningful differences only
- Enhance UI with count indicators and better visual organization
- Replace sensitive data display with boolean flags (hasIdentity, hasMnemonic)
- Fix settings migration logic for proper DID field handling
- Improve verification message to show detailed breakdown by type
- Add proper filtering for missing records in all data types
Security: Prevents exposure of mnemonics, private keys, and identity data
UI/UX: Cleaner display with better information hierarchy and counts
Migration: More accurate comparison results and better debugging visibility
- Add migrateAll() function that handles complete migration in correct order:
Accounts → Settings → Contacts to avoid foreign key constraint issues
- Add prominent "Migrate All (Recommended)" button to migration UI
- Add informational section explaining migration order and rationale
- Add info icon to icon set for UI clarity
- Improve migration logic to handle overwriteExisting parameter properly:
- New records are always migrated regardless of checkbox setting
- Existing records are only updated when overwriteExisting=true
- Clear warning messages when records are skipped
- Maintain backward compatibility with individual migration buttons
- All code linted and formatted according to project standards
Co-authored-by: Matthew Raymer
- Fix import in src/db-sql/migration.ts to use named imports and alias runMigrations to avoid naming conflict
- Add missing migration management functions (registerMigration, runMigrations) to migrationService with full typing and logging
- Update function signatures to accept SQL parameters for compatibility with AbsurdSqlDatabaseService
- Clean up Prettier formatting issues in migrationService and migration.ts
- Confirmed dev server and linter run cleanly
Co-authored-by: Matthew Raymer
- Add account migration support to migrationService with importFromMnemonic integration
- Extend DataComparison and MigrationResult interfaces to include accounts
- Add getDexieAccounts() and getSqliteAccounts() functions for account retrieval
- Implement compareAccounts() and migrateAccounts() functions with proper error handling
- Update DatabaseMigration.vue UI to support account migration:
- Add "Migrate Accounts" button with lock icon
- Extend summary cards grid to show Dexie/SQLite account counts
- Add Account Differences section with added/modified/missing indicators
- Update success message to include account migration counts
- Enhance grid layouts to accommodate 6 summary cards and 3 difference sections
The migration service now provides complete data migration capabilities
for contacts, settings, and accounts, with enhanced reliability through
the importFromMnemonic utility for mnemonic-based account handling.
- Create DatabaseMigration.vue with vue-facing-decorator and Tailwind CSS
- Add complete UI for comparing and migrating data between Dexie and SQLite
- Implement real-time loading states, error handling, and success feedback
- Add navigation link to Account page for easy access
- Include export functionality for comparison data
- Create comprehensive documentation in doc/database-migration-guide.md
- Fix all linting issues and ensure code quality standards
- Support both contact and settings migration with overwrite options
- Add visual difference analysis with summary cards and detailed breakdowns
The component provides a professional interface for the migrationService.ts,
enabling users to safely transfer data between database systems during
the transition from Dexie to SQLite storage.
- Add migrationService.ts with functions to compare and transfer data between Dexie and SQLite
- Implement data comparison with detailed difference analysis (added/modified/missing)
- Add contact migration with overwrite options and error handling
- Add settings migration focusing on key user fields (firstName, isRegistered, profileImageUrl, showShortcutBvc, searchBoxes)
- Include YAML export functionality for data inspection
- Add comprehensive JSDoc documentation with examples and usage instructions
- Support both INSERT and UPDATE operations with parameterized SQL generation
- Include detailed logging and error reporting for migration operations
This service enables safe migration of user data from the legacy Dexie (IndexedDB)
database to the new SQLite implementation, with full comparison capabilities
and rollback safety through detailed reporting.
- Add contactsToExportJson utility function for standardized data export
- Replace CSV export with JSON format in DataExportSection
- Update file extension and MIME type to application/json
- Remove Dexie-specific export logic in favor of unified SQLite/Dexie approach
- Update success notifications to reflect JSON format
- Add TypeScript interfaces for export data structure
This change improves data portability and standardization by:
- Using a consistent JSON format for data export/import
- Supporting both SQLite and Dexie databases
- Including all contact fields in export
- Properly handling contactMethods as stringified JSON
- Maintaining backward compatibility with existing import tools
Security: No sensitive data exposure, maintains existing access controls
- Replace any[] with SqlValue[] type for SQL parameters in runMigrations
- Update import to use QueryExecResult from interfaces/database
- Add proper typing for SQL parameter values (string | number | null | Uint8Array)
This change improves type safety and helps catch potential SQL parameter
type mismatches at compile time, reducing the risk of runtime errors
or data corruption.
