# Migration Guide: Dexie to absurd-sql ## Overview This document outlines the migration process from Dexie.js to absurd-sql for the TimeSafari app's storage implementation. The migration aims to provide a consistent SQLite-based storage solution across all platforms while maintaining data integrity and ensuring a smooth transition for users. **Current Status**: The migration is in **Phase 2** with a well-defined migration fence in place. Core settings and account data have been migrated, with contact migration in progress. **ActiveDid migration has been implemented** to ensure user identity continuity. **⚠️ UPDATE**: The migration fence is now implemented through the **PlatformServiceMixin** rather than a `USE_DEXIE_DB` constant. This provides a cleaner, more maintainable approach to database access control. ## Migration Goals 1. **Data Integrity** - Preserve all existing data - Maintain data relationships - Ensure data consistency - **Preserve user's active identity** 2. **Performance** - Improve query performance - Reduce storage overhead - Optimize for platform-specific capabilities 3. **User Experience** - Seamless transition with no data loss - Maintain user's active identity and preferences - Preserve application state ## Migration Architecture ### Migration Fence The migration fence is now defined by the **PlatformServiceMixin** in `src/utils/PlatformServiceMixin.ts`: - **PlatformServiceMixin**: Centralized database access with caching and utilities - **Migration Tools**: Exclusive interface between legacy and new databases - **Service Layer**: All database operations go through PlatformService ### Migration Order The migration follows a specific order to maintain data integrity: 1. **Accounts** (foundational - contains DIDs) 2. **Settings** (references accountDid, activeDid) 3. **ActiveDid** (depends on accounts and settings) ⭐ **NEW** 4. **Contacts** (independent, but migrated after accounts for consistency) ## ActiveDid Migration ⭐ **NEW FEATURE** ### Problem Solved Previously, the `activeDid` setting was not migrated from Dexie to SQLite, causing users to lose their active identity after migration. ### Solution Implemented The migration now includes a dedicated step for migrating the `activeDid`: 1. **Detection**: Identifies the `activeDid` from Dexie master settings 2. **Validation**: Verifies the `activeDid` exists in SQLite accounts 3. **Migration**: Updates SQLite master settings with the `activeDid` 4. **Error Handling**: Graceful handling of missing accounts ### Implementation Details #### New Function: `migrateActiveDid()` ```typescript export async function migrateActiveDid(): Promise { // 1. Get Dexie settings to find the activeDid const dexieSettings = await getDexieSettings(); const masterSettings = dexieSettings.find(setting => !setting.accountDid); // 2. Verify the activeDid exists in SQLite accounts const accountExists = await platformService.dbQuery( "SELECT did FROM accounts WHERE did = ?", [dexieActiveDid], ); // 3. Update SQLite master settings await updateDefaultSettings({ activeDid: dexieActiveDid }); } ``` #### Enhanced `migrateSettings()` Function The settings migration now includes activeDid handling: - Extracts `activeDid` from Dexie master settings - Validates account existence in SQLite - Updates SQLite master settings with the `activeDid` #### Updated `migrateAll()` Function The complete migration now includes a dedicated step for activeDid: ```typescript // Step 3: Migrate ActiveDid (depends on accounts and settings) logger.info("[MigrationService] Step 3: Migrating activeDid..."); const activeDidResult = await migrateActiveDid(); ``` ### Benefits - ✅ **User Identity Preservation**: Users maintain their active identity - ✅ **Seamless Experience**: No need to manually select identity after migration - ✅ **Data Consistency**: Ensures all identity-related settings are preserved - ✅ **Error Resilience**: Graceful handling of edge cases ## Migration Process ### Phase 1: Preparation ✅ - [x] PlatformServiceMixin implementation - [x] Implement data comparison tools - [x] Create migration service structure ### Phase 2: Core Migration ✅ - [x] Account migration with `importFromMnemonic` - [x] Settings migration (excluding activeDid) - [x] **ActiveDid migration** ⭐ **COMPLETED** - [x] Contact migration framework ### Phase 3: Validation and Cleanup 🔄 - [ ] Comprehensive data validation - [ ] Performance testing - [ ] User acceptance testing - [ ] Dexie removal ## Usage ### Manual Migration ```typescript import { migrateAll, migrateActiveDid } from '../services/indexedDBMigrationService'; // Complete migration const result = await migrateAll(); // Or migrate just the activeDid const activeDidResult = await migrateActiveDid(); ``` ### Migration Verification ```typescript import { compareDatabases } from '../services/indexedDBMigrationService'; const comparison = await compareDatabases(); console.log('Migration differences:', comparison.differences); ``` ### PlatformServiceMixin Integration After migration, use the mixin for all database operations: ```typescript // Use mixin methods for database access const contacts = await this.$contacts(); const settings = await this.$settings(); const result = await this.$db("SELECT * FROM contacts WHERE did = ?", [accountDid]); ``` ## Error Handling ### ActiveDid Migration Errors - **Missing Account**: If the `activeDid` from Dexie doesn't exist in SQLite accounts - **Database Errors**: Connection or query failures - **Settings Update Failures**: Issues updating SQLite master settings ### Recovery Strategies 1. **Automatic Recovery**: Migration continues even if activeDid migration fails 2. **Manual Recovery**: Users can manually select their identity after migration 3. **Fallback**: System creates new identity if none exists ## Security Considerations ### Data Protection - All sensitive data (mnemonics, private keys) are encrypted - Migration preserves encryption standards - No plaintext data exposure during migration ### Identity Verification - ActiveDid migration validates account existence - Prevents setting non-existent identities as active - Maintains cryptographic integrity ## Testing ### Migration Testing ```bash # Run migration npm run migrate # Verify results npm run test:migration ``` ### ActiveDid Testing ```typescript // Test activeDid migration specifically const result = await migrateActiveDid(); expect(result.success).toBe(true); expect(result.warnings).toContain('Successfully migrated activeDid'); ``` ### PlatformServiceMixin Testing ```typescript // Test mixin integration describe('PlatformServiceMixin', () => { it('should provide database access methods', async () => { const contacts = await this.$contacts(); const settings = await this.$settings(); expect(contacts).toBeDefined(); expect(settings).toBeDefined(); }); }); ``` ## Troubleshooting ### Common Issues 1. **ActiveDid Not Found** - Ensure accounts were migrated before activeDid migration - Check that the Dexie activeDid exists in SQLite accounts 2. **Migration Failures** - Verify Dexie database is accessible - Check SQLite database permissions - Review migration logs for specific errors 3. **Data Inconsistencies** - Use `compareDatabases()` to identify differences - Re-run migration if necessary - Check for duplicate or conflicting records 4. **PlatformServiceMixin Issues** - Ensure mixin is properly imported and used - Check that all database operations use mixin methods - Verify caching and error handling work correctly ### Debugging ```typescript // Debug migration process import { logger } from '../utils/logger'; logger.debug('[Migration] Starting migration process...'); const result = await migrateAll(); logger.debug('[Migration] Migration completed:', result); ``` ## Benefits of PlatformServiceMixin Approach 1. **Centralized Access**: Single point of control for all database operations 2. **Caching**: Built-in caching for performance optimization 3. **Type Safety**: Enhanced TypeScript integration 4. **Error Handling**: Consistent error handling across components 5. **Code Reduction**: Up to 80% reduction in database boilerplate 6. **Maintainability**: Single source of truth for database patterns ## Migration Status Checklist ### ✅ Completed - [x] PlatformServiceMixin implementation - [x] SQLite database service - [x] Migration tools - [x] Settings migration - [x] Account migration - [x] ActiveDid migration ### 🔄 In Progress - [ ] Contact migration - [ ] DatabaseUtil to PlatformServiceMixin migration - [ ] File-by-file migration ### ❌ Not Started - [ ] Legacy Dexie removal - [ ] Final cleanup and validation --- **Author**: Matthew Raymer **Created**: 2025-07-05 **Status**: Active Migration Phase **Last Updated**: 2025-07-05 **Note**: Migration fence now implemented through PlatformServiceMixin instead of USE_DEXIE_DB constant