8.9 KiB
Migration Guide: Dexie to absurd-sql
Overview
This document outlines the migration process from Dexie.js to absurd-sql for the TimeSafari app's storage implementation. The migration aims to provide a consistent SQLite-based storage solution across all platforms while maintaining data integrity and ensuring a smooth transition for users.
Current Status: The migration is in Phase 2 with a well-defined migration fence in place. Core settings and account data have been migrated, with contact migration in progress. ActiveDid migration has been implemented to ensure user identity continuity.
⚠️ UPDATE: The migration fence is now implemented through the PlatformServiceMixin rather than a USE_DEXIE_DB
constant. This provides a cleaner, more maintainable approach to database access control.
Migration Goals
-
Data Integrity
- Preserve all existing data
- Maintain data relationships
- Ensure data consistency
- Preserve user's active identity
-
Performance
- Improve query performance
- Reduce storage overhead
- Optimize for platform-specific capabilities
-
User Experience
- Seamless transition with no data loss
- Maintain user's active identity and preferences
- Preserve application state
Migration Architecture
Migration Fence
The migration fence is now defined by the PlatformServiceMixin in src/utils/PlatformServiceMixin.ts
:
- PlatformServiceMixin: Centralized database access with caching and utilities
- Migration Tools: Exclusive interface between legacy and new databases
- Service Layer: All database operations go through PlatformService
Migration Order
The migration follows a specific order to maintain data integrity:
- Accounts (foundational - contains DIDs)
- Settings (references accountDid, activeDid)
- ActiveDid (depends on accounts and settings) ⭐ NEW
- Contacts (independent, but migrated after accounts for consistency)
ActiveDid Migration ⭐ NEW FEATURE
Problem Solved
Previously, the activeDid
setting was not migrated from Dexie to SQLite, causing users to lose their active identity after migration.
Solution Implemented
The migration now includes a dedicated step for migrating the activeDid
:
- Detection: Identifies the
activeDid
from Dexie master settings - Validation: Verifies the
activeDid
exists in SQLite accounts - Migration: Updates SQLite master settings with the
activeDid
- Error Handling: Graceful handling of missing accounts
Implementation Details
New Function: migrateActiveDid()
export async function migrateActiveDid(): Promise<MigrationResult> {
// 1. Get Dexie settings to find the activeDid
const dexieSettings = await getDexieSettings();
const masterSettings = dexieSettings.find(setting => !setting.accountDid);
// 2. Verify the activeDid exists in SQLite accounts
const accountExists = await platformService.dbQuery(
"SELECT did FROM accounts WHERE did = ?",
[dexieActiveDid],
);
// 3. Update SQLite master settings
await updateDefaultSettings({ activeDid: dexieActiveDid });
}
Enhanced migrateSettings()
Function
The settings migration now includes activeDid handling:
- Extracts
activeDid
from Dexie master settings - Validates account existence in SQLite
- Updates SQLite master settings with the
activeDid
Updated migrateAll()
Function
The complete migration now includes a dedicated step for activeDid:
// Step 3: Migrate ActiveDid (depends on accounts and settings)
logger.info("[MigrationService] Step 3: Migrating activeDid...");
const activeDidResult = await migrateActiveDid();
Benefits
- ✅ User Identity Preservation: Users maintain their active identity
- ✅ Seamless Experience: No need to manually select identity after migration
- ✅ Data Consistency: Ensures all identity-related settings are preserved
- ✅ Error Resilience: Graceful handling of edge cases
Migration Process
Phase 1: Preparation ✅
- PlatformServiceMixin implementation
- Implement data comparison tools
- Create migration service structure
Phase 2: Core Migration ✅
- Account migration with
importFromMnemonic
- Settings migration (excluding activeDid)
- ActiveDid migration ⭐ COMPLETED
- Contact migration framework
Phase 3: Validation and Cleanup 🔄
- Comprehensive data validation
- Performance testing
- User acceptance testing
- Dexie removal
Usage
Manual Migration
import { migrateAll, migrateActiveDid } from '../services/indexedDBMigrationService';
// Complete migration
const result = await migrateAll();
// Or migrate just the activeDid
const activeDidResult = await migrateActiveDid();
Migration Verification
import { compareDatabases } from '../services/indexedDBMigrationService';
const comparison = await compareDatabases();
console.log('Migration differences:', comparison.differences);
PlatformServiceMixin Integration
After migration, use the mixin for all database operations:
// Use mixin methods for database access
const contacts = await this.$contacts();
const settings = await this.$settings();
const result = await this.$db("SELECT * FROM contacts WHERE did = ?", [accountDid]);
Error Handling
ActiveDid Migration Errors
- Missing Account: If the
activeDid
from Dexie doesn't exist in SQLite accounts - Database Errors: Connection or query failures
- Settings Update Failures: Issues updating SQLite master settings
Recovery Strategies
- Automatic Recovery: Migration continues even if activeDid migration fails
- Manual Recovery: Users can manually select their identity after migration
- Fallback: System creates new identity if none exists
Security Considerations
Data Protection
- All sensitive data (mnemonics, private keys) are encrypted
- Migration preserves encryption standards
- No plaintext data exposure during migration
Identity Verification
- ActiveDid migration validates account existence
- Prevents setting non-existent identities as active
- Maintains cryptographic integrity
Testing
Migration Testing
# Run migration
npm run migrate
# Verify results
npm run test:migration
ActiveDid Testing
// Test activeDid migration specifically
const result = await migrateActiveDid();
expect(result.success).toBe(true);
expect(result.warnings).toContain('Successfully migrated activeDid');
PlatformServiceMixin Testing
// Test mixin integration
describe('PlatformServiceMixin', () => {
it('should provide database access methods', async () => {
const contacts = await this.$contacts();
const settings = await this.$settings();
expect(contacts).toBeDefined();
expect(settings).toBeDefined();
});
});
Troubleshooting
Common Issues
-
ActiveDid Not Found
- Ensure accounts were migrated before activeDid migration
- Check that the Dexie activeDid exists in SQLite accounts
-
Migration Failures
- Verify Dexie database is accessible
- Check SQLite database permissions
- Review migration logs for specific errors
-
Data Inconsistencies
- Use
compareDatabases()
to identify differences - Re-run migration if necessary
- Check for duplicate or conflicting records
- Use
-
PlatformServiceMixin Issues
- Ensure mixin is properly imported and used
- Check that all database operations use mixin methods
- Verify caching and error handling work correctly
Debugging
// Debug migration process
import { logger } from '../utils/logger';
logger.debug('[Migration] Starting migration process...');
const result = await migrateAll();
logger.debug('[Migration] Migration completed:', result);
Benefits of PlatformServiceMixin Approach
- Centralized Access: Single point of control for all database operations
- Caching: Built-in caching for performance optimization
- Type Safety: Enhanced TypeScript integration
- Error Handling: Consistent error handling across components
- Code Reduction: Up to 80% reduction in database boilerplate
- Maintainability: Single source of truth for database patterns
Migration Status Checklist
✅ Completed
- PlatformServiceMixin implementation
- SQLite database service
- Migration tools
- Settings migration
- Account migration
- ActiveDid migration
🔄 In Progress
- Contact migration
- DatabaseUtil to PlatformServiceMixin migration
- File-by-file migration
❌ Not Started
- Legacy Dexie removal
- Final cleanup and validation
Author: Matthew Raymer Created: 2025-07-05 Status: Active Migration Phase Last Updated: 2025-07-05 Note: Migration fence now implemented through PlatformServiceMixin instead of USE_DEXIE_DB constant