You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 
 
 
 

8.9 KiB

Migration Guide: Dexie to absurd-sql

Overview

This document outlines the migration process from Dexie.js to absurd-sql for the TimeSafari app's storage implementation. The migration aims to provide a consistent SQLite-based storage solution across all platforms while maintaining data integrity and ensuring a smooth transition for users.

Current Status: The migration is in Phase 2 with a well-defined migration fence in place. Core settings and account data have been migrated, with contact migration in progress. ActiveDid migration has been implemented to ensure user identity continuity.

⚠️ UPDATE: The migration fence is now implemented through the PlatformServiceMixin rather than a USE_DEXIE_DB constant. This provides a cleaner, more maintainable approach to database access control.

Migration Goals

  1. Data Integrity

    • Preserve all existing data
    • Maintain data relationships
    • Ensure data consistency
    • Preserve user's active identity
  2. Performance

    • Improve query performance
    • Reduce storage overhead
    • Optimize for platform-specific capabilities
  3. User Experience

    • Seamless transition with no data loss
    • Maintain user's active identity and preferences
    • Preserve application state

Migration Architecture

Migration Fence

The migration fence is now defined by the PlatformServiceMixin in src/utils/PlatformServiceMixin.ts:

  • PlatformServiceMixin: Centralized database access with caching and utilities
  • Migration Tools: Exclusive interface between legacy and new databases
  • Service Layer: All database operations go through PlatformService

Migration Order

The migration follows a specific order to maintain data integrity:

  1. Accounts (foundational - contains DIDs)
  2. Settings (references accountDid, activeDid)
  3. ActiveDid (depends on accounts and settings) NEW
  4. Contacts (independent, but migrated after accounts for consistency)

ActiveDid Migration NEW FEATURE

Problem Solved

Previously, the activeDid setting was not migrated from Dexie to SQLite, causing users to lose their active identity after migration.

Solution Implemented

The migration now includes a dedicated step for migrating the activeDid:

  1. Detection: Identifies the activeDid from Dexie master settings
  2. Validation: Verifies the activeDid exists in SQLite accounts
  3. Migration: Updates SQLite master settings with the activeDid
  4. Error Handling: Graceful handling of missing accounts

Implementation Details

New Function: migrateActiveDid()

export async function migrateActiveDid(): Promise<MigrationResult> {
  // 1. Get Dexie settings to find the activeDid
  const dexieSettings = await getDexieSettings();
  const masterSettings = dexieSettings.find(setting => !setting.accountDid);
  
  // 2. Verify the activeDid exists in SQLite accounts
  const accountExists = await platformService.dbQuery(
    "SELECT did FROM accounts WHERE did = ?",
    [dexieActiveDid],
  );
  
  // 3. Update SQLite master settings
  await updateDefaultSettings({ activeDid: dexieActiveDid });
   }

Enhanced migrateSettings() Function

The settings migration now includes activeDid handling:

  • Extracts activeDid from Dexie master settings
  • Validates account existence in SQLite
  • Updates SQLite master settings with the activeDid

Updated migrateAll() Function

The complete migration now includes a dedicated step for activeDid:

// Step 3: Migrate ActiveDid (depends on accounts and settings)
logger.info("[MigrationService] Step 3: Migrating activeDid...");
const activeDidResult = await migrateActiveDid();

Benefits

  • User Identity Preservation: Users maintain their active identity
  • Seamless Experience: No need to manually select identity after migration
  • Data Consistency: Ensures all identity-related settings are preserved
  • Error Resilience: Graceful handling of edge cases

Migration Process

Phase 1: Preparation

  • PlatformServiceMixin implementation
  • Implement data comparison tools
  • Create migration service structure

Phase 2: Core Migration

  • Account migration with importFromMnemonic
  • Settings migration (excluding activeDid)
  • ActiveDid migration COMPLETED
  • Contact migration framework

Phase 3: Validation and Cleanup 🔄

  • Comprehensive data validation
  • Performance testing
  • User acceptance testing
  • Dexie removal

Usage

Manual Migration

import { migrateAll, migrateActiveDid } from '../services/indexedDBMigrationService';

// Complete migration
const result = await migrateAll();

// Or migrate just the activeDid
const activeDidResult = await migrateActiveDid();

Migration Verification

import { compareDatabases } from '../services/indexedDBMigrationService';

const comparison = await compareDatabases();
console.log('Migration differences:', comparison.differences);

PlatformServiceMixin Integration

After migration, use the mixin for all database operations:

// Use mixin methods for database access
const contacts = await this.$contacts();
const settings = await this.$settings();
const result = await this.$db("SELECT * FROM contacts WHERE did = ?", [accountDid]);

Error Handling

ActiveDid Migration Errors

  • Missing Account: If the activeDid from Dexie doesn't exist in SQLite accounts
  • Database Errors: Connection or query failures
  • Settings Update Failures: Issues updating SQLite master settings

Recovery Strategies

  1. Automatic Recovery: Migration continues even if activeDid migration fails
  2. Manual Recovery: Users can manually select their identity after migration
  3. Fallback: System creates new identity if none exists

Security Considerations

Data Protection

  • All sensitive data (mnemonics, private keys) are encrypted
  • Migration preserves encryption standards
  • No plaintext data exposure during migration

Identity Verification

  • ActiveDid migration validates account existence
  • Prevents setting non-existent identities as active
  • Maintains cryptographic integrity

Testing

Migration Testing

# Run migration
npm run migrate

# Verify results
npm run test:migration

ActiveDid Testing

// Test activeDid migration specifically
const result = await migrateActiveDid();
expect(result.success).toBe(true);
expect(result.warnings).toContain('Successfully migrated activeDid');

PlatformServiceMixin Testing

// Test mixin integration
describe('PlatformServiceMixin', () => {
  it('should provide database access methods', async () => {
    const contacts = await this.$contacts();
    const settings = await this.$settings();
    expect(contacts).toBeDefined();
    expect(settings).toBeDefined();
  });
});

Troubleshooting

Common Issues

  1. ActiveDid Not Found

    • Ensure accounts were migrated before activeDid migration
    • Check that the Dexie activeDid exists in SQLite accounts
  2. Migration Failures

    • Verify Dexie database is accessible
    • Check SQLite database permissions
    • Review migration logs for specific errors
  3. Data Inconsistencies

    • Use compareDatabases() to identify differences
    • Re-run migration if necessary
    • Check for duplicate or conflicting records
  4. PlatformServiceMixin Issues

    • Ensure mixin is properly imported and used
    • Check that all database operations use mixin methods
    • Verify caching and error handling work correctly

Debugging

// Debug migration process
import { logger } from '../utils/logger';

logger.debug('[Migration] Starting migration process...');
const result = await migrateAll();
logger.debug('[Migration] Migration completed:', result);

Benefits of PlatformServiceMixin Approach

  1. Centralized Access: Single point of control for all database operations
  2. Caching: Built-in caching for performance optimization
  3. Type Safety: Enhanced TypeScript integration
  4. Error Handling: Consistent error handling across components
  5. Code Reduction: Up to 80% reduction in database boilerplate
  6. Maintainability: Single source of truth for database patterns

Migration Status Checklist

Completed

  • PlatformServiceMixin implementation
  • SQLite database service
  • Migration tools
  • Settings migration
  • Account migration
  • ActiveDid migration

🔄 In Progress

  • Contact migration
  • DatabaseUtil to PlatformServiceMixin migration
  • File-by-file migration

Not Started

  • Legacy Dexie removal
  • Final cleanup and validation

Author: Matthew Raymer Created: 2025-07-05 Status: Active Migration Phase Last Updated: 2025-07-05 Note: Migration fence now implemented through PlatformServiceMixin instead of USE_DEXIE_DB constant