crowd-funder-from-jason/docs/migration-to-wa-sqlite.md

# Migration Guide: Dexie to wa-sqlite

## Overview

This document outlines the migration process from Dexie.js to wa-sqlite for the TimeSafari app's storage implementation. The migration aims to provide a consistent SQLite-based storage solution across all platforms while maintaining data integrity and ensuring a smooth transition for users.

## Migration Goals

1. **Data Integrity**
   - Preserve all existing data
   - Maintain data relationships
   - Ensure data consistency

2. **Performance**
   - Improve query performance
   - Reduce storage overhead
   - Optimize for platform-specific features

3. **Security**
   - Maintain or improve encryption
   - Preserve access controls
   - Enhance data protection

4. **User Experience**
   - Zero data loss
   - Minimal downtime
   - Automatic migration where possible

## Prerequisites

1. **Backup Requirements**
   ```typescript
   interface MigrationBackup {
     timestamp: number;
     accounts: Account[];
     settings: Setting[];
     contacts: Contact[];
     metadata: {
       version: string;
       platform: string;
       dexieVersion: string;
     };
   }
   ```

2. **Storage Requirements**
   - Sufficient IndexedDB quota
   - Available disk space for SQLite
   - Backup storage space

3. **Platform Support**
   - Web: Modern browser with IndexedDB support
   - iOS: iOS 13+ with SQLite support
   - Android: Android 5+ with SQLite support
   - Electron: Latest version with SQLite support

## Migration Process

### 1. Preparation

```typescript
// src/services/storage/migration/MigrationService.ts
export class MigrationService {
  private static instance: MigrationService;
  private backup: MigrationBackup | null = null;

  async prepare(): Promise<void> {
    try {
      // 1. Check prerequisites
      await this.checkPrerequisites();
      
      // 2. Create backup
      this.backup = await this.createBackup();
      
      // 3. Verify backup integrity
      await this.verifyBackup();
      
      // 4. Initialize wa-sqlite
      await this.initializeWaSqlite();
    } catch (error) {
      throw new StorageError(
        'Migration preparation failed',
        StorageErrorCodes.MIGRATION_FAILED,
        error
      );
    }
  }

  private async checkPrerequisites(): Promise<void> {
    // Check IndexedDB availability
    if (!window.indexedDB) {
      throw new StorageError(
        'IndexedDB not available',
        StorageErrorCodes.INITIALIZATION_FAILED
      );
    }

    // Check storage quota
    const quota = await navigator.storage.estimate();
    if (quota.quota && quota.usage && quota.usage > quota.quota * 0.9) {
      throw new StorageError(
        'Insufficient storage space',
        StorageErrorCodes.STORAGE_FULL
      );
    }

    // Check platform support
    const capabilities = await PlatformDetection.getCapabilities();
    if (!capabilities.hasFileSystem) {
      throw new StorageError(
        'Platform does not support required features',
        StorageErrorCodes.INITIALIZATION_FAILED
      );
    }
  }

  private async createBackup(): Promise<MigrationBackup> {
    const dexieDB = new Dexie('TimeSafariDB');
    
    return {
      timestamp: Date.now(),
      accounts: await dexieDB.accounts.toArray(),
      settings: await dexieDB.settings.toArray(),
      contacts: await dexieDB.contacts.toArray(),
      metadata: {
        version: '1.0.0',
        platform: await PlatformDetection.getPlatform(),
        dexieVersion: Dexie.version
      }
    };
  }
}
```

### 2. Data Migration

```typescript
// src/services/storage/migration/DataMigration.ts
export class DataMigration {
  async migrate(backup: MigrationBackup): Promise<void> {
    try {
      // 1. Create new database schema
      await this.createSchema();
      
      // 2. Migrate accounts
      await this.migrateAccounts(backup.accounts);
      
      // 3. Migrate settings
      await this.migrateSettings(backup.settings);
      
      // 4. Migrate contacts
      await this.migrateContacts(backup.contacts);
      
      // 5. Verify migration
      await this.verifyMigration(backup);
    } catch (error) {
      // 6. Handle failure
      await this.handleMigrationFailure(error, backup);
    }
  }

  private async migrateAccounts(accounts: Account[]): Promise<void> {
    const db = await this.getWaSqliteConnection();
    
    // Use transaction for atomicity
    await db.transaction(async (tx) => {
      for (const account of accounts) {
        await tx.execute(`
          INSERT INTO accounts (did, public_key_hex, created_at, updated_at)
          VALUES (?, ?, ?, ?)
        `, [
          account.did,
          account.publicKeyHex,
          account.createdAt,
          account.updatedAt
        ]);
      }
    });
  }

  private async verifyMigration(backup: MigrationBackup): Promise<void> {
    const db = await this.getWaSqliteConnection();
    
    // Verify account count
    const accountCount = await db.selectValue(
      'SELECT COUNT(*) FROM accounts'
    );
    if (accountCount !== backup.accounts.length) {
      throw new StorageError(
        'Account count mismatch',
        StorageErrorCodes.VERIFICATION_FAILED
      );
    }

    // Verify data integrity
    await this.verifyDataIntegrity(backup);
  }
}
```

### 3. Rollback Strategy

```typescript
// src/services/storage/migration/RollbackService.ts
export class RollbackService {
  async rollback(backup: MigrationBackup): Promise<void> {
    try {
      // 1. Stop all database operations
      await this.stopDatabaseOperations();
      
      // 2. Restore from backup
      await this.restoreFromBackup(backup);
      
      // 3. Verify restoration
      await this.verifyRestoration(backup);
      
      // 4. Clean up wa-sqlite
      await this.cleanupWaSqlite();
    } catch (error) {
      throw new StorageError(
        'Rollback failed',
        StorageErrorCodes.ROLLBACK_FAILED,
        error
      );
    }
  }

  private async restoreFromBackup(backup: MigrationBackup): Promise<void> {
    const dexieDB = new Dexie('TimeSafariDB');
    
    // Restore accounts
    await dexieDB.accounts.bulkPut(backup.accounts);
    
    // Restore settings
    await dexieDB.settings.bulkPut(backup.settings);
    
    // Restore contacts
    await dexieDB.contacts.bulkPut(backup.contacts);
  }
}
```

## Migration UI

```vue
<!-- src/components/MigrationProgress.vue -->
<template>
  <div class="migration-progress">
    <h2>Database Migration</h2>
    
    <div class="progress-container">
      <div class="progress-bar" :style="{ width: `${progress}%` }" />
      <div class="progress-text">{{ progress }}%</div>
    </div>
    
    <div class="status-message">{{ statusMessage }}</div>
    
    <div v-if="error" class="error-message">
      {{ error }}
      <button @click="retryMigration">Retry</button>
    </div>
  </div>
</template>

<script setup lang="ts">
import { ref, onMounted } from 'vue';
import { MigrationService } from '@/services/storage/migration/MigrationService';

const progress = ref(0);
const statusMessage = ref('Preparing migration...');
const error = ref<string | null>(null);

const migrationService = MigrationService.getInstance();

async function startMigration() {
  try {
    // 1. Preparation
    statusMessage.value = 'Creating backup...';
    await migrationService.prepare();
    progress.value = 20;
    
    // 2. Data migration
    statusMessage.value = 'Migrating data...';
    await migrationService.migrate();
    progress.value = 80;
    
    // 3. Verification
    statusMessage.value = 'Verifying migration...';
    await migrationService.verify();
    progress.value = 100;
    
    statusMessage.value = 'Migration completed successfully!';
  } catch (err) {
    error.value = err instanceof Error ? err.message : 'Migration failed';
    statusMessage.value = 'Migration failed';
  }
}

async function retryMigration() {
  error.value = null;
  progress.value = 0;
  await startMigration();
}

onMounted(() => {
  startMigration();
});
</script>

<style scoped>
.migration-progress {
  padding: 2rem;
  max-width: 600px;
  margin: 0 auto;
}

.progress-container {
  position: relative;
  height: 20px;
  background: #eee;
  border-radius: 10px;
  overflow: hidden;
  margin: 1rem 0;
}

.progress-bar {
  position: absolute;
  height: 100%;
  background: #4CAF50;
  transition: width 0.3s ease;
}

.progress-text {
  position: absolute;
  width: 100%;
  text-align: center;
  line-height: 20px;
  color: #000;
}

.status-message {
  text-align: center;
  margin: 1rem 0;
}

.error-message {
  color: #f44336;
  text-align: center;
  margin: 1rem 0;
}

button {
  margin-top: 1rem;
  padding: 0.5rem 1rem;
  background: #2196F3;
  color: white;
  border: none;
  border-radius: 4px;
  cursor: pointer;
}

button:hover {
  background: #1976D2;
}
</style>
```

## Testing Strategy

1. **Unit Tests**
   ```typescript
   // src/services/storage/migration/__tests__/MigrationService.spec.ts
   describe('MigrationService', () => {
     it('should create valid backup', async () => {
       const service = MigrationService.getInstance();
       const backup = await service.createBackup();
       
       expect(backup).toBeDefined();
       expect(backup.accounts).toBeInstanceOf(Array);
       expect(backup.settings).toBeInstanceOf(Array);
       expect(backup.contacts).toBeInstanceOf(Array);
     });

     it('should migrate data correctly', async () => {
       const service = MigrationService.getInstance();
       const backup = await service.createBackup();
       
       await service.migrate(backup);
       
       // Verify migration
       const accounts = await service.getMigratedAccounts();
       expect(accounts).toHaveLength(backup.accounts.length);
     });

     it('should handle rollback correctly', async () => {
       const service = MigrationService.getInstance();
       const backup = await service.createBackup();
       
       // Simulate failed migration
       await service.migrate(backup);
       await service.simulateFailure();
       
       // Perform rollback
       await service.rollback(backup);
       
       // Verify rollback
       const accounts = await service.getOriginalAccounts();
       expect(accounts).toHaveLength(backup.accounts.length);
     });
   });
   ```

2. **Integration Tests**
   ```typescript
   // src/services/storage/migration/__tests__/integration/Migration.spec.ts
   describe('Migration Integration', () => {
     it('should handle concurrent access during migration', async () => {
       const service = MigrationService.getInstance();
       
       // Start migration
       const migrationPromise = service.migrate();
       
       // Simulate concurrent access
       const accessPromises = Array(5).fill(null).map(() =>
         service.getAccount('did:test:123')
       );
       
       // Wait for all operations
       const [migrationResult, ...accessResults] = await Promise.allSettled([
         migrationPromise,
         ...accessPromises
       ]);
       
       // Verify results
       expect(migrationResult.status).toBe('fulfilled');
       expect(accessResults.some(r => r.status === 'rejected')).toBe(true);
     });

     it('should maintain data integrity during platform transition', async () => {
       const service = MigrationService.getInstance();
       
       // Simulate platform change
       await service.simulatePlatformChange();
       
       // Verify data
       const accounts = await service.getAllAccounts();
       const settings = await service.getAllSettings();
       const contacts = await service.getAllContacts();
       
       expect(accounts).toBeDefined();
       expect(settings).toBeDefined();
       expect(contacts).toBeDefined();
     });
   });
   ```

## Success Criteria

1. **Data Integrity**
   - [ ] All accounts migrated successfully
   - [ ] All settings preserved
   - [ ] All contacts transferred
   - [ ] No data corruption

2. **Performance**
   - [ ] Migration completes within acceptable time
   - [ ] No significant performance degradation
   - [ ] Efficient storage usage
   - [ ] Smooth user experience

3. **Security**
   - [ ] Encrypted data remains secure
   - [ ] Access controls maintained
   - [ ] No sensitive data exposure
   - [ ] Secure backup process

4. **User Experience**
   - [ ] Clear migration progress
   - [ ] Informative error messages
   - [ ] Automatic recovery from failures
   - [ ] No data loss

## Rollback Plan

1. **Automatic Rollback**
   - Triggered by migration failure
   - Restores from verified backup
   - Maintains data consistency
   - Logs rollback reason

2. **Manual Rollback**
   - Available through settings
   - Requires user confirmation
   - Preserves backup data
   - Provides rollback status

3. **Emergency Recovery**
   - Manual backup restoration
   - Database repair tools
   - Data recovery procedures
   - Support contact information

## Post-Migration

1. **Verification**
   - Data integrity checks
   - Performance monitoring
   - Error rate tracking
   - User feedback collection

2. **Cleanup**
   - Remove old database
   - Clear migration artifacts
   - Update application state
   - Archive backup data

3. **Monitoring**
   - Track migration success rate
   - Monitor performance metrics
   - Collect error reports
   - Gather user feedback

## Support

For assistance with migration:
1. Check the troubleshooting guide
2. Review error logs
3. Contact support team
4. Submit issue report

## Timeline

1. **Preparation Phase** (1 week)
   - Backup system implementation
   - Migration service development
   - Testing framework setup

2. **Testing Phase** (2 weeks)
   - Unit testing
   - Integration testing
   - Performance testing
   - Security testing

3. **Deployment Phase** (1 week)
   - Staged rollout
   - Monitoring
   - Support preparation
   - Documentation updates

4. **Post-Deployment** (2 weeks)
   - Monitoring
   - Bug fixes
   - Performance optimization
   - User feedback collection