docs: add comprehensive migration guide for Dexie to wa-sqlite

- Add detailed migration process documentation including preparation, data migration, and rollback strategies\n- Include TypeScript implementation examples for MigrationService, DataMigration, and RollbackService\n- Add Vue component for migration progress tracking with error handling\n- Document testing strategy with unit and integration test examples\n- Define clear success criteria and timeline for migration\n- Include platform-specific considerations and prerequisites\n- Add post-migration verification and monitoring guidelines
4 weeks ago · 28e848e386
2 changed files with 821 additions and 0 deletions
--- a/.cursor/rules/wa-sqlite.mdc
+++ b/.cursor/rules/wa-sqlite.mdc
@ -0,0 +1,267 @@
+---
+description: 
+globs: 
+alwaysApply: true
+---
+# wa-sqlite Usage Guide
+
+## Table of Contents
+- [1. Overview](#1-overview)
+- [2. Installation](#2-installation)
+- [3. Basic Setup](#3-basic-setup)
+  - [3.1 Import and Initialize](#31-import-and-initialize)
+  - [3.2 Basic Database Operations](#32-basic-database-operations)
+- [4. Virtual File Systems (VFS)](#4-virtual-file-systems-vfs)
+  - [4.1 Available VFS Options](#41-available-vfs-options)
+  - [4.2 Using a VFS](#42-using-a-vfs)
+- [5. Best Practices](#5-best-practices)
+  - [5.1 Error Handling](#51-error-handling)
+  - [5.2 Transaction Management](#52-transaction-management)
+  - [5.3 Prepared Statements](#53-prepared-statements)
+- [6. Performance Considerations](#6-performance-considerations)
+- [7. Common Issues and Solutions](#7-common-issues-and-solutions)
+- [8. TypeScript Support](#8-typescript-support)
+
+## 1. Overview
+wa-sqlite is a WebAssembly build of SQLite that enables SQLite database operations in web browsers and JavaScript environments. It provides both synchronous and asynchronous builds, with support for custom virtual file systems (VFS) for persistent storage.
+
+## 2. Installation
+```bash
+npm install wa-sqlite
+# or
+yarn add wa-sqlite
+```
+
+## 3. Basic Setup
+
+### 3.1 Import and Initialize
+```javascript
+// Choose one of these imports based on your needs:
+// - wa-sqlite.mjs: Synchronous build
+// - wa-sqlite-async.mjs: Asynchronous build (required for async VFS)
+// - wa-sqlite-jspi.mjs: JSPI-based async build (experimental, Chromium only)
+import SQLiteESMFactory from 'wa-sqlite/dist/wa-sqlite.mjs';
+import * as SQLite from 'wa-sqlite';
+
+async function initDatabase() {
+  // Initialize SQLite module
+  const module = await SQLiteESMFactory();
+  const sqlite3 = SQLite.Factory(module);
+  
+  // Open database (returns a Promise)
+  const db = await sqlite3.open_v2('myDatabase');
+  return { sqlite3, db };
+}
+```
+
+### 3.2 Basic Database Operations
+```javascript
+async function basicOperations() {
+  const { sqlite3, db } = await initDatabase();
+  
+  try {
+    // Create a table
+    await sqlite3.exec(db, `
+      CREATE TABLE IF NOT EXISTS users (
+        id INTEGER PRIMARY KEY,
+        name TEXT NOT NULL,
+        email TEXT UNIQUE
+      )
+    `);
+    
+    // Insert data
+    await sqlite3.exec(db, `
+      INSERT INTO users (name, email) 
+      VALUES ('John Doe', 'john@example.com')
+    `);
+    
+    // Query data
+    const results = [];
+    await sqlite3.exec(db, 'SELECT * FROM users', (row, columns) => {
+      results.push({ row, columns });
+    });
+    
+    return results;
+  } finally {
+    // Always close the database when done
+    await sqlite3.close(db);
+  }
+}
+```
+
+## 4. Virtual File Systems (VFS)
+
+### 4.1 Available VFS Options
+wa-sqlite provides several VFS implementations for persistent storage:
+
+1. **IDBBatchAtomicVFS** (Recommended for general use)
+   - Uses IndexedDB with batch atomic writes
+   - Works in all contexts (Window, Worker, Service Worker)
+   - Supports WAL mode
+   - Best performance with `PRAGMA synchronous=normal`
+
+2. **IDBMirrorVFS**
+   - Keeps files in memory, persists to IndexedDB
+   - Works in all contexts
+   - Good for smaller databases
+
+3. **OPFS-based VFS** (Origin Private File System)
+   - Various implementations available:
+     - AccessHandlePoolVFS
+     - OPFSAdaptiveVFS
+     - OPFSCoopSyncVFS
+     - OPFSPermutedVFS
+   - Better performance but limited to Worker contexts
+
+### 4.2 Using a VFS
+```javascript
+import { IDBBatchAtomicVFS } from 'wa-sqlite/src/examples/IDBBatchAtomicVFS.js';
+import SQLiteESMFactory from 'wa-sqlite/dist/wa-sqlite-async.mjs';
+import * as SQLite from 'wa-sqlite';
+
+async function initDatabaseWithVFS() {
+  const module = await SQLiteESMFactory();
+  const sqlite3 = SQLite.Factory(module);
+  
+  // Register VFS
+  const vfs = await IDBBatchAtomicVFS.create('myApp', module);
+  sqlite3.vfs_register(vfs, true);
+  
+  // Open database with VFS
+  const db = await sqlite3.open_v2('myDatabase');
+  
+  // Configure for better performance
+  await sqlite3.exec(db, 'PRAGMA synchronous = normal');
+  await sqlite3.exec(db, 'PRAGMA journal_mode = WAL');
+  
+  return { sqlite3, db };
+}
+```
+
+## 5. Best Practices
+
+### 5.1 Error Handling
+```javascript
+async function safeDatabaseOperation() {
+  const { sqlite3, db } = await initDatabase();
+  
+  try {
+    await sqlite3.exec(db, 'SELECT * FROM non_existent_table');
+  } catch (error) {
+    if (error.code === SQLite.SQLITE_ERROR) {
+      console.error('SQL error:', error.message);
+    } else {
+      console.error('Database error:', error);
+    }
+  } finally {
+    await sqlite3.close(db);
+  }
+}
+```
+
+### 5.2 Transaction Management
+```javascript
+async function transactionExample() {
+  const { sqlite3, db } = await initDatabase();
+  
+  try {
+    await sqlite3.exec(db, 'BEGIN TRANSACTION');
+    
+    // Perform multiple operations
+    await sqlite3.exec(db, 'INSERT INTO users (name) VALUES (?)', ['Alice']);
+    await sqlite3.exec(db, 'INSERT INTO users (name) VALUES (?)', ['Bob']);
+    
+    await sqlite3.exec(db, 'COMMIT');
+  } catch (error) {
+    await sqlite3.exec(db, 'ROLLBACK');
+    throw error;
+  } finally {
+    await sqlite3.close(db);
+  }
+}
+```
+
+### 5.3 Prepared Statements
+```javascript
+async function preparedStatementExample() {
+  const { sqlite3, db } = await initDatabase();
+  
+  try {
+    // Prepare statement
+    const stmt = await sqlite3.prepare(db, 'SELECT * FROM users WHERE id = ?');
+    
+    // Execute with different parameters
+    await sqlite3.bind(stmt, 1, 1);
+    while (await sqlite3.step(stmt) === SQLite.SQLITE_ROW) {
+      const row = sqlite3.row(stmt);
+      console.log(row);
+    }
+    
+    // Reset and reuse
+    await sqlite3.reset(stmt);
+    await sqlite3.bind(stmt, 1, 2);
+    // ... execute again
+    
+    await sqlite3.finalize(stmt);
+  } finally {
+    await sqlite3.close(db);
+  }
+}
+```
+
+## 6. Performance Considerations
+
+1. **VFS Selection**
+   - Use IDBBatchAtomicVFS for general-purpose applications
+   - Consider OPFS-based VFS for better performance in Worker contexts
+   - Use MemoryVFS for temporary databases
+
+2. **Configuration**
+   - Set appropriate page size (default is usually fine)
+   - Use WAL mode for better concurrency
+   - Consider `PRAGMA synchronous=normal` for better performance
+   - Adjust cache size based on your needs
+
+3. **Concurrency**
+   - Use transactions for multiple operations
+   - Be aware of VFS-specific concurrency limitations
+   - Consider using Web Workers for heavy database operations
+
+## 7. Common Issues and Solutions
+
+1. **Database Locking**
+   - Use appropriate transaction isolation levels
+   - Implement retry logic for busy errors
+   - Consider using WAL mode
+
+2. **Storage Limitations**
+   - Be aware of browser storage quotas
+   - Implement cleanup strategies
+   - Monitor database size
+
+3. **Cross-Context Access**
+   - Use appropriate VFS for your context
+   - Consider message passing for cross-context communication
+   - Be aware of storage access limitations
+
+## 8. TypeScript Support
+wa-sqlite includes TypeScript definitions. The main types are:
+
+```typescript
+type SQLiteCompatibleType = number | string | Uint8Array | Array<number> | bigint | null;
+
+interface SQLiteAPI {
+  open_v2(filename: string, flags?: number, zVfs?: string): Promise<number>;
+  exec(db: number, sql: string, callback?: (row: any[], columns: string[]) => void): Promise<number>;
+  close(db: number): Promise<number>;
+  // ... other methods
+}
+```
+
+## Additional Resources
+
+- [Official GitHub Repository](https://github.com/rhashimoto/wa-sqlite)
+- [Online Demo](https://rhashimoto.github.io/wa-sqlite/demo/)
+- [API Reference](https://rhashimoto.github.io/wa-sqlite/docs/)
+- [FAQ](https://github.com/rhashimoto/wa-sqlite/issues?q=is%3Aissue+label%3Afaq+)
+- [Discussion Forums](https://github.com/rhashimoto/wa-sqlite/discussions) 
--- a/docs/migration-to-wa-sqlite.md
+++ b/docs/migration-to-wa-sqlite.md
@ -0,0 +1,554 @@
+# Migration Guide: Dexie to wa-sqlite
+
+## Overview
+
+This document outlines the migration process from Dexie.js to wa-sqlite for the TimeSafari app's storage implementation. The migration aims to provide a consistent SQLite-based storage solution across all platforms while maintaining data integrity and ensuring a smooth transition for users.
+
+## Migration Goals
+
+1. **Data Integrity**
+   - Preserve all existing data
+   - Maintain data relationships
+   - Ensure data consistency
+
+2. **Performance**
+   - Improve query performance
+   - Reduce storage overhead
+   - Optimize for platform-specific features
+
+3. **Security**
+   - Maintain or improve encryption
+   - Preserve access controls
+   - Enhance data protection
+
+4. **User Experience**
+   - Zero data loss
+   - Minimal downtime
+   - Automatic migration where possible
+
+## Prerequisites
+
+1. **Backup Requirements**
+   ```typescript
+   interface MigrationBackup {
+     timestamp: number;
+     accounts: Account[];
+     settings: Setting[];
+     contacts: Contact[];
+     metadata: {
+       version: string;
+       platform: string;
+       dexieVersion: string;
+     };
+   }
+   ```
+
+2. **Storage Requirements**
+   - Sufficient IndexedDB quota
+   - Available disk space for SQLite
+   - Backup storage space
+
+3. **Platform Support**
+   - Web: Modern browser with IndexedDB support
+   - iOS: iOS 13+ with SQLite support
+   - Android: Android 5+ with SQLite support
+   - Electron: Latest version with SQLite support
+
+## Migration Process
+
+### 1. Preparation
+
+```typescript
+// src/services/storage/migration/MigrationService.ts
+export class MigrationService {
+  private static instance: MigrationService;
+  private backup: MigrationBackup | null = null;
+
+  async prepare(): Promise<void> {
+    try {
+      // 1. Check prerequisites
+      await this.checkPrerequisites();
+      
+      // 2. Create backup
+      this.backup = await this.createBackup();
+      
+      // 3. Verify backup integrity
+      await this.verifyBackup();
+      
+      // 4. Initialize wa-sqlite
+      await this.initializeWaSqlite();
+    } catch (error) {
+      throw new StorageError(
+        'Migration preparation failed',
+        StorageErrorCodes.MIGRATION_FAILED,
+        error
+      );
+    }
+  }
+
+  private async checkPrerequisites(): Promise<void> {
+    // Check IndexedDB availability
+    if (!window.indexedDB) {
+      throw new StorageError(
+        'IndexedDB not available',
+        StorageErrorCodes.INITIALIZATION_FAILED
+      );
+    }
+
+    // Check storage quota
+    const quota = await navigator.storage.estimate();
+    if (quota.quota && quota.usage && quota.usage > quota.quota * 0.9) {
+      throw new StorageError(
+        'Insufficient storage space',
+        StorageErrorCodes.STORAGE_FULL
+      );
+    }
+
+    // Check platform support
+    const capabilities = await PlatformDetection.getCapabilities();
+    if (!capabilities.hasFileSystem) {
+      throw new StorageError(
+        'Platform does not support required features',
+        StorageErrorCodes.INITIALIZATION_FAILED
+      );
+    }
+  }
+
+  private async createBackup(): Promise<MigrationBackup> {
+    const dexieDB = new Dexie('TimeSafariDB');
+    
+    return {
+      timestamp: Date.now(),
+      accounts: await dexieDB.accounts.toArray(),
+      settings: await dexieDB.settings.toArray(),
+      contacts: await dexieDB.contacts.toArray(),
+      metadata: {
+        version: '1.0.0',
+        platform: await PlatformDetection.getPlatform(),
+        dexieVersion: Dexie.version
+      }
+    };
+  }
+}
+```
+
+### 2. Data Migration
+
+```typescript
+// src/services/storage/migration/DataMigration.ts
+export class DataMigration {
+  async migrate(backup: MigrationBackup): Promise<void> {
+    try {
+      // 1. Create new database schema
+      await this.createSchema();
+      
+      // 2. Migrate accounts
+      await this.migrateAccounts(backup.accounts);
+      
+      // 3. Migrate settings
+      await this.migrateSettings(backup.settings);
+      
+      // 4. Migrate contacts
+      await this.migrateContacts(backup.contacts);
+      
+      // 5. Verify migration
+      await this.verifyMigration(backup);
+    } catch (error) {
+      // 6. Handle failure
+      await this.handleMigrationFailure(error, backup);
+    }
+  }
+
+  private async migrateAccounts(accounts: Account[]): Promise<void> {
+    const db = await this.getWaSqliteConnection();
+    
+    // Use transaction for atomicity
+    await db.transaction(async (tx) => {
+      for (const account of accounts) {
+        await tx.execute(`
+          INSERT INTO accounts (did, public_key_hex, created_at, updated_at)
+          VALUES (?, ?, ?, ?)
+        `, [
+          account.did,
+          account.publicKeyHex,
+          account.createdAt,
+          account.updatedAt
+        ]);
+      }
+    });
+  }
+
+  private async verifyMigration(backup: MigrationBackup): Promise<void> {
+    const db = await this.getWaSqliteConnection();
+    
+    // Verify account count
+    const accountCount = await db.selectValue(
+      'SELECT COUNT(*) FROM accounts'
+    );
+    if (accountCount !== backup.accounts.length) {
+      throw new StorageError(
+        'Account count mismatch',
+        StorageErrorCodes.VERIFICATION_FAILED
+      );
+    }
+
+    // Verify data integrity
+    await this.verifyDataIntegrity(backup);
+  }
+}
+```
+
+### 3. Rollback Strategy
+
+```typescript
+// src/services/storage/migration/RollbackService.ts
+export class RollbackService {
+  async rollback(backup: MigrationBackup): Promise<void> {
+    try {
+      // 1. Stop all database operations
+      await this.stopDatabaseOperations();
+      
+      // 2. Restore from backup
+      await this.restoreFromBackup(backup);
+      
+      // 3. Verify restoration
+      await this.verifyRestoration(backup);
+      
+      // 4. Clean up wa-sqlite
+      await this.cleanupWaSqlite();
+    } catch (error) {
+      throw new StorageError(
+        'Rollback failed',
+        StorageErrorCodes.ROLLBACK_FAILED,
+        error
+      );
+    }
+  }
+
+  private async restoreFromBackup(backup: MigrationBackup): Promise<void> {
+    const dexieDB = new Dexie('TimeSafariDB');
+    
+    // Restore accounts
+    await dexieDB.accounts.bulkPut(backup.accounts);
+    
+    // Restore settings
+    await dexieDB.settings.bulkPut(backup.settings);
+    
+    // Restore contacts
+    await dexieDB.contacts.bulkPut(backup.contacts);
+  }
+}
+```
+
+## Migration UI
+
+```vue
+<!-- src/components/MigrationProgress.vue -->
+<template>
+  <div class="migration-progress">
+    <h2>Database Migration</h2>
+    
+    <div class="progress-container">
+      <div class="progress-bar" :style="{ width: `${progress}%` }" />
+      <div class="progress-text">{{ progress }}%</div>
+    </div>
+    
+    <div class="status-message">{{ statusMessage }}</div>
+    
+    <div v-if="error" class="error-message">
+      {{ error }}
+      <button @click="retryMigration">Retry</button>
+    </div>
+  </div>
+</template>
+
+<script setup lang="ts">
+import { ref, onMounted } from 'vue';
+import { MigrationService } from '@/services/storage/migration/MigrationService';
+
+const progress = ref(0);
+const statusMessage = ref('Preparing migration...');
+const error = ref<string | null>(null);
+
+const migrationService = MigrationService.getInstance();
+
+async function startMigration() {
+  try {
+    // 1. Preparation
+    statusMessage.value = 'Creating backup...';
+    await migrationService.prepare();
+    progress.value = 20;
+    
+    // 2. Data migration
+    statusMessage.value = 'Migrating data...';
+    await migrationService.migrate();
+    progress.value = 80;
+    
+    // 3. Verification
+    statusMessage.value = 'Verifying migration...';
+    await migrationService.verify();
+    progress.value = 100;
+    
+    statusMessage.value = 'Migration completed successfully!';
+  } catch (err) {
+    error.value = err instanceof Error ? err.message : 'Migration failed';
+    statusMessage.value = 'Migration failed';
+  }
+}
+
+async function retryMigration() {
+  error.value = null;
+  progress.value = 0;
+  await startMigration();
+}
+
+onMounted(() => {
+  startMigration();
+});
+</script>
+
+<style scoped>
+.migration-progress {
+  padding: 2rem;
+  max-width: 600px;
+  margin: 0 auto;
+}
+
+.progress-container {
+  position: relative;
+  height: 20px;
+  background: #eee;
+  border-radius: 10px;
+  overflow: hidden;
+  margin: 1rem 0;
+}
+
+.progress-bar {
+  position: absolute;
+  height: 100%;
+  background: #4CAF50;
+  transition: width 0.3s ease;
+}
+
+.progress-text {
+  position: absolute;
+  width: 100%;
+  text-align: center;
+  line-height: 20px;
+  color: #000;
+}
+
+.status-message {
+  text-align: center;
+  margin: 1rem 0;
+}
+
+.error-message {
+  color: #f44336;
+  text-align: center;
+  margin: 1rem 0;
+}
+
+button {
+  margin-top: 1rem;
+  padding: 0.5rem 1rem;
+  background: #2196F3;
+  color: white;
+  border: none;
+  border-radius: 4px;
+  cursor: pointer;
+}
+
+button:hover {
+  background: #1976D2;
+}
+</style>
+```
+
+## Testing Strategy
+
+1. **Unit Tests**
+   ```typescript
+   // src/services/storage/migration/__tests__/MigrationService.spec.ts
+   describe('MigrationService', () => {
+     it('should create valid backup', async () => {
+       const service = MigrationService.getInstance();
+       const backup = await service.createBackup();
+       
+       expect(backup).toBeDefined();
+       expect(backup.accounts).toBeInstanceOf(Array);
+       expect(backup.settings).toBeInstanceOf(Array);
+       expect(backup.contacts).toBeInstanceOf(Array);
+     });
+
+     it('should migrate data correctly', async () => {
+       const service = MigrationService.getInstance();
+       const backup = await service.createBackup();
+       
+       await service.migrate(backup);
+       
+       // Verify migration
+       const accounts = await service.getMigratedAccounts();
+       expect(accounts).toHaveLength(backup.accounts.length);
+     });
+
+     it('should handle rollback correctly', async () => {
+       const service = MigrationService.getInstance();
+       const backup = await service.createBackup();
+       
+       // Simulate failed migration
+       await service.migrate(backup);
+       await service.simulateFailure();
+       
+       // Perform rollback
+       await service.rollback(backup);
+       
+       // Verify rollback
+       const accounts = await service.getOriginalAccounts();
+       expect(accounts).toHaveLength(backup.accounts.length);
+     });
+   });
+   ```
+
+2. **Integration Tests**
+   ```typescript
+   // src/services/storage/migration/__tests__/integration/Migration.spec.ts
+   describe('Migration Integration', () => {
+     it('should handle concurrent access during migration', async () => {
+       const service = MigrationService.getInstance();
+       
+       // Start migration
+       const migrationPromise = service.migrate();
+       
+       // Simulate concurrent access
+       const accessPromises = Array(5).fill(null).map(() =>
+         service.getAccount('did:test:123')
+       );
+       
+       // Wait for all operations
+       const [migrationResult, ...accessResults] = await Promise.allSettled([
+         migrationPromise,
+         ...accessPromises
+       ]);
+       
+       // Verify results
+       expect(migrationResult.status).toBe('fulfilled');
+       expect(accessResults.some(r => r.status === 'rejected')).toBe(true);
+     });
+
+     it('should maintain data integrity during platform transition', async () => {
+       const service = MigrationService.getInstance();
+       
+       // Simulate platform change
+       await service.simulatePlatformChange();
+       
+       // Verify data
+       const accounts = await service.getAllAccounts();
+       const settings = await service.getAllSettings();
+       const contacts = await service.getAllContacts();
+       
+       expect(accounts).toBeDefined();
+       expect(settings).toBeDefined();
+       expect(contacts).toBeDefined();
+     });
+   });
+   ```
+
+## Success Criteria
+
+1. **Data Integrity**
+   - [ ] All accounts migrated successfully
+   - [ ] All settings preserved
+   - [ ] All contacts transferred
+   - [ ] No data corruption
+
+2. **Performance**
+   - [ ] Migration completes within acceptable time
+   - [ ] No significant performance degradation
+   - [ ] Efficient storage usage
+   - [ ] Smooth user experience
+
+3. **Security**
+   - [ ] Encrypted data remains secure
+   - [ ] Access controls maintained
+   - [ ] No sensitive data exposure
+   - [ ] Secure backup process
+
+4. **User Experience**
+   - [ ] Clear migration progress
+   - [ ] Informative error messages
+   - [ ] Automatic recovery from failures
+   - [ ] No data loss
+
+## Rollback Plan
+
+1. **Automatic Rollback**
+   - Triggered by migration failure
+   - Restores from verified backup
+   - Maintains data consistency
+   - Logs rollback reason
+
+2. **Manual Rollback**
+   - Available through settings
+   - Requires user confirmation
+   - Preserves backup data
+   - Provides rollback status
+
+3. **Emergency Recovery**
+   - Manual backup restoration
+   - Database repair tools
+   - Data recovery procedures
+   - Support contact information
+
+## Post-Migration
+
+1. **Verification**
+   - Data integrity checks
+   - Performance monitoring
+   - Error rate tracking
+   - User feedback collection
+
+2. **Cleanup**
+   - Remove old database
+   - Clear migration artifacts
+   - Update application state
+   - Archive backup data
+
+3. **Monitoring**
+   - Track migration success rate
+   - Monitor performance metrics
+   - Collect error reports
+   - Gather user feedback
+
+## Support
+
+For assistance with migration:
+1. Check the troubleshooting guide
+2. Review error logs
+3. Contact support team
+4. Submit issue report
+
+## Timeline
+
+1. **Preparation Phase** (1 week)
+   - Backup system implementation
+   - Migration service development
+   - Testing framework setup
+
+2. **Testing Phase** (2 weeks)
+   - Unit testing
+   - Integration testing
+   - Performance testing
+   - Security testing
+
+3. **Deployment Phase** (1 week)
+   - Staged rollout
+   - Monitoring
+   - Support preparation
+   - Documentation updates
+
+4. **Post-Deployment** (2 weeks)
+   - Monitoring
+   - Bug fixes
+   - Performance optimization
+   - User feedback collection