crowd-funder-for-time-pwa/doc/migration-to-wa-sqlite.md

# Migration Guide: Dexie to absurd-sql

## Overview

This document outlines the migration process from Dexie.js to absurd-sql for the TimeSafari app's storage implementation. The migration aims to provide a consistent SQLite-based storage solution across all platforms while maintaining data integrity and ensuring a smooth transition for users.

**Current Status**: The migration is in **Phase 2** with a well-defined migration fence in place. Core settings and account data have been migrated, with contact migration in progress. **ActiveDid migration has been implemented** to ensure user identity continuity.

**⚠️ UPDATE**: The migration fence is now implemented through the **PlatformServiceMixin** rather than a `USE_DEXIE_DB` constant. This provides a cleaner, more maintainable approach to database access control.

## Migration Goals

1. **Data Integrity**
   - Preserve all existing data
   - Maintain data relationships
   - Ensure data consistency
   - **Preserve user's active identity**

2. **Performance**
   - Improve query performance
   - Reduce storage overhead
   - Optimize for platform-specific capabilities

3. **User Experience**
   - Seamless transition with no data loss
   - Maintain user's active identity and preferences
   - Preserve application state

## Migration Architecture

### Migration Fence

The migration fence is now defined by the **PlatformServiceMixin** in `src/utils/PlatformServiceMixin.ts`:

- **PlatformServiceMixin**: Centralized database access with caching and utilities
- **Migration Tools**: Exclusive interface between legacy and new databases
- **Service Layer**: All database operations go through PlatformService

### Migration Order

The migration follows a specific order to maintain data integrity:

1. **Accounts** (foundational - contains DIDs)
2. **Settings** (references accountDid, activeDid)
3. **ActiveDid** (depends on accounts and settings) ⭐ **NEW**
4. **Contacts** (independent, but migrated after accounts for consistency)

## ActiveDid Migration ⭐ **NEW FEATURE**

### Problem Solved

Previously, the `activeDid` setting was not migrated from Dexie to SQLite, causing users to lose their active identity after migration.

### Solution Implemented

The migration now includes a dedicated step for migrating the `activeDid`:

1. **Detection**: Identifies the `activeDid` from Dexie master settings
2. **Validation**: Verifies the `activeDid` exists in SQLite accounts
3. **Migration**: Updates SQLite master settings with the `activeDid`
4. **Error Handling**: Graceful handling of missing accounts

### Implementation Details

#### New Function: `migrateActiveDid()`

```typescript
export async function migrateActiveDid(): Promise<MigrationResult> {
  // 1. Get Dexie settings to find the activeDid
  const dexieSettings = await getDexieSettings();
  const masterSettings = dexieSettings.find(setting => !setting.accountDid);
  
  // 2. Verify the activeDid exists in SQLite accounts
  const accountExists = await platformService.dbQuery(
    "SELECT did FROM accounts WHERE did = ?",
    [dexieActiveDid],
  );
  
  // 3. Update SQLite master settings
  await updateDefaultSettings({ activeDid: dexieActiveDid });
   }
   ```

#### Enhanced `migrateSettings()` Function

The settings migration now includes activeDid handling:

- Extracts `activeDid` from Dexie master settings
- Validates account existence in SQLite
- Updates SQLite master settings with the `activeDid`

#### Updated `migrateAll()` Function

The complete migration now includes a dedicated step for activeDid:

```typescript
// Step 3: Migrate ActiveDid (depends on accounts and settings)
logger.info("[MigrationService] Step 3: Migrating activeDid...");
const activeDidResult = await migrateActiveDid();
```

### Benefits

- ✅ **User Identity Preservation**: Users maintain their active identity
- ✅ **Seamless Experience**: No need to manually select identity after migration
- ✅ **Data Consistency**: Ensures all identity-related settings are preserved
- ✅ **Error Resilience**: Graceful handling of edge cases

## Migration Process

### Phase 1: Preparation ✅

- [x] PlatformServiceMixin implementation
- [x] Implement data comparison tools
- [x] Create migration service structure

### Phase 2: Core Migration ✅

- [x] Account migration with `importFromMnemonic`
- [x] Settings migration (excluding activeDid)
- [x] **ActiveDid migration** ⭐ **COMPLETED**
- [x] Contact migration framework

### Phase 3: Validation and Cleanup 🔄

- [ ] Comprehensive data validation
- [ ] Performance testing
- [ ] User acceptance testing
- [ ] Dexie removal

## Usage

### Manual Migration

```typescript
import { migrateAll, migrateActiveDid } from '../services/indexedDBMigrationService';

// Complete migration
const result = await migrateAll();

// Or migrate just the activeDid
const activeDidResult = await migrateActiveDid();
```

### Migration Verification

```typescript
import { compareDatabases } from '../services/indexedDBMigrationService';

const comparison = await compareDatabases();
console.log('Migration differences:', comparison.differences);
```

### PlatformServiceMixin Integration

After migration, use the mixin for all database operations:

```typescript
// Use mixin methods for database access
const contacts = await this.$contacts();
const settings = await this.$settings();
const result = await this.$db("SELECT * FROM contacts WHERE did = ?", [accountDid]);
```

## Error Handling

### ActiveDid Migration Errors

- **Missing Account**: If the `activeDid` from Dexie doesn't exist in SQLite accounts
- **Database Errors**: Connection or query failures
- **Settings Update Failures**: Issues updating SQLite master settings

### Recovery Strategies

1. **Automatic Recovery**: Migration continues even if activeDid migration fails
2. **Manual Recovery**: Users can manually select their identity after migration
3. **Fallback**: System creates new identity if none exists

## Security Considerations

### Data Protection

- All sensitive data (mnemonics, private keys) are encrypted
- Migration preserves encryption standards
- No plaintext data exposure during migration

### Identity Verification

- ActiveDid migration validates account existence
- Prevents setting non-existent identities as active
- Maintains cryptographic integrity

## Testing

### Migration Testing

```bash
# Run migration
npm run migrate

# Verify results
npm run test:migration
```

### ActiveDid Testing

   ```typescript
// Test activeDid migration specifically
const result = await migrateActiveDid();
expect(result.success).toBe(true);
expect(result.warnings).toContain('Successfully migrated activeDid');
```

### PlatformServiceMixin Testing

```typescript
// Test mixin integration
describe('PlatformServiceMixin', () => {
  it('should provide database access methods', async () => {
    const contacts = await this.$contacts();
    const settings = await this.$settings();
    expect(contacts).toBeDefined();
    expect(settings).toBeDefined();
  });
});
```

## Troubleshooting

### Common Issues

1. **ActiveDid Not Found**
   - Ensure accounts were migrated before activeDid migration
   - Check that the Dexie activeDid exists in SQLite accounts

2. **Migration Failures**
   - Verify Dexie database is accessible
   - Check SQLite database permissions
   - Review migration logs for specific errors

3. **Data Inconsistencies**
   - Use `compareDatabases()` to identify differences
   - Re-run migration if necessary
   - Check for duplicate or conflicting records

4. **PlatformServiceMixin Issues**
   - Ensure mixin is properly imported and used
   - Check that all database operations use mixin methods
   - Verify caching and error handling work correctly

### Debugging

```typescript
// Debug migration process
import { logger } from '../utils/logger';

logger.debug('[Migration] Starting migration process...');
const result = await migrateAll();
logger.debug('[Migration] Migration completed:', result);
   ```

## Benefits of PlatformServiceMixin Approach

1. **Centralized Access**: Single point of control for all database operations
2. **Caching**: Built-in caching for performance optimization
3. **Type Safety**: Enhanced TypeScript integration
4. **Error Handling**: Consistent error handling across components
5. **Code Reduction**: Up to 80% reduction in database boilerplate
6. **Maintainability**: Single source of truth for database patterns

## Migration Status Checklist

### ✅ Completed

- [x] PlatformServiceMixin implementation
- [x] SQLite database service
- [x] Migration tools
- [x] Settings migration
- [x] Account migration
- [x] ActiveDid migration

### 🔄 In Progress

- [ ] Contact migration
- [ ] DatabaseUtil to PlatformServiceMixin migration
- [ ] File-by-file migration

### ❌ Not Started

- [ ] Legacy Dexie removal
- [ ] Final cleanup and validation

---

**Author**: Matthew Raymer
**Created**: 2025-07-05
**Status**: Active Migration Phase
**Last Updated**: 2025-07-05
**Note**: Migration fence now implemented through PlatformServiceMixin instead of USE_DEXIE_DB constant