crowd-funder-for-time-pwa/doc/migration-fence-definition.md

# Migration Fence Definition: Dexie to SQLite

## Overview

This document defines the **migration fence** - the boundary between the legacy Dexie (IndexedDB) storage system and the new SQLite-based storage system in TimeSafari. The fence ensures controlled migration while maintaining data integrity and application stability.

## Current Migration Status

### ✅ Completed Components
- **SQLite Database Service**: Fully implemented with absurd-sql
- **Platform Service Layer**: Unified database interface across platforms
- **Migration Tools**: Data comparison and transfer utilities
- **Schema Migration**: Complete table structure migration
- **Data Export/Import**: Backup and restore functionality

### 🔄 Active Migration Components
- **Settings Migration**: Core user settings transferred
- **Account Migration**: Identity and key management
- **Contact Migration**: User contact data (via import interface)

### ❌ Legacy Components (Fence Boundary)
- **Dexie Database**: Legacy IndexedDB storage (disabled by default)
- **Dexie-Specific Code**: Direct database access patterns
- **Legacy Migration Paths**: Old data transfer methods

## Migration Fence Definition

### 1. Configuration Boundary

```typescript
// src/constants/app.ts
export const USE_DEXIE_DB = false; // FENCE: Controls legacy database access
```

**Fence Rule**: When `USE_DEXIE_DB = false`:
- All new data operations use SQLite
- Legacy Dexie database is not initialized
- Migration tools are the only path to legacy data

**Fence Rule**: When `USE_DEXIE_DB = true`:
- Legacy database is available for migration
- Dual-write operations may be enabled
- Migration tools can access both databases

### 2. Service Layer Boundary

```typescript
// src/services/PlatformServiceFactory.ts
export class PlatformServiceFactory {
  public static getInstance(): PlatformService {
    // FENCE: All database operations go through platform service
    // No direct Dexie access outside migration tools
  }
}
```

**Fence Rule**: All database operations must use:
- `PlatformService.dbQuery()` for read operations
- `PlatformService.dbExec()` for write operations
- No direct `db.` or `accountsDBPromise` access in application code

### 3. Data Access Patterns

#### ✅ Allowed (Inside Fence)
```typescript
// Use platform service for all database operations
const platformService = PlatformServiceFactory.getInstance();
const contacts = await platformService.dbQuery(
  "SELECT * FROM contacts WHERE did = ?",
  [accountDid]
);
```

#### ❌ Forbidden (Outside Fence)
```typescript
// Direct Dexie access (legacy pattern)
const contacts = await db.contacts.where('did').equals(accountDid).toArray();

// Direct database reference
const result = await accountsDBPromise;
```

### 4. Migration Tool Boundary

```typescript
// src/services/indexedDBMigrationService.ts
// FENCE: Only migration tools can access both databases
export async function compareDatabases(): Promise<DataComparison> {
  // This is the ONLY place where both databases are accessed
}
```

**Fence Rule**: Migration tools are the exclusive interface between:
- Legacy Dexie database
- New SQLite database
- Data comparison and transfer operations

## Migration Fence Guidelines

### 1. Code Development Rules

#### New Feature Development
- **Always** use `PlatformService` for database operations
- **Never** import or reference Dexie directly
- **Always** test with `USE_DEXIE_DB = false`

#### Legacy Code Maintenance
- **Only** modify Dexie code for migration purposes
- **Always** add migration tests for schema changes
- **Never** add new Dexie-specific features

### 2. Data Integrity Rules

#### Migration Safety
- **Always** create backups before migration
- **Always** verify data integrity after migration
- **Never** delete legacy data until verified

#### Rollback Strategy
- **Always** maintain ability to rollback to Dexie
- **Always** preserve migration logs
- **Never** assume migration is irreversible

### 3. Testing Requirements

#### Migration Testing
```typescript
// Required test pattern for migration
describe('Database Migration', () => {
  it('should migrate data without loss', async () => {
    // 1. Enable Dexie
    // 2. Create test data
    // 3. Run migration
    // 4. Verify data integrity
    // 5. Disable Dexie
  });
});
```

#### Application Testing
```typescript
// Required test pattern for application features
describe('Feature with Database', () => {
  it('should work with SQLite only', async () => {
    // Test with USE_DEXIE_DB = false
    // Verify all operations use PlatformService
  });
});
```

## Migration Fence Enforcement

### 1. Static Analysis

#### ESLint Rules
```json
{
  "rules": {
    "no-restricted-imports": [
      "error",
      {
        "patterns": [
          {
            "group": ["../db/index"],
            "message": "Use PlatformService instead of direct Dexie access"
          }
        ]
      }
    ]
  }
}
```

#### TypeScript Rules
```json
{
  "compilerOptions": {
    "strict": true,
    "noImplicitAny": true
  }
}
```

### 2. Runtime Checks

#### Development Mode Validation
```typescript
// Development-only fence validation
if (import.meta.env.DEV && USE_DEXIE_DB) {
  console.warn('⚠️ Dexie is enabled - migration mode active');
}
```

#### Production Safety
```typescript
// Production fence enforcement
if (import.meta.env.PROD && USE_DEXIE_DB) {
  throw new Error('Dexie cannot be enabled in production');
}
```

## Migration Fence Timeline

### Phase 1: Fence Establishment ✅
- [x] Define migration fence boundaries
- [x] Implement PlatformService layer
- [x] Create migration tools
- [x] Set `USE_DEXIE_DB = false` by default

### Phase 2: Data Migration 🔄
- [x] Migrate core settings
- [x] Migrate account data
- [ ] Complete contact migration
- [ ] Verify all data integrity

### Phase 3: Code Cleanup 📋
- [ ] Remove unused Dexie imports
- [ ] Clean up legacy database code
- [ ] Update all documentation
- [ ] Remove migration tools

### Phase 4: Fence Removal 🎯
- [ ] Remove `USE_DEXIE_DB` constant
- [ ] Remove Dexie dependencies
- [ ] Remove migration service
- [ ] Finalize SQLite-only architecture

## Security Considerations

### 1. Data Protection
- **Encryption**: Maintain encryption standards across migration
- **Access Control**: Preserve user privacy during migration
- **Audit Trail**: Log all migration operations

### 2. Error Handling
- **Graceful Degradation**: Handle migration failures gracefully
- **User Communication**: Clear messaging about migration status
- **Recovery Options**: Provide rollback mechanisms

## Performance Considerations

### 1. Migration Performance
- **Batch Operations**: Use transactions for bulk data transfer
- **Progress Indicators**: Show migration progress to users
- **Background Processing**: Non-blocking migration operations

### 2. Application Performance
- **Query Optimization**: Optimize SQLite queries for performance
- **Indexing Strategy**: Maintain proper database indexes
- **Memory Management**: Efficient memory usage during migration

## Documentation Requirements

### 1. Code Documentation
- **Migration Fence Comments**: Document fence boundaries in code
- **API Documentation**: Update all database API documentation
- **Migration Guides**: Comprehensive migration documentation

### 2. User Documentation
- **Migration Instructions**: Clear user migration steps
- **Troubleshooting**: Common migration issues and solutions
- **Rollback Instructions**: How to revert if needed

## Conclusion

The migration fence provides a controlled boundary between legacy and new database systems, ensuring:
- **Data Integrity**: No data loss during migration
- **Application Stability**: Consistent behavior across platforms
- **Development Clarity**: Clear guidelines for code development
- **Migration Safety**: Controlled and reversible migration process

This fence will remain in place until all data is successfully migrated and verified, at which point the legacy system can be safely removed.