crowd-funder-for-time-pwa/doc/activeDid-migration-plan.md

# ActiveDid Migration Plan - Separate Table Architecture

**Author**: Matthew Raymer
**Date**: 2025-08-29T15:00Z
**Status**: 🎯 **IMPLEMENTATION READY** - Streamlined for existing migration service

## Objective

Move the `activeDid` field from the `settings` table to a dedicated
`active_identity` table to improve database architecture, prevent data corruption,
and separate identity selection from user preferences.

## Result

This document serves as the focused implementation guide for the ActiveDid
migration, integrated with the existing IndexedDB migration service.

## Use/Run

Reference this document during implementation to ensure all migration
steps are followed correctly. **Critical**: This plan integrates with the
existing `indexedDBMigrationService.ts` and maintains backward compatibility.

## Context & Scope

- **In scope**:
  - Database schema modification for active_identity table with proper constraints
  - Migration of existing activeDid data with validation
  - Updates to PlatformServiceMixin API layer
  - Integration with existing IndexedDB migration service
- **Out of scope**:
  - Changes to user interface for identity selection
  - Modifications to identity creation logic
  - Changes to authentication flow
  - Updates to individual components (handled by API layer)

## Environment & Preconditions

- **OS/Runtime**: All platforms (Web, Electron, iOS, Android)
- **Versions/Builds**: Current development branch, SQLite database
- **Services/Endpoints**: Local database, PlatformServiceMixin, indexedDBMigrationService
- **Auth mode**: Existing authentication system unchanged

## Architecture / Process Overview

The migration integrates with the existing IndexedDB migration service:

```mermaid
flowchart TD
    A[Current State<br/>activeDid in settings] --> B[Phase 1: Schema Creation<br/>Add active_identity table with constraints]
    B --> C[Phase 2: Data Migration<br/>Copy activeDid data with validation]
    C --> D[Phase 3: API Updates<br/>Update PlatformServiceMixin methods]
    D --> E[Final State<br/>Separate active_identity table + dual-write]

    F[Existing IndexedDB Migration] --> G[Enhanced with active_identity support]
    G --> H[Maintains backward compatibility]
    H --> I[Preserves existing migration paths]
```

## Current Codebase Assessment

### ✅ What's Already Implemented

- **Database Schema**: `activeDid` field exists in `settings` table (`src/db-sql/migration.ts:67`)
- **Constants**: `MASTER_SETTINGS_KEY = "1"` is properly defined (`src/db/tables/settings.ts:88`)
- **Types**: Settings type includes `activeDid?: string` (`src/db/tables/settings.ts:25`)
- **Migration Infrastructure**: SQLite migration system exists (`src/db-sql/migration.ts:31`)
- **IndexedDB Migration Service**: Complete service exists (`src/services/indexedDBMigrationService.ts`)
- **PlatformServiceMixin**: Basic structure exists with `$updateActiveDid()` method

### ❌ What Needs Implementation

- **Missing Table**: `active_identity` table doesn't exist in current schema
- **Missing API Methods**: Core PlatformServiceMixin methods need implementation
- **Missing Types**: `ActiveIdentity` interface needs creation

## Required Changes

### **1. Database Schema via migration.ts**

Add migration 003 to existing MIGRATIONS array:

```typescript
// Add to MIGRATIONS array in src/db-sql/migration.ts
{
  name: "003_active_did_separate_table",
  sql: `
    -- Create new active_identity table with proper constraints
    CREATE TABLE IF NOT EXISTS active_identity (
      id INTEGER PRIMARY KEY CHECK (id = 1),
      activeDid TEXT NOT NULL,
      lastUpdated TEXT NOT NULL DEFAULT (datetime('now')),
      FOREIGN KEY (activeDid) REFERENCES accounts(did) ON DELETE CASCADE
    );

    -- Add performance indexes
    CREATE INDEX IF NOT EXISTS idx_active_identity_activeDid ON active_identity(activeDid);
    CREATE UNIQUE INDEX IF NOT EXISTS idx_active_identity_single_record ON active_identity(id);

    -- Insert default record (will be updated during migration)
    INSERT OR IGNORE INTO active_identity (id, activeDid, lastUpdated) VALUES (1, '', datetime('now'));

    -- MIGRATE EXISTING DATA: Copy activeDid from settings to active_identity
    -- This prevents data loss when migration runs on existing databases
    UPDATE active_identity
    SET activeDid = (SELECT activeDid FROM settings WHERE id = 1),
        lastUpdated = datetime('now')
    WHERE id = 1
    AND EXISTS (SELECT 1 FROM settings WHERE id = 1 AND activeDid IS NOT NULL AND activeDid != '');
  `,
},
```

**Critical Data Migration Logic**: This migration includes data transfer to
prevent users from losing their active identity selection when the migration
runs on existing databases.

### **2. Type Definitions**

Create ActiveIdentity interface in `src/db/tables/settings.ts`:

```typescript
// Add to src/db/tables/settings.ts
export interface ActiveIdentity {
  id: number;
  activeDid: string;
  lastUpdated: string;
}
```

### **3. PlatformServiceMixin Methods**

Implement required methods in `src/utils/PlatformServiceMixin.ts`:

```typescript
// Add to PlatformServiceMixin methods section
async $getActiveIdentity(): Promise<ActiveIdentity> {
  try {
    const result = await this.$dbQuery(
      "SELECT id, activeDid, lastUpdated FROM active_identity WHERE id = 1"
    );

    if (result?.values?.length) {
      const [id, activeDid, lastUpdated] = result.values[0];
      return { id: id as number, activeDid: activeDid as string, lastUpdated: lastUpdated as string };
    }

    // Return default if no record exists
    return { id: 1, activeDid: '', lastUpdated: new Date().toISOString() };
  } catch (error) {
    logger.error("[PlatformServiceMixin] Error getting active identity:", error);
    throw error;
  }
}

async $accountSettings(did?: string, defaults: Settings = {}): Promise<Settings> {
  try {
    // Get settings without activeDid
    const settings = await this.$getSettings(MASTER_SETTINGS_KEY, defaults);

    if (!settings) {
      return defaults;
    }

    // Get activeDid from new table>
// Enhanced update activeDid method with dual-write pattern
async $updateActiveDid(newDid: string | null): Promise<boolean> {
  try {
    if (newDid === null) {
      // Clear active identity in both tables
      await this.$dbExec(
        "UPDATE active_identity SET activeDid = '', lastUpdated = datetime('now') WHERE id = 1"
      );

      // Keep legacy field in sync (backward compatibility)
      await this.$dbExec(
        "UPDATE settings SET activeDid = '' WHERE id = ?",
        [MASTER_SETTINGS_KEY]
      );
    } else {
      // Validate DID exists before setting
      const accountExists = await this.$dbQuery(
        "SELECT did FROM accounts WHERE did = ?",
        [newDid]
      );

      if (!accountExists?.values?.length) {
        logger.error(`[PlatformServiceMixin] Cannot set activeDid to non-existent DID: ${newDid}`);
        return false;
      }

      // Update active identity in new table
      await this.$dbExec(
        "UPDATE active_identity SET activeDid = ?, lastUpdated = datetime('now') WHERE id = 1",
        [newDid]
      );

      // Keep legacy field in sync (backward compatibility)
      await this.$dbExec(
        "UPDATE settings SET activeDid = ? WHERE id = ?",
        [newDid, MASTER_SETTINGS_KEY]
      );
    }

    // Update internal tracking (existing functionality)
    await this._updateInternalActiveDid(newDid);
    return true;
  } catch (error) {
    logger.error("[PlatformServiceMixin] Error updating activeDid:", error);
    return false;
  }
}
```

### **4. Integration with Existing IndexedDB Migration Service**

The existing `indexedDBMigrationService.ts` already handles activeDid migration
from Dexie to SQLite. This plan adds the separate table architecture while
maintaining compatibility.

**No changes needed** to the existing migration service - it will continue to
work with the dual-write pattern.

### **5. Data Migration Strategy**

The migration 003 includes **automatic data migration** to prevent data loss:

1. **Schema Creation**: Creates `active_identity` table with proper constraints
2. **Data Transfer**: Automatically copies existing `activeDid` from `settings` table
3. **Validation**: Ensures data exists before attempting migration
4. **Atomic Operation**: Schema and data migration happen together

**Benefits of Single Migration Approach**:

- **No Data Loss**: Existing users keep their active identity selection
- **Atomic Operation**: If it fails, nothing is partially migrated
- **Simpler Tracking**: Only one migration to track and manage
- **Rollback Safety**: Complete rollback if issues arise

## What Doesn't Need to Change

- **All Vue components** - API layer handles migration transparently
- **Platform services** - Use PlatformServiceMixin, no direct access
- **User interface** - No changes to identity selection UI
- **Authentication flow** - Existing system unchanged
- **Component logic** - All activeDid handling through API methods
- **Migration system** - Use existing migration.ts approach
- **IndexedDB migration service** - Continues working unchanged
- **Existing database operations** - All current queries continue working

## Implementation Steps

### **Step 1: Add Migration**

- Add migration 003 to `MIGRATIONS` array in `src/db-sql/migration.ts`
- Deploy migration to create `active_identity` table

### **Step 2: Implement API Methods**

- Create `ActiveIdentity` interface in `src/db/tables/settings.ts`
- Implement `$getActiveIdentity()` method in PlatformServiceMixin
- Implement `$accountSettings()` method in PlatformServiceMixin
- Enhance `$updateActiveDid()` method with dual-write pattern

### **Step 3: Test Integration**

- Test new methods with existing components
- Verify dual-write pattern works correctly
- Validate backward compatibility

## Backward Compatibility

### **Critical Requirements**

- **IndexedDB Migration Service**: Must continue working unchanged
- **MASTER_SETTINGS_KEY = "1"**: Must be preserved for legacy support
- **Dual-Write Pattern**: Ensures both old and new systems stay in sync
- **Existing Queries**: All current database operations continue working

### **Migration Strategy**

- **Phase 1**: Add new table alongside existing system
- **Phase 2**: Use dual-write pattern during transition
- **Phase 3**: Future cleanup (not in current scope)

## Rollback Strategy

If migration fails, the existing `activeDid` field in settings table remains functional:

```sql
-- Rollback: Remove new table
DROP TABLE IF EXISTS active_identity;
```

No data loss risk - the legacy field continues working unchanged.

## Success Criteria

- [ ] `active_identity` table created with proper constraints
- [ ] All new PlatformServiceMixin methods implemented and tested
- [ ] Dual-write pattern working correctly
- [ ] Existing IndexedDB migration service continues working
- [ ] No breaking changes to existing functionality
- [ ] All platforms tested and verified
- [ ] Data migration validation successful (existing activeDid data preserved)

## Risks & Mitigation

### **Low Risk: Migration Failure**

- **Mitigation**: Rollback removes new table, legacy system continues working
- **Impact**: No data loss, no service interruption
- **Data Safety**: Existing activeDid data is preserved in settings table

### **Low Risk: Data Loss**

- **Mitigation**: Migration 003 includes automatic data transfer from settings
  to active_identity
- **Impact**: Users maintain their active identity selection
- **Validation**: Migration only runs if data exists and is valid

### **Low Risk: API Changes**

- **Mitigation**: Dual-write pattern maintains backward compatibility
- **Impact**: Existing components continue working unchanged

### **Low Risk: Performance Impact**

- **Mitigation**: Proper indexing and minimal additional queries
- **Impact**: Negligible performance change

## Summary

This migration plan:

1. **Adds new architecture** without breaking existing functionality
2. **Integrates seamlessly** with existing IndexedDB migration service
3. **Maintains full backward compatibility** during transition
4. **Requires minimal changes** to existing codebase
5. **Provides clear rollback path** if issues arise

The plan focuses only on necessary changes while preserving all existing
functionality and migration paths.