@ -1,8 +1,8 @@
# ActiveDid Migration Plan - Separate Table Architecture
**Author**: Matthew Raymer
**Date**: 2025-08-29T08:03 Z
**Status**: 🎯 **PLANNING** - Active migration planning phas e
**Date**: 2025-08-29T15:00 Z
**Status**: 🎯 **IMPLEMENTATION READY** - Streamlined for existing migration servic e
## Objective
@ -12,15 +12,14 @@ and separate identity selection from user preferences.
## Result
This document serves as the comprehensive planning and implementation
guide for the ActiveDid migration with enhanced data integrity and
rollback capabilities.
This document serves as the focused implementation guide for the ActiveDid
migration, integrated with the existing IndexedDB migration service.
## Use/Run
Reference this document during implementation to ensure all migration
steps are followed correctly and all stakeholders are aligned on the
approach .
steps are followed correctly. **Critical** : This plan integrates with the
existing `indexedDBMigrationService.ts` and maintains backward compatibility .
## Context & Scope
@ -28,9 +27,7 @@ approach.
- Database schema modification for active_identity table with proper constraints
- Migration of existing activeDid data with validation
- Updates to PlatformServiceMixin API layer
- Type definition updates
- Testing across all platforms
- Comprehensive rollback procedures
- Integration with existing IndexedDB migration service
- **Out of scope** :
- Changes to user interface for identity selection
- Modifications to identity creation logic
@ -41,51 +38,47 @@ approach.
- **OS/Runtime** : All platforms (Web, Electron, iOS, Android)
- **Versions/Builds** : Current development branch, SQLite database
- **Services/Endpoints** : Local database, PlatformServiceMixin
- **Services/Endpoints** : Local database, PlatformServiceMixin, indexedDBMigrationService
- **Auth mode** : Existing authentication system unchanged
## Architecture / Process Overview
The migration follows a phased approach to minimize risk and ensure
data integrity with enhanced validation and rollback capabilities:
The migration integrates with the existing IndexedDB migration service:
```mermaid
flowchart TD
A[Current State< br / > activeDid in settings] --> B[Phase 1: Schema Creation< br / > Add active_identity table with constraints]
B --> C[Phase 2: Data Migration< br / > Copy activeDid data with validation]
C --> D[Phase 3: API Updates< br / > Update PlatformServiceMixin methods]
D --> E[Phase 4: Cleanup< br / > Remove activeDid from settings]
E --> F[Final State< br / > Separate active_identity table]
D --> E[Final State< br / > Separate active_identity table + dual-write]
G[Enhanced Rollback Plan< br / > Schema and data rollback] --> H[Data Validation< br / > Verify integrity at each step]
H --> I[Platform Testing< br / > Test all platforms]
I --> J[Production Deployment< br / > Gradual rollout with monitoring]
K[Foreign Key Constraints< br / > Prevent future corruption] --> L[Performance Optimization< br / > Proper indexing]
L --> M[Error Recovery< br / > Graceful failure handling]
F[Existing IndexedDB Migration] --> G[Enhanced with active_identity support]
G --> H[Maintains backward compatibility]
H --> I[Preserves existing migration paths]
```
## Interfaces & Contracts
## Current Codebase Assessment
### ✅ What's Already Implemented
### Database Schema Changes
- **Database Schema** : `activeDid` field exists in `settings` table (`src/db-sql/migration.ts:67`)
- **Constants** : `MASTER_SETTINGS_KEY = "1"` is properly defined (`src/db/tables/settings.ts:88`)
- **Types** : Settings type includes `activeDid?: string` (`src/db/tables/settings.ts:25`)
- **Migration Infrastructure** : SQLite migration system exists (`src/db-sql/migration.ts:31`)
- **IndexedDB Migration Service** : Complete service exists (`src/services/indexedDBMigrationService.ts`)
- **PlatformServiceMixin** : Basic structure exists with `$updateActiveDid()` method
| Table | Current Schema | New Schema | Migration Required |
|-------|----------------|------------|-------------------|
| `settings` | `activeDid TEXT` | Field removed | Yes - data migration |
| `active_identity` | Does not exist | New table with `activeDid TEXT` + constraints | Yes - table creation |
### ❌ What Needs Implementation
### Enhanced API Contract Changes
- **Missing Table** : `active_identity` table doesn't exist in current schema
- **Missing API Methods** : Core PlatformServiceMixin methods need implementation
- **Missing Types** : `ActiveIdentity` interface needs creation
| Method | Current Behavior | New Behavior | Breaking Change |
|---------|------------------|--------------|-----------------|
| `$accountSettings()` | Returns settings with activeDid | Returns settings with activeDid from new table | No - backward compatible |
| `$saveSettings()` | Updates settings.activeDid | Updates active_identity.activeDid | Yes - requires updates |
| `$updateActiveDid()` | Updates internal tracking | Updates active_identity table | Yes - requires updates |
| `$getActiveIdentity()` | Does not exist | New method for active identity management | No - new functionality |
## Required Changes
## Repro: End-to-End Procedure
### **1. Database Schema via migration.ts**
### Phase 1: Enhanced Schema Creation via migration.ts
Add migration 003 to existing MIGRATIONS array:
```typescript
// Add to MIGRATIONS array in src/db-sql/migration.ts
@ -110,88 +103,54 @@ flowchart TD
},
```
### Phase 2: Enhanced Data Migration with Validation
### **2. Type Definitions**
Create ActiveIdentity interface in `src/db/tables/settings.ts` :
```typescript
// Enhanced migration function with comprehensive validation
async function migrateActiveDidToSeparateTable(): Promise< MigrationResult > {
const result: MigrationResult = {
success: false,
errors: [],
warnings: [],
dataMigrated: 0
};
// Add to src/db/tables/settings.ts
export interface ActiveIdentity {
id: number;
activeDid: string;
lastUpdated: string;
}
```
try {
// 1. Get current activeDid from settings (legacy approach)
const currentSettings = await retrieveSettingsForDefaultAccount();
const activeDid = currentSettings.activeDid;
### **3. PlatformServiceMixin Methods**
if (!activeDid) {
result.warnings.push("No activeDid found in current settings");
return result;
}
Implement required methods in `src/utils/PlatformServiceMixin.ts` :
// 2. Validate activeDid exists in accounts table
const accountExists = await dbQuery(
"SELECT did FROM accounts WHERE did = ?",
[activeDid]
```typescript
// Add to PlatformServiceMixin methods section
async $getActiveIdentity(): Promise< ActiveIdentity > {
try {
const result = await this.$dbQuery(
"SELECT id, activeDid, lastUpdated FROM active_identity WHERE id = 1"
);
if (!accountExists ?.values?.length) {
result.errors.push(`ActiveDid ${activeDid} not found in accounts table - data corruption detected`) ;
return result ;
if (result ?.values?.length) {
const [id, activeDid, lastUpdated] = result.values[0] ;
return { id: id as number, activeDid: activeDid as string, lastUpdated: lastUpdated as string } ;
}
// 3. Update active_identity table (new system)
await dbExec(
"UPDATE active_identity SET activeDid = ?, lastUpdated = datetime('now') WHERE id = 1",
[activeDid]
);
// 4. Ensure legacy settings.activeDid stays in sync (backward compatibility)
// This maintains compatibility with IndexedDB migration service
await dbExec(
"UPDATE settings SET activeDid = ? WHERE id = ?",
[activeDid, MASTER_SETTINGS_KEY]
);
dataMigrated = 1;
result.warnings.push(`Successfully migrated activeDid: ${activeDid}`);
// Return default if no record exists
return { id: 1, activeDid: '', lastUpdated: new Date().toISOString() };
} catch (error) {
result.errors.push(`Migration failed: ${error}` );
logger.error("[ActiveDid Migration] Critical error during migration:", error) ;
logger.error("[PlatformServiceMixin] Error getting active identity:", error);
throw error;
}
return result;
}
```
### Phase 3: Focused API Updates with Dual-Write Pattern
```typescript
// Updated PlatformServiceMixin method - maintains backward compatibility
async $accountSettings(did?: string, defaults: Settings = {}): Promise< Settings > {
try {
// Get settings without activeDid (unchanged logic)
const settings = await this._getSettingsWithoutActiveDid( );
// Get settings without activeDid
const settings = await this.$getSettings(MASTER_SETTINGS_KEY, defaults);
if (!settings) {
return defaults;
}
// Get activeDid from new table (new logic)
const activeIdentity = await this._getActiveIdentity();
// Return combined result (maintains backward compatibility)
return { ...settings, activeDid: activeIdentity.activeDid };
} catch (error) {
logger.error("[Settings Trace] ❌ Error in $accountSettings:", error);
return defaults;
}
}
// Get activeDid from new table>
// Enhanced update activeDid method with dual-write pattern
async $updateActiveDid(newDid: string | null): Promise< boolean > {
try {
@ -231,7 +190,7 @@ async $updateActiveDid(newDid: string | null): Promise<boolean> {
);
}
// Update internal tracking
// Update internal tracking (existing functionality)
await this._updateInternalActiveDid(newDid);
return true;
} catch (error) {
@ -241,336 +200,107 @@ async $updateActiveDid(newDid: string | null): Promise<boolean> {
}
```
### **Master Settings Functions Implementation Strategy **
### **4. Integration with Existing IndexedDB Migration Service **
#### **1. Update `retrieveSettingsForDefaultAccount()` **
The existing `indexedDBMigrationService.ts` already handles activeDid migration
from Dexie to SQLite. This plan adds the separate table architecture while
maintaining compatibility.
```typescript
// Enhanced implementation with active_identity table integration
export async function retrieveSettingsForDefaultAccount(): Promise< Settings > {
const platform = PlatformServiceFactory.getInstance();
// Get settings without activeDid
const sql = "SELECT id, accountDid, apiServer, filterFeedByNearby, filterFeedByVisible, " +
"finishedOnboarding, firstName, hideRegisterPromptOnNewContact, isRegistered, " +
"lastName, lastAckedOfferToUserJwtId, lastAckedOfferToUserProjectsJwtId, " +
"lastNotifiedClaimId, lastViewedClaimId, notifyingNewActivityTime, " +
"notifyingReminderMessage, notifyingReminderTime, partnerApiServer, " +
"passkeyExpirationMinutes, profileImageUrl, searchBoxes, showContactGivesInline, " +
"showGeneralAdvanced, showShortcutBvc, vapid, warnIfProdServer, warnIfTestServer, " +
"webPushServer FROM settings WHERE id = ?";
const result = await platform.dbQuery(sql, [MASTER_SETTINGS_KEY]);
if (!result) {
return DEFAULT_SETTINGS;
} else {
const settings = mapColumnsToValues(result.columns, result.values)[0] as Settings;
// Handle JSON parsing
if (settings.searchBoxes) {
settings.searchBoxes = JSON.parse(settings.searchBoxes);
}
**No changes needed** to the existing migration service - it will continue to
work with the dual-write pattern.
// Get activeDid from separate table
const activeIdentityResult = await platform.dbQuery(
"SELECT activeDid FROM active_identity WHERE id = 1"
);
## What Doesn't Need to Change
if (activeIdentityResult?.values?.length) {
const activeDid = activeIdentityResult.values[0][0] as string;
if (activeDid) {
// Validate activeDid exists in accounts
const accountExists = await platform.dbQuery(
"SELECT did FROM accounts WHERE did = ?",
[activeDid]
);
if (accountExists?.values?.length) {
settings.activeDid = activeDid;
} else {
logger.warn(`[databaseUtil] ActiveDid ${activeDid} not found in accounts, clearing`);
// Clear corrupted activeDid
await platform.dbExec(
"UPDATE active_identity SET activeDid = '', lastUpdated = datetime('now') WHERE id = 1"
);
}
}
}
- **All Vue components** - API layer handles migration transparently
- **Platform services** - Use PlatformServiceMixin, no direct access
- **User interface** - No changes to identity selection UI
- **Authentication flow** - Existing system unchanged
- **Component logic** - All activeDid handling through API methods
- **Migration system** - Use existing migration.ts approach
- **IndexedDB migration service** - Continues working unchanged
- **Existing database operations** - All current queries continue working
return settings;
}
}
```
## Implementation Steps
#### **2. Update `$getMergedSettings()` Method **
### **Step 1: Add Migration**
```typescript
// Enhanced implementation with active_identity table integration
async $getMergedSettings(defaultKey: string, accountDid?: string, defaultFallback: Settings = {}): Promise< Settings > {
try {
// Get default settings (now without activeDid)
const defaultSettings = await this.$getSettings(defaultKey, defaultFallback);
// If no account DID, return defaults with activeDid from separate table
if (!accountDid) {
if (defaultSettings) {
// Get activeDid from separate table
const activeIdentityResult = await this.$dbQuery(
"SELECT activeDid FROM active_identity WHERE id = 1"
);
if (activeIdentityResult?.values?.length) {
const activeDid = activeIdentityResult.values[0][0] as string;
if (activeDid) {
// Validate activeDid exists in accounts
const accountExists = await this.$dbQuery(
"SELECT did FROM accounts WHERE did = ?",
[activeDid]
);
if (accountExists?.values?.length) {
defaultSettings.activeDid = activeDid;
} else {
logger.warn(`[Settings Trace] ActiveDid ${activeDid} not found in accounts, clearing`);
// Clear corrupted activeDid
await this.$dbExec(
"UPDATE active_identity SET activeDid = '', lastUpdated = datetime('now') WHERE id = 1"
);
}
}
}
}
return defaultSettings || defaultFallback;
}
- Add migration 003 to `MIGRATIONS` array in `src/db-sql/migration.ts`
- Deploy migration to create `active_identity` table
// ... rest of existing implementation for account-specific settings
} catch (error) {
logger.error(`[Settings Trace] ❌ Failed to get merged settings:`, { defaultKey, accountDid, error });
return defaultFallback;
}
}
```
### **Step 2: Implement API Methods**
## What Works (Evidence)
- Create `ActiveIdentity` interface in `src/db/tables/settings.ts`
- Implement `$getActiveIdentity()` method in PlatformServiceMixin
- Implement `$accountSettings()` method in PlatformServiceMixin
- Enhance `$updateActiveDid()` method with dual-write pattern
- ✅ **Current activeDid storage** in settings table
- **Time** : 2025-08-29T08:03Z
- **Evidence** : `src/db-sql/migration.ts:67` - activeDid field exists in initial migration
- **Verify at** : Current database schema and Settings type definition
### **Step 3: Test Integration**
- ✅ **PlatformServiceMixin integration** with activeDid
- **Time** : 2025-08-29T08:03Z
- **Evidence** : `src/utils/PlatformServiceMixin.ts:108` - activeDid tracking
- **Verify at** : Component usage across all platforms
- Test new methods with existing components
- Verify dual-write pattern works correctly
- Validate backward compatibility
- ✅ **Database migration infrastructure** exists
- **Time** : 2025-08-29T08:03Z
- **Evidence** : `src/db-sql/migration.ts:31` - migration system in place
- **Verify at** : Existing migration scripts and database versioning
## Backward Compatibility
## What Doesn't (Evidence & Hypotheses)
### **Critical Requirements**
- ❌ **No separate active_identity table** exists
- **Time** : 2025-08-29T08:03Z
- **Evidence** : Database schema only shows settings table
- **Hypothesis** : Table needs to be created as part of migration
- **Next probe** : Add migration to existing MIGRATIONS array
- **IndexedDB Migration Service** : Must continue working unchanged
- **MASTER_SETTINGS_KEY = "1"** : Must be preserved for legacy support
- **Dual-Write Pattern** : Ensures both old and new systems stay in sync
- **Existing Queries** : All current database operations continue working
- ❌ **Data corruption issues** with orphaned activeDid references
- **Time** : 2025-08-29T08:03Z
- **Evidence** : `IdentitySwitcherView.vue:175` - `hasCorruptedIdentity` detection
- **Hypothesis** : Current schema allows activeDid to point to non-existent accounts
- **Next probe** : Implement foreign key constraints in new table
### **Migration Strategy**
## Risks, Limits, Assumptions
- **Phase 1** : Add new table alongside existing system
- **Phase 2** : Use dual-write pattern during transition
- **Phase 3** : Future cleanup (not in current scope)
- **Data Loss Risk** : Migration failure could lose activeDid values
- **Breaking Changes** : API updates required in PlatformServiceMixin
- **Rollback Complexity** : Schema changes make rollback difficult
- **Testing Overhead** : All platforms must be tested with new structure
- **Performance Impact** : Additional table join for activeDid retrieval
- **Migration Timing** : Must be coordinated with other database changes
- **Data Corruption** : Current system has documented corruption issues
- **Foreign Key Constraints** : New constraints may prevent some operations
## Rollback Strategy
## Enhanced Rollback Strategy
If migration fails, the existing `activeDid` field in settings table remains functional:
### **Schema Rollback**
```sql
-- If migration fails, restore original schema
-- Rollback: Remove new table
DROP TABLE IF EXISTS active_identity;
-- Restore activeDid field to settings table if needed
ALTER TABLE settings ADD COLUMN activeDid TEXT;
```
### **Data Rollback**
```typescript
// Rollback function to restore activeDid to settings table
async function rollbackActiveDidMigration(): Promise< boolean > {
try {
// Get activeDid from active_identity table
const activeIdentityResult = await dbQuery(
"SELECT activeDid FROM active_identity WHERE id = 1"
);
No data loss risk - the legacy field continues working unchanged.
if (activeIdentityResult?.values?.length) {
const activeDid = activeIdentityResult.values[0][0] as string;
## Success Criteria
// Restore to settings table
await dbExec(
"UPDATE settings SET activeDid = ? WHERE id = ?",
[activeDid, MASTER_SETTINGS_KEY]
);
- [ ] `active_identity` table created with proper constraints
- [ ] All new PlatformServiceMixin methods implemented and tested
- [ ] Dual-write pattern working correctly
- [ ] Existing IndexedDB migration service continues working
- [ ] No breaking changes to existing functionality
- [ ] All platforms tested and verified
return true;
}
## Risks & Mitigation
return false;
} catch (error) {
logger.error("[Rollback] Failed to restore activeDid:", error);
return false;
}
}
```
### **Rollback Triggers**
- Migration validation fails
- Data integrity checks fail
- Performance regression detected
- User reports data loss
- Cross-platform inconsistencies found
## Next Steps
| Owner | Task | Exit Criteria | Target Date (UTC) |
|-------|------|---------------|-------------------|
| Development Team | Add migration to existing MIGRATIONS array | Migration script integrated with existing system | 2025-08-30 |
| Development Team | Update type definitions | Settings type updated, ActiveIdentity type created | 2025-08-30 |
| Development Team | Update PlatformServiceMixin | Core methods updated and tested | 2025-08-31 |
| Development Team | Implement foreign key constraints | Schema validation prevents corruption | 2025-08-31 |
| QA Team | Platform testing | All platforms tested and verified | 2025-09-01 |
| Development Team | Deploy migration | Production deployment successful | 2025-09-02 |
## References
- [Database Migration Guide ](./database-migration-guide.md )
- [Dexie to SQLite Mapping ](./dexie-to-sqlite-mapping.md )
- [PlatformServiceMixin Documentation ](./component-communication-guide.md )
- [Migration Templates ](./migration-templates/ )
## Competence Hooks
- *Why this works* : Separates concerns between identity selection and
user preferences, prevents data corruption with foreign key constraints,
centralizes identity management through API layer
- *Common pitfalls* : Forgetting to implement foreign key constraints, not
testing rollback scenarios, missing data validation during migration,
over-engineering component updates when API layer handles everything
- *Next skill unlock* : Advanced database schema design with constraints,
migration planning with rollback strategies
- *Teach-back* : Explain the four-phase migration approach and why each
phase is necessary, especially the foreign key constraints
## Collaboration Hooks
- **Sign-off checklist** :
- [ ] Migration script integrated with existing MIGRATIONS array
- [ ] Foreign key constraints implemented and tested
- [ ] PlatformServiceMixin updated and tested
- [ ] Rollback procedures validated
- [ ] Performance impact assessed
- [ ] All stakeholders approve deployment timeline
## Assumptions & Limits
- Current activeDid values are valid and should be preserved
- All platforms can handle the additional database table
- Migration can be completed without user downtime
- Rollback to previous schema is acceptable if needed
- Performance impact of additional table join is minimal
- Foreign key constraints will prevent future corruption
- API layer updates will handle component compatibility
## What Needs to Change
### **1. Database Schema via migration.ts**
- Add migration to existing MIGRATIONS array in `src/db-sql/migration.ts`
- Create `active_identity` table with foreign key constraints
- Add performance indexes
- **Keep `activeDid` field in `settings` table temporarily** for backward compatibility
- **Preserve `MASTER_SETTINGS_KEY = "1"`** for legacy migration support
### **2. PlatformServiceMixin Methods**
- `$accountSettings()` - integrate with new table while maintaining backward compatibility
- `$saveSettings()` - handle activeDid in new table, sync with legacy field
- `$updateActiveDid()` - validate and update new table, sync with legacy field
- `$getActiveIdentity()` - new method for identity management
### **3. Master Settings Functions**
- `retrieveSettingsForDefaultAccount()` - integrate with new table while preserving legacy support
- `$getMergedSettings()` - integrate with new table while preserving legacy support
### **4. Type Definitions**
- **Keep `activeDid` in Settings type temporarily** for backward compatibility
- Create ActiveIdentity type for new table
- Update related interfaces
### **5. Legacy Compatibility**
- **Preserve `MASTER_SETTINGS_KEY = "1"`** for IndexedDB migration service
- **Maintain dual-write pattern** during transition period
- **Ensure legacy clients can still migrate** from Dexie to SQLite
### **Low Risk: Migration Failure**
## What Doesn't Need to Change
- **All Vue components** - API layer handles migration transparently
- **Platform services** - Use PlatformServiceMixin, no direct access
- **User interface** - No changes to identity selection UI
- **Authentication flow** - Existing system unchanged
- **Component logic** - All activeDid handling through API methods
- **Migration system** - Use existing migration.ts approach, not separate files
- **IndexedDB migration service** - Must continue working for legacy clients
- **Mitigation** : Rollback removes new table, legacy system continues working
- **Impact** : No data loss, no service interruption
## Enhanced Architecture: Dual-Write Pattern
### **Low Risk: API Changes**
### **Phase 1: Add New Table (Current)**
```typescript
// Create active_identity table
// Keep existing settings.activeDid for backward compatibility
// Use dual-write pattern during transition
```
- **Mitigation** : Dual-write pattern maintains backward compatibility
- **Impact** : Existing components continue working unchanged
### **Phase 2: Dual-Write Pattern**
```typescript
// When updating activeDid:
// 1. Update active_identity table (new system)
// 2. Update settings.activeDid (legacy compatibility)
// 3. Ensure both stay in sync
```
### **Low Risk: Performance Impact**
### **Phase 3: Future Cleanup (Not in Current Scope)**
```typescript
// Eventually:
// 1. Remove activeDid from settings table
// 2. Deprecate MASTER_SETTINGS_KEY
// 3. Use pure accountDid IS NULL pattern
// 4. Update IndexedDB migration service
```
- **Mitigation** : Proper indexing and minimal additional queries
- **Impact** : Negligible performance change
## Backward Compatibility Requirements
## Summary
### **Critical: IndexedDB Migration Service**
- **Must continue working** for users migrating from Dexie
- **Must recognize `id = "1"`** as master settings
- **Must preserve existing migration paths**
This migration plan:
### **Important: Legacy Database Operations**
- **Must continue working** for existing SQLite databases
- **Must handle both old and new patterns**
- **Must not break existing queries**
1. **Adds new architecture** without breaking existing functionality
2. **Integrates seamlessly** with existing IndexedDB migration service
3. **Maintains full backward compatibility** during transition
4. **Requires minimal changes** to existing codebase
5. **Provides clear rollback path** if issues arise
### **Desired: Cleaner Architecture**
- **New operations** use `accountDid IS NULL` pattern
- **Legacy operations** continue using `MASTER_SETTINGS_KEY`
- **Gradual migration** toward cleaner patterns
The plan focuses only on necessary changes while preserving all existing
functionality and migration paths.
Matthew Raymer
3 weeks ago