# Database Migration Guide

## Overview

The Database Migration feature allows you to compare and migrate data between Dexie (IndexedDB) and SQLite databases in the TimeSafari application. This is particularly useful during the transition from the old Dexie-based storage system to the new SQLite-based system.

## Features

### 1. Database Comparison

- Compare data between Dexie and SQLite databases
- View detailed differences in contacts and settings
- Identify added, modified, and missing records
- Export comparison results for analysis

### 2. Data Migration

- Migrate contacts from Dexie to SQLite
- Migrate settings from Dexie to SQLite
- Option to overwrite existing records or skip them
- Comprehensive error handling and reporting

### 3. User Interface

- Modern, responsive UI built with Tailwind CSS
- Real-time loading states and progress indicators
- Clear success and error messaging
- Export functionality for comparison data

## Prerequisites

### Enable Dexie Database

Before using the migration features, you must enable the Dexie database by setting:

```typescript
// In constants/app.ts
export const USE_DEXIE_DB = true;
```

**Note**: This should only be enabled temporarily during migration. Remember to set it back to `false` after migration is complete.

## Accessing the Migration Interface

1. Navigate to the **Account** page in the TimeSafari app
2. Scroll down to find the **Database Migration** link
3. Click the link to open the migration interface

## Using the Migration Interface

### Step 1: Compare Databases

1. Click the **"Compare Databases"** button
2. The system will retrieve data from both Dexie and SQLite databases
3. Review the comparison results showing:
   - Summary counts for each database
   - Detailed differences (added, modified, missing records)
   - Specific records that need attention

### Step 2: Review Differences

The comparison results are displayed in several sections:

#### Summary Cards

- **Dexie Contacts**: Number of contacts in Dexie database
- **SQLite Contacts**: Number of contacts in SQLite database
- **Dexie Settings**: Number of settings in Dexie database
- **SQLite Settings**: Number of settings in SQLite database

#### Contact Differences

- **Added**: Contacts in Dexie but not in SQLite
- **Modified**: Contacts that differ between databases
- **Missing**: Contacts in SQLite but not in Dexie

#### Settings Differences

- **Added**: Settings in Dexie but not in SQLite
- **Modified**: Settings that differ between databases
- **Missing**: Settings in SQLite but not in Dexie

### Step 3: Configure Migration Options

Before migrating data, configure the migration options:

- **Overwrite existing records**: When enabled, existing records in SQLite will be updated with data from Dexie. When disabled, existing records will be skipped.

### Step 4: Migrate Data

#### Migrate Contacts

1. Click the **"Migrate Contacts"** button
2. The system will transfer contacts from Dexie to SQLite
3. Review the migration results showing:
   - Number of contacts successfully migrated
   - Any warnings or errors encountered

#### Migrate Settings

1. Click the **"Migrate Settings"** button
2. The system will transfer settings from Dexie to SQLite
3. Review the migration results showing:
   - Number of settings successfully migrated
   - Any warnings or errors encountered

### Step 5: Export Comparison (Optional)

1. Click the **"Export Comparison"** button
2. A JSON file will be downloaded containing the complete comparison data
3. This file can be used for analysis or backup purposes

## Migration Process Details

### Contact Migration

The contact migration process:

1. **Retrieves** all contacts from Dexie database
2. **Checks** for existing contacts in SQLite by DID
3. **Inserts** new contacts or **updates** existing ones (if overwrite is enabled)
4. **Handles** complex fields like `contactMethods` (JSON arrays)
5. **Reports** success/failure for each contact

### Settings Migration

The settings migration process:

1. **Retrieves** all settings from Dexie database
2. **Focuses** on key user-facing settings:
   - `firstName`
   - `isRegistered`
   - `profileImageUrl`
   - `showShortcutBvc`
   - `searchBoxes`
3. **Preserves** other settings in SQLite
4. **Reports** success/failure for each setting

## Error Handling

### Common Issues

#### Dexie Database Not Enabled

**Error**: "Dexie database is not enabled"
**Solution**: Set `USE_DEXIE_DB = true` in `constants/app.ts`

#### Database Connection Issues

**Error**: "Failed to retrieve Dexie contacts"
**Solution**: Check that the Dexie database is properly initialized and accessible

#### SQLite Query Errors

**Error**: "Failed to retrieve SQLite contacts"
**Solution**: Verify that the SQLite database is properly set up and the platform service is working

#### Migration Failures

**Error**: "Migration failed: [specific error]"
**Solution**: Review the error details and check data integrity in both databases

### Error Recovery

1. **Review** the error messages carefully
2. **Check** the browser console for additional details
3. **Verify** database connectivity and permissions
4. **Retry** the operation if appropriate
5. **Export** comparison data for manual review if needed

## Best Practices

### Before Migration

1. **Backup** your data if possible
2. **Test** the migration on a small dataset first
3. **Verify** that both databases are accessible
4. **Review** the comparison results before migrating

### During Migration

1. **Don't** interrupt the migration process
2. **Monitor** the progress and error messages
3. **Note** any warnings or skipped records
4. **Export** comparison data for reference

### After Migration

1. **Verify** that data was migrated correctly
2. **Test** the application functionality
3. **Disable** Dexie database (`USE_DEXIE_DB = false`)
4. **Clean up** any temporary files or exports

## Technical Details

### Database Schema

The migration handles the following data structures:

#### Contacts Table

```typescript
interface Contact {
  did: string;                    // Decentralized Identifier
  name: string;                   // Contact name
  contactMethods: ContactMethod[]; // Array of contact methods
  nextPubKeyHashB64: string;      // Next public key hash
  notes: string;                  // Contact notes
  profileImageUrl: string;        // Profile image URL
  publicKeyBase64: string;        // Public key in base64
  seesMe: boolean;                // Visibility flag
  registered: boolean;            // Registration status
}
```

#### Settings Table

```typescript
interface Settings {
  id: number;                     // Settings ID
  accountDid: string;             // Account DID
  activeDid: string;              // Active DID
  firstName: string;              // User's first name
  isRegistered: boolean;          // Registration status
  profileImageUrl: string;        // Profile image URL
  showShortcutBvc: boolean;       // UI preference
  searchBoxes: any[];             // Search configuration
  // ... other fields
}
```

### Migration Logic

The migration service uses sophisticated comparison logic:

1. **Primary Key Matching**: Uses DID for contacts, ID for settings
2. **Deep Comparison**: Compares all fields including complex objects
3. **JSON Handling**: Properly handles JSON fields like `contactMethods` and `searchBoxes`
4. **Conflict Resolution**: Provides options for handling existing records

### Performance Considerations

- **Batch Processing**: Processes records one by one for reliability
- **Error Isolation**: Individual record failures don't stop the entire migration
- **Memory Management**: Handles large datasets efficiently
- **Progress Reporting**: Provides real-time feedback during migration

## Troubleshooting

### Migration Stuck

If the migration appears to be stuck:

1. **Check** the browser console for errors
2. **Refresh** the page and try again
3. **Verify** database connectivity
4. **Check** for large datasets that might take time

### Incomplete Migration

If migration doesn't complete:

1. **Review** error messages
2. **Check** data integrity in both databases
3. **Export** comparison data for manual review
4. **Consider** migrating in smaller batches

### Data Inconsistencies

If you notice data inconsistencies:

1. **Export** comparison data
2. **Review** the differences carefully
3. **Manually** verify critical records
4. **Consider** selective migration of specific records

## Support

For issues with the Database Migration feature:

1. **Check** this documentation first
2. **Review** the browser console for error details
3. **Export** comparison data for analysis
4. **Contact** the development team with specific error details

## Security Considerations

- **Data Privacy**: Migration data is processed locally and not sent to external servers
- **Access Control**: Only users with access to the account can perform migration
- **Data Integrity**: Migration preserves data integrity and handles conflicts gracefully
- **Audit Trail**: Export functionality provides an audit trail of migration operations

---

**Note**: This migration tool is designed for the transition period between database systems. Once migration is complete and verified, the Dexie database should be disabled to avoid confusion and potential data conflicts.