forked from trent_larson/crowd-funder-for-time-pwa
feat: Add comprehensive Database Migration UI component
- Create DatabaseMigration.vue with vue-facing-decorator and Tailwind CSS - Add complete UI for comparing and migrating data between Dexie and SQLite - Implement real-time loading states, error handling, and success feedback - Add navigation link to Account page for easy access - Include export functionality for comparison data - Create comprehensive documentation in doc/database-migration-guide.md - Fix all linting issues and ensure code quality standards - Support both contact and settings migration with overwrite options - Add visual difference analysis with summary cards and detailed breakdowns The component provides a professional interface for the migrationService.ts, enabling users to safely transfer data between database systems during the transition from Dexie to SQLite storage.
This commit is contained in:
Matthew Raymer
295
doc/database-migration-guide.md
Normal file
295
doc/database-migration-guide.md
Normal file
@@ -0,0 +1,295 @@
|
||||
# Database Migration Guide
|
||||
|
||||
## Overview
|
||||
|
||||
The Database Migration feature allows you to compare and migrate data between Dexie (IndexedDB) and SQLite databases in the TimeSafari application. This is particularly useful during the transition from the old Dexie-based storage system to the new SQLite-based system.
|
||||
|
||||
## Features
|
||||
|
||||
### 1. Database Comparison
|
||||
|
||||
- Compare data between Dexie and SQLite databases
|
||||
- View detailed differences in contacts and settings
|
||||
- Identify added, modified, and missing records
|
||||
- Export comparison results for analysis
|
||||
|
||||
### 2. Data Migration
|
||||
|
||||
- Migrate contacts from Dexie to SQLite
|
||||
- Migrate settings from Dexie to SQLite
|
||||
- Option to overwrite existing records or skip them
|
||||
- Comprehensive error handling and reporting
|
||||
|
||||
### 3. User Interface
|
||||
|
||||
- Modern, responsive UI built with Tailwind CSS
|
||||
- Real-time loading states and progress indicators
|
||||
- Clear success and error messaging
|
||||
- Export functionality for comparison data
|
||||
|
||||
## Prerequisites
|
||||
|
||||
### Enable Dexie Database
|
||||
|
||||
Before using the migration features, you must enable the Dexie database by setting:
|
||||
|
||||
```typescript
|
||||
// In constants/app.ts
|
||||
export const USE_DEXIE_DB = true;
|
||||
```
|
||||
|
||||
**Note**: This should only be enabled temporarily during migration. Remember to set it back to `false` after migration is complete.
|
||||
|
||||
## Accessing the Migration Interface
|
||||
|
||||
1. Navigate to the **Account** page in the TimeSafari app
|
||||
2. Scroll down to find the **Database Migration** link
|
||||
3. Click the link to open the migration interface
|
||||
|
||||
## Using the Migration Interface
|
||||
|
||||
### Step 1: Compare Databases
|
||||
|
||||
1. Click the **"Compare Databases"** button
|
||||
2. The system will retrieve data from both Dexie and SQLite databases
|
||||
3. Review the comparison results showing:
|
||||
- Summary counts for each database
|
||||
- Detailed differences (added, modified, missing records)
|
||||
- Specific records that need attention
|
||||
|
||||
### Step 2: Review Differences
|
||||
|
||||
The comparison results are displayed in several sections:
|
||||
|
||||
#### Summary Cards
|
||||
|
||||
- **Dexie Contacts**: Number of contacts in Dexie database
|
||||
- **SQLite Contacts**: Number of contacts in SQLite database
|
||||
- **Dexie Settings**: Number of settings in Dexie database
|
||||
- **SQLite Settings**: Number of settings in SQLite database
|
||||
|
||||
#### Contact Differences
|
||||
|
||||
- **Added**: Contacts in Dexie but not in SQLite
|
||||
- **Modified**: Contacts that differ between databases
|
||||
- **Missing**: Contacts in SQLite but not in Dexie
|
||||
|
||||
#### Settings Differences
|
||||
|
||||
- **Added**: Settings in Dexie but not in SQLite
|
||||
- **Modified**: Settings that differ between databases
|
||||
- **Missing**: Settings in SQLite but not in Dexie
|
||||
|
||||
### Step 3: Configure Migration Options
|
||||
|
||||
Before migrating data, configure the migration options:
|
||||
|
||||
- **Overwrite existing records**: When enabled, existing records in SQLite will be updated with data from Dexie. When disabled, existing records will be skipped.
|
||||
|
||||
### Step 4: Migrate Data
|
||||
|
||||
#### Migrate Contacts
|
||||
|
||||
1. Click the **"Migrate Contacts"** button
|
||||
2. The system will transfer contacts from Dexie to SQLite
|
||||
3. Review the migration results showing:
|
||||
- Number of contacts successfully migrated
|
||||
- Any warnings or errors encountered
|
||||
|
||||
#### Migrate Settings
|
||||
|
||||
1. Click the **"Migrate Settings"** button
|
||||
2. The system will transfer settings from Dexie to SQLite
|
||||
3. Review the migration results showing:
|
||||
- Number of settings successfully migrated
|
||||
- Any warnings or errors encountered
|
||||
|
||||
### Step 5: Export Comparison (Optional)
|
||||
|
||||
1. Click the **"Export Comparison"** button
|
||||
2. A JSON file will be downloaded containing the complete comparison data
|
||||
3. This file can be used for analysis or backup purposes
|
||||
|
||||
## Migration Process Details
|
||||
|
||||
### Contact Migration
|
||||
|
||||
The contact migration process:
|
||||
|
||||
1. **Retrieves** all contacts from Dexie database
|
||||
2. **Checks** for existing contacts in SQLite by DID
|
||||
3. **Inserts** new contacts or **updates** existing ones (if overwrite is enabled)
|
||||
4. **Handles** complex fields like `contactMethods` (JSON arrays)
|
||||
5. **Reports** success/failure for each contact
|
||||
|
||||
### Settings Migration
|
||||
|
||||
The settings migration process:
|
||||
|
||||
1. **Retrieves** all settings from Dexie database
|
||||
2. **Focuses** on key user-facing settings:
|
||||
- `firstName`
|
||||
- `isRegistered`
|
||||
- `profileImageUrl`
|
||||
- `showShortcutBvc`
|
||||
- `searchBoxes`
|
||||
3. **Preserves** other settings in SQLite
|
||||
4. **Reports** success/failure for each setting
|
||||
|
||||
## Error Handling
|
||||
|
||||
### Common Issues
|
||||
|
||||
#### Dexie Database Not Enabled
|
||||
|
||||
**Error**: "Dexie database is not enabled"
|
||||
**Solution**: Set `USE_DEXIE_DB = true` in `constants/app.ts`
|
||||
|
||||
#### Database Connection Issues
|
||||
|
||||
**Error**: "Failed to retrieve Dexie contacts"
|
||||
**Solution**: Check that the Dexie database is properly initialized and accessible
|
||||
|
||||
#### SQLite Query Errors
|
||||
|
||||
**Error**: "Failed to retrieve SQLite contacts"
|
||||
**Solution**: Verify that the SQLite database is properly set up and the platform service is working
|
||||
|
||||
#### Migration Failures
|
||||
|
||||
**Error**: "Migration failed: [specific error]"
|
||||
**Solution**: Review the error details and check data integrity in both databases
|
||||
|
||||
### Error Recovery
|
||||
|
||||
1. **Review** the error messages carefully
|
||||
2. **Check** the browser console for additional details
|
||||
3. **Verify** database connectivity and permissions
|
||||
4. **Retry** the operation if appropriate
|
||||
5. **Export** comparison data for manual review if needed
|
||||
|
||||
## Best Practices
|
||||
|
||||
### Before Migration
|
||||
|
||||
1. **Backup** your data if possible
|
||||
2. **Test** the migration on a small dataset first
|
||||
3. **Verify** that both databases are accessible
|
||||
4. **Review** the comparison results before migrating
|
||||
|
||||
### During Migration
|
||||
|
||||
1. **Don't** interrupt the migration process
|
||||
2. **Monitor** the progress and error messages
|
||||
3. **Note** any warnings or skipped records
|
||||
4. **Export** comparison data for reference
|
||||
|
||||
### After Migration
|
||||
|
||||
1. **Verify** that data was migrated correctly
|
||||
2. **Test** the application functionality
|
||||
3. **Disable** Dexie database (`USE_DEXIE_DB = false`)
|
||||
4. **Clean up** any temporary files or exports
|
||||
|
||||
## Technical Details
|
||||
|
||||
### Database Schema
|
||||
|
||||
The migration handles the following data structures:
|
||||
|
||||
#### Contacts Table
|
||||
|
||||
```typescript
|
||||
interface Contact {
|
||||
did: string; // Decentralized Identifier
|
||||
name: string; // Contact name
|
||||
contactMethods: ContactMethod[]; // Array of contact methods
|
||||
nextPubKeyHashB64: string; // Next public key hash
|
||||
notes: string; // Contact notes
|
||||
profileImageUrl: string; // Profile image URL
|
||||
publicKeyBase64: string; // Public key in base64
|
||||
seesMe: boolean; // Visibility flag
|
||||
registered: boolean; // Registration status
|
||||
}
|
||||
```
|
||||
|
||||
#### Settings Table
|
||||
|
||||
```typescript
|
||||
interface Settings {
|
||||
id: number; // Settings ID
|
||||
accountDid: string; // Account DID
|
||||
activeDid: string; // Active DID
|
||||
firstName: string; // User's first name
|
||||
isRegistered: boolean; // Registration status
|
||||
profileImageUrl: string; // Profile image URL
|
||||
showShortcutBvc: boolean; // UI preference
|
||||
searchBoxes: any[]; // Search configuration
|
||||
// ... other fields
|
||||
}
|
||||
```
|
||||
|
||||
### Migration Logic
|
||||
|
||||
The migration service uses sophisticated comparison logic:
|
||||
|
||||
1. **Primary Key Matching**: Uses DID for contacts, ID for settings
|
||||
2. **Deep Comparison**: Compares all fields including complex objects
|
||||
3. **JSON Handling**: Properly handles JSON fields like `contactMethods` and `searchBoxes`
|
||||
4. **Conflict Resolution**: Provides options for handling existing records
|
||||
|
||||
### Performance Considerations
|
||||
|
||||
- **Batch Processing**: Processes records one by one for reliability
|
||||
- **Error Isolation**: Individual record failures don't stop the entire migration
|
||||
- **Memory Management**: Handles large datasets efficiently
|
||||
- **Progress Reporting**: Provides real-time feedback during migration
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Migration Stuck
|
||||
|
||||
If the migration appears to be stuck:
|
||||
|
||||
1. **Check** the browser console for errors
|
||||
2. **Refresh** the page and try again
|
||||
3. **Verify** database connectivity
|
||||
4. **Check** for large datasets that might take time
|
||||
|
||||
### Incomplete Migration
|
||||
|
||||
If migration doesn't complete:
|
||||
|
||||
1. **Review** error messages
|
||||
2. **Check** data integrity in both databases
|
||||
3. **Export** comparison data for manual review
|
||||
4. **Consider** migrating in smaller batches
|
||||
|
||||
### Data Inconsistencies
|
||||
|
||||
If you notice data inconsistencies:
|
||||
|
||||
1. **Export** comparison data
|
||||
2. **Review** the differences carefully
|
||||
3. **Manually** verify critical records
|
||||
4. **Consider** selective migration of specific records
|
||||
|
||||
## Support
|
||||
|
||||
For issues with the Database Migration feature:
|
||||
|
||||
1. **Check** this documentation first
|
||||
2. **Review** the browser console for error details
|
||||
3. **Export** comparison data for analysis
|
||||
4. **Contact** the development team with specific error details
|
||||
|
||||
## Security Considerations
|
||||
|
||||
- **Data Privacy**: Migration data is processed locally and not sent to external servers
|
||||
- **Access Control**: Only users with access to the account can perform migration
|
||||
- **Data Integrity**: Migration preserves data integrity and handles conflicts gracefully
|
||||
- **Audit Trail**: Export functionality provides an audit trail of migration operations
|
||||
|
||||
---
|
||||
|
||||
**Note**: This migration tool is designed for the transition period between database systems. Once migration is complete and verified, the Dexie database should be disabled to avoid confusion and potential data conflicts.
|
||||
Reference in New Issue
Block a user