jsnbuchanan/crowd-funder-for-time-pwa
1
0
Fork 1
forked from trent_larson/crowd-funder-for-time-pwa
Code Issues Pull Requests Projects Releases Wiki Activity

Finalize Dexie-to-SQLite migration prep: docs, circular dep removal, SQL helpers, tests

Browse Source 
- Removed all vestigial Dexie/USE_DEXIE_DB references from code and docs
- Centralized DB logic in PlatformServiceMixin; resolved logger/databaseUtil circular dependency
- Modularized SQL helpers (`$generateInsertStatement`, `$generateUpdateStatement`) and added unit tests
- Created/updated migration tracking docs and helper script for cross-machine progress
- Confirmed all lint/type checks and tests pass; ready for systematic file migration
This commit is contained in:
Matthew Raymer
2025-07-06 09:44:20 +00:00
parent 72041f29e1
commit 64e78fdbce
27 changed files with 8854 additions and 295 deletions
Download Patch File Download Diff File Expand all files Collapse all files

165
doc/migration-fence-definition.md
View File

@@ -4,11 +4,14 @@
This document defines the **migration fence** - the boundary between the legacy Dexie (IndexedDB) storage system and the new SQLite-based storage system in TimeSafari. The fence ensures controlled migration while maintaining data integrity and application stability.
**⚠️ UPDATE**: The migration fence is now implemented through the **PlatformServiceMixin** rather than a `USE_DEXIE_DB` constant. This provides a cleaner, more maintainable approach to database access control.
## Current Migration Status
### ✅ Completed Components
- **SQLite Database Service**: Fully implemented with absurd-sql
- **Platform Service Layer**: Unified database interface across platforms
- **PlatformServiceMixin**: Centralized database access with caching and utilities
- **Migration Tools**: Data comparison and transfer utilities
- **Schema Migration**: Complete table structure migration
- **Data Export/Import**: Backup and restore functionality
@@ -17,6 +20,7 @@ This document defines the **migration fence** - the boundary between the legacy
- **Settings Migration**: Core user settings transferred
- **Account Migration**: Identity and key management
- **Contact Migration**: User contact data (via import interface)
- **DatabaseUtil Migration**: Moving functions to PlatformServiceMixin
### ❌ Legacy Components (Fence Boundary)
- **Dexie Database**: Legacy IndexedDB storage (disabled by default)
@@ -25,22 +29,27 @@ This document defines the **migration fence** - the boundary between the legacy
## Migration Fence Definition
### 1. Configuration Boundary
### 1. PlatformServiceMixin Boundary
```typescript
// src/constants/app.ts
export const USE_DEXIE_DB = false; // FENCE: Controls legacy database access
// src/utils/PlatformServiceMixin.ts
export const PlatformServiceMixin = {
computed: {
platformService(): PlatformService {
// FENCE: All database operations go through platform service
// No direct Dexie access outside migration tools
return PlatformServiceFactory.getInstance();
}
}
}
```
**Fence Rule**: When `USE_DEXIE_DB = false`:
- All new data operations use SQLite
- Legacy Dexie database is not initialized
- Migration tools are the only path to legacy data
**Fence Rule**: When `USE_DEXIE_DB = true`:
- Legacy database is available for migration
- Dual-write operations may be enabled
- Migration tools can access both databases
**Fence Rule**: All database operations must use:
- `this.$db()` for read operations
- `this.$exec()` for write operations
- `this.$settings()` for settings access
- `this.$contacts()` for contact access
- No direct `db.` or `accountsDBPromise` access in application code
### 2. Service Layer Boundary
@@ -63,12 +72,10 @@ export class PlatformServiceFactory {
#### ✅ Allowed (Inside Fence)
```typescript
// Use platform service for all database operations
const platformService = PlatformServiceFactory.getInstance();
const contacts = await platformService.dbQuery(
"SELECT * FROM contacts WHERE did = ?",
[accountDid]
);
// Use PlatformServiceMixin for all database operations
const contacts = await this.$contacts();
const settings = await this.$settings();
const result = await this.$db("SELECT * FROM contacts WHERE did = ?", [accountDid]);
```
#### ❌ Forbidden (Outside Fence)
@@ -100,9 +107,9 @@ export async function compareDatabases(): Promise<DataComparison> {
### 1. Code Development Rules
#### New Feature Development
- **Always** use `PlatformService` for database operations
- **Always** use `PlatformServiceMixin` for database operations
- **Never** import or reference Dexie directly
- **Always** test with `USE_DEXIE_DB = false`
- **Always** use mixin methods like `this.$settings()`, `this.$contacts()`
#### Legacy Code Maintenance
- **Only** modify Dexie code for migration purposes
@@ -128,11 +135,10 @@ export async function compareDatabases(): Promise<DataComparison> {
// Required test pattern for migration
describe('Database Migration', () => {
it('should migrate data without loss', async () => {
// 1. Enable Dexie
// 2. Create test data
// 3. Run migration
// 4. Verify data integrity
// 5. Disable Dexie
// 1. Create test data in Dexie
// 2. Run migration
// 3. Verify data integrity in SQLite
// 4. Verify PlatformServiceMixin access
});
});
```
@@ -141,9 +147,9 @@ describe('Database Migration', () => {
```typescript
// Required test pattern for application features
describe('Feature with Database', () => {
it('should work with SQLite only', async () => {
// Test with USE_DEXIE_DB = false
// Verify all operations use PlatformService
it('should work with PlatformServiceMixin', async () => {
// Test with PlatformServiceMixin methods
// Verify all operations use mixin methods
});
});
```
@@ -162,7 +168,7 @@ describe('Feature with Database', () => {
"patterns": [
{
"group": ["../db/index"],
"message": "Use PlatformService instead of direct Dexie access"
"message": "Use PlatformServiceMixin instead of direct Dexie access"
}
]
}
@@ -186,87 +192,52 @@ describe('Feature with Database', () => {
#### Development Mode Validation
```typescript
// Development-only fence validation
if (import.meta.env.DEV && USE_DEXIE_DB) {
console.warn('⚠️ Dexie is enabled - migration mode active');
if (import.meta.env.DEV) {
console.warn('⚠️ Using PlatformServiceMixin for all database operations');
}
```
#### Production Safety
```typescript
// Production fence enforcement
if (import.meta.env.PROD && USE_DEXIE_DB) {
throw new Error('Dexie cannot be enabled in production');
if (import.meta.env.PROD) {
// All database operations must go through PlatformServiceMixin
// Direct Dexie access is not allowed
}
```
## Migration Fence Timeline
## Migration Status Checklist
### Phase 1: Fence Establishment ✅
- [x] Define migration fence boundaries
- [x] Implement PlatformService layer
- [x] Create migration tools
- [x] Set `USE_DEXIE_DB = false` by default
### ✅ Completed
- [x] PlatformServiceMixin implementation
- [x] SQLite database service
- [x] Migration tools
- [x] Settings migration
- [x] Account migration
- [x] ActiveDid migration
### Phase 2: Data Migration 🔄
- [x] Migrate core settings
- [x] Migrate account data
- [ ] Complete contact migration
- [ ] Verify all data integrity
### 🔄 In Progress
- [ ] Contact migration
- [ ] DatabaseUtil to PlatformServiceMixin migration
- [ ] File-by-file migration
### Phase 3: Code Cleanup 📋
- [ ] Remove unused Dexie imports
- [ ] Clean up legacy database code
- [ ] Update all documentation
- [ ] Remove migration tools
### ❌ Not Started
- [ ] Legacy Dexie removal
- [ ] Final cleanup and validation
### Phase 4: Fence Removal 🎯
- [ ] Remove `USE_DEXIE_DB` constant
- [ ] Remove Dexie dependencies
- [ ] Remove migration service
- [ ] Finalize SQLite-only architecture
## Benefits of PlatformServiceMixin Approach
## Security Considerations
1. **Centralized Access**: Single point of control for all database operations
2. **Caching**: Built-in caching for performance optimization
3. **Type Safety**: Enhanced TypeScript integration
4. **Error Handling**: Consistent error handling across components
5. **Code Reduction**: Up to 80% reduction in database boilerplate
6. **Maintainability**: Single source of truth for database patterns
### 1. Data Protection
- **Encryption**: Maintain encryption standards across migration
- **Access Control**: Preserve user privacy during migration
- **Audit Trail**: Log all migration operations
---
### 2. Error Handling
- **Graceful Degradation**: Handle migration failures gracefully
- **User Communication**: Clear messaging about migration status
- **Recovery Options**: Provide rollback mechanisms
## Performance Considerations
### 1. Migration Performance
- **Batch Operations**: Use transactions for bulk data transfer
- **Progress Indicators**: Show migration progress to users
- **Background Processing**: Non-blocking migration operations
### 2. Application Performance
- **Query Optimization**: Optimize SQLite queries for performance
- **Indexing Strategy**: Maintain proper database indexes
- **Memory Management**: Efficient memory usage during migration
## Documentation Requirements
### 1. Code Documentation
- **Migration Fence Comments**: Document fence boundaries in code
- **API Documentation**: Update all database API documentation
- **Migration Guides**: Comprehensive migration documentation
### 2. User Documentation
- **Migration Instructions**: Clear user migration steps
- **Troubleshooting**: Common migration issues and solutions
- **Rollback Instructions**: How to revert if needed
## Conclusion
The migration fence provides a controlled boundary between legacy and new database systems, ensuring:
- **Data Integrity**: No data loss during migration
- **Application Stability**: Consistent behavior across platforms
- **Development Clarity**: Clear guidelines for code development
- **Migration Safety**: Controlled and reversible migration process
This fence will remain in place until all data is successfully migrated and verified, at which point the legacy system can be safely removed.
**Author**: Matthew Raymer
**Created**: 2025-07-05
**Status**: Active Migration Phase
**Last Updated**: 2025-07-05
**Note**: Migration fence now implemented through PlatformServiceMixin instead of USE_DEXIE_DB constant