This commit significantly improves SQLite database management and IPC communication
in the TimeSafari Electron app. Key changes include:
- Add new IPC handlers for database lifecycle management:
- sqlite-open: Open database connections
- sqlite-close: Close database connections
- sqlite-is-db-open: Check database connection status
- get-path: Retrieve database path
- get-base-path: Get base directory path
- Enhance SQLite initialization with:
- Improved error handling and recovery mechanisms
- Detailed logging for all database operations
- State verification and tracking
- Proper cleanup of IPC handlers
- Transaction state management
- Security improvements:
- Validate all IPC channels
- Implement proper file permissions (0o755)
- Add connection state verification
- Secure error handling and logging
- Performance optimizations:
- Implement WAL journal mode
- Configure optimal PRAGMA settings
- Add connection pooling support
- Implement retry logic with exponential backoff
Technical details:
- Add SQLiteError class for detailed error tracking
- Implement handler registration tracking
- Add comprehensive logging with operation tagging
- Update preload script with new valid channels
- Add type definitions for all SQLite operations
Testing:
- All handlers include proper error handling
- State verification before operations
- Recovery mechanisms for failed operations
- Logging for debugging and monitoring
Author: Matthew Raymer
- Comment out all info and debug logs in sqlite-migrations.ts
- Maintain logging structure for future debugging
- Reduce console output during normal operation
- Keep error handling and logging infrastructure intact
This change reduces noise in the console while maintaining the ability
to re-enable detailed logging for debugging purposes.
- Add structured SQLite configuration in main process with separate settings for
initialization and operations
- Implement proper retry logic with configurable attempts and delays
- Add connection pool management to prevent resource exhaustion
- Reduce initialization timeout from 30s to 10s for faster feedback
- Add proper cleanup of timeouts and resources
- Maintain consistent retry behavior in preload script
This change addresses the cascade of SQLite timeout errors seen in the logs
by implementing proper timing controls and resource management. The main
process now handles initialization more robustly with configurable retries,
while the preload script maintains its existing retry behavior for
compatibility.
Security Impact:
- No security implications
- Improves application stability
- Prevents resource exhaustion
Testing:
- Verify SQLite initialization completes within new timeout
- Confirm retry behavior works as expected
- Check that connection pool limits are respected
- Ensure proper cleanup of resources
Current Status:
- SQLite plugin successfully initializes in main process
- Preload script and context bridge working correctly
- IPC handlers for SQLite operations not registered
- Type definitions out of sync with implementation
Changes Made:
- Added comprehensive debug logging in preload script
- Implemented retry logic for SQLite operations (3 attempts, 1s delay)
- Added proper type definitions for SQLite connection options
- Defined strict channel validation for IPC communication
- Enhanced error handling and logging throughout
Type Definitions Updates:
- Aligned ElectronAPI interface with actual implementation
- Added proper typing for SQLite operations
- Structured IPC renderer interface with correct method signatures
Next Steps:
- Register missing IPC handlers in main process
- Update type definitions to match implementation
- Add proper error recovery for SQLite operations
- Address Content Security Policy warnings
Affected Files:
- electron/src/preload.ts
- src/types/electron.d.ts
- src/utils/debug-electron.ts
- src/services/platforms/ElectronPlatformService.ts
Note: This is a transitional commit. While the structure is improved,
database operations are not yet functional due to missing IPC handlers.
This commit updates the Electron preload script and type definitions to improve
SQLite integration and IPC communication. The changes include:
- Enhanced preload script (electron/src/preload.ts):
* Added detailed logging for SQLite operations and IPC communication
* Implemented retry logic for SQLite operations (3 attempts, 1s delay)
* Added proper type definitions for SQLite connection options
* Defined strict channel validation for IPC communication
* Improved error handling and logging throughout
- Updated type definitions (src/types/electron.d.ts):
* Aligned ElectronAPI interface with actual implementation
* Added proper typing for all SQLite operations
* Added environment variables (platform, isDev)
* Structured IPC renderer interface with proper method signatures
Current Status:
- Preload script initializes successfully
- SQLite availability check works (returns true)
- SQLite ready signal is properly received
- Database operations are failing with two types of errors:
1. "CapacitorSQLite not available" during initialization
2. "Cannot read properties of undefined" for SQLite methods
Next Steps:
- Verify context bridge exposure in renderer process
- Check main process SQLite handlers
- Debug database initialization
- Address Content Security Policy warning
Affected Files:
- Modified: electron/src/preload.ts
- Modified: src/types/electron.d.ts
Note: This is a transitional commit. While the preload script and type
definitions are now properly structured, database operations are not yet
functional. Further debugging and fixes are required to resolve the
SQLite integration issues.
The preload script (src/electron/preload.js) was removed as part of a refactor to
separate web and electron builds. This script was previously responsible for:
- Secure IPC communication between electron main and renderer processes
- SQLite database access bridge for the renderer process
- Context isolation and API exposure for electron-specific features
Current state:
- Web app builds successfully without preload script
- Electron builds fail due to missing preload script
- SQLite initialization works in main process but renderer can't access it
- Database operations fail with "Cannot read properties of undefined"
This commit is a breaking change for electron builds. The preload script will need
to be recreated to restore electron database functionality.
Affected files:
- Deleted: src/electron/preload.js
- Modified: src/main.electron.ts (removed DatabaseManager import)
- Modified: src/utils/logger.ts (simplified logging implementation)
- Modified: src/types/electron.d.ts (updated ElectronAPI interface)
- Modified: src/types/global.d.ts (updated window.electron type definition)
Next steps:
- Recreate preload script with proper SQLite bridge
- Update electron build configuration
- Restore database access in renderer process
- Consolidate preload script with secure IPC bridge and channel validation
- Ensure single initialization path in main process
- Add robust error handling and user feedback
- Fix race conditions in window creation and SQLite ready signal
Current state:
- SQLite initializes successfully in main process
- IPC bridge is established and events are transmitted
- Window creation and loading sequence is correct
- Renderer receives ready signal and mounts app
- Database operations still fail in renderer due to connection issues
Known issues:
- SQLite proxy not properly initialized in renderer
- Database connection not established in renderer
- Error logging attempts to use database before ready
- Connection state management needs improvement
This commit represents a stable point where IPC communication
is working but database operations need to be fixed.
- Remove connection object and connection pool logic
- Call all database methods directly on window.CapacitorSQLite with db name
- Refactor migrations, queries, and exec to match Capacitor SQLite Electron API
- Ensure preload script exposes both window.electron and window.CapacitorSQLite
- Fixes runtime errors related to missing query/run methods on connection
- Improves security and cross-platform compatibility
Co-authored-by: Matthew Raymer
- Add robust logging and error tracking to migration system
- Implement idempotent migrations with transaction safety
- Add detailed progress tracking and state verification
- Improve error handling with recoverable/non-recoverable states
- Add migration version validation and sequential checks
- Implement proper rollback handling with error recording
- Add table state verification and debugging
- Fix migration SQL parsing and parameter handling
- Add connection pool management and retry logic
- Add proper transaction isolation and state tracking
The migration system now provides:
- Atomic transactions per migration
- Automatic rollback on failure
- Detailed error logging and context
- State verification before/after operations
- Proper connection management
- Idempotent operations for safety
This commit improves database reliability and makes debugging
easier while maintaining proper process isolation. The changes
are focused on the migration system and do not require
restructuring the existing ElectronPlatformService architecture.
Technical details:
- Added MigrationError interface for better error tracking
- Added logMigrationProgress helper for consistent logging
- Added debugTableState for verification
- Added executeWithRetry for connection resilience
- Added validateMigrationVersions for safety
- Enhanced SQL parsing with better error handling
- Added proper transaction state management
- Added connection pool with retry logic
- Added detailed logging throughout migration process
Note: This commit addresses the database initialization issues
while maintaining the current architecture. Further improvements
to the ElectronPlatformService initialization will be handled
in a separate commit to maintain clear separation of concerns.
Add proper secret generation using Node's crypto module and initial data setup
for the electron environment. This commit:
- Implements secure random secret generation using crypto.randomBytes()
- Adds initial data migrations (002) with:
- Secret table with cryptographically secure random key
- Settings table with default API server
- Contacts, logs, and temp tables
- Improves SQL parameter handling for migrations
- Adds proper transaction safety and rollback support
- Includes comprehensive logging and error handling
Security:
- Uses Node's crypto module for secure random generation
- Implements proper base64 encoding for secrets
- Maintains transaction safety for all operations
Testing:
- Verified database structure via sqlite3 CLI
- Confirmed successful migration execution
- Validated initial data insertion
- Checked index creation and constraints
Note: This is a temporary solution for secret storage until a more
secure storage mechanism is implemented.
Split initial schema into two sequential migrations to prevent duplicate
table creation and improve migration clarity.
Changes:
- Separate initial schema into two distinct migrations:
* 001_initial_accounts (v1): Create accounts table & index
* 002_secret_and_settings (v2): Create remaining tables (secret, settings, contacts, logs, temp)
- Add version conflict detection to prevent duplicate migration versions
- Ensure migrations are sequential (no gaps)
- Update rollback scripts to only drop relevant tables
Technical Details:
- Add validateMigrationVersions() to check for:
* Duplicate version numbers
* Sequential version ordering
* Gaps in version numbers
- Validate migrations both at definition time and runtime
- Update schema_version tracking to reflect new versioning
Testing:
- Verified no duplicate table creation
- Confirmed migrations run in correct order
- Validated rollback procedures
- Checked version conflict detection
Add robust SQLite migration system with initial schema for TimeSafari desktop app.
Includes comprehensive error handling, transaction safety, and detailed logging.
Key Changes:
- Add migration system with version tracking and rollback support
- Implement initial schema with accounts, secret, settings, contacts tables
- Configure SQLite PRAGMAs for optimal performance and reliability
- Add detailed logging and state verification
- Set up WAL journal mode and connection pooling
Technical Details:
- Use @capacitor-community/sqlite for native SQLite integration
- Implement atomic transactions per migration
- Add SQL validation and parsing utilities
- Configure PRAGMAs:
* foreign_keys = ON
* journal_mode = WAL
* synchronous = NORMAL
* temp_store = MEMORY
* page_size = 4096
* cache_size = 2000
* busy_timeout = 15000
* wal_autocheckpoint = 1000
Note: Current version has duplicate migration v1 entries that need to be
addressed in a follow-up commit to ensure proper versioning.
Testing:
- Verified migrations run successfully
- Confirmed table creation and index setup
- Validated transaction safety and rollback
- Checked logging and error handling
- Add comprehensive logging for database operations
- Implement proper database path handling and permissions
- Set up WAL journal mode and PRAGMA configurations
- Create initial database schema with tables and triggers
- Add retry logic for database operations
- Implement proper error handling and state management
Current state:
- Database initialization works in main process
- Connection creation succeeds with proper permissions
- Schema creation and table setup complete
- Logging system fully implemented
- Known issue: Property name mismatch between main process and renderer
causing read-only mode conflicts (to be fixed in next commit)
Technical details:
- Uses WAL journal mode for better concurrency
- Implements proper file permissions checking
- Sets up foreign key constraints
- Creates tables: users, time_entries, time_goals, time_goal_entries
- Adds automatic timestamp triggers
- Implements proper connection lifecycle management
Security:
- Proper file permissions (755 for directory)
- No hardcoded credentials
- Proper error handling and logging
- Safe file path handling
Author: Matthew Raymer
Changes to sqlite-init.ts:
- Standardized connection options format between main and renderer processes
- Added explicit mode: 'rwc' to force read-write-create mode
- Added connection registration verification
- Added detailed logging of actual file paths
- Added journal mode verification to detect read-only state
- Removed redundant PRAGMA settings (now handled in main process)
- Added more detailed error reporting for connection failures
Security:
- Ensures consistent read-write permissions across processes
- Verifies database is not opened in read-only mode
- Maintains proper file permissions (644) and directory permissions (755)
Testing:
- Verified connection creation in both main and renderer processes
- Confirmed journal mode is set correctly
- Validated connection registration
- Tested error handling for invalid states
Author: Matthew Raymer
- Implement XDG Base Directory Specification for data storage
- Use $XDG_DATA_HOME (defaults to ~/.local/share) for data files
- Add proper directory permissions (700) for security
- Fallback to ~/.timesafari if XDG paths fail
- Add graceful degradation for SQLite failures
- Allow app to boot even if SQLite initialization fails
- Track and expose initialization errors via IPC
- Add availability checks to all SQLite operations
- Improve error reporting and logging
- Security improvements
- Set secure permissions (700) on data directories
- Verify directory permissions on existing paths
- Add proper error handling for permission issues
TODO:
- Fix database creation
- Add retry logic for initialization
- Add reinitialization capability
- Add more detailed error reporting
- Consider fallback storage options
- Remove unused isDev variable
- Add proper type annotations for event handlers
- Add explicit Promise<void> return type for loadIndexHtml
- Use unknown type for error parameters instead of any
- Fix event handler parameters by prefixing unused params with underscore
- Improve error handling type safety throughout the file
This commit improves type safety and removes unused code while maintaining
existing functionality. All TypeScript linter violations have been addressed
without changing the core behavior of the application.
Database paths are still broken here.
Security Impact: None - changes are type-level only
Testing: No functional changes, existing tests should pass
- Make database initialization non-blocking to prevent app crashes
- Add proper window lifecycle management and error handling
- Implement retry logic for index.html loading
- Add detailed logging for debugging
- Fix type safety issues in error handling
- Add proper cleanup on window close
WIP: Database path resolution still needs fixing
- Current issue: Path conflict between ~/.local/share and ~/.config
- Database connection failing with invalid response
- Multiple connection attempts occurring
This commit improves app stability but database connectivity needs to be addressed in a follow-up commit.
- Add logic in main process to resolve and create the correct database directory and file path using Electron's app, path, and fs modules
- Pass absolute dbPath to CapacitorSQLite plugin for reliable database creation
- Add extensive logging for debugging database location, permissions, and initialization
- Remove redundant open() call after createConnection in Electron platform service
- Add IPC handlers for essential SQLite operations (echo, createConnection, execute, query, closeConnection, isAvailable)
- Improve error handling and logging throughout initialization and IPC
- Still investigating database file creation and permissions issues
- Assign CapacitorSQLite instance to the global sqlitePlugin variable so it is accessible in IPC handlers.
- Enhance error logging in the IPC handler to include JSON stringification and stack trace for better debugging.
- Reveal that the generic .handle() method is not available on the plugin, clarifying the next steps for correct IPC wiring.
- Added "type": "module" to package.json to support ES module imports for Electron SQLite
- Renamed postcss.config.js to postcss.config.cjs to maintain CommonJS syntax
- This ensures build tools can properly load the PostCSS configuration
- Ensures devToolsWebContents is not null before calling focus() after opening DevTools in detached mode.
- Prevents runtime and linter errors in Electron main process.
- Changed from direct plugin usage to SQLiteConnection pattern
- Matches how platform services use the SQLite plugin
- Removed handle() method dependency
- Added proper method routing in IPC handler
The app now launches without initialization errors. Next steps:
- Test actual SQLite operations (createConnection, query, etc.)
- Verify database creation and access
- Add error handling for database operations
- Remove worker thread approach in favor of direct plugin initialization
- Initialize CapacitorSQLiteElectron in main process
- Set up IPC handler to forward SQLite operations
- Remove unused worker files and build config
This reverts to a simpler, more reliable approach for SQLite in Electron,
using the plugin as intended in the main process.
Matthew Raymer
Matthew Raymer
Trent Larson