README-first/tech/progress/PROJECT-mesh-network.md

Utilizing a P2P network

Time Safari currently records all data on an Endorser server, routed through the general internet. This project will allow people to send their data directly P2P. The first iteration will most likely be over a LoRa mesh radio network run with Meshtastic devices.

Overview

The goal is to create a decentralized version of the Endorser system that can operate without internet connectivity, enabling local communities to share claims, confirmations, and visibility permissions through mesh networks. This is particularly valuable for:

• Remote areas with limited internet connectivity
• Emergency situations where infrastructure is compromised
• Privacy-conscious communities wanting local-first data sharing
• Resilient social networks that don't depend on centralized services

Task 1: P2P Claim Protocol Design

1.1 Message Structure & Packaging

Objective: Design efficient message formats for transmitting Endorser claims over bandwidth-constrained mesh networks.

Key Components:

Message Types

• Claim Broadcast: New claims being shared with neighbors
• Claim Request: Request for specific claims by ID, user, or criteria
• Claim Response: Response to requests with claim data
• Visibility Update: Changes to visibility permissions between users
• Merkle Sync: Synchronization of merkle hash chains for integrity verification
• Peer Discovery: Announcement of available data and capabilities

Message Format

{
  "version": "1.0",
  "type": "claim_broadcast|claim_request|claim_response|visibility_update|merkle_sync|peer_discovery",
  "timestamp": "ISO8601",
  "sender_did": "did:ethr:0x...",
  "message_id": "unique_id",
  "ttl": 5,
  "payload": {
    // Type-specific payload
  },
  "signature": "cryptographic_signature"
}

Payload Structures

Claim Broadcast Payload:

• Compressed JWT claim data
• Visibility hints (who can see what)
• Related claim IDs for context
• Geographic bounds (if location-based)
• Priority level (urgent vs. routine)

Claim Request Payload:

• Request type: by_id, by_user, by_location, by_type, recent_updates
• Filter criteria
• Maximum response size
• Requester's merkle hash for sync

Peer Discovery Payload:

• Available claim count by type
• Supported protocol versions
• Storage capacity and retention policy
• Geographic coverage area
• Last activity timestamp

1.2 Data Selection & Filtering

Objective: Implement intelligent data selection to maximize utility within bandwidth constraints.

Priority System:

1. High Priority: Recent claims from visible users, confirmations of user's own claims
2. Medium Priority: Claims matching user's interests/location, visibility updates
3. Low Priority: Older claims, claims from users with limited visibility

Filtering Mechanisms:

• Spatial Filtering: Only share claims within geographic relevance
• Temporal Filtering: Prioritize recent activity, age-out old claims
• Social Filtering: Prioritize claims from users in visibility network
• Type Filtering: Focus on specific claim types (gives, offers, projects)
• Size Filtering: Compress or truncate large claims

Adaptive Bandwidth Management:

• Monitor network conditions and adjust message frequency
• Implement exponential backoff for failed transmissions
• Use compression for claim content (gzip, custom dictionary)
• Batch multiple small claims into single messages

1.3 Privacy & Visibility Control

Objective: Maintain Endorser's privacy model in P2P environment.

Visibility Propagation:

• Encrypt claim content for intended recipients only
• Use public key cryptography for visibility-controlled data
• Maintain visibility graphs locally and sync changes
• Implement "visibility hints" to help routing without exposing content

Privacy-Preserving Discovery:

• Use bloom filters for private set intersection
• Implement zero-knowledge proofs for claim existence without revealing content
• Support anonymous claim requests through onion routing
• Enable selective disclosure of claim attributes

Trust & Reputation:

• Implement web-of-trust model for peer reliability
• Track and share reputation scores for claim accuracy
• Support claim verification through multiple confirmations
• Enable reporting of malicious or spam claims

1.4 Synchronization & Consistency

Objective: Ensure data consistency across the mesh network using merkle chains.

Merkle Chain Synchronization:

• Each peer maintains local merkle chain of all known claims
• Periodic sync requests to compare merkle roots with neighbors
• Efficient delta synchronization for missing claims
• Conflict resolution for competing claim versions

Eventual Consistency Model:

• Accept that perfect consistency isn't possible in P2P networks
• Implement vector clocks for causality tracking
• Support claim versioning and conflict resolution
• Enable manual conflict resolution for important discrepancies

Data Integrity:

• Cryptographic verification of all claims using JWT signatures
• Merkle proof verification for claim authenticity
• Detection and isolation of corrupted or malicious data
• Backup and recovery mechanisms for critical claims

Task 2: Meshtastic Network Integration

2.1 Meshtastic Protocol Adaptation

Objective: Integrate the P2P claim protocol with Meshtastic's LoRa mesh networking.

Protocol Mapping:

• Map claim message types to Meshtastic packet types
• Implement custom protobuf definitions for Endorser messages
• Handle Meshtastic's packet size limitations (237 bytes max)
• Implement message fragmentation for large claims

Meshtastic Integration Points:

• Use Meshtastic's MQTT gateway for phone connectivity
• Leverage Meshtastic's routing algorithms for message propagation
• Integrate with Meshtastic's encryption for transport security
• Utilize Meshtastic's node database for peer discovery

Message Fragmentation:

Fragment Header:
- Message ID (4 bytes)
- Fragment number (1 byte)  
- Total fragments (1 byte)
- Fragment payload (up to 230 bytes)

2.2 Device Architecture & Connectivity

Objective: Design the hardware and software architecture for mesh-enabled Time Safari.

Hardware Components:

• Meshtastic Node: T-Beam, Heltec, or similar LoRa device
• Mobile Device: Android/iOS phone running Time Safari
• Connection: Bluetooth or WiFi between phone and Meshtastic node

Software Architecture:

Time Safari App (Phone)
    ↓ Bluetooth/WiFi
Mesh Bridge Service (Phone or Node)
    ↓ Serial/MQTT
Meshtastic Firmware (Node)
    ↓ LoRa Radio
Mesh Network

Mesh Bridge Service:

• Translates between Endorser API calls and mesh messages
• Manages local claim storage and caching
• Handles message queuing and retry logic
• Provides offline-first operation with sync when connected

2.3 Mobile App Integration

Objective: Seamlessly integrate mesh networking into existing Time Safari interface.

User Interface Changes:

• Network Status Indicator: Show mesh connectivity and peer count
• Offline Mode: Clear indication when operating without internet
• Sync Status: Progress indicator for claim synchronization
• Mesh Settings: Configure mesh behavior, privacy settings, data limits

Data Management:

• Local Storage: SQLite database for offline claim storage
• Sync Queue: Queue outgoing claims for mesh transmission
• Conflict Resolution UI: Interface for resolving claim conflicts
• Data Usage: Monitor and limit mesh data consumption

Fallback Behavior:

• Graceful degradation when mesh is unavailable
• Automatic switching between mesh and internet connectivity
• Queuing of actions for later mesh transmission
• User notification of connectivity status changes

2.4 Network Management & Optimization

Objective: Optimize mesh network performance and reliability.

Routing Optimization:

• Implement claim-aware routing (route claims toward interested peers)
• Use geographic routing for location-based claims
• Implement store-and-forward for intermittent connectivity
• Support multi-hop routing with TTL management

Performance Monitoring:

• Track message delivery rates and latency
• Monitor network topology and connectivity
• Measure bandwidth utilization and efficiency
• Log and analyze network performance metrics

Network Health:

• Implement heartbeat messages for peer liveness
• Detect and handle network partitions
• Support network healing and topology changes
• Enable manual network diagnostics and troubleshooting

Quality of Service:

• Prioritize urgent messages (emergency claims)
• Implement fair queuing for multiple users
• Support bandwidth allocation policies
• Enable traffic shaping and congestion control

Implementation Phases

Phase 1: Protocol Foundation (4-6 weeks)

• Design and implement basic message formats
• Create claim serialization/deserialization
• Implement cryptographic signing and verification
• Build basic peer discovery mechanism

Phase 2: Meshtastic Integration (3-4 weeks)

• Develop Meshtastic protocol adapter
• Implement message fragmentation
• Create mesh bridge service
• Test basic claim transmission over LoRa

Phase 3: Mobile Integration (4-5 weeks)

• Integrate mesh service with Time Safari app
• Implement offline storage and sync
• Create mesh-specific UI components
• Add network status and configuration

Phase 4: Advanced Features (6-8 weeks)

• Implement merkle chain synchronization
• Add privacy-preserving features
• Optimize routing and performance
• Create comprehensive testing suite

Phase 5: Field Testing & Optimization (4-6 weeks)

• Deploy test network with multiple nodes
• Conduct real-world performance testing
• Optimize based on field results
• Prepare for production deployment

Technical Considerations

Security

• All claims must maintain cryptographic integrity
• Implement defense against replay attacks
• Support key rotation and revocation
• Enable secure bootstrapping of new nodes

Scalability

• Design for networks of 50-100 active nodes
• Implement efficient data structures for large claim sets
• Support hierarchical network organization
• Plan for future integration with other mesh protocols

Reliability

• Handle node failures gracefully
• Implement redundant storage across multiple nodes
• Support network partitioning and healing
• Enable data recovery from partial failures

Interoperability

• Maintain compatibility with existing Endorser API
• Support migration between mesh and internet modes
• Enable bridging between different mesh networks
• Plan for integration with other P2P protocols (IPFS, etc.)

This detailed plan provides a comprehensive roadmap for implementing P2P mesh networking in Time Safari while maintaining the privacy, security, and usability principles of the existing Endorser system.