Limitations

Understanding LivenDB's design constraints and operational limits.

LivenDB is optimized for real-time event processing and time-series workloads. Like all databases, it has trade-offs and limitations that are important to understand for production deployments.

Architectural Limitations

Single-Node Design

LivenDB is designed as a single-node database without built-in replication or distributed coordination.

• No automatic failover — High availability requires external solutions
• No distributed transactions — All operations are local to a single node
• No sharding — Data partitioning must be handled at the application level

Memory Constraints

LivenDB maintains an in-memory index of all active keys using a concurrent skip-list structure.

• Index memory scales with active keys — Not total data size
• Configurable limit — Set max_index_ram_mb to control memory usage (see Configuration)
• Monitor regularly — Use status() to track memory usage

Operational Limits

Connection Management

LivenDB enforces connection limits to prevent resource exhaustion.

• Default limit — 10,000 concurrent connections
• Graceful queuing — New connections queue when limit is reached
• Monitor saturation — Check connection metrics viastatus()

Segment Management

Data is organized into fixed-size segment files with configurable limits.

• Fixed segment size — Configured viamax_segment_mb (default 16MB)
• Rollover behavior — Segments roll over when size limit is reached
• Active segment — Only one writable segment per stream

Query and Data Model Limitations

Query Language

LivenDB uses a custom query language optimized for its data model.

• Not SQL-compatible — Requires learning LivenDB-specific syntax
• Pipeline-based processing — Complex operations require chaining stages
• Limited join support — No cross-stream joins; use enrichment operations instead

Data Model Constraints

The key-value data model has specific characteristics.

• Single primary index — Only stream_name:key combination is indexed
• No secondary indexes — Filtering on other fields requires full scans
• No schema enforcement — Flexible but requires application discipline

Consistency and Durability

Eventual Consistency

Certain operations have eventual consistency characteristics.

• Background compaction — Runs automatically but not immediately
• Tombstone handling — Deleted records may appear in historical queries until compacted
• No multi-operation transactions — Atomicity is per-operation

Durability Guarantees

LivenDB provides strong durability for individual operations.

• Write-ahead logging — All operations are durably logged before acknowledgment
• Fsync configuration — Tunable durability vs. performance trade-offs
• No built-in backups — Requires external backup solutions

Performance Considerations

Write Performance

Write performance depends on segment size and compaction frequency.

• Segment rollover overhead — Frequent rollovers can impact performance
• Compaction I/O — Background compaction uses disk and CPU resources
• Write amplification — Updates create new records rather than in-place modifications

Read Performance

Read performance varies by query type and data distribution.

• Point queries — Fast (O(log n) via in-memory index)
• Range scans — Slower (O(n) scan of segment files)
• Pipeline queries — Performance depends on number of stages and data volume

When to Consider LivenDB

Ideal Use Cases

• Real-time event processing and analytics
• Time-series data with moderate cardinality
• Applications needing simple key-value with real-time updates
• Use cases where eventual consistency is acceptable
• Systems requiring a balance of performance and simplicity

When to Consider Alternatives

• Complex analytical queries requiring SQL
• High-write applications with millions of active keys
• Applications requiring strong consistency guarantees
• Distributed deployments needing automatic failover
• Use cases requiring secondary indexes or joins

Mitigation Strategies

Many limitations can be mitigated with proper configuration and design:

• Monitor regularly — Use status() query
• Tune configuration — Adjust segment sizes, memory limits, and compaction thresholds
• Design for scale — Use appropriate stream naming and key strategies
• Implement caching — For frequently accessed data patterns
• Plan for backups — Implement regular backup procedures
• Consider partitioning — Distribute load across multiple LivenDB instances if needed