Data Storage Technologies Comparison: Redis, SQLite, and IndexedDB

by

🌏 閱讀中文版本

Why Understanding These Three Data Storage Technologies Matters

In modern application development, data storage strategy directly impacts system performance, user experience, and development costs. Choosing the right storage technology enables you to:

  • Boost Performance: Select optimal storage solutions based on access patterns to reduce latency
  • Reduce Costs: Avoid over-provisioning server resources or bandwidth
  • Enhance User Experience: Enable offline functionality, fast responses, smooth interactions
  • Simplify Architecture: The right tool reduces system complexity

Redis, SQLite, and IndexedDB represent three distinct storage scenarios: server-side caching, local databases, and browser-side storage. Understanding their differences and use cases helps engineers make informed technical decisions.

Core Characteristics of the Three Technologies

Redis: High-Performance In-Memory Database

Technical Background:

Redis (Remote Dictionary Server) is an open-source in-memory key-value database renowned for its exceptional read/write performance. All data resides in memory with optional disk persistence mechanisms.

Core Features:

  • Rich Data Structures: Supports String, Hash, List, Set, Sorted Set, and more
  • Atomic Operations: All operations are atomic, ensuring data consistency
  • High Performance: Capable of tens to hundreds of thousands of operations per second
  • Persistence Options: RDB snapshots and AOF logging mechanisms
  • Master-Slave Replication: Supports data replication and high availability configurations
  • Pub/Sub Messaging: Built-in publish/subscribe mechanism

Ideal Use Cases:

  • Session Management: Store user login states
  • Cache Layer: Reduce database query load
  • Leaderboards: Real-time ranking using Sorted Sets
  • Counters: Page views, API rate limiting
  • Real-time Messaging: Chat rooms, push notifications

Limitations and Considerations:

  • Memory Cost: All data in memory costs more than disk storage
  • Single-Threaded: Despite high performance, long-running commands block other requests
  • Data Volume Limit: Constrained by server memory capacity
  • Persistence Trade-offs: RDB may lose recent minutes of data; AOF impacts write performance

SQLite: Lightweight Embedded Database

Technical Background:

SQLite is a fully embedded relational database that stores data in a single file without requiring a separate database server. It supports complete SQL syntax and is widely used in mobile apps and desktop software.

Core Features:

  • Zero Configuration: No installation or setup required
  • Full SQL Support: Supports JOIN, Transaction, View, Trigger, and more
  • ACID Compliance: Ensures Atomicity, Consistency, Isolation, Durability
  • Cross-Platform: Runs on Windows, macOS, Linux, iOS, Android
  • Compact and Lightweight: Library size is only hundreds of KB
  • Reliable and Stable: Extensively tested, widely deployed in production

Ideal Use Cases:

  • Mobile Apps: Local data storage for iOS and Android apps
  • Desktop Software: Configuration files, user data management
  • Embedded Systems: IoT devices, automotive systems
  • Prototyping: Rapid validation of data models
  • Small Websites: Databases for low-traffic sites

Limitations and Considerations:

  • Concurrent Writes: Only one write operation allowed at a time
  • Data Volume Limit: Recommended single database size under several GB (theoretical limit 281 TB)
  • Network Access: Not suitable for network file systems (NFS)
  • Complex Queries: Heavy JOIN operations perform worse than professional databases

IndexedDB: Browser-Side Structured Storage

Technical Background:

IndexedDB is a browser-provided client-side database API supporting large amounts of structured data with indexing for efficient queries. Designed with asynchronous APIs to avoid blocking the main thread.

Core Features:

  • Large Capacity: Typically stores hundreds of MB to several GB
  • Index Support: Create multiple indexes to accelerate queries
  • Transaction Mechanism: Ensures data operation consistency
  • Asynchronous API: Prevents UI rendering blocks
  • Cross-Tab Sharing: Multiple same-origin tabs can access shared data
  • Key-Value and Object Storage: Directly stores JavaScript objects

Ideal Use Cases:

  • Offline-First Apps: Progressive Web App (PWA) data caching
  • Large Forms: Draft storage to prevent data loss
  • Data Sync: Local caching of server data to reduce network requests
  • Media Content: Storing image and audio metadata and indexes
  • Game State: Save functionality for browser games

Limitations and Considerations:

  • Browser Compatibility: Check target browser support
  • API Complexity: More complex than localStorage, requires handling callbacks or Promises
  • Security: Subject to same-origin policy but vulnerable to XSS attacks
  • Capacity Limits: Implementation varies by browser; may be cleared when storage is low
  • No SQL Support: Query functionality is simpler than relational databases

Detailed Comparison Table

Comparison Item Redis SQLite IndexedDB
Data Model Key-value + multiple data structures Relational (tables, columns, indexes) Key-value + object storage + indexes
Query Language Redis command set Standard SQL JavaScript API (no SQL)
Transaction Support Yes (MULTI/EXEC) Full ACID transactions Yes (Transaction API)
Persistence Optional (RDB, AOF) Default persistence to file Default persistence to browser storage
Concurrency Single-threaded + multiplexing Multiple readers, single writer Browser-dependent
Typical Capacity Several GB (memory limited) Several GB ~ tens of GB 50MB ~ several GB (browser-dependent)
Read Performance Extremely high (microsecond level) High (millisecond level) Medium (depends on data volume and indexes)
Write Performance Extremely high Medium (affected by locking) Medium
Data Security Network transmission needs encryption File permission protection Same-origin policy isolation
Deployment Complexity Requires dedicated server No deployment needed (embedded) No deployment needed (browser built-in)
Development Difficulty Medium Low (SQL familiarity sufficient) Medium-High (complex API)
Maintenance Cost Requires monitoring and tuning Almost no maintenance No maintenance

Real-World Use Cases

Case 1: E-Commerce Session Management (Redis)

Requirement:
E-commerce platform needs to manage tens of thousands of concurrent user sessions, including shopping carts, login status, browsing history.

Why Redis:

  • High concurrent read/write capability to handle traffic spikes during promotions
  • Session auto-expiration (TTL mechanism)
  • Master-slave replication ensures high availability
  • Cross-server session sharing supports horizontal scaling

Implementation Highlights:

# Set session (30-minute expiration)
SET session:user123 "{\"cart\":[1,2,3],\"logged_in\":true}" EX 1800

# Read session
GET session:user123

# Extend session TTL
EXPIRE session:user123 1800

Case 2: Note App Offline Storage (SQLite)

Requirement:
Mobile note application needs to function offline, storing notes content, categories, tags, attachments, and other structured data.

Why SQLite:

  • Complete relational database functionality supporting complex queries
  • ACID compliance ensures data integrity even during app crashes
  • Cross-platform support, shared codebase between iOS and Android
  • No network connection required, fully local operation

Implementation Highlights:

-- Create notes table
CREATE TABLE notes (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  title TEXT NOT NULL,
  content TEXT,
  created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
  updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
);

-- Create tags table
CREATE TABLE tags (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  name TEXT UNIQUE NOT NULL
);

-- Create note-tag relationship table
CREATE TABLE note_tags (
  note_id INTEGER,
  tag_id INTEGER,
  FOREIGN KEY (note_id) REFERENCES notes(id),
  FOREIGN KEY (tag_id) REFERENCES tags(id),
  PRIMARY KEY (note_id, tag_id)
);

-- Query notes with specific tag
SELECT n.* FROM notes n
JOIN note_tags nt ON n.id = nt.note_id
JOIN tags t ON nt.tag_id = t.id
WHERE t.name = 'work';

Case 3: PWA Offline Map Application (IndexedDB)

Requirement:
Progressive Web App map service needs to cache map tiles, location information, user markers for offline browsing functionality.

Why IndexedDB:

  • Can store large amounts of binary data (map images)
  • Index functionality supports fast queries for specific geographic ranges
  • Asynchronous API doesn’t affect map interaction smoothness
  • No server-side support needed, operates entirely in browser

Implementation Highlights:

// Open database
const request = indexedDB.open('MapDatabase', 1);

request.onupgradeneeded = function(event) {
  const db = event.target.result;
  
  // Create tile storage
  const tileStore = db.createObjectStore('tiles', { keyPath: 'id' });
  tileStore.createIndex('zoom', 'zoom', { unique: false });
  tileStore.createIndex('coordinates', ['x', 'y'], { unique: false });
  
  // Create place storage
  const placeStore = db.createObjectStore('places', { keyPath: 'id' });
  placeStore.createIndex('category', 'category', { unique: false });
};

// Save map tile
function saveTile(db, tile) {
  const transaction = db.transaction(['tiles'], 'readwrite');
  const store = transaction.objectStore('tiles');
  store.put(tile);
}

// Query tiles by zoom level
function getTilesByZoom(db, zoom) {
  return new Promise((resolve, reject) => {
    const transaction = db.transaction(['tiles'], 'readonly');
    const store = transaction.objectStore('tiles');
    const index = store.index('zoom');
    const request = index.getAll(zoom);
    
    request.onsuccess = () => resolve(request.result);
    request.onerror = () => reject(request.error);
  });
}

Decision Flow Chart

Quick Decision Steps:

  1. Identify Runtime Environment
    • Server-side → Consider Redis or other server databases
    • Client application (App) → Consider SQLite
    • Browser-side → Consider IndexedDB
  2. Evaluate Data Characteristics
    • Need ultra-fast read/write → Redis
    • Need complex SQL queries → SQLite
    • Need large browser storage → IndexedDB
  3. Consider Data Persistence
    • Temporary data, caching → Redis
    • Long-term storage, critical data → SQLite or backend database
    • Offline-first apps → IndexedDB
  4. Estimate Data Volume and Traffic
    • High concurrency, small data → Redis
    • Medium traffic, structured data → SQLite
    • Frontend data caching → IndexedDB

Best Practice Recommendations

Redis Best Practices

  • Set Appropriate Expiration: Avoid memory waste using EXPIRE or SETEX commands
  • Monitor Memory Usage: Configure maxmemory and eviction policies (e.g., allkeys-lru)
  • Avoid Large Keys: Single keys shouldn’t exceed 10MB; consider splitting data
  • Use Pipeline: Batch operations to reduce network round trips
  • Choose Right Data Structure: Use String for counters, Sorted Set for leaderboards
  • Enable Persistence: Production environments should use both RDB and AOF
  • Master-Slave Replication: Configure at least one slave node for high availability

SQLite Best Practices

  • Use WAL Mode: Improve concurrent read performance (PRAGMA journal_mode=WAL)
  • Create Appropriate Indexes: Speed up common queries but avoid too many indexes affecting writes
  • Use Transactions: Wrap batch writes with BEGIN/COMMIT to boost performance
  • Regular VACUUM: Clean up fragmentation, reclaim space
  • Avoid Using on NFS: Network file systems may cause file locking issues
  • Backup Strategy: Regularly copy database file or use .backup command
  • Limit Database Size: Single database recommended under several GB

IndexedDB Best Practices

  • Use Promise Wrappers: Simplify async operations with libraries like idb
  • Create Appropriate Indexes: Speed up queries, but each index increases storage
  • Handle Version Upgrades: Carefully manage schema changes in onupgradeneeded
  • Error Handling: Always handle onerror and onblocked events
  • Batch Operations: Use single transactions for multiple records to improve performance
  • Clean Expired Data: Regularly delete unnecessary cached data
  • Check Browser Support: Use feature detection to ensure compatibility
  • Consider Capacity Limits: Check available space, handle QuotaExceededError

Frequently Asked Questions

Q1: Is Redis suitable as a primary database?

A: Generally not recommended. Redis is primarily designed as a cache and high-speed data access layer. While it supports persistence, these reasons make it less suitable as a primary database:

  • High memory cost makes storing large amounts of data uneconomical
  • Persistence mechanisms have data loss risks (RDB may lose minutes of data)
  • Lacks complex query capabilities (no JOIN, no complete SQL)
  • Single-threaded architecture may become a bottleneck in some scenarios

Recommended to use with traditional relational or NoSQL databases, with Redis as a cache layer or storage for specific features (like leaderboards, sessions).

Q2: How much concurrency can SQLite handle?

A: SQLite uses file locking with limited concurrent write capability:

  • Reads: Supports multiple processes reading simultaneously
  • Writes: Only one write operation allowed at a time
  • WAL Mode: Reads and writes can occur simultaneously, but writes are mutually exclusive

Recommended scenarios:

  • ✅ Read-heavy applications (like content management systems)
  • ✅ Single-user applications (desktop software, mobile apps)
  • ✅ Low-traffic websites (thousands to tens of thousands of requests per day)
  • ❌ High concurrent writes (use MySQL, PostgreSQL instead)
  • ❌ Multi-server environments (need centralized database)

Q3: Will browsers clear IndexedDB data?

A: Possibly, depending on these factors:

  • Insufficient Storage: Browsers may clear IndexedDB data to free space
  • User Action: Clearing browsing data also deletes IndexedDB
  • Persistent Storage API: Can request browsers not to auto-clear data

Recommended approach:

// Request persistent storage permission
if (navigator.storage && navigator.storage.persist) {
  navigator.storage.persist().then(granted => {
    if (granted) {
      console.log('Data will not be automatically cleared');
    } else {
      console.log('Browser may clear data when space is low');
    }
  });
}

// Check current storage status
navigator.storage.persisted().then(isPersisted => {
  console.log('Persistence status:', isPersisted);
});

Q4: How to choose between Redis, SQLite, and IndexedDB?

A: Quick decision guide based on use case:

Need Recommended Technology Reason
Session management, API rate limiting Redis High concurrency, expiration support
Mobile app local data SQLite Full SQL, ACID compliance
PWA offline functionality IndexedDB Large capacity, browser native
Real-time leaderboards Redis Sorted Set efficient sorting
Embedded system database SQLite Lightweight, no server needed
Large form drafts IndexedDB Avoid frequent server requests
Distributed cache Redis Master-slave replication, Cluster support

Q5: Can you use multiple storage technologies simultaneously?

A: Yes, and this is common best practice. Mixed-use examples:

  • E-Commerce Platform:
    • Redis: Sessions, shopping carts, popular product cache
    • MySQL: Orders, products, user master data
    • IndexedDB: User browsing history, drafts
  • Note Application:
    • SQLite: App local database
    • PostgreSQL: Cloud sync data
    • Redis: Real-time collaboration state
  • Map Service:
    • IndexedDB: Map tile cache
    • Redis: Real-time traffic data
    • PostgreSQL + PostGIS: Geographic data master database

The key is selecting appropriate technology based on data characteristics (read/write frequency, data volume, persistence needs).

Q6: What happens when Redis loses memory data?

A: Impact depends on persistence configuration:

  • No Persistence: All data lost after restart; suitable only for pure cache scenarios
  • RDB Snapshot: Recovers to last snapshot time; may lose minutes to hours of data
  • AOF Log:
    • appendfsync always: Almost no loss but worst performance
    • appendfsync everysec: Lose at most 1 second of data (recommended)
    • appendfsync no: OS decides, may lose more data
  • RDB + AOF: Combines benefits of both; recovery prioritizes AOF

Production environment recommendations:

# redis.conf settings
save 900 1          # Snapshot if at least 1 change in 900 seconds
save 300 10         # Snapshot if at least 10 changes in 300 seconds
save 60 10000       # Snapshot if at least 10000 changes in 60 seconds

appendonly yes                # Enable AOF
appendfsync everysec          # Sync every second
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

Q7: What’s the difference between IndexedDB and localStorage?

A: Both are browser-side storage, but characteristics differ significantly:

Feature IndexedDB localStorage
Storage Capacity 50MB ~ several GB 5 ~ 10 MB
Data Types Objects, arrays, binary Strings only
Query Features Indexes, range queries Key-value queries only
API Design Asynchronous Synchronous
Transaction Support Yes No
Performance Better for large data Simpler for small data
Ease of Use Medium-High Low

Selection Advice:

  • ✅ localStorage: Store simple settings, small user preferences
  • ✅ IndexedDB: Offline data, large caches, structured data

Summary and Selection Advice

Redis, SQLite, and IndexedDB each have their optimal use cases. When choosing, consider:

  1. Runtime Environment: Server, client application, or browser
  2. Data Characteristics: Structure level, data volume, access patterns
  3. Performance Requirements: Concurrency, latency requirements, read/write ratio
  4. Persistence Needs: Temporary cache or long-term storage
  5. Development and Maintenance Costs: Team tech stack, operational capabilities

Core Principles:

  • Redis: Pursue ultimate performance for server-side caching and real-time data processing
  • SQLite: Need complete SQL functionality for local and embedded scenarios
  • IndexedDB: Browser-side large-capacity data storage and offline functionality

In practice, most complex applications mix multiple storage technologies, each serving its purpose to maximize benefits. Correct technology choices significantly improve system performance, reduce costs, and enhance user experience.

Related Articles

Related posts:

  1. 資料儲存技術比較：Redis、SQLite 與 IndexedDB
  2. 深入了解 Redis：資料結構與 CRUD 操作完整指南
  3. Deep Dive into Redis: Data Structures and CRUD Operations Complete Guide
  4. Azure SQL Post-Migration Performance Optimization: Query Statistics, Top SQL Analysis, and Index Tuning Guide

Leave a Comment