Redis

Redis (Remote Dictionary Server) is an open-source, high-performance key-value in-memory database that also supports persistent data storage. Boasting core advantages of ultra-fast read/write speeds, rich data structures, high concurrency support, and strong functional scalability, it is widely used in scenarios such as caching, session storage, message queuing, and real-time statistics, serving as an indispensable middleware in modern distributed systems.

I. Background and Fundamentals

1. Development Background

The birth of Redis stemmed from the urgent demand for high-performance data access. In the early 21st century, with the rise of Web 2.0 applications, user traffic and data volume exploded exponentially. Traditional relational databases faced prominent performance bottlenecks in high-concurrency read/write scenarios—disk I/O speeds were slow, and they could not support high-frequency queries and data updates.

  • In 2009, Italian developer Salvatore Sanfilippo released the first version of Redis, aiming to create a lightweight in-memory database to address data access efficiency issues in high-concurrency scenarios.
  • Compared with Memcached, an early key-value storage system that only supported simple key-value pairs, Redis expanded with rich data structures and added features like persistence, master-slave replication, and clustering, quickly becoming the mainstream in-memory database.
  • After years of iteration, the Redis ecosystem has been continuously improved. It supports client libraries for multiple programming languages and provides rich extension modules (e.g., Redis Search, Redis JSON) to meet complex business requirements.

Today, Redis has become one of the most popular in-memory databases worldwide, widely adopted in industries such as the Internet, e-commerce, and finance.

2. Core Fundamentals

  • Key-Value Storage Model: Redis’s core data model is key-value pairs. Keys are unique strings, while values support multiple data structures, breaking the functional limitations of traditional key-value databases.
  • Memory-First Storage: Data is stored in memory by default. Memory read/write speeds are orders of magnitude faster than disk, which is the core reason for Redis’s high performance. It also supports persisting data to disk to avoid data loss in case of restarts.
  • Single-Threaded Architecture: Redis adopts a single-threaded + I/O multiplexing model to process command requests. This avoids the overhead of context switching and lock contention in multi-threaded systems, ensuring stable performance in high-concurrency scenarios.
  • Open-Source License: Redis follows the BSD open-source license, allowing users to use, modify, and distribute the source code for free without commercial licensing restrictions. It has a vibrant community and comprehensive community support.

II. Components and Technical Principles

Redis’s architecture is designed around three core goals: high performance, high availability, and scalability. It consists of key components such as data structures, memory management, persistence, and high-availability mechanisms.

1. Core Components

Module Core Function Key Technical Details
Data Structure Layer Provides rich value data types to meet diverse business needs Supports five basic structures: Strings, Hashes, Lists, Sets, and Sorted Sets; also includes extended structures like Bitmaps, Geospatial (Geo), and HyperLogLog for cardinality counting
Memory Management Module Responsible for memory allocation, reclamation, and optimization Uses the jemalloc memory allocator to reduce memory fragmentation; supports expired key eviction policies (e.g., volatile-lru, allkeys-random) to automatically evict infrequently used data when memory reaches the threshold
Persistence Module Synchronizes in-memory data to disk to ensure data durability Offers two persistence methods: RDB (Redis Database) and AOF (Append Only File) , which can be used independently or in combination
High-Availability Module Implements automatic failover to ensure service continuity Includes master-slave replication (master for writes, slaves for reads to achieve read-write separation), Sentinel (monitors master-slave status and performs automatic failover), and Cluster (shards data across nodes to support horizontal scaling)
Network Communication Module Handles client connections and command transmission Communicates based on the TCP protocol and adopts an I/O multiplexing model (epoll/kqueue, etc.) to efficiently process a large number of concurrent connections with a single thread

2. Core Technical Principles

(1) Core Data Structures

Redis’s strength lies in its support for multiple data structures, each optimized for specific scenarios:

  • Strings: The most basic data type, which can store strings, numbers, and binary data. Supports operations like increment/decrement and batch read/write, suitable for caching user information, counters, etc.
  • Hashes: A collection of key-value pairs, ideal for storing objects (e.g., user name, age, email). Allows independent operations on individual fields of an object, saving memory space.
  • Lists: Ordered string lists implemented based on doubly linked lists. Supports inserting/deleting data from both ends, making it suitable for message queues, latest message lists, etc.
  • Sets: Unordered collections of unique strings. Supports set operations such as intersection, union, and difference, applicable for tag management, mutual friend recommendations, etc.
  • Sorted Sets: Collections with associated scores, where elements are sorted by score. Supports range queries by score, suitable for leaderboards, delayed tasks, etc.

(2) Persistence Mechanisms

To solve the problem of volatile in-memory data, Redis provides two persistence solutions:

  • RDB Persistence: Takes a snapshot of the entire dataset in memory and writes it to disk at specified time intervals. Its advantages include small file size and fast recovery speed; the disadvantage is the risk of losing data modified after the last snapshot.
  • AOF Persistence: Records all write commands in a log file. Data is restored by re-executing the commands in the log when Redis restarts. It supports three synchronization strategies (always for real-time sync, everysec for per-second sync, no for OS-controlled sync), offering higher data security but larger log files and slower recovery speed.

(3) High-Availability and Clustering Technologies

  • Master-Slave Replication: A master node can be paired with multiple slave nodes. The master handles write operations, while slaves synchronize data from the master and provide read services, achieving read-write separation and improving concurrent read capability.
  • Sentinel Mechanism: Sentinel nodes monitor the health status of master and slave nodes. When the master node fails, it automatically promotes a slave node to the new master and notifies clients to switch connections, enabling failover without manual intervention.
  • Redis Cluster: Adopts sharding technology to distribute data across multiple nodes based on 16384 hash slots, with each node responsible for a subset of slots. It supports dynamic addition/removal of nodes, enabling horizontal scaling to support massive data storage and ultra-high concurrent access.

(4) Single-Threaded Architecture and I/O Multiplexing

Redis uses a single thread to process commands, but this does not hinder its high-concurrency performance for two key reasons:

  • Memory operations are extremely fast, so a single thread is sufficient to handle a large number of commands.
  • It adopts an I/O multiplexing model to monitor multiple client network connections simultaneously. Operations are triggered only when a connection has data to read or write, avoiding the overhead of thread switching.

III. Application Scenarios

With its high performance and rich functionality, Redis has become a core middleware in distributed systems, with typical application scenarios including:

  1. Caching System: This is Redis’s most mainstream application. Hot data (e.g., product information, homepage ads, user sessions) is cached in Redis to replace high-frequency queries to relational databases, reducing database load and improving system response speed. It supports setting expiration times to enable automatic cache invalidation.
  2. Session Storage: In distributed systems, user session information can be stored in Redis, making it accessible to all application nodes and solving the session sharing problem. Examples include user login status and shopping cart data on e-commerce platforms.
  3. Message Queue: Lightweight message queuing can be implemented using Lists or Pub/Sub (Publish/Subscribe). Lists are suitable for point-to-point message delivery, while Pub/Sub is ideal for one-to-many message broadcasting, enabling system decoupling and asynchronous task processing.
  4. Real-Time Statistics and Counting: Using the increment/decrement functions of Strings and the cardinality counting capability of HyperLogLog, real-time counting scenarios can be implemented, such as article view counts, website UV statistics, and product like counts.
  5. Leaderboards and Rate Limiting: Leveraging the sorting feature of Sorted Sets, real-time leaderboards (e.g., game power rankings, product sales rankings) can be built. Combined with Redis’s atomic operations, it can implement API rate limiting (e.g., restricting each user to a maximum of 10 requests per second).
  6. Distributed Locks: Using Redis’s SET NX (set if not exists) command, distributed locks can be implemented to resolve conflicts when multiple nodes concurrently operate the same resource.

IV. Summary

Redis is a high-performance, multi-functional in-memory database. Its core advantages lie in memory-level read/write speeds, rich data structures, and comprehensive high-availability solutions. It can not only act as a cache to improve system performance but also take on roles such as message queuing, distributed locking, and real-time statistics, making it an indispensable component in modern Internet architectures. Mastering Redis’s core principles and usage is crucial for building high-concurrency, high-availability distributed systems.

Redis Core Learning Guide

 22 articles