Deadlock Phenomenon in Distributed Locks Implemented with Redis

In distributed systems, multiple service instances frequently require mutually exclusive access to shared resources. Implementing distributed locks using Redis stands out as one of the most common approaches. However, improper implementation can easily lead to "deadlock" scenarios where locks are never released, causing subsequent requests to block and business logic to stall. This article examines the root causes of this phenomenon and outlines robust solutions for engineering practice.

Consider a common flawed implementation:

public boolean tryLock(String key) {
    // SETNX: Set if not exists
    return Boolean.TRUE.equals(redis.setIfAbsent(key, "locked"));
}

public void unlock(String key) {
    redis.delete(key);
}

After Client A successfully acquires the lock via SETNX, if any of the following situations occur during business execution:

• The process suddenly crashes (due to OOM, kill -9, or a power outage);
• An uncaught exception is thrown between lock acquisition and release, without a finally block acting as a safety net;
• A network partition prevents the lock-release command from reaching Redis;

Then this lock will remain in Redis forever. Subsequent lock attempts by Client B and Client C will all fail, creating a de facto deadlock. Strictly speaking, this is not a traditional circular-wait deadlock, but rather permanent mutual exclusion caused by "lock leakage," carrying an equally severe impact.

1. Absence of Expiration Times

SETNX by itself does not allow keys to expire automatically. Once the holder exits abnormally, the lock turns into a "zombie lock," representing the most direct source of deadlocks.

2. Non-Atomic Lock Acquisition and Expiration Assignment

For the sake of "safety," some implementations split the operation into two steps:

SETNX lock:key 1
EXPIRE lock:key 10

If the client crashes after executing SETNX but before EXPIRE, the lock still lacks an expiration time, leaving the deadlock issue unresolved.

3. Releasing Locks That Do Not Belong to the Caller

Even with an expiration time configured, hidden risks remain: if Client A's business execution takes too long, the lock may expire and get acquired by Client B. At this point, if Client A executes DEL, it will mistakenly delete Client B's lock, breaking mutual exclusion and triggering concurrency issues or even chained deadlocks.

Solutions

1. Atomic Lock Acquisition with Expiration

Since Redis 2.6.12, the SET command has supported the NX and EX options, allowing a single command to complete both lock acquisition and timeout setup:

SET lock:order:1001 <uniqueValue> NX EX 10

• NX: Sets the key only if it does not exist, ensuring mutual exclusion;
• EX 10: Automatically expires the key after 10 seconds as a safety net against deadlocks;
• <uniqueValue>: It is recommended to use a UUID combined with a thread ID to identify the lock holder.

2. Secure Lock Release via Lua Scripts

Releasing a lock must follow a "verify holder then delete" pattern, with both steps executed atomically. Lua scripts provide the standard approach:

if redis.call("GET", KEYS[1]) == ARGV[1] then
    return redis.call("DEL", KEYS[1])
else
    return 0
end

In Java, this script can be executed through DefaultRedisScript to ensure that only the lock held by the current client is deleted.

3. Automatic Renewal via Watchdog Mechanisms

Setting an expiration time solves deadlocks but introduces a new problem: the lock might expire before the business logic finishes executing. Mainstream clients (such as Redisson) introduce a "Watchdog" mechanism: after a successful lock acquisition, a background thread starts up and periodically checks whether the business is still running (typically at 1/3 of the lock validity period), automatically renewing the lease if it is. This guarantees that locks expire automatically upon exceptions while preventing premature expiration during normal execution.

Redisson example:

RLock lock = redisson.getLock("lock:order:1001");
lock.lock(); // Default 30 seconds, watchdog automatically renews every 10 seconds
try {
    // Business logic
} finally {
    lock.unlock();
}

Best Practices

• Always use the atomic SET ... NX EX command for locking; never separate SETNX and EXPIRE.
• The lock value must be a unique identifier, and releasing the lock must involve validation through a Lua script before deletion.
• Lock release code must always reside inside a finally block.
• Reasonably estimate business execution time and coordinate with watchdog renewals to prevent premature lock expiration.
• For scenarios requiring extremely high consistency, evaluate the RedLock algorithm or switch to alternatives equipped with session-expiration mechanisms like ZooKeeper or etcd.

Conclusion

Deadlocks in Redis-based distributed locks fundamentally stem from a lack of fallback expiration mechanisms or unsafe release operations. By combining atomic lock acquisition with expiration, Lua script-based verification during release, and automatic watchdog renewal, systems can simultaneously avoid deadlocks and accidental lock deletions, building production-ready distributed locks.