Publish-Subscribe

Publish/Subscribe (often shortened to Pub/Sub) is a messaging pattern that decouples the components that produce data from those that consume it. Instead of a direct, point‑to‑point connection, publishers emit messages to a topic or channel and subscribers listen for the messages they care about. The broker—an intermediary service—handles routing, storage, and delivery, allowing the system to scale and evolve without tight coupling between producers and consumers.

Core Principles

• Publisher
  Generates events or data and sends them to a topic. It has no knowledge of who will receive the messages, nor how many subscribers exist.

• Subscriber
  Registers interest in one or more topics. It receives only the messages that match its subscriptions, without caring about the source.

• Broker
  Receives, stores (optionally), and forwards messages from publishers to all matching subscribers. It may provide guarantees such as persistence, ordering, or delivery semantics.

The typical flow is:

1. Subscribers register with the broker for specific topics.
2. Publishers send messages to those topics.
3. The broker pushes the messages to every subscriber that has expressed interest.

Because the broker mediates the communication, the system naturally supports one‑to‑many and many‑to‑many patterns. This makes Pub/Sub ideal for log aggregation, real‑time dashboards, event‑driven microservices, and any scenario where many consumers need to react to the same event.

Common Implementation Strategies

Choosing the right implementation hinges on factors such as latency requirements, durability, throughput, operational complexity, and cost.

Practical Code Samples

1. Node.js EventEmitter (Local)

const { EventEmitter } = require('events');

class AppBus extends EventEmitter {}
const bus = new AppBus();

// Subscriber
bus.on('userSignedUp', (user) => {
  console.log(`Welcome email sent to ${user.email}`);
});

// Publisher
function signUp(user) {
  // ... create user in DB ...
  bus.emit('userSignedUp', user);
}

signUp({ id: 42, email: 'alice@example.com' });

This pattern is perfect for intra‑process communication where network latency is irrelevant.

2. Redis Pub/Sub (Real‑time Push)

# Terminal 1 – subscribe
redis-cli SUBSCRIBE updates:orders

# Terminal 2 – publish
redis-cli PUBLISH updates:orders "Order #1234 shipped"

// Node.js client
const redis = require('redis');
const subscriber = redis.createClient();
const publisher = redis.createClient();

subscriber.subscribe('updates:orders');
subscriber.on('message', (channel, message) => {
  console.log(`[${channel}] ${message}`);
});

publisher.publish('updates:orders', 'Order #5678 delivered');

Redis forwards messages instantly but does not persist them, making it suitable for notifications where occasional loss is acceptable.

3. Kafka (High‑Throughput, Durable)

# Create a topic with 3 partitions
kafka-topics.sh --create --topic user-events \
  --partitions 3 --replication-factor 2 \
  --bootstrap-server localhost:9092

// Producer
Properties p = new Properties();
p.put("bootstrap.servers", "localhost:9092");
p.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
p.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
KafkaProducer<String, String> producer = new KafkaProducer<>(p);
producer.send(new ProducerRecord<>("user-events", "user1", "login"));
producer.close();

// Consumer (group)
Properties c = new Properties();
c.put("bootstrap.servers", "localhost:9092");
c.put("group.id", "analytics");
c.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
c.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
KafkaConsumer<String, String> consumer = new KafkaConsumer<>(c);
consumer.subscribe(Arrays.asList("user-events"));
while (true) {
  ConsumerRecords<String, String> records = consumer.poll(Duration.ofMillis(100));
  for (ConsumerRecord<String, String> r : records) {
    System.out.printf("topic=%s key=%s value=%s%n", r.topic(), r.key(), r.value());
  }
}

Kafka’s persistence, partitioning, and offset tracking make it a go‑to choice for audit logs, analytics pipelines, and any scenario that demands replayability.

Design Guidelines

• Topic Granularity
  Over‑splitting topics increases administrative overhead; too few topics lead to unnecessary message processing. A common pattern is service.eventType, e.g., order.created, order.cancelled.

• Reliability vs. Latency

  • Durable: Use Kafka or RabbitMQ when message loss is unacceptable.
  • Transient: Redis or EventEmitter works when speed trumps reliability.

• Push vs. Pull

  • Push (broker‑initiated) suits low‑volume, low‑latency use cases.
  • Pull (consumer‑initiated) is preferable for high‑volume streams where back‑pressure control is needed.

• Idempotency
  Subscribers should process messages idempotently to guard against duplicates caused by retries or network glitches.

• Security
  Enable TLS, authentication (SASL, OAuth), and fine‑grained ACLs to prevent unauthorized publishing or subscribing, especially in multi‑tenant environments.

• Monitoring & Metrics
  Track publish/subscribe rates, lag, and error counts. Most brokers expose JMX, Prometheus, or Cloud‑native dashboards.

Real‑World Use Case: A Lightweight Chat Service with Redis

Goal: Build a multi‑room chat that delivers messages instantly without persisting history.

Architecture

1. Client → WebSocket → Node.js server.
2. Node.js server uses Redis Pub/Sub to broadcast messages to all clients in the same room.
3. Each WebSocket connection subscribes to a Redis channel named after the room (room:{id}).

Key Code

// chatServer.js
const WebSocket = require('ws');
const redis = require('redis');

const wss = new WebSocket.Server({ port: 8080 });
const pub = redis.createClient();
const sub = redis.createClient();

wss.on('connection', (ws) => {
  let currentRoom = null;

  ws.on('message', (msg) => {
    const payload = JSON.parse(msg);

    if (payload.type === 'join') {
      currentRoom = `room:${payload.roomId}`;
      sub.subscribe(currentRoom);
    } else if (payload.type === 'message' && currentRoom) {
      const broadcast = JSON.stringify({
        user: payload.user,
        text: payload.text,
        timestamp: Date.now(),
      });
      pub.publish(currentRoom, broadcast);
    }
  });

  sub.on('message', (channel, message) => {
    ws.send(message);
  });

  ws.on('close', () => {
    if (currentRoom) sub.unsubscribe(currentRoom);
  });
});

Outcome
When users A, B, and C join room:1, any message sent by one user is immediately pushed to all others with sub‑millisecond latency. No message history is stored, keeping the system lightweight and horizontally scalable.

Takeaway

Publish/Subscribe is a versatile pattern that empowers developers to build loosely coupled, scalable systems. By selecting the appropriate broker—whether an in‑process emitter, an in‑memory store, a robust queue, or a cloud‑managed service—you can tailor the trade‑offs between latency, durability, and operational overhead to fit your specific needs. Mastery of topic design, message semantics, consumer strategies, and idempotent processing unlocks powerful event‑driven architectures across logging, analytics, real‑time dashboards, and beyond.