A Simple Guide to Canal and Spring Boot

In modern microservices architectures, maintaining data consistency across various downstream systems—such as distribute

In modern microservices architectures, maintaining data consistency across various downstream systems—such as distributed caches or search engines—is a perennial challenge. The traditional "dual-write" approach, where an application attempts to write to both a database and a cache simultaneously, is notoriously prone to consistency issues due to partial failures or network latency.

To solve this, Alibaba open-sourced Canal, a powerful tool designed to handle data synchronization seamlessly. By mimicking a MySQL Slave, Canal intercepts the MySQL Binlog (binary log) and transforms it into structured data, allowing downstream systems to react to database changes in real-time without impacting the primary database's performance.

In this guide, we will walk through a practical implementation of Canal using a Spring Boot application.
Before diving into the code, it is essential to understand the three fundamental components of the Canal ecosystem:

  • MySQL Binlog: This is the source of truth. It is a binary file that records all Data Definition Language (DDL) and Data Manipulation Language (DML) statements executed on the MySQL server.
  • Canal Server: This component acts as a "pseudo-slave." It uses the MySQL replication protocol to connect to the Master database, pulls the Binlog, and parses the raw binary data into a structured format.
  • Canal Client: This is the consumer. It connects to the Canal Server to subscribe to specific data changes and performs business logic based on those changes. In our example, our Spring Boot application serves as the Client.

Environment Setup

1. Configuring MySQL

For Canal to function, MySQL must be configured to enable binary logging and use the ROW format. The ROW format is critical because it provides the actual data changes rather than just the SQL statements.

Modify your my.cnf (Linux) or my.ini (Windows) file with the following settings:

[mysqld]
# Enable binary logging
log-bin=mysql-bin
# Set format to ROW to capture actual data changes
binlog-format=ROW
# Ensure a unique server ID
server-id=1

After modifying the configuration, restart your MySQL service.

2. Creating a Synchronization Account

Canal requires a dedicated user with specific permissions to read the replication logs. Execute the following SQL commands:

CREATE USER 'canal'@'%' IDENTIFIED BY 'canal';
GRANT SELECT, REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'canal'@'%';
FLUSH PRIVILEGES;

3. Deploying Canal Server

  1. Download and extract the Canal Server package.
  2. Navigate to the conf/example/instance.properties file.
  3. Update the connection details (IP, port, username, and password) to match your MySQL configuration.
  4. Start the Canal Server.

Implementing the Spring Boot Client

To simplify the integration, we will use a Spring Boot starter that wraps the official canal-client. This allows us to use annotations to listen for database events, making the code much cleaner.

1. Add Maven Dependencies

Include the following dependencies in your pom.xml to bring in Spring Boot and the Canal starter:

<dependencies>
    <!-- Spring Boot Web Starter -->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>

    <!-- Canal Spring Boot Starter -->
    <dependency>
        <groupId>top.socket</groupId>
        <artifactId>canal-spring-boot-starter</artifactId>
        <version>1.1.6</version>
    </dependency>
</dependencies>

2. Configure Application Properties

In your src/main/resources/application.yml, define the connection parameters for the Canal Server and specify the destination (the instance name configured in Canal):

server:
  port: 8080

canal:
  server: 127.0.0.1:11111  # The address of your Canal Server
  destination: example     # The instance name from instance.properties
  socketTimeout: 30000

3. Create the Data Event Listener

The most efficient way to handle data changes is to implement a listener class. By using the @CanalEventListener and @ListenPoint annotations, we can target specific tables and event types (INSERT, UPDATE, DELETE).

import com.alibaba.otter.canal.protocol.CanalEntry;
import org.springframework.stereotype.Component;
import top.socket.canal.annotation.CanalEventListener;
import top.socket.canal.annotation.ListenPoint;

@Component
@CanalEventListener
public class MyCanalClientExample {

    /**
     * Handles INSERT events for a specific table.
     */
    @ListenPoint(destination = "example", schema = "test_db", table = "t_user", eventType = CanalEntry.EventType.INSERT)
    public void onInsert(CanalEntry.EventType eventType, CanalEntry.RowData rowData) {
        System.out.println(">>> Detected INSERT event:");
        rowData.getAfterColumnsList().forEach((column) -> {
            System.out.println("Column: " + column.getName() + " | Value: " + column.getValue());
        });
    }

    /**
     * Handles UPDATE events, providing both pre-change and post-change data.
     */
    @ListenPoint(destination = "example", schema = "test_db", table = "t_user", eventType = CanalEntry.EventType.UPDATE)
    public void onUpdate(CanalEntry.EventType eventType, CanalEntry.RowData rowData) {
        System.out.println(">>> Detected UPDATE event:");
        
        System.out.println("--- Before Change ---");
        rowData.getBeforeColumnsList().forEach((column) -> {
            System.out.println(column.getName() + " = " + column.getValue());
        });

        System.out.println("--- After Change ---");
        rowData.getAfterColumnsList().forEach((column) -> {
            System.out.println(column.getName() + " = " + column.getValue());
        });
    }
}

Testing the Implementation

To verify that the synchronization is working correctly, follow these steps:

  1. Start the Spring Boot Application: Ensure the console shows that the application has started successfully and the Canal client has connected.
  2. Execute SQL Commands: Connect to your MySQL instance and run an insertion command:
    INSERT INTO test_db.t_user (id, name, age) VALUES (1, 'John Doe', 30);
  3. Verify Output: Check your Spring Boot console. You should see the printed column names and values appearing in real-time.
  4. Test Updates: Run an UPDATE statement:
    UPDATE test_db.t_user SET age = 31 WHERE id = 1;
    The console should display both the "Before" and "After" states of the record.

Summary and Use Cases

By leveraging Canal with Spring Boot, you can move away from the risky "dual-write" pattern and embrace an event-driven architecture. This approach ensures that your downstream systems are eventually consistent with your primary database without adding complexity to your core business logic.

Common real-world applications include:

  • Cache Invalidation: Automatically updating or deleting entries in Redis when the underlying database changes.
  • Search Engine Synchronization: Pushing data changes to Elasticsearch to keep search results up to date.
  • Data Auditing and Decoupling: Creating an asynchronous audit log or triggering secondary business workflows (like sending notifications) based on database mutations.