Scaling and Shrinking the Doris Cluster (Best Practices for Dynamic Node Addition)

What is Cluster Scaling?

Cluster scaling refers to the dynamic addition or removal of nodes in a Doris cluster, enabling the cluster's scalability. This mechanism can be adjusted according to business requirements, improving cluster availability and efficiency.

Why Scale In and Out

Scaling in and out offers many benefits, such as:

  • Enhanced Availability: When increased processing power is needed, nodes can be dynamically added to improve cluster availability.
  • Cost Reduction: There is no longer a need to maintain multiple separate Doris instances, thereby reducing storage and maintenance costs.
  • Increased Elasticity: The cluster can be adjusted according to business requirements, achieving scalability.

Best Practices

1. Preparation Before Node Expansion

Before preparing for node expansion, the following steps must be completed:

  • Check Node Availability: Ensure that the node to be added is running normally and can connect to the Doris management interface.
  • Update Management Instance: Add the new node to the Doris management instance so that the management instance knows about the new node to be joined.

2. Node Expansion

When executing node expansion, the following steps must be used:

  • Obtain Node Information: Retrieve information about the new node from the Doris management interface, including IP address, port number, and user credentials.
  • Execute Expansion Command: Use the expansion command provided by Doris to add the new node to the cluster. Execution must be performed based on specific configuration files and parameters.

3. Node Contraction

Before preparing for node contraction, the following steps must be completed:

  • Check Node Status: Ensure that the node to be deleted has stopped normally and is no longer used by any other instance.
  • Update Management Instance: Remove the node from the Doris management instance to inform the management instance that the node has been contracted.

4. Node Contraction

When executing node contraction, the following steps must be used:

  • Obtain Node Information: Retrieve information about the node to be deleted from the Doris management interface.
  • Execute Contraction Command: Use the contraction command provided by Doris to delete the node and remove it from the cluster.