Implementation of Cross-Cluster Queries for Doris (Configuration and Management)

Doris is an open-source columnar database system that supports cross-cluster queries between multiple clusters. This feature allows you to start a query at one point in one cluster and reach another point in another cluster, thereby solving the problem of data dispersion.

What is Cross Cluster Query?

Cross Cluster Query means you can start a query from one cluster and then retrieve data from other clusters within the same query. This feature requires two or more Doris clusters, and these clusters need to share the same namespace.

How to Configure Cross Cluster Query?

1. Enable Cross Cluster Query

To enable cross-cluster queries, you need to create a named space on each cluster and associate them. A named space is a namespace within Doris, and its name must be globally unique.

-- Create a new named space
CREATE SPACE `my_space` FOR 'row_format=fixed_size' AS 'size=1024';

-- Create another cluster and named space
CREATE SPACE `my_space1` FOR 'row_format=fixed_size' AS 'size=1024';

2. Associate Two Clusters

You need to associate the two clusters. This can be done using the following commands:

-- Associate cluster1 with cluster2
ALTER CLUSTER GROUP 'my_group' ADD MASTER 'cluster1';
ALTER CLUSTER GROUP 'my_group' ADD SLAVE 'cluster2';

3. Configure Cross Cluster Query

Configuring cross-cluster queries requires creating a system view on each cluster and specifying the data to be retrieved from other clusters.

-- Create a new system view
CREATE SYSTEM VIEW `my_view` FOR 'row_format=fixed_size' AS 'size=1024';

-- Specify data retrieval from another cluster
ALTER SYSTEM VIEW `my_view` ADD FROM 'cluster1';

How to Manage Cross Cluster Query?

1. Query System View

You can view configuration information by querying the system view.

SELECT * FROM `my_view`;

2. Handle Exceptions

If there are exceptions, you need to check the log and handle the error.

-- View log
SELECT * FROM `my_log`;

-- Handle exception
ALTER SYSTEM VIEW `my_view` HANDLE ERROR;

3. Analyze Performance

You can understand the performance of cross-cluster queries by analyzing the query log.

-- View query log
SELECT * FROM `my_query_log`;

4. Offset Data

You can offset data on one cluster to reduce data reading from another cluster.

-- Offset data
ALTER CLUSTER GROUP 'my_group' OFFSET 'cluster1';

Conclusion

Cross-cluster query is an important feature of Doris that allows you to start a query at one point in one cluster and reach another point in another cluster, thereby solving the problem of data dispersion. Configuring and managing these features can better handle large-scale data.