Introduction to the Apache Airflow Command Line Interface (CLI)

Apache Airflow, as an industry-leading workflow orchestration tool, provides its executable command-line interface (CLI) as the most direct window for users to interact with the scheduler. Compared to operations performed via the Web UI, the CLI offers more powerful command-line control capabilities, a higher degree of automation, and greater flexibility in script writing. Mastering Airflow CLI is not only an essential skill for daily operations but also key to building efficient automated operational workflows. This article delves into the core concepts, common commands, and practical applications of the Airflow CLI.

The Airflow CLI is essentially a toolkit that encapsulates the logic of both the Scheduler (scheduler) and the Webserver. It allows users to directly execute task triggers, status queries, log viewing operations in their local terminal without relying on remote server network services. This design enables seamless integration of Airflow into existing CI/CD pipelines or automation scripts, realizing the vision of "code as infrastructure."
Before using the Airflow CLI, it is necessary to understand its basic working mode. Airflow provides a rich set of commands covering the entire lifecycle from initialization and deployment to daily operations. Here are some of the most common and core commands:

• airflow init
  This is the first step in starting a new Airflow environment. The command creates necessary database table structures, configuration files, and directories required by the Webserver. After executing this command, the system can proceed to subsequent operational phases.

  # Initialize default configurations
  airflow init
  
  # Specify custom database path and config directory
  airflow init --db-path /var/lib/airflow/db \
              --core-config /etc/airflow/airflow.cfg
  

• airflow dags
  This command is used to manage DAG (Directed Acyclic Graph) files. It can list all registered DAGs, create new DAGs, or validate the syntax correctness of DAG files. This is an indispensable tool during the development phase.

  # List all DAGs in the current environment
  airflow dags list
  
  # View detailed information and status for a specific DAG
  airflow dags show my_dag_name
  
  # Check if there are logical errors in the DAG file
  airflow dags validate /path/to/my_dag.py
  

• airflow tasks
  Used to manage individual task instances. Users can view execution history, trigger new tasks, or cancel running tasks. This is crucial for debugging failed workflows.

  # List all tasks belonging to a specific DAG
  airflow tasks list my_dag_name
  
  # View detailed status and log links for a single task
  airflow tasks show task_id=task_1 run_id=xcom_value
  
  # Manually trigger a task even if upstream dependencies have not been executed
  airflow tasks trigger task_id=task_2 dag_id=my_dag_name
  

• airflow logs
  Used to retrieve and view execution logs for tasks. Airflow supports filtering logs by time range or task ID, which is very useful for troubleshooting runtime errors.

  # View the last 10 log entries
  airflow logs --dag my_dag_name --limit 10
  
  # View complete logs for a specific task ID
  airflow logs task_id=task_3
  

Advanced Usage and Automation Integration

Beyond basic task management, Airflow CLI supports more advanced interaction modes, particularly when batch operations or programmatic control are required.

1. Interactive Command Mode
Airflow CLI supports the -i parameter to enter interactive mode. In this mode, users can directly input Python code to execute complex logic and dynamic operations without needing to write scripts in advance. This is highly efficient for temporary debugging or quickly verifying logic.

# Enter an interactive shell
airflow -i

# Execute Python code within the interactive environment
>>> from airflow.models import DAG, task_instance
>>> dag = get_dag('my_dag')
>>> ti = task_instance(dag, 'task_1', start_date=datetime(2023, 10, 1))
>>> ti.run()

2. Automation Script Integration
In actual production environments, Airflow operations are often encapsulated into Python scripts. By utilizing the airflow.cli module, these commands can be converted into function calls to achieve fully automated workflow control. For example, in a CI/CD pipeline, deployment tasks can be automatically triggered via API or CLI after the build is complete.

3. Permission and Configuration Management
To ensure security, Airflow CLI typically runs under the same user permissions as the Webserver (e.g., airflow-webuser). When executing sensitive operations, it is recommended to combine configurations with environment variables rather than hardcoding passwords or keys in code. Additionally, utilizing the --user parameter allows specifying an operating system user for command execution, achieving fine-grained permission control.

Summary and Best Practices

The Airflow CLI serves as a bridge connecting the Airflow scheduler with external automation systems. By mastering core commands such as airflow init, dags, tasks, and logs, developers can more efficiently manage complex business processes. Furthermore, combining interactive modes with script integration significantly enhances operational efficiency and reduces human error.

In practical application, it is recommended to follow these best practices:

1. Prioritize Web UI: For daily monitoring and simple task viewing, the Web UI provides a friendlier graphical interface; CLI should serve as an auxiliary tool or automation mechanism.
2. Handle Logs with Prioritization: Use the filtering capabilities of the CLI to quickly locate critical errors instead of wasting time sifting through massive amounts of logs.
3. Secure Configuration: Always pass sensitive information via environment variables and restrict execution permissions for CLI commands.

By reasonably utilizing Airflow CLI, organizations can advance workflow orchestration from "manual configuration" to "code-driven," building a more robust and scalable data processing platform.