Viewing and Monitoring

In distributed systems, reliability work is not only about fixing problems after they appear. It is also about building the habits and tooling that help teams understand what is happening. Two closely related practices often appear together: viewing and monitoring.

Viewing focuses on the present. It asks, “What is happening right now?” Monitoring focuses on change over time. It asks, “Is the system behaving as expected, and is it starting to fail?”

Two Ways to Look at System Health

Viewing is usually ad hoc and interactive. A person triggers it when something feels wrong, looks odd, or needs confirmation. Monitoring is usually continuous and automated. Systems collect data on a schedule, compare it with expectations, and raise alerts when needed.

• Viewing tends to be temporary, manual, and targeted.
• Monitoring tends to be continuous, automatic, and preventive.

Starting with Viewing

Viewing is the first response muscle. It is interactive and should be practical. Before digging deeper, clarify the time range, service instance, and error keywords. Then narrow the scope step by step.

Useful viewing targets include:

• Logs: application logs, system logs, access logs, audit logs.
• Processes and resources: CPU, memory, disk, network, file descriptors.
• Service status: port listening, health checks, dependency connectivity.
• Events: Kubernetes events, release records, system audit, configuration changes.

Example commands:

# Follow application logs
tail -f /var/log/app/app.log

# Show systemd logs for nginx from the last 10 minutes
journalctl -u nginx -f --since "10 min ago"

# Inspect listening ports and processes
ss -lntp | grep 8080

# Check health endpoint
curl -s http://localhost:8080/healthz

Viewing should not rely on a single metric. High CPU may be normal batch processing. Many error logs may not mean a global outage. Context matters.

Monitoring: Metrics, Logs, and Traces

Monitoring is continuous collection and automatic analysis. Observability often rests on three pillars:

• Metrics: numeric time series data, useful for trends and alerts.
• Logs: text events, useful for detail and audit.
• Traces: request paths, useful for distributed latency analysis.

Metric design can follow established methods:

• RED: Rate, Errors, Duration, focused on service.
• USE: Utilization, Saturation, Errors, focused on resource.
• Four golden signals: latency, traffic, errors, saturation.

Example PromQL queries:

sum(rate(http_requests_total[5m])) by (service)

sum(rate(http_requests_total{status=~"5.."}[5