Log Management Commands

Logs are the foundation of system troubleshooting, security auditing, and performance analysis. In a Linux environment, log data is typically managed jointly by rsyslog, systemd-journald, and the applications themselves. Mastering common log management commands allows engineers to pinpoint errors quickly, analyze access trends, and prevent logs from consuming all available disk space.

Common Log Files and Locations

Log paths vary slightly across distributions and services, but several locations are standard:

• /var/log/messages — general system log on RHEL/CentOS
• /var/log/syslog — general system log on Debian/Ubuntu
• /var/log/auth.log or /var/log/secure — authentication and security events
• /var/log/kern.log — kernel messages
• /var/log/nginx/, /var/log/mysql/ — application-specific log directories
• journalctl — systemd binary logs, not stored as plain text files

To inspect the log directory and check disk usage:

ls -lh /var/log
du -sh /var/log/* | sort -h

Basic Viewing and Real-Time Monitoring

The most frequently used commands for reading logs are cat, less, head, and tail. For large files, less or tail is preferable to avoid flooding the terminal with excessive output.

tail -n 50 /var/log/syslog
tail -f /var/log/nginx/access.log
less +F /var/log/messages

tail -f continuously streams new lines as they are written, making it ideal for live observation. less +F offers similar follow-mode functionality; press Ctrl+C to exit follow mode and return to normal browsing.

Filtering and Searching

grep enables fast keyword filtering:

grep -i "error" /var/log/syslog
grep -E "error|fail|timeout" /var/log/app.log
grep -C 3 "OutOfMemory" /var/log/app.log

To count HTTP 404 responses in an access log:

grep -c "404" /var/log/nginx/access.log

You can also extract specific fields with awk when the log format is consistent:

awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -nr | head
awk '$9 == 500 {print $1, $7}' /var/log/nginx/access.log

Field positions differ between services, so adjust column numbers according to the actual log layout.

Managing systemd Logs with journalctl

Modern distributions rely heavily on systemd-journald, and journalctl is the primary interface for querying its binary logs.

journalctl -b
journalctl -u nginx.service -f
journalctl --since "2024-05-01 10:00" --until "2024-05-01 11:00"
journalctl -p err -b
journalctl -k
journalctl -o json-pretty

Common options include:

• -b — show logs from the current boot
• -u — filter by a specific systemd unit
• -f — follow logs in real time
• -p — filter by priority, such as err or warning
• -k — display kernel messages only
• --since and --until — restrict output to a time window

To check how much space the journal occupies:

journalctl --disk-usage
journalctl --vacuum-time=7d
journalctl --vacuum-size=500M

For persistent storage across reboots, edit /etc/systemd/journald.conf, set Storage=persistent, and restart systemd-journald.

Kernel and Hardware Logs

dmesg reads the kernel ring buffer and is useful for diagnosing hardware, driver, and boot-time issues.

dmesg -T
dmesg -T | tail -n 50
dmesg -T | grep -i "usb"

The -T flag converts timestamps into a human-readable format. On some systems, elevated privileges are required to view the full output.

Log Rotation and Cleanup

Unchecked log growth can exhaust disk space. logrotate is the standard rotation utility. Its main configuration lives in /etc/logrotate.conf, while service-specific rules are usually placed in /etc/logrotate.d/.

Example configuration:

/var/log/myapp/*.log {
    daily
    rotate 7
    compress
    delaycompress
    missingok
    notifempty
    create 0640 www-data adm
    sharedscripts
    postrotate
        systemctl reload myapp >/dev/null 2>&1 || true
    endscript
}

Test or force rotation with:

logrotate -d /etc/logrotate.d/myapp
logrotate -f /etc/logrotate.d/myapp

To manually empty a file that is actively being written to, use:

truncate -s 0 /var/log/app.log

Avoid deleting the log file outright while a process holds it open; doing so can prevent file-handle release and may not free disk space as expected.

Centralized Logging and Best Practices

In multi-host environments, forwarding logs to a centralized platform—such as an rsyslog server, ELK stack, or Loki—simplifies correlation and retention. An rsyslog TCP forwarding rule looks like this:

*.* @@logserver:514

Here, @@ denotes TCP, while @ indicates UDP. It is advisable to standardize timestamps, log levels, and field formats across services. Combine rotation, permission controls, and disk monitoring with tools like journalctl, grep, awk, and logrotate to build a stable, maintainable logging infrastructure. Periodically review /var/log utilization to catch growth issues before they affect system stability.