DevOps Lab: Run Your Own Log Server

By Sudheer S

syslog

Syslog is a standard for logging system events on Unix and Linux systems. It is typically used to collect and store log messages from various applications and system components, such as the kernel, system libraries, and applications. Syslog uses a client-server model, where each client application sends log messages to a central syslog server, which then stores the messages in a log file. The syslog server can also forward the log messages to other syslog servers or send them to a log management system for further analysis. Syslog uses a simple text-based format for its log messages, which makes it easy to read and analyze. It also supports multiple levels of severity, allowing applications to categorize their log messages based on importance.

To implement syslog on a Linux system, you’ll need to first install the syslog daemon (syslogd) on the system. This daemon is responsible for receiving, storing, and forwarding log messages from syslog clients. Once the syslog daemon is installed, you can configure the applications and system components on your system to send log messages to it. This typically involves editing the applications’ configuration files and specifying the syslog server’s IP address and port number. You can then use the syslog daemon’s configuration file to control how the daemon behaves, such as where it stores log messages and how it formats them. You can also use the syslog daemon to forward log messages to other syslog servers or to a log management system for further analysis.

rsyslog

rsyslog is an enhanced version of the syslog daemon that is commonly used on Unix and Linux systems. It is designed to be compatible with the standard syslog protocol, but offers additional features and performance improvements. Some key differences between rsyslog and syslogd include:

  • Rsyslog supports multiple threading and can handle a higher volume of log messages than syslogd.
  • Rsyslog has a more flexible configuration system, allowing you to specify complex rules for processing log messages.
  • Rsyslog supports newer protocols and features, such as TCP and SSL/TLS encryption, and the ability to send and receive log messages over the network.
  • Rsyslog can store log messages in a variety of formats, such as plain text, JSON, and databases.

Modern versions of Linux distributions ship rsyslog in the default installation profiles. Your Linux installation already has the program logger installed via the bsdutils package.

To implement rsyslog on a Unix or Linux system, you’ll need to first install the rsyslog daemon. Once it is installed, you can configure the applications and system components on your system to send log messages to it, using the same procedures as you would for syslogd. You can then use rsyslog’s configuration file to control how the daemon behaves and to specify how log messages should be processed and stored.

Sending Log Messages

Sending a sample log message from the shell using the logger command:

logger "this is a test message"

Sending a sample log message from a Python program:

import syslog
syslog.syslog("This is a log messages from Python")

Viewing The Log Message

tail /var/log/syslog

The Project Steps

  • Create a centralized log server. You could use a VM for this on your laptop. Configure rsyslog to receive log messages on a network interface.
  • Write a shell script or an application using Python or another language to send log messages using the syslog protocol.
  • On the application VMs collect the logs using the rsyslog daemon and forward them to the centralized log server.
  • View the logs on the central log server.
  • Optional step: display the log messages on a web interface. Use open source projects and write your own rudimentary web interface to show and search log messages.

IAC It

Like most of our DevOps lab projects, use IAC tools such as Ansible to install and configure rsyslog.

References

Resources