Configuring the Apache Error and Access Logs

Introduction

Apache is a cross-platform and open-source HTTP server. It has many great features that can be enhanced by numerous other modules. Examining the log files is one of the most regular duties involved in administering Apache web servers.

When troubleshooting server or application problems, being able to configure and examine the logs is particularly helpful because they contain in-depth debugging information.

The Apache error and access logs are files that record information about errors and access attempts made to your Apache web server. The error log records any problems encountered by the server, while the access log keeps track of all incoming requests to the server.

Configuring the Access Log

For each processed request, the Apache web server creates a new entry in the access log. A timestamp and numerous details about the client and the requested resource are included in each event record. The location of visitors, the pages they view, how long they stay on each page, and other information are all displayed in access logs.

The CustomLog directive specifies both the location of the log file and the message format.

The CustomLog directive's most basic syntax is as follows:

CustomLog log_file format [condition];

The log_file can either be a full path to the log file or be relative to the ServerRoot. The pipe symbol | can be used to pipe the log messages to another application.

The second argument, format determines the log message format. Both an explicit format specification and a nickname specified by the LogFormat directive are acceptable.

LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
CustomLog logs/access.log combined

CustomLog logs/access.log "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\""

Preferably, define the LogFormat directive and use it as a nickname in the CustomLog directive to prevent repeating the same code several times.

Check the “mod_log_config” module documentation for a complete list of all format strings and modifiers.

You can only write log messages when a particular condition is met by using the third argument [condition], which is optional. Usually, environment variables are used for this. The ! symbol can be used to disregard the condition.

For instance, you might use the following if you wanted to prevent requests for css files from being recorded in the log file:

SetEnvIf Request_URI \.css$ css-file
CustomLog logs/access.log custom env=!css-file

Either define a new LogFormat directive or override the default one to alter the logging format. In most cases, it is preferable to specify a new format.

Even while the access log contains a lot of valuable information, it uses up disk space and could have an impact on server performance. You might want to disable the access log if your server has few resources and your website is frequently visited.

Simply remove the CustomLog directive from the main server configuration and virtual server sections to accomplish this.

Set the first argument of the CustomLog directive to /dev/null if you only want to disable the access log for one virtual host:

CustomLog /dev/null combined

Configuring the Error Log

In the error log file, Apache records messages pertaining to the application and common server errors. The error log is the best place to start when troubleshooting problems with your web application if it is facing errors.

The name and location of the error log are specified by the ErrorLog directive. It appears in the following way:

ErrorLog log_file

The path of the log_file is specified as relative to the ServerRoot if it is not absolute. The pipe symbol | can be used to pipe the error messages to another program.

The level of logging is determined by the LogLevel variable. Levels are given below in order of increasing severity, from low to high:

Trace messages - trace1 - trace8
Debugging messages - debug
Informational messages - info
Notices - notice
Warnings - warn
Errors while processing a request - error
Critical issues. Requires a prompt action - crit
Alerts. Action must be taken immediately - alert
Emergency situation. The system is in an unusable state - emerg

The higher levels are included in each log level. For instance, Apache also writes the error, crit, alert, and emerg messages if the log level is set to warn.
The default value for the LogLevel parameter is warn when it is not supplied. Setting the level to at least crit is advised.

The format of the error log is specified by the ErrorLogFormat directive. The default format used by the Apache server on the majority of Linux distributions is adequate in most situations.

Virtual Hosts and Global Logging

Both the logging behavior and the file locations can be customized on a global or per-virtual host basis.

As soon as the CustomLog or ErrorLog directives are set in the main server context, the server writes all log messages to the same access and error log files. Otherwise, just the log messages for that virtual host are written to the designated file if the directives are contained inside a <VirtualHost> block.

The server context's log directive is superseded by the one set in the <VirtualHost> block's log directive.

Log messages from virtual hosts without CustomLog or ErrorLog directives will be written to the global server logs.

It is advised to set different access and error log files for each virtual host to improve readability. Here's an example:

<VirtualHost *:80>
     ServerName example.com
     ServerAlias www.example.com
     ServerAdmin [email protected]
     DocumentRoot /var/www/example.com/public
     LogLevel warn
     ErrorLog /var/www/example.com/logs/error.log
     CustomLog /var/www/example.com/logs/access.log combined
</VirtualHost>

You must restart the Apache service each time you make changes to the configuration file in order for the changes to take effect.

Location of the Log Files

The /var/log/apache2 directory is the default location for access and error logs on Debian-based systems like Ubuntu. The log files are stored in the /var/log/httpd directory on CentOS.

Reading and Understanding the Apache Log Files

cat , less , grep , cut , awk and other common commands can be used to open and parse log files.

Here is an example of a record from the access log file that uses the combine log format of Debian:

Output

192.168.33.1 - - [08/Jan/2020:21:39:03 +0000] "GET / HTTP/1.1" 200 6169 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.88 Safari/537.36"

Let us examine what each record field implies individually:

%h - 192.168.33.1 - The client making the request's hostname or IP address.
%l - - - Remote logname. This field displays - if the username is not specified.
%u - - - The remote username is displayed if the request was authenticated.
%t - [08/Jan/2020:21:39:03 +0000] - Time on the local server.
\"%r\" - "GET / HTTP/1.1" - The first line of request. The protocol, request type, and path.
%>s - 200 - The server's final response code. The status of the original request will be displayed if the > symbol is not used and the request has been internally redirected.
%O - 396 - The server response size in bytes.
\"%{Referer}i\" - "-" - The referral's URL.
\"%{User-Agent}i\" - Mozilla/5.0 ... - The client's user agent (web browser).

In order to view the log file in real-time, use thetail command:

tail -f access.log

FAQs to Configure the Apache Error and Access Logs

Where can I find the Apache error and access log files?

By default, the Apache error log file is typically located at /var/log/apache2/error.log on Linux systems. The access log file can be found at /var/log/apache2/access.log.

How can I log remote IP addresses behind a load balancer or proxy server?

To log the real IP addresses of clients when your Apache server is behind a load balancer or proxy server, you can use the RemoteIPHeader and RemoteIPProxiesHeader directives. This allows Apache to extract the client IP from the specified header field and log it accordingly.

How can I configure the Apache error and access logs to store logs in a different location?

To configure the error and access logs to store logs in a different location, you can edit the ErrorLog and CustomLog directives in the Apache configuration file (httpd.conf or apache2.conf). Specify the desired file path for both directives accordingly.

Can I separate error logs for different virtual hosts on my Apache server?

Yes, you can separate error logs for different virtual hosts on your Apache server. In the virtual host configuration block, you can specify a different ErrorLog directive to set the path for the error log file for that particular virtual host.

How can I enable or disable logging specific types of errors in the Apache error log?

To enable or disable logging specific types of errors, you can modify the LogLevel directive in the Apache configuration file. Increase the log level to include more errors or decrease it to filter out less severe errors from being logged.

Is it possible to log additional information in the Apache access logs?

Yes, you can log additional information in the Apache access logs. Apache provides various log formats like Combined, Common, and Custom. You can create a custom log format using the LogFormat directive to include specific information such as the client's IP address, requested URL, response code, referer, and more.

Is it possible to analyze and view Apache error and access logs using log analysis tools?

Yes, you can analyze and view Apache error and access logs using log analysis tools like AWStats, Webalizer, or ELK Stack (Elasticsearch, Logstash, and Kibana). These tools provide visual representation, filtering options, and detailed insights into the logs, helping you understand server performance, track errors, and perform security analysis.