NGINX Logs to Graylog - Quick and Dirty
Sometimes I have a casual personal project where I just want to get the logs into something where I can easily run queries and create visualizations. The popular content packs on the Graylog Marketplace seem to be deprecated or designed for the NGINX Docker container. This guide will get NGINX logs forwarding to Graylog in just a few minutes. Security and reliability are not guaranteed.
Configuring NGINX
Updated: Since many NGINX content packs are outdated and do not target baremetal servers, I created my own. These instructions have been modified to use my version.
Edit /etc/nginx/nginx.conf and add the following lines in the Logging Settings section. Replace logging.example.com with the domain or IP address of your Graylog server. Restart the service.
log_format graylog_json escape=json '{ "nginx_timestamp": "$time_iso8601", '
'"remote_addr": "$remote_addr", '
'"connection": "$connection", '
'"connection_requests": $connection_requests, '
'"pipe": "$pipe", '
'"body_bytes_sent": $body_bytes_sent, '
'"request_length": $request_length, '
'"request_time": $request_time, '
'"response_status": $status, '
'"request": "$request", '
'"request_method": "$request_method", '
'"host": "$host", '
'"upstream_cache_status": "$upstream_cache_status", '
'"upstream_addr": "$upstream_addr", '
'"http_x_forwarded_for": "$http_x_forwarded_for", '
'"http_referrer": "$http_referer", '
'"http_user_agent": "$http_user_agent", '
'"http_version": "$server_protocol", '
'"remote_user": "$remote_user", '
'"http_x_forwarded_proto": "$http_x_forwarded_proto", '
'"upstream_response_time": "$upstream_response_time", '
'"nginx_access": true }';
access_log syslog:server=logging.example.com:12401 graylog_json;
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
Graylog Content Pack
Download this content pack and upload it to Graylog by going to System → Content Packs then install it. This will create the nginx-syslog input. The extractors attached to the input parse the JSON in the syslog message and also replace the message field with a short readable summary.
Update (2021-01-28): The nginx ISO 8601 timestamp is not parseable by Graylog. To ensure a correct timestamp, Graylog adds a gl2_processing_error field and replaces the timestamp field with the current time. To correct the timestamp before this happens, the provided nginx.conf was modified to use the field nginx_timestamp. An extractor was added to the content pack to copy the field to the timestamp field. It uses the Flexible Date converter to correctly parse the date and convert it to UTC.
Wrap Up
Make sure logs are flowing in using the Show received messages for the nginx-syslog input. You can now search fields such as http_user_agent, request_method, and response_status.