// Web Server · Nginx
Nginx Logs
Nginx writes access and error logs. The format is defined via log_format in nginx.conf. Nginx's error log also covers upstream proxy failures, making it central to debugging reverse proxy setups.
Access Log
Default: /var/log/nginx/access.log. Nginx's default format is called "combined" and is nearly identical to Apache's.
192.168.1.100 - -
[10/Apr/2025:16:22:11 +0000]
"POST /api/login HTTP/2.0"
401
189
"-"
"python-requests/2.28.0"
rt=0.003 uct=0.001 urt=0.002
$remote_addr
$time_local
$request
$status
$body_bytes_sent
$http_referer
$http_user_agent
$request_time / $upstream_*
Variable Reference
| Variable | Type | Description |
|---|---|---|
| $remote_addr | IP | Client IP. Behind proxy/CDN, real IP is in $http_x_forwarded_for or $http_x_real_ip (use set_real_ip_from). |
| $remote_user | String / - | HTTP Basic Auth user. Hyphen if unauthenticated. |
| $time_local | Timestamp | Local time in Combined Log Format. Use $time_iso8601 for ISO 8601 with timezone. |
| $request | String | Full first request line: method URI protocol. E.g. GET /path HTTP/1.1. |
| $request_method | String | HTTP method alone: GET, POST, PUT, DELETE, HEAD, OPTIONS, PATCH. |
| $request_uri | String | Full URI including query string. Immutable — unchanged by rewrites. |
| $uri | String | Current URI after normalization and rewrites. May differ from $request_uri. |
| $args | String | Query string parameters (same as $query_string). |
| $status | Integer | HTTP response status code. |
| $body_bytes_sent | Integer | Bytes sent in response body (headers excluded). Use $bytes_sent for total including headers. |
| $http_referer | String / - | Referer header value. |
| $http_user_agent | String | User-Agent header value. |
| $request_time | Float (s) | Total time from first client byte received to last response byte sent. Millisecond precision. |
| $upstream_response_time | Float / - | Time waiting on the upstream backend (PHP-FPM, Node, gunicorn). Only set for proxied requests. |
| $upstream_connect_time | Float / - | Time to establish connection to upstream. Spike = backend overloaded or network issue. |
| $upstream_header_time | Float / - | Time from upstream connection to first response header byte received. |
| $upstream_addr | IP:port | Backend address that handled the request. Useful for debugging load balancer routing. |
| $upstream_status | Integer / - | Status code from backend. May differ from $status if Nginx intercepts the error. |
| $host | String | Host header or server_name. Essential in multi-vhost setups. |
| $ssl_protocol | String / - | TLS version: TLSv1.2, TLSv1.3, or empty for non-HTTPS. |
| $ssl_cipher | String / - | Negotiated cipher suite. |
| $connection | Integer | Connection sequence number. Identifies keepalive reuse across multiple requests. |
Error Log
Default: /var/log/nginx/error.log. Every entry includes: timestamp, level, PID#TID, and message.
2025/04/10 16:22:11
[error]
7890#7890:
*1234 connect() failed (111: Connection refused) while connecting to upstream, client: 10.0.0.5, server: api.example.com, request: "GET /health HTTP/1.1", upstream: "http://127.0.0.1:3000/health"
Error Levels
| Level | Meaning |
|---|---|
| emerg | System-wide emergency. Nginx cannot proceed. Port/socket binding failure. |
| alert | Requires immediate attention. Worker crash, core dump. |
| crit | Critical. SSL cert load failure, listen() failures. |
| error | Most common actionable level. Upstream failures, permission denied, client reset. |
| warn | Recoverable. Upstream keepalive pool exhausted, header too large. |
| notice | Operational: signal received, workers started/stopped, config reloaded. |
| info | Detailed info. Connection details, upstream selection. |
| debug | Full trace. Use debug_connection for a specific IP to avoid log floods. |
Common Nginx Errors Decoded
| Error | Cause | Fix |
|---|---|---|
| connect() failed (111: Connection refused) | Backend not running or wrong port in proxy_pass/fastcgi_pass | Start backend service. Verify upstream address and port. |
| upstream timed out (110) | Backend too slow; exceeds proxy_read_timeout | Increase proxy_read_timeout / fastcgi_read_timeout. Fix slow queries in app. |
| no live upstreams while connecting | All upstream servers marked down (max_fails reached) | Check backend health. Review max_fails and fail_timeout settings. |
| recv() failed (104: Connection reset by peer) | Client abruptly closed connection (browser cancel/timeout) | Usually benign. High volume may indicate DDoS or network issues. |
| SSL_do_handshake() failed | TLS failure — expired cert, wrong cert for SNI, cipher mismatch | Check ssl_certificate path, openssl x509 -enddate -noout -in cert.pem. |
| too many open files | Process hit file descriptor limit | Increase worker_rlimit_nofile in nginx.conf and OS fs.file-max. |
| client intended to send too large body | Upload exceeds client_max_body_size | Set client_max_body_size 50m; in location/server block. |
| directory index forbidden | No index file and autoindex off | Add index.html or enable autoindex on; (avoid in production). |
log_format Configuration
nginx.conf
http { ## Default combined format log_format combined '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" "$http_user_agent"'; ## Extended with upstream timing — recommended for production log_format extended '$remote_addr - $remote_user [$time_local] "$request" $status $body_bytes_sent ' '"$http_referer" "$http_user_agent" ' 'rt=$request_time uct=$upstream_connect_time uht=$upstream_header_time urt=$upstream_response_time ' 'upstream=$upstream_addr us=$upstream_status'; ## JSON — ideal for Filebeat, Fluentd, Loki, Splunk log_format json_logs escape=json '{' '"time":"$time_iso8601",' '"remote_addr":"$remote_addr",' '"request":"$request",' '"status":$status,' '"bytes":$body_bytes_sent,' '"request_time":$request_time,' '"upstream_time":"$upstream_response_time",' '"host":"$host",' '"referer":"$http_referer",' '"ua":"$http_user_agent",' '"ssl":"$ssl_protocol/$ssl_cipher"' '}'; access_log /var/log/nginx/access.log extended; error_log /var/log/nginx/error.log warn; server { ## Per-vhost logs access_log /var/log/nginx/example.com.access.log json_logs; error_log /var/log/nginx/example.com.error.log error; } }
💡 Pro Tip
Enable debug logging only for a specific IP using debug_connection 10.0.0.5; inside the events {} block. This avoids flooding your error.log while debugging a specific client connection.