Skip to main content

Log files reference

Contributors netapp-lhalbert
PDFs

StorageGRID provides logs that are used to capture events, diagnostic messages, and error conditions. You might be asked to collect log files and forward them to technical support to assist with troubleshooting.

The logs are categorized as follows:

Note The details provided for each log type are for reference only. The logs are intended for advanced troubleshooting by technical support. Advanced techniques that involve reconstructing the problem history using the audit logs and the application log files are beyond the scope of these instructions.

Access the logs

To access the logs, you can collect log files and system data from one or more nodes as a single log file archive. Or, if the primary Admin Node is unavailable or unable to reach a specific node, you can access individual log files for each grid node as follows:

  1. Enter the following command: ssh admin@grid_node_IP

  2. Enter the password listed in the Passwords.txt file.

  3. Enter the following command to switch to root: su -

  4. Enter the password listed in the Passwords.txt file.

Export logs to the syslog server

Exporting the logs to the syslog server provides these capabilities:

  • Receive a list of all Grid Manager and Tenant Manager requests, in addition to S3 and Swift requests.

  • Better visibility into S3 requests that return errors, without the performance impact caused by audit logging methods.

  • Access to HTTP-layer requests and error codes that are easy to parse.

  • Better visibility into requests that were blocked by traffic classifiers at the load balancer.

To export the logs, refer to Configure audit messages and log destinations.

Log file categories

The StorageGRID log file archive contains the logs described for each category and additional files that contain metrics and debug command output.

Archive location Description

audit

Audit messages generated during normal system operation.

base-os-logs

Base operating system information, including StorageGRID image versions.

bundles

Global configuration information (bundles).

cassandra

Cassandra database information and Reaper repair logs.

ec

VCSs information about the current node and EC group information by profile ID.

grid

General grid logs including debug (bycast.log) and servermanager logs.

grid.json

Grid configuration file shared across all nodes. Additionally, node.json is specific to the current node.

hagroups

High availability groups metrics and logs.

install

Gdu-server and install logs.

Lambda-arbitrator

Logs related to the S3 Select proxy request.

lumberjack.log

Debug messages related to log collection.

Metrics

Service logs for Grafana, Jaeger, node exporter, and Prometheus.

miscd

Miscd access and error logs.

mysql

The mariaDB database configuration and related logs.

net

Logs generated by networking-related scripts and the Dynip service.

nginx

Load balancer and grid federation configuration files and logs. Also includes Grid Manager and Tenant Manager traffic logs.

nginx-gw

  • access.log: Grid Manager and Tenant manager request log messages.

    • These messages are prefixed with mgmt: when exported using syslog.

    • The format of these log messages is [$time_iso8601] $remote_addr $status $bytes_sent $request_length $request_time "$endpointId" "$request" "$http_host" "$http_user_agent" "$http_referer"

  • cgr-access.log.gz: Inbound cross-grid replication requests.

    • These messages are prefixed with cgr: when exported using syslog.

    • The format of these log messages is [$time_iso8601] $remote_addr $status $bytes_sent $request_length $request_time "$endpointId" "$upstream_addr" "$request" "$http_host"

  • endpoint-access.log.gz: S3 and Swift requests to load balancer endpoints.

    • These messages are prefixed with endpoint: when exported using syslog.

    • The format of these log messages is [$time_iso8601] $remote_addr $status $bytes_sent $request_length $request_time "$endpointId" "$upstream_addr" "$request" "$http_host"

  • nginx-gw-dns-check.log: Related to the new DNS check alert.

ntp

NTP configuration file and logs.

Orphaned objects

Logs pertaining to orphaned objects.

os

Node and grid state file, including services pid.

other

Log files under /var/local/log that aren't collected in other folders.

perf

Peformance information for CPU, networking, and disk I/O.

prometheus-data

Current Prometheus metrics, if the log collection includes Prometheus data.

provisioning

Logs related to grid provisioning process.

raft

Logs from Raft cluster used in platform services.

ssh

Logs related to SSH configuration and service.

snmp

SNMP agent configuration used for sending SNMP notifications.

sockets-data

Sockets data for network debug.

system-commands.txt

Output of StorageGRID container commands. Contains system information, such as networking and disk usage.

synchronize-recovery-package

Related to maintaining consistency of the latest Recovery Package across all Admin Nodes and Storage Nodes that host the ADC service.