Documentation

Logs

Access, search, and analyze logs from your DanubeData resources for debugging, auditing, and compliance.

Overview

Log management features:

  • Centralized Logging: All logs in one place
  • Real-Time Streaming: Live log viewing
  • Search and Filter: Find specific entries
  • Retention: 30 days included
  • Export: Download logs for archival

Log Types

System Logs

Operating system logs:

  • syslog: System messages
  • auth.log: Authentication attempts
  • kernel.log: Kernel messages
  • dmesg: Boot and hardware messages

Application Logs

Application-specific logs:

  • Web Server: Nginx, Apache logs
  • Application: Custom application logs
  • Database: PostgreSQL, MySQL logs
  • Cache: Redis logs

Access Logs

HTTP request logs:

192.168.1.100 - - [12/Oct/2024:10:15:23 +0000] "GET /api/users HTTP/1.1" 200 1234 "-" "Mozilla/5.0"

Error Logs

Application and system errors:

[2024-10-12 10:15:23] ERROR: Database connection failed
[2024-10-12 10:15:24] CRITICAL: Out of memory

Accessing Logs

Via Dashboard

  1. Navigate to resource
  2. Click Logs tab
  3. Select log type
  4. View real-time or historical logs

Log Viewer Features

  • Real-Time: Live log streaming
  • Search: Full-text search
  • Filter: By level, source, time
  • Highlight: Syntax highlighting
  • Download: Export logs

Log Levels

Filter by severity:

  • DEBUG: Detailed debug information
  • INFO: Informational messages
  • WARNING: Warning messages
  • ERROR: Error conditions
  • CRITICAL: Critical issues

Searching Logs

Basic Search

Search for specific text:

Search: "database connection"
Results: All log lines containing phrase

Advanced Filters

Combine multiple criteria:

Level: ERROR
Time: Last 24 hours
Source: application
Search: "timeout"

Example Searches

Find errors:

level:ERROR OR level:CRITICAL

Find slow queries:

"execution time" AND duration > 1000

Authentication failures:

source:auth.log AND "Failed password"

Log Retention

Included Retention

  • Duration: 30 days
  • Included: No additional cost
  • Automatic: Old logs auto-deleted

Extended Retention

Coming soon:

  • 90 days ($10/month)
  • 1 year ($30/month)
  • Custom retention available

Exporting Logs

Download for long-term storage:

  1. Click Export
  2. Select date range
  3. Choose format (JSON, CSV, text)
  4. Download file

Log Analysis

Common Patterns

Error Tracking:

grep ERROR application.log | wc -l
# Count error occurrences

Performance Analysis:

grep "execution time" | awk '{sum+=$NF} END {print sum/NR}'
# Average execution time

Traffic Analysis:

awk '{print $1}' access.log | sort | uniq -c | sort -rn | head -10
# Top 10 IP addresses

Best Practices

Logging Strategy

  1. Log Appropriately: Not too verbose, not too sparse
  2. Structured Logging: Use consistent formats
  3. Context: Include relevant information
  4. Levels: Use appropriate log levels
  5. Sensitive Data: Don't log passwords, keys

Log Monitoring

  1. Regular Review: Check logs daily
  2. Error Tracking: Monitor error rates
  3. Pattern Detection: Look for anomalies
  4. Alerting: Set up alerts for critical errors
  5. Archive Important Logs: Export for compliance

Integration

Send Logs to External Services

Forward logs to third-party services:

Supported Integrations:

  • Datadog
  • Splunk
  • ELK Stack
  • Loggly
  • Papertrail

Configure in Settings > Log Forwarding.

Related Documentation