Logging Levels: Debug, Info, Warn, Error & Structured Logging

Problem

Applications generate vast amounts of runtime information. Without proper logging levels and structure, developers face:

• Information overload: Can't distinguish critical issues from routine operations
• Performance degradation: Excessive logging slows production systems
• Debugging difficulty: Unstructured logs are hard to parse and analyze
• Compliance issues: Lack of audit trails for security and regulatory requirements
• Scalability problems: Centralized log aggregation becomes unwieldy with unstructured data

Solution

Implement a multi-level logging strategy combined with structured logging to:

1. Control verbosity through severity levels
2. Standardize format for machine parsing
3. Enable filtering at collection and analysis stages
4. Improve searchability across distributed systems
5. Maintain performance in production environments

Logging Levels Hierarchy

DEBUG (0) → INFO (1) → WARN (2) → ERROR (3)

Code

Python Implementation

import logging
import json
from datetime import datetime
from typing import Any, Dict

class StructuredLogger:
    """Structured logging with multiple levels"""

    def __init__(self, name: str, level: str = "INFO"):
        self.logger = logging.getLogger(name)
        self.logger.setLevel(getattr(logging, level))

        # JSON formatter for structured output
        handler = logging.StreamHandler()
        formatter = logging.Formatter(
            '%(message)s'
        )
        handler.setFormatter(formatter)
        self.logger.addHandler(handler)

    def _format_log(self, level: str, message: str, 
                    **context: Any) -> str:
        """Format log entry as structured JSON"""
        log_entry = {
            "timestamp": datetime.utcnow().isoformat(),
            "level": level,
            "message": message,
            "context": context
        }
        return json.dumps(log_entry)

    def debug(self, message: str, **context: Any) -> None:
        """Log debug-level message"""
        self.logger.debug(self._format_log("DEBUG", message, **context))

    def info(self, message: str, **context: Any) -> None:
        """Log info-level message"""
        self.logger.info(self._format_log("INFO", message, **context))

    def warn(self, message: str, **context: Any) -> None:
        """Log warning-level message"""
        self.logger.warning(self._format_log("WARN", message, **context))

    def error(self, message: str, exception: Exception = None, 
              **context: Any) -> None:
        """Log error-level message with optional exception"""
        if exception:
            context["exception"] = str(exception)
            context["exception_type"] = type(exception).__name__
        self.logger.error(self._format_log("ERROR", message, **context))

# Usage Example
logger = StructuredLogger("app", level="DEBUG")

# Debug: Detailed diagnostic info
logger.debug("Database connection initialized", 
             host="localhost", port=5432, pool_size=10)

# Info: General application flow
logger.info("User authentication successful", 
            user_id=12345, ip_address="192.168.1.1")

# Warn: Recoverable issues
logger.warn("Cache miss detected", 
            cache_key="user:12345", retry_count=2)

# Error: Critical failures
try:
    result = 1 / 0
except ZeroDivisionError as e:
    logger.error("Calculation failed", 
                 exception=e, operation="division", 
                 operands=[1, 0])

JavaScript/Node.js Implementation

const fs = require('fs');

class StructuredLogger {
  constructor(name, level = 'INFO') {
    this.name = name;
    this.levels = { DEBUG: 0, INFO: 1, WARN: 2, ERROR: 3 };
    this.currentLevel = this.levels[level];
  }

  _formatLog(level, message, context = {}) {
    return JSON.stringify({
      timestamp: new Date().toISOString(),
      level,
      logger: this.name,
      message,
      context
    });
  }

  _shouldLog(level) {
    return this.levels[level] >= this.currentLevel;
  }

  debug(message, context = {}) {
    if (this._shouldLog('DEBUG')) {
      console.log(this._formatLog('DEBUG', message, context));
    }
  }

  info(message, context = {}) {
    if (this._shouldLog('INFO')) {
      console.log(this._formatLog('INFO', message, context));
    }
  }

  warn(message, context = {}) {
    if (this._shouldLog('WARN')) {
      console.warn(this._formatLog('WARN', message, context));
    }
  }

  error(message, exception = null, context = {}) {
    if (this._shouldLog('ERROR')) {
      if (exception) {
        context.exception = exception.message;
        context.stack = exception.stack;
      }
      console.error(this._formatLog('ERROR', message, context));
    }
  }
}

// Usage
const logger = new StructuredLogger('api-server', 'DEBUG');

logger.debug('Request received', { 
  method: 'POST', 
  path: '/api/users' 
});

logger.info('User created successfully', { 
  userId: 789, 
  email: 'user@example.com' 
});

logger.warn('Slow query detected', { 
  query: 'SELECT * FROM users', 
  duration: 2500 
});

try {
  throw new Error('Database connection timeout');
} catch (err) {
  logger.error('Database operation failed', err, { 
    operation: 'INSERT', 
    table: 'orders' 
  });
}

Java Implementation

import com.fasterxml.jackson.databind.ObjectMapper;
import java.time.Instant;
import java.util.HashMap;
import java.util.Map;

public class StructuredLogger {
    private static final ObjectMapper mapper = new ObjectMapper();
    private String name;
    private LogLevel currentLevel;

    enum LogLevel {
        DEBUG(0), INFO(1), WARN(2), ERROR(3);
        final int value;
        LogLevel(int value) { this.value = value; }
    }

    public StructuredLogger(String name, String level) {
        this.name = name;
        this.currentLevel = LogLevel.valueOf(level);
    }

    private String formatLog(LogLevel level, String message, 
                            Map<String, Object> context) {
        try {
            Map<String, Object> logEntry = new HashMap<>();
            logEntry.put("timestamp", Instant.now().toString());
            logEntry.put("level", level.name());
            logEntry.put("logger", name);
            logEntry.put("message", message);
            logEntry.put("context", context);
            return mapper.writeValueAsString(logEntry);
        } catch (Exception e) {
            return "{\"error\": \"Serialization failed\"}";
        }
    }

    private boolean shouldLog(LogLevel level) {
        return level.value >= currentLevel.value;
    }

    public void debug(String message, Map<String, Object> context) {
        if (shouldLog(LogLevel.DEBUG)) {
            System.out.println(formatLog(LogLevel.DEBUG, message, context));
        }
    }

    public void info(String message, Map<String, Object> context) {
        if (shouldLog(LogLevel.INFO)) {
            System.out.println(formatLog(LogLevel.INFO, message, context));
        }
    }

    public void warn(String message, Map<String, Object> context) {
        if (shouldLog(LogLevel.WARN)) {
            System.out.println(formatLog(LogLevel.WARN, message, context));
        }
    }

    public void error(String message, Exception exception, 
                     Map<String, Object> context) {
        if (shouldLog(LogLevel.ERROR)) {
            if (exception != null) {
                context.put("exception", exception.getMessage());
                context.put("exceptionType", exception.getClass().getName());
            }
            System.err.println(formatLog(LogLevel.ERROR, message, context));
        }
    }

    // Usage
    public static void main(String[] args) {
        StructuredLogger logger = new StructuredLogger("app", "DEBUG");

        Map<String, Object> ctx1 = new HashMap<>();
        ctx1.put("service", "auth");
        ctx1.put("duration_ms", 145);
        logger.info("Authentication service started", ctx1);

        Map<String, Object> ctx2 = new HashMap<>();
        ctx2.put("userId", 456);
        ctx2.put("action", "login");
        logger.debug("User login attempt", ctx2);
    }
}

Tips

1. Level Selection Guidelines

• DEBUG: Variable values, function entry/exit, loop iterations
• INFO: Application startup, configuration loaded, user actions
• WARN: Deprecated API usage, retry attempts, resource limits
• ERROR: Exceptions, failed operations, data inconsistencies

2. Structured Logging Best Practices

# ✅ Good: Structured context
logger.info("Payment processed", 
            user_id=123, amount=99.99, currency="USD", 
            transaction_id="txn_abc123")

# ❌ Bad: Unstructured string concatenation
logger.info(f"Payment processed for user 123: $99.99 USD")

3. Performance Optimization

• Set production level to WARN or ERROR to reduce I/O
• Use lazy evaluation: logger.debug(lambda: expensive_operation())
• Implement log sampling for high-frequency events
• Buffer logs before writing to reduce system calls

4. Context Propagation

Use correlation IDs across distributed systems:

import uuid
request_id = str(uuid.uuid4())
logger.info("Request started", request_id=request_id, endpoint="/api/users")
# Pass request_id to downstream services

5. Sensitive Data Handling

# ❌ Never log sensitive data
logger.info("User login", password=user_password)

# ✅ Redact or exclude sensitive fields
logger.info("User login", user_id=user_id, 
            password="***REDACTED***")

6. Log Aggregation Integration

Structure logs for ELK Stack, Splunk, or CloudWatch:

{
  "timestamp": "2024-01-15T10:30:45.123Z",
  "level": "ERROR",
  "service": "payment-api",
  "trace_id": "abc123def456",
  "message": "Payment gateway timeout",
  "context": {
    "gateway": "stripe",
    "timeout_ms": 5000,
    "retry_count": 3
  }
}

7. Testing Logs

import logging
from io import StringIO

def test_logging():
    log_stream = StringIO()
    handler = logging.StreamHandler(log_stream)
    logger.logger.addHandler(handler)

    logger.info("Test message", key="value")
    log_output = log_stream.getvalue()

    assert "Test message" in log_output
    assert "key" in log_output

---

Key Takeaway: Structured logging with proper levels transforms raw log data into queryable, actionable intelligence. Combine JSON formatting with appropriate level selection to balance visibility, performance, and maintainability across development and production environments.