Microservices Observability: Mastering Logs, Metrics, and Traces

The distributed nature of microservices architecture has fundamentally changed how we build and deploy applications. While this approach offers unprecedented scalability and flexibility, it introduces a critical challenge: understanding what's happening across dozens or hundreds of interconnected services. Traditional monitoring approaches that worked perfectly for monolithic applications crumble under the complexity of distributed systems, leaving developers blind when issues arise.

The 2026 Observability Crisis

By 2026, the average enterprise application comprises 47 microservices, according to recent industry surveys. Each service generates logs, emits metrics, and participates in distributed transactions. When a user reports a slow checkout process, you're no longer debugging a single application—you're investigating a chain of events spanning authentication services, inventory management, payment processing, and notification systems.

The problem intensifies when you consider that a single user request might trigger 15-20 service calls, each with its own latency, error rate, and resource consumption. Without proper observability, identifying the root cause becomes an exercise in frustration, often taking hours or days instead of minutes.

Traditional monitoring tools fail here because they were designed for a different era. They treat each service as an isolated entity, making it nearly impossible to correlate events across service boundaries or understand the complete journey of a request through your system.

Why Traditional Monitoring Falls Short

Legacy monitoring solutions typically focus on infrastructure metrics—CPU usage, memory consumption, disk I/O. While these remain important, they don't tell you why your checkout service is slow or which upstream dependency is causing cascading failures.

Traditional logging presents another challenge. When each service writes logs independently, you end up with fragmented information scattered across multiple files or systems. Searching through these logs to reconstruct a user's journey requires manual correlation, often involving grep commands across dozens of log files and educated guesses about timing.

The fundamental issue is the lack of context. Without understanding how services interact and how requests flow through your system, you're essentially trying to solve a puzzle with missing pieces. You might see that Service A is experiencing high latency, but without traces, you can't determine if it's waiting on Service B, struggling with database queries, or dealing with resource constraints.

The Three Pillars of Modern Observability

Modern observability rests on three interconnected pillars: logs, metrics, and traces. Each provides a different lens for understanding system behavior, and together they offer comprehensive visibility into distributed systems.

Logs provide detailed, event-level information about what happened in your application. They're invaluable for debugging specific issues and understanding the context around errors.

Metrics offer aggregated, numerical data about system behavior over time. They're perfect for identifying trends, setting alerts, and understanding overall system health.

Traces show the path of requests through your distributed system, revealing how services interact and where time is spent. They're essential for understanding performance bottlenecks and dependencies.

The key insight is that these pillars work together. A metric might alert you to increased latency, traces help you identify which service is slow, and logs provide the detailed context about what went wrong.

Implementing Observability with TypeScript

Let's build a production-ready observability solution using TypeScript, OpenTelemetry, and modern best practices. OpenTelemetry has emerged as the industry standard for observability instrumentation, providing vendor-neutral APIs for generating telemetry data.

Setting Up OpenTelemetry

First, install the necessary dependencies:

// package.json dependencies
{
  "@opentelemetry/api": "^1.7.0",
  "@opentelemetry/sdk-node": "^0.45.0",
  "@opentelemetry/auto-instrumentations-node": "^0.39.0",
  "@opentelemetry/exporter-trace-otlp-http": "^0.45.0",
  "@opentelemetry/exporter-metrics-otlp-http": "^0.45.0",
  "pino": "^8.16.0",
  "pino-opentelemetry-transport": "^0.2.0"
}

Unified Observability Configuration

Create a centralized observability setup that initializes all three pillars:

// observability/config.ts
import { NodeSDK } from '@opentelemetry/sdk-node';
import { getNodeAutoInstrumentations } from '@opentelemetry/auto-instrumentations-node';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';
import { OTLPMetricExporter } from '@opentelemetry/exporter-metrics-otlp-http';
import { PeriodicExportingMetricReader } from '@opentelemetry/sdk-metrics';
import { Resource } from '@opentelemetry/resources';
import { SemanticResourceAttributes } from '@opentelemetry/semantic-conventions';

export function initializeObservability(serviceName: string) {
  const resource = new Resource({
    [SemanticResourceAttributes.SERVICE_NAME]: serviceName,
    [SemanticResourceAttributes.SERVICE_VERSION]: process.env.SERVICE_VERSION || '1.0.0',
    [SemanticResourceAttributes.DEPLOYMENT_ENVIRONMENT]: process.env.NODE_ENV || 'development',
  });

  const sdk = new NodeSDK({
    resource,
    traceExporter: new OTLPTraceExporter({
      url: process.env.OTEL_EXPORTER_OTLP_ENDPOINT || 'http://localhost:4318/v1/traces',
    }),
    metricReader: new PeriodicExportingMetricReader({
      exporter: new OTLPMetricExporter({
        url: process.env.OTEL_EXPORTER_OTLP_ENDPOINT || 'http://localhost:4318/v1/metrics',
      }),
      exportIntervalMillis: 60000,
    }),
    instrumentations: [
      getNodeAutoInstrumentations({
        '@opentelemetry/instrumentation-fs': { enabled: false },
      }),
    ],
  });

  sdk.start();

  process.on('SIGTERM', () => {
    sdk.shutdown()
      .then(() => console.log('Observability SDK shut down successfully'))
      .catch((error) => console.error('Error shutting down SDK', error))
      .finally(() => process.exit(0));
  });

  return sdk;
}

Structured Logging with Correlation

Implement structured logging that automatically includes trace context:

// observability/logger.ts
import pino from 'pino';
import { trace, context } from '@opentelemetry/api';

export const logger = pino({
  level: process.env.LOG_LEVEL || 'info',
  formatters: {
    log(object) {
      const span = trace.getSpan(context.active());
      if (span) {
        const spanContext = span.spanContext();
        return {
          ...object,
          trace_id: spanContext.traceId,
          span_id: spanContext.spanId,
          trace_flags: spanContext.traceFlags,
        };
      }
      return object;
    },
  },
});

Custom Metrics and Spans

Create reusable utilities for adding custom observability:

// observability/instrumentation.ts
import { trace, metrics, SpanStatusCode } from '@opentelemetry/api';

const tracer = trace.getTracer('app-tracer');
const meter = metrics.getMeter('app-meter');

// Custom metrics
export const requestCounter = meter.createCounter('http_requests_total', {
  description: 'Total number of HTTP requests',
});

export const requestDuration = meter.createHistogram('http_request_duration_ms', {
  description: 'HTTP request duration in milliseconds',
});

// Decorator for automatic tracing
export function Traced(spanName?: string) {
  return function (
    target: any,
    propertyKey: string,
    descriptor: PropertyDescriptor
  ) {
    const originalMethod = descriptor.value;

    descriptor.value = async function (...args: any[]) {
      const name = spanName || `${target.constructor.name}.${propertyKey}`;
      return tracer.startActiveSpan(name, async (span) => {
        try {
          const result = await originalMethod.apply(this, args);
          span.setStatus({ code: SpanStatusCode.OK });
          return result;
        } catch (error) {
          span.setStatus({
            code: SpanStatusCode.ERROR,
            message: error instanceof Error ? error.message : 'Unknown error',
          });
          span.recordException(error as Error);
          throw error;
        } finally {
          span.end();
        }
      });
    };

    return descriptor;
  };
}

Practical Service Implementation

Here's how to use these observability tools in a real service:

// services/order.service.ts
import { logger } from '../observability/logger';
import { Traced, requestCounter, requestDuration } from '../observability/instrumentation';
import { trace } from '@opentelemetry/api';

export class OrderService {
  @Traced('OrderService.createOrder')
  async createOrder(userId: string, items: OrderItem[]): Promise<Order> {
    const startTime = Date.now();
    const span = trace.getActiveSpan();

    try {
      logger.info({ userId, itemCount: items.length }, 'Creating order');

      span?.setAttributes({
        'user.id': userId,
        'order.item_count': items.length,
      });

      // Validate inventory
      await this.validateInventory(items);

      // Process payment
      const paymentResult = await this.processPayment(userId, items);

      // Create order record
      const order = await this.saveOrder(userId, items, paymentResult);

      requestCounter.add(1, { operation: 'create_order', status: 'success' });
      requestDuration.record(Date.now() - startTime, { operation: 'create_order' });

      logger.info({ orderId: order.id }, 'Order created successfully');
      return order;
    } catch (error) {
      requestCounter.add(1, { operation: 'create_order', status: 'error' });
      logger.error({ error, userId }, 'Failed to create order');
      throw error;
    }
  }

  @Traced('OrderService.validateInventory')
  private async validateInventory(items: OrderItem[]): Promise<void> {
    // Implementation with automatic tracing
  }
}

Common Pitfalls and How to Avoid Them

Over-instrumentation: Adding traces and logs everywhere creates noise and performance overhead. Focus on service boundaries, external calls, and critical business operations.

Ignoring cardinality: High-cardinality labels in metrics (like user IDs or request IDs) can overwhelm your monitoring system. Use dimensions wisely and aggregate appropriately.

Missing context propagation: Ensure trace context is propagated across all service boundaries, including message queues and async operations. Without this, your distributed traces will be fragmented.

Inconsistent naming: Establish naming conventions for spans, metrics, and log fields across all services. This makes querying and correlation significantly easier.

Neglecting sampling: In high-traffic systems, tracing every request is impractical. Implement intelligent sampling strategies that capture errors and slow requests while sampling normal traffic.

Best Practices for Production

Always include correlation IDs in logs and propagate them through your entire request chain. This single practice dramatically simplifies debugging.

Set up alerts based on metrics, but use traces and logs for investigation. Metrics tell you that something is wrong; traces and logs tell you what and why.

Implement structured logging from day one. JSON-formatted logs with consistent field names enable powerful querying and analysis.

Use semantic conventions from OpenTelemetry for naming spans, attributes, and metrics. This ensures consistency and enables better tooling support.

Monitor your observability pipeline itself. If your telemetry collection fails, you're flying blind.

Frequently Asked Questions

Q: How much performance overhead does observability add? A: With proper sampling and configuration, overhead is typically 1-5%. Auto-instrumentation adds minimal latency, and the benefits far outweigh the costs.

Q: Should I use a commercial observability platform or open-source tools? A: Start with open-source tools like Jaeger, Prometheus, and Grafana for learning. Commercial platforms offer better integration and support but at significant cost. OpenTelemetry ensures you're not locked in.

Q: How long should I retain telemetry data? A: Traces: 7-30 days; Metrics: 13-18 months with downsampling; Logs: 30-90 days for detailed logs, longer for aggregated data. Adjust based on compliance requirements and storage costs.

Q: What's the difference between monitoring and observability? A: Monitoring tells you when something is wrong based on predefined metrics. Observability lets you ask arbitrary questions about system behavior, even for scenarios you didn't anticipate.

Q: How do I handle observability in local development? A: Use lightweight local stacks like Jaeger all-in-one or Grafana Cloud's free tier. Ensure your code works with observability disabled for developers who prefer simpler setups.

Q: Should every microservice have its own observability configuration? A: No. Create a shared observability library that all services import. This ensures consistency and makes updates easier.

Q: How do I correlate logs from different services? A: Use trace IDs in your logs. When a request enters your system, generate or extract a trace ID and include it in all logs related to that request across all services.

Conclusion

Observability isn't optional for microservices—it's fundamental to operating distributed systems successfully. The three pillars of logs, metrics, and traces provide complementary views into system behavior, enabling you to understand, debug, and optimize your applications effectively.

By implementing observability with TypeScript and OpenTelemetry, you're building on industry standards that ensure flexibility and avoid vendor lock-in. The investment in proper instrumentation pays dividends every time you need to debug a production issue or optimize system performance.

Start small: instrument your most critical services first, establish patterns and conventions, then expand systematically. The goal isn't perfect observability from day one—it's building a foundation that grows with your system and provides the insights you need when problems arise.

Remember that observability is a journey, not a destination. As your system evolves, your observability practices should evolve with it, continuously providing the visibility you need to build reliable, performant distributed systems.

---

Metadata

```json { "seo_title": "Microservices Observability: Logs, Metrics & Traces Guide", "meta_description": "Master microservices observability with TypeScript and OpenTelemetry. Learn how logs, metrics, and traces work together to debug distributed systems effectively.", "primary_keyword": "microservices observability", "secondary_keywords": [ "distributed tracing", "OpenTelemetry TypeScript", "microservices monitoring", "structured logging", "observability best practices", "logs metrics traces", "distributed systems debugging", "telemetry instrumentation" ], "tags": [ "microservices", "observability", "TypeScript", "OpenTelemetry", "distributed-systems", "monitoring", "DevOps" ] }