Content Role: pillar

Event-Driven Architecture: Decoupling Microservices

Event sourcing and CQRS patterns with Kafka and EventBridge

Distributed systems fail when services become tightly coupled through synchronous HTTP calls. A single slow dependency cascades into system-wide latency. Database locks spread across service boundaries. Deployment windows require coordinating multiple teams. These problems stem from treating microservices as a distributed monolith.

Event-driven architecture patterns solve this by inverting the dependency model. Services publish events about state changes without knowing who consumes them. Consumers process events asynchronously at their own pace. This fundamental shift enables true service autonomy, but introduces new challenges around consistency, ordering, and operational complexity.

Why Request-Response Patterns Break at Scale

Traditional REST-based microservices create hidden coupling. When Service A calls Service B synchronously, A inherits B's availability characteristics. If B experiences latency, A's response time degrades. If B goes down, A fails or needs complex circuit breaker logic.

The coupling extends beyond runtime. Schema changes in B require coordinated deployments with A. Testing A requires running B or maintaining elaborate mocks. Teams lose independence despite organizational boundaries.

Distributed transactions compound these issues. Two-phase commit protocols don't scale in cloud environments with network partitions. Saga patterns implemented over synchronous calls create complex state machines that are difficult to reason about and debug.

Core Event-Driven Architecture Patterns

Event-driven systems rely on three foundational patterns that work together to achieve loose coupling while maintaining data consistency.

Event Notification

The simplest pattern involves services emitting lightweight notifications when state changes occur. Other services listen for relevant events and react accordingly.

// Order service publishes events
interface OrderCreatedEvent {
  eventId: string;
  eventType: 'order.created';
  timestamp: Date;
  orderId: string;
  customerId: string;
  totalAmount: number;
}

class OrderService {
  async createOrder(orderData: CreateOrderDTO): Promise<Order> {
    const order = await this.repository.save(orderData);

    await this.eventBus.publish({
      eventId: crypto.randomUUID(),
      eventType: 'order.created',
      timestamp: new Date(),
      orderId: order.id,
      customerId: order.customerId,
      totalAmount: order.total
    });

    return order;
  }
}

// Inventory service reacts to events
class InventoryEventHandler {
  @EventHandler('order.created')
  async handleOrderCreated(event: OrderCreatedEvent): Promise<void> {
    const order = await this.orderClient.getOrder(event.orderId);
    await this.reserveInventory(order.items);
  }
}

This pattern works for simple notifications but requires consumers to fetch additional data, creating temporal coupling. If the order service is unavailable when inventory service processes the event, the operation fails.

Event-Carried State Transfer

Instead of minimal notifications, events carry complete state changes. Consumers have all necessary data without additional service calls.

interface OrderCreatedEvent {
  eventId: string;
  eventType: 'order.created';
  timestamp: Date;
  orderId: string;
  customerId: string;
  items: Array<{
    productId: string;
    quantity: number;
    price: number;
  }>;
  shippingAddress: Address;
  totalAmount: number;
}

class InventoryEventHandler {
  @EventHandler('order.created')
  async handleOrderCreated(event: OrderCreatedEvent): Promise<void> {
    // No external calls needed - all data in event
    await this.reserveInventory(event.items);
  }
}

This eliminates runtime coupling but increases event size and creates data duplication. Services maintain local caches of data from other domains, trading consistency for autonomy.

Event Sourcing

Rather than storing current state, event sourcing persists every state change as an immutable event. Current state is derived by replaying events.

// Event store implementation
interface Event {
  aggregateId: string;
  aggregateType: string;
  eventType: string;
  eventData: unknown;
  version: number;
  timestamp: Date;
}

class EventStore {
  async appendEvents(
    aggregateId: string,
    expectedVersion: number,
    events: Event[]
  ): Promise<void> {
    // Optimistic concurrency check
    const currentVersion = await this.getVersion(aggregateId);
    if (currentVersion !== expectedVersion) {
      throw new ConcurrencyError('Version mismatch');
    }

    await this.db.transaction(async (tx) => {
      for (const event of events) {
        await tx.insert('events', event);
      }
    });

    // Publish to event bus for consumers
    await this.eventBus.publishBatch(events);
  }

  async getEvents(aggregateId: string): Promise<Event[]> {
    return this.db
      .select('*')
      .from('events')
      .where({ aggregateId })
      .orderBy('version', 'asc');
  }
}

// Aggregate reconstructed from events
class Order {
  private id: string;
  private status: OrderStatus;
  private items: OrderItem[] = [];
  private version: number = 0;
  private uncommittedEvents: Event[] = [];

  static async load(eventStore: EventStore, orderId: string): Promise<Order> {
    const events = await eventStore.getEvents(orderId);
    const order = new Order(orderId);

    for (const event of events) {
      order.applyEvent(event, false);
    }

    return order;
  }

  createOrder(items: OrderItem[]): void {
    this.applyEvent({
      aggregateId: this.id,
      aggregateType: 'Order',
      eventType: 'OrderCreated',
      eventData: { items },
      version: this.version + 1,
      timestamp: new Date()
    }, true);
  }

  private applyEvent(event: Event, isNew: boolean): void {
    switch (event.eventType) {
      case 'OrderCreated':
        this.items = event.eventData.items;
        this.status = 'pending';
        break;
      case 'OrderConfirmed':
        this.status = 'confirmed';
        break;
    }

    this.version = event.version;
    if (isNew) {
      this.uncommittedEvents.push(event);
    }
  }

  async save(eventStore: EventStore): Promise<void> {
    await eventStore.appendEvents(
      this.id,
      this.version - this.uncommittedEvents.length,
      this.uncommittedEvents
    );
    this.uncommittedEvents = [];
  }
}

Event sourcing provides complete audit trails and enables temporal queries. You can reconstruct system state at any point in time. However, it requires careful schema evolution and increases storage requirements.

CQRS: Separating Reads from Writes

Command Query Responsibility Segregation pairs naturally with event sourcing. Write operations append events to the event store. Read operations query optimized projections built from those events.

// Write side - command handler
class CreateOrderHandler {
  async handle(command: CreateOrderCommand): Promise<void> {
    const order = new Order(command.orderId);
    order.createOrder(command.items);
    await this.eventStore.save(order);
  }
}

// Read side - projection builder
class OrderProjection {
  @EventHandler('OrderCreated')
  async handleOrderCreated(event: OrderCreatedEvent): Promise<void> {
    await this.db.insert('order_summary', {
      orderId: event.aggregateId,
      customerId: event.eventData.customerId,
      totalAmount: event.eventData.items.reduce(
        (sum, item) => sum + item.price * item.quantity, 
        0
      ),
      status: 'pending',
      createdAt: event.timestamp
    });
  }

  @EventHandler('OrderConfirmed')
  async handleOrderConfirmed(event: OrderConfirmedEvent): Promise<void> {
    await this.db
      .update('order_summary')
      .set({ status: 'confirmed' })
      .where({ orderId: event.aggregateId });
  }
}

// Read model optimized for queries
class OrderQueryService {
  async getCustomerOrders(customerId: string): Promise<OrderSummary[]> {
    return this.db
      .select('*')
      .from('order_summary')
      .where({ customerId })
      .orderBy('createdAt', 'desc');
  }
}

This separation allows independent scaling. Write operations remain consistent and transactional. Read models can be denormalized, cached aggressively, and scaled horizontally without affecting write performance.

Infrastructure: Kafka vs EventBridge

Apache Kafka excels at high-throughput event streaming with strong ordering guarantees within partitions. Use Kafka when you need:

• Guaranteed message ordering per partition key
• Event replay capabilities
• High throughput (millions of events per second)
• Stream processing with Kafka Streams or Flink

// Kafka producer with idempotency
const producer = kafka.producer({
  idempotent: true,
  maxInFlightRequests: 5,
  transactionalId: 'order-service-producer'
});

await producer.send({
  topic: 'orders',
  messages: [{
    key: order.customerId, // Ensures ordering per customer
    value: JSON.stringify(event),
    headers: {
      'event-type': event.eventType,
      'correlation-id': correlationId
    }
  }]
});

AWS EventBridge provides serverless event routing with schema registry and built-in integrations. Choose EventBridge for:

• AWS-native architectures
• Schema validation and discovery
• Cross-account event routing
• Lower operational overhead

// EventBridge with schema validation
const eventBridge = new EventBridgeClient({ region: 'us-east-1' });

await eventBridge.send(new PutEventsCommand({
  Entries: [{
    Source: 'order.service',
    DetailType: 'OrderCreated',
    Detail: JSON.stringify(event),
    EventBusName: 'application-events',
    Resources: [`order/${order.id}`]
  }]
}));

Common Pitfalls and Solutions

Eventual consistency confusion: Developers expect immediate consistency across services. Document consistency boundaries clearly. Use correlation IDs to trace event flows. Implement compensating transactions for business-critical workflows.

Event schema evolution: Breaking changes to event schemas break consumers. Use schema versioning with backward compatibility. Include schema version in event metadata. Maintain multiple event versions during transitions.

interface BaseEvent {
  schemaVersion: string;
  eventId: string;
  eventType: string;
}

interface OrderCreatedEventV1 extends BaseEvent {
  schemaVersion: '1.0';
  orderId: string;
  amount: number;
}

interface OrderCreatedEventV2 extends BaseEvent {
  schemaVersion: '2.0';
  orderId: string;
  items: OrderItem[]; // Breaking change
  totalAmount: number;
}

class EventHandler {
  async handle(event: BaseEvent): Promise<void> {
    switch (event.schemaVersion) {
      case '1.0':
        return this.handleV1(event as OrderCreatedEventV1);
      case '2.0':
        return this.handleV2(event as OrderCreatedEventV2);
    }
  }
}

Duplicate event processing: Network retries and at-least-once delivery guarantees cause duplicate events. Implement idempotent handlers using event IDs. Store processed event IDs in a deduplication table with TTL.

Event ordering across aggregates: Events from different aggregates have no guaranteed order. Design aggregates to be self-contained. Avoid business logic that depends on cross-aggregate ordering.

Best Practices Checklist

• Define clear aggregate boundaries - Each aggregate is a consistency boundary with its own event stream
• Use correlation IDs - Track related events across service boundaries for debugging and monitoring
• Implement dead letter queues - Capture failed events for manual intervention and replay
• Version all events - Include schema version in every event for backward compatibility
• Monitor event lag - Track consumer lag to detect processing bottlenecks
• Design for idempotency - All event handlers must safely handle duplicate events
• Document event contracts - Maintain a schema registry with event documentation
• Test with chaos engineering - Verify system behavior under message delays and failures
• Implement circuit breakers - Prevent cascading failures in event processing pipelines
• Use structured logging - Include event metadata in logs for correlation

Frequently Asked Questions

When should I use event-driven architecture instead of REST APIs?

Use event-driven patterns when services need to react to state changes without tight coupling, when you need audit trails, or when scaling read and write operations independently. Stick with REST for simple request-response interactions where immediate consistency is required.

How do I handle transactions across multiple services?

Implement the Saga pattern using event choreography or orchestration. Each service performs its local transaction and publishes events. Compensating transactions roll back changes if any step fails. Avoid distributed transactions in favor of eventual consistency.

What's the difference between event sourcing and event-driven architecture?

Event-driven architecture uses events for communication between services. Event sourcing stores events as the source of truth for application state. You can use event-driven architecture without event sourcing, but event sourcing naturally fits event-driven systems.

How do I query data across multiple event-sourced aggregates?

Build read models (projections) that denormalize data from multiple event streams. These projections are eventually consistent but optimized for queries. Use CQRS to separate write operations on aggregates from read operations on projections.

How do I handle event schema changes without breaking consumers?

Use schema versioning with backward-compatible changes when possible. For breaking changes, publish both old and new event versions during a transition period. Implement version-aware consumers that handle multiple schema versions.

What's the best way to test event-driven systems?

Use contract testing to verify event schemas between producers and consumers. Implement integration tests with test containers running Kafka or LocalStack. Test idempotency by processing events multiple times. Use chaos engineering to verify resilience.

How do I debug issues in event-driven systems?

Implement distributed tracing with correlation IDs across all events. Use structured logging with event metadata. Build monitoring dashboards showing event flow and consumer lag. Maintain event replay capabilities for reproducing issues in test environments.