Integration Patterns and Strategies: Building Robust System Integrations

By X402 Team | Last Updated: February 2026

Direct Answer

Integration patterns are proven architectural approaches for connecting different systems. The main patterns include: Request-Response (synchronous API calls, best for real-time needs), Event-Driven (asynchronous events via message queues, best for decoupling), ETL (Extract-Transform-Load for batch data sync, best for analytics), and Webhooks (push notifications, best for real-time updates). Choose based on: latency requirements (real-time vs batch), data volume, system coupling needs, and failure handling requirements. Always implement idempotency, retry logic, and comprehensive error handling.

Table of Contents

1. Common Integration Patterns
2. Choosing the Right Pattern
3. Data Synchronization Strategies
4. Handling Failures and Rollbacks
5. Testing Integrations
6. Integration Best Practices

Common Integration Patterns

1. Request-Response Pattern (Synchronous)

Description: Client sends request, waits for immediate response.

When to Use:

• Real-time operations (user registration, payment processing)
• Simple CRUD operations
• Operations requiring immediate confirmation
• Low data volume

Example: User Registration Flow

// API Gateway receives registration request
app.post('/api/register', async (req, res) => {
  try {
    // 1. Validate input
    const { email, password, name } = req.body;

// 2. Call authentication service (synchronous)
    const authResponse = await authService.createUser({
      email,
      password
    });

// 3. Call profile service (synchronous)
    const profileResponse = await profileService.createProfile({
      userId: authResponse.userId,
      name,
      email
    });

// 4. Call email service (synchronous)
    await emailService.sendWelcomeEmail(email, name);

// 5. Return success immediately
    res.status(201).json({
      userId: authResponse.userId,
      message: 'Registration successful'
    });

} catch (error) {
    // Handle errors synchronously
    res.status(400).json({ error: error.message });
  }
});

Pros:

• Simple to implement and understand
• Immediate feedback
• Easy error handling
• Natural transaction boundaries

Cons:

• Tight coupling between services
• Slower response times (waits for all operations)
• Cascade failures (if one service down, request fails)
• Not suitable for long-running operations

2. Event-Driven Pattern (Asynchronous)

Description: Services communicate via events published to message queues or event buses.

When to Use:

• Decoupling services
• Multiple consumers for same event
• Long-running background tasks
• High throughput requirements

Example: Order Processing with Events

// Order service publishes event
const EventEmitter = require('events');
const RabbitMQ = require('amqplib');

class OrderService {
  constructor(eventBus) {
    this.eventBus = eventBus;
  }

async createOrder(orderData) {
    // 1. Create order in database
    const order = await db.orders.create({
      userId: orderData.userId,
      items: orderData.items,
      total: orderData.total,
      status: 'pending'
    });

// 2. Publish event (fire and forget)
    await this.eventBus.publish('order.created', {
      orderId: order.id,
      userId: order.userId,
      total: order.total,
      items: order.items,
      timestamp: new Date()
    });

// 3. Return immediately (don't wait for consumers)
    return order;
  }
}

// Inventory service consumes event
class InventoryConsumer {
  async handleOrderCreated(event) {
    try {
      console.log('Processing order for inventory:', event.orderId);

// Reserve inventory
      for (const item of event.items) {
        await this.reserveInventory(item.productId, item.quantity);
      }

// Publish inventory reserved event
      await eventBus.publish('inventory.reserved', {
        orderId: event.orderId,
        items: event.items
      });

} catch (error) {
      console.error('Inventory reservation failed:', error);

// Publish failure event
      await eventBus.publish('inventory.reservation.failed', {
        orderId: event.orderId,
        error: error.message
      });
    }
  }
}

// Email service consumes event
class EmailConsumer {
  async handleOrderCreated(event) {
    try {
      const user = await db.users.findById(event.userId);
      await this.sendOrderConfirmationEmail(user.email, event.orderId);

await eventBus.publish('email.sent', {
        orderId: event.orderId,
        type: 'order_confirmation'
      });

} catch (error) {
      console.error('Email sending failed:', error);
      // Will be retried automatically by message queue
    }
  }
}

// Payment service consumes event
class PaymentConsumer {
  async handleInventoryReserved(event) {
    try {
      const order = await db.orders.findById(event.orderId);

// Process payment
      const payment = await stripeService.createCharge({
        amount: order.total,
        userId: order.userId
      });

if (payment.status === 'succeeded') {
        await eventBus.publish('payment.succeeded', {
          orderId: event.orderId,
          paymentId: payment.id
        });

// Update order status
        await db.orders.update(event.orderId, { status: 'paid' });
      }

} catch (error) {
      await eventBus.publish('payment.failed', {
        orderId: event.orderId,
        error: error.message
      });
    }
  }
}

Event Bus Implementation (RabbitMQ)

class EventBus {
  constructor(rabbitmqUrl) {
    this.url = rabbitmqUrl;
    this.connection = null;
    this.channel = null;
  }

async connect() {
    this.connection = await RabbitMQ.connect(this.url);
    this.channel = await this.connection.createChannel();

// Create exchange for events
    await this.channel.assertExchange('events', 'topic', { durable: true });
  }

async publish(eventType, data) {
    const message = JSON.stringify({
      type: eventType,
      data,
      timestamp: new Date().toISOString(),
      id: generateId()
    });

this.channel.publish(
      'events',
      eventType,
      Buffer.from(message),
      { persistent: true }
    );

console.log(Published event: ${eventType});
  }

async subscribe(eventPattern, handler) {
    // Create queue for this consumer
    const queue = await this.channel.assertQueue('', {
      exclusive: true,
      durable: false
    });

// Bind queue to event pattern
    await this.channel.bindQueue(queue.queue, 'events', eventPattern);

// Consume messages
    this.channel.consume(queue.queue, async (msg) => {
      try {
        const event = JSON.parse(msg.content.toString());
        await handler(event.data);

// Acknowledge message
        this.channel.ack(msg);

} catch (error) {
        console.error('Event handler error:', error);

// Negative acknowledge (will be requeued)
        this.channel.nack(msg, false, true);
      }
    });
  }
}

// Usage
const eventBus = new EventBus('amqp://localhost');
await eventBus.connect();

// Subscribe to events
await eventBus.subscribe('order.created', (data) => {
  inventoryConsumer.handleOrderCreated(data);
});

await eventBus.subscribe('inventory.reserved', (data) => {
  paymentConsumer.handleInventoryReserved(data);
});

Pros:

• Loose coupling (services don't know about each other)
• High scalability (add consumers easily)
• Fault tolerance (messages persisted in queue)
• Multiple consumers for same event
• Built-in retry mechanisms

Cons:

• More complex architecture
• Eventual consistency (not immediate)
• Harder to debug (distributed tracing needed)
• Message ordering challenges

3. ETL Pattern (Extract-Transform-Load)

Description: Batch processing for data synchronization between systems.

When to Use:

• Data warehousing and analytics
• Large data volumes
• Non-time-sensitive synchronization
• System migrations
• Reporting pipelines

Example: Syncing Sales Data to Data Warehouse

const cron = require('node-cron');

class SalesDataETL {
  constructor(sourceDB, targetDB) {
    this.sourceDB = sourceDB;  // Production database
    this.targetDB = targetDB;  // Data warehouse
  }

// Schedule ETL job (runs daily at 2 AM)
  schedule() {
    cron.schedule('0 2   ', async () => {
      console.log('Starting ETL job:', new Date());
      await this.runETL();
    });
  }


async runETL() {
    const batchSize = 1000;
    let offset = 0;
    let totalProcessed = 0;

try {
      // Get last successful sync timestamp
      const lastSync = await this.getLastSyncTimestamp();
      console.log(Syncing data since: ${lastSync});

while (true) {
        // EXTRACT: Fetch batch from source
        const orders = await this.extract(lastSync, offset, batchSize);

if (orders.length === 0) break;

// TRANSFORM: Clean and enrich data
        const transformed = await this.transform(orders);

// LOAD: Insert into data warehouse
        await this.load(transformed);

totalProcessed += orders.length;
        offset += batchSize;

console.log(Processed ${totalProcessed} orders);
      }

// Update sync timestamp
      await this.updateSyncTimestamp(new Date());

console.log(ETL complete. Total records: ${totalProcessed});

} catch (error) {
      console.error('ETL job failed:', error);
      // Alert team, retry later
      await this.alertFailure(error);
    }
  }

// EXTRACT: Get data from source database
  async extract(since, offset, limit) {
    return await this.sourceDB.query(`
      SELECT
        o.id,
        o.user_id,
        o.total,
        o.status,
        o.created_at,
        u.email,
        u.name
      FROM orders o
      JOIN users u ON o.user_id = u.id
      WHERE o.created_at > $1
      ORDER BY o.created_at
      OFFSET $2 LIMIT $3
    `, [since, offset, limit]);
  }

// TRANSFORM: Clean, enrich, and format data
  async transform(orders) {
    return orders.map(order => ({
      // Standardize field names
      order_id: order.id,
      customer_id: order.user_id,
      customer_email: order.email,
      customer_name: order.name,
      order_total: parseFloat(order.total),
      order_status: order.status.toUpperCase(),

// Add calculated fields
      order_year: new Date(order.created_at).getFullYear(),
      order_month: new Date(order.created_at).getMonth() + 1,
      order_quarter: Math.ceil((new Date(order.created_at).getMonth() + 1) / 3),

// Add metadata
      etl_loaded_at: new Date(),
      etl_source: 'production_db',
      etl_version: '1.0'
    }));
  }

// LOAD: Insert into data warehouse
  async load(records) {
    // Bulk insert for performance
    const query = `
      INSERT INTO warehouse.orders (
        order_id, customer_id, customer_email, customer_name,
        order_total, order_status, order_year, order_month,
        order_quarter, etl_loaded_at, etl_source, etl_version
      ) VALUES ${records.map((_, i) =>
        `($${i  12 + 1}, $${i  12 + 2}, $${i  12 + 3}, $${i  12 + 4},
          $${i  12 + 5}, $${i  12 + 6}, $${i  12 + 7}, $${i  12 + 8},
          $${i  12 + 9}, $${i  12 + 10}, $${i  12 + 11}, $${i  12 + 12})`
      ).join(', ')}
      ON CONFLICT (order_id)
      DO UPDATE SET
        order_status = EXCLUDED.order_status,
        etl_loaded_at = EXCLUDED.etl_loaded_at
    `;


const values = records.flatMap(r => [
      r.order_id, r.customer_id, r.customer_email, r.customer_name,
      r.order_total, r.order_status, r.order_year, r.order_month,
      r.order_quarter, r.etl_loaded_at, r.etl_source, r.etl_version
    ]);

await this.targetDB.query(query, values);
  }

async getLastSyncTimestamp() {
    const result = await this.targetDB.query(
      'SELECT MAX(etl_loaded_at) as last_sync FROM warehouse.orders'
    );
    return result.rows[0].last_sync || new Date('2020-01-01');
  }

async updateSyncTimestamp(timestamp) {
    await this.targetDB.query(
      'INSERT INTO warehouse.etl_runs (job_name, completed_at) VALUES ($1, $2)',
      ['sales_data_sync', timestamp]
    );
  }
}

// Run ETL
const etl = new SalesDataETL(productionDB, warehouseDB);
etl.schedule();

import pandas as pd
from datetime import datetime, timedelta
import schedule
import time

class SalesDataETL:
    def __init__(self, source_conn, target_conn):
        self.source = source_conn
        self.target = target_conn

def run_etl(self):
        """Run complete ETL pipeline"""
        try:
            print(f"Starting ETL: {datetime.now()}")

# Extract
            df = self.extract()
            print(f"Extracted {len(df)} records")

# Transform
            df = self.transform(df)
            print(f"Transformed {len(df)} records")

# Load
            self.load(df)
            print(f"Loaded {len(df)} records")

print("ETL complete")

except Exception as e:
            print(f"ETL failed: {str(e)}")
            self.alert_failure(e)

def extract(self) -> pd.DataFrame:
        """Extract data from source database"""
        last_sync = self.get_last_sync_timestamp()

query = """
            SELECT
                o.id,
                o.user_id,
                o.total,
                o.status,
                o.created_at,
                u.email,
                u.name
            FROM orders o
            JOIN users u ON o.user_id = u.id
            WHERE o.created_at > %s
        """

return pd.read_sql(query, self.source, params=[last_sync])

def transform(self, df: pd.DataFrame) -> pd.DataFrame:
        """Transform and enrich data"""
        # Rename columns
        df = df.rename(columns={
            'id': 'order_id',
            'user_id': 'customer_id',
            'email': 'customer_email',
            'name': 'customer_name',
            'total': 'order_total'
        })

# Add calculated fields
        df['created_at'] = pd.to_datetime(df['created_at'])
        df['order_year'] = df['created_at'].dt.year
        df['order_month'] = df['created_at'].dt.month
        df['order_quarter'] = df['created_at'].dt.quarter

# Add metadata
        df['etl_loaded_at'] = datetime.now()
        df['etl_source'] = 'production_db'
        df['etl_version'] = '1.0'

return df

def load(self, df: pd.DataFrame):
        """Load data into warehouse"""
        df.to_sql(
            'orders',
            self.target,
            schema='warehouse',
            if_exists='append',
            index=False,
            method='multi',  # Bulk insert
            chunksize=1000
        )

def schedule_job(self):
        """Schedule daily ETL at 2 AM"""
        schedule.every().day.at("02:00").do(self.run_etl)

while True:
            schedule.run_pending()
            time.sleep(60)

// Example: User Registration

// PATTERN 1: Request-Response (Synchronous)
app.post('/register', async (req, res) => {
  const user = await createUser(req.body);        // Wait
  await sendWelcomeEmail(user);                   // Wait
  await createDefaultSettings(user);              // Wait
  res.json({ success: true, userId: user.id });   // Then respond (slow)
});
// Pros: Simple, immediate confirmation
// Cons: Slow (waits for all operations), tight coupling

// PATTERN 2: Event-Driven (Asynchronous)
app.post('/register', async (req, res) => {
  const user = await createUser(req.body);
  await eventBus.publish('user.registered', { userId: user.id });
  res.json({ success: true, userId: user.id });   // Fast response
});
// Email and settings services consume event separately
// Pros: Fast response, decoupled, scalable
// Cons: Eventually consistent, complex

// PATTERN 3: Hybrid (Best of both)
app.post('/register', async (req, res) => {
  const user = await createUser(req.body);        // Critical: wait
  res.json({ success: true, userId: user.id });   // Respond fast

// Non-critical: background
  backgroundQueue.add('send-welcome-email', { userId: user.id });
  backgroundQueue.add('create-default-settings', { userId: user.id });
});
// Pros: Fast, simple, reliable for critical operations
// Cons: Some coupling remains

async function fullSync() {
  // 1. Extract all data from source
  const allRecords = await source.getAllRecords();

// 2. Clear target (or use staging table)
  await target.truncate();

// 3. Load all data to target
  await target.bulkInsert(allRecords);
}

async function incrementalSync() {
  // 1. Get last sync timestamp
  const lastSync = await getLastSyncTimestamp();

// 2. Extract only new/modified records
  const newRecords = await source.getRecordsSince(lastSync);

// 3. Upsert to target
  for (const record of newRecords) {
    await target.upsert(record);
  }

// 4. Update sync timestamp
  await setLastSyncTimestamp(new Date());
}

// Using Debezium or database triggers

// PostgreSQL trigger example
CREATE OR REPLACE FUNCTION notify_change()
RETURNS trigger AS $$
BEGIN
  PERFORM pg_notify(
    'data_changes',
    json_build_object(
      'operation', TG_OP,
      'table', TG_TABLE_NAME,
      'data', row_to_json(NEW)
    )::text
  );
  RETURN NEW;
END;
$$ LANGUAGE plpgsql;

CREATE TRIGGER users_change_trigger
AFTER INSERT OR UPDATE OR DELETE ON users
FOR EACH ROW EXECUTE FUNCTION notify_change();

// Listen for changes
const { Client } = require('pg');

const client = new Client({ / connection config / });
await client.connect();


client.query('LISTEN data_changes');

client.on('notification', async (msg) => {
  const change = JSON.parse(msg.payload);

if (change.table === 'users') {
    if (change.operation === 'INSERT' || change.operation === 'UPDATE') {
      await targetDB.upsert('users', change.data);
    } else if (change.operation === 'DELETE') {
      await targetDB.delete('users', change.data.id);
    }
  }
});

// ❌ NOT Idempotent
async function processPayment(orderId, amount) {
  await payments.create({ orderId, amount });
  // Calling twice creates two charges!
}

// ✅ Idempotent
async function processPayment(orderId, amount) {
  // Check if already processed
  const existing = await payments.findByOrderId(orderId);
  if (existing) {
    return existing;  // Return existing charge
  }

return await payments.create({
    orderId,
    amount,
    idempotencyKey: orderId  // Provider handles duplicates
  });
}

async function retryWithBackoff(fn, maxRetries = 5) {
  for (let attempt = 1; attempt <= maxRetries; attempt++) {
    try {
      return await fn();
    } catch (error) {
      if (attempt === maxRetries) throw error;

// Don't retry client errors (4xx)
      if (error.statusCode >= 400 && error.statusCode < 500) {
        throw error;
      }

// Exponential backoff: 2^attempt seconds + jitter
      const delay = Math.min(1000  Math.pow(2, attempt) + Math.random() * 1000, 60000);
      console.log(Retry ${attempt}/${maxRetries} in ${delay}ms);
      await sleep(delay);
    }
  }
}

// Usage
await retryWithBackoff(() => externalAPI.createUser(userData));

3. Circuit Breaker Pattern

class CircuitBreaker {
  constructor(threshold = 5, timeout = 60000) {
    this.failureCount = 0;
    this.threshold = threshold;
    this.timeout = timeout;
    this.state = 'CLOSED';  // CLOSED, OPEN, HALF_OPEN
    this.nextAttempt = Date.now();
  }

async execute(fn) {
    if (this.state === 'OPEN') {
      if (Date.now() < this.nextAttempt) {
        throw new Error('Circuit breaker is OPEN');
      }
      this.state = 'HALF_OPEN';
    }

try {
      const result = await fn();

// Success - reset
      if (this.state === 'HALF_OPEN') {
        this.state = 'CLOSED';
        this.failureCount = 0;
      }

return result;

} catch (error) {
      this.failureCount++;

if (this.failureCount >= this.threshold) {
        this.state = 'OPEN';
        this.nextAttempt = Date.now() + this.timeout;
        console.log(Circuit breaker OPEN for ${this.timeout}ms);
      }

throw error;
    }
  }
}

// Usage
const breaker = new CircuitBreaker(5, 60000);
await breaker.execute(() => externalService.call());

4. Saga Pattern (Distributed Transactions)

// Compensating transactions for rollback

class OrderSaga {
  async execute(orderData) {
    const compensations = [];

try {
      // Step 1: Reserve inventory
      const inventory = await inventoryService.reserve(orderData.items);
      compensations.push(() => inventoryService.release(inventory.id));

// Step 2: Process payment
      const payment = await paymentService.charge(orderData.total);
      compensations.push(() => paymentService.refund(payment.id));

// Step 3: Create shipment
      const shipment = await shippingService.create(orderData);
      compensations.push(() => shippingService.cancel(shipment.id));

// Step 4: Confirm order
      const order = await orderService.confirm(orderData);

return order;

} catch (error) {
      console.error('Saga failed, rolling back:', error);

// Execute compensations in reverse order
      for (const compensate of compensations.reverse()) {
        try {
          await compensate();
        } catch (compensationError) {
          console.error('Compensation failed:', compensationError);
          // Log for manual intervention
        }
      }

throw error;
    }
  }
}

Testing Integrations

1. Integration Tests with Test Containers

const { GenericContainer } = require('testcontainers');

describe('Integration Tests', () => {
  let rabbitmqContainer;
  let eventBus;

beforeAll(async () => {
    // Start RabbitMQ container
    rabbitmqContainer = await new GenericContainer('rabbitmq:3-management')
      .withExposedPorts(5672)
      .start();

const port = rabbitmqContainer.getMappedPort(5672);
    eventBus = new EventBus(amqp://localhost:${port});
    await eventBus.connect();
  });

afterAll(async () => {
    await rabbitmqContainer.stop();
  });

test('order creation triggers inventory reservation', async () => {
    const orderCreated = new Promise((resolve) => {
      eventBus.subscribe('inventory.reserved', (data) => {
        resolve(data);
      });
    });

// Create order
    await orderService.create({
      userId: 'user_123',
      items: [{ productId: 'prod_1', quantity: 2 }],
      total: 100
    });

// Wait for event
    const event = await orderCreated;
    expect(event.orderId).toBeDefined();
  });
});

2. Contract Testing

// Producer contract (Order Service)
describe('Order Service Events', () => {
  test('order.created event schema', () => {
    const event = {
      orderId: 'order_123',
      userId: 'user_456',
      total: 100,
      items: [{ productId: 'prod_1', quantity: 2, price: 50 }],
      timestamp: new Date().toISOString()
    };

// Validate against schema
    expect(event).toMatchSchema(OrderCreatedSchema);
  });
});

// Consumer contract (Inventory Service)
describe('Inventory Service Consumers', () => {
  test('handles order.created event', async () => {
    const event = {
      orderId: 'order_123',
      items: [{ productId: 'prod_1', quantity: 2 }]
    };

await inventoryConsumer.handleOrderCreated(event);

// Verify inventory was reserved
    const inventory = await db.inventory.findByOrderId('order_123');
    expect(inventory.status).toBe('reserved');
  });
});

3. Mocking External Services

const nock = require('nock');

describe('External API Integration', () => {
  test('handles external service failure gracefully', async () => {
    // Mock external API failure
    nock('https://external-api.com')
      .post('/users')
      .reply(500, { error: 'Internal server error' });

// Should retry and eventually fail gracefully
    await expect(integration.syncUser(userData))
      .rejects.toThrow('External service unavailable');

// Verify retry attempts
    expect(nock.isDone()).toBe(true);
  });
});

Integration Best Practices

✅ Do This

1. Implement idempotency - Always check for duplicates
2. Use retry logic - Exponential backoff for transient failures
3. Log everything - Comprehensive logging for debugging
4. Monitor health - Track success rates, latency, errors
5. Version APIs - Backwards compatibility for consumers
6. Validate data - Check inputs and outputs
7. Handle timeouts - Set reasonable timeouts (30-60s)
8. Circuit breakers - Prevent cascade failures
9. Test failures - Simulate failures in tests
10. Document contracts - Clear event schemas and API specs

❌ Avoid This

1. No retries - Transient failures will cause permanent failures
2. Synchronous long operations - Blocks requests, times out
3. No idempotency - Duplicate processing on retries
4. Tight coupling - Services can't evolve independently
5. No monitoring - Can't detect issues early
6. Ignoring errors - Silent failures lead to data inconsistency
7. No versioning - Breaking changes break consumers
8. No timeouts - Hanging requests consume resources
9. No compensation - Can't rollback distributed transactions
10. No testing - Production failures inevitable

Summary Checklist

When building integrations, ensure you:

• [ ] Choose appropriate pattern - Request-response, event-driven, ETL, or hybrid
• [ ] Implement idempotency - Handle duplicate requests safely
• [ ] Add retry logic - Exponential backoff with max retries
• [ ] Handle failures gracefully - Circuit breakers, fallbacks, compensation
• [ ] Monitor integration health - Success rates, latency, error rates
• [ ] Version APIs/events - Backwards compatibility
• [ ] Comprehensive logging - Debug distributed systems
• [ ] Test failure scenarios - Network failures, timeouts, service unavailable
• [ ] Document contracts - Clear schemas and expectations
• [ ] Plan for rollbacks - Saga pattern or compensation logic

Tags: integration patterns, event-driven architecture, ETL, request-response, data synchronization, distributed systems, microservices, message queues, failure handling, saga pattern, circuit breaker, idempotency

Related Guides:

• API Development Best Practices
• Webhook Implementation Guide
• Building SDK Wrappers
• Developer Tools and Workflows

Start Building with X402

Get our free X402 Implementation Starter Kit with ready-to-use templates, code examples, and best practices.

What is included:

• Quick-start implementation templates
• API integration examples
• Configuration best practices guide

Get the Free Starter Kit