API Design & Development - 3/3

Oct 11, 2025

Beyond Traditional REST: Advanced API Patterns

You’ve mastered REST principles, built robust validation systems, and documented your APIs professionally. Your REST endpoints handle CRUD operations elegantly, validate inputs securely, and respond with consistent error formats. But here’s where most developers plateau: REST isn’t the solution to every API problem.

The REST limitation reality check:

// Your beautiful REST API
app.get("/users/:id/posts", getAllPostsByUser); // Works great
app.post("/posts/:id/comments", addCommentToPost); // Perfect
app.put("/users/:id/profile", updateUserProfile); // Excellent

// But then real requirements hit:
// "We need to show live notifications"
// "The mobile app needs to fetch user, posts, and comments in one request"
// "We need to notify external systems when orders are placed"
// "Different clients need different data fields"
// "We need to handle 10,000 requests per second"

The uncomfortable truth: REST is fantastic for CRUD operations on resources, but modern applications require patterns that REST doesn’t handle well:

Real-time communication: Chat systems, live notifications, collaborative editing
Complex data fetching: Mobile apps needing multiple related resources in one request
Event-driven architectures: Systems that react to changes rather than poll for them
High-throughput scenarios: APIs handling thousands of concurrent requests
Flexible data requirements: Different clients needing different subsets of data

Professional backend engineering means knowing when REST is perfect and when to reach for alternatives. GraphQL solves over-fetching and under-fetching problems. WebSockets enable real-time bidirectional communication. Rate limiting protects your infrastructure. API gateways provide enterprise-scale management. Webhooks enable event-driven architectures.

This final part of our API trilogy explores the advanced patterns that separate basic CRUD APIs from sophisticated, scalable systems that handle complex real-world requirements.

GraphQL vs REST vs SOAP: Choosing the Right Paradigm

Understanding GraphQL

GraphQL isn’t a replacement for REST—it’s a different approach to API design that solves specific problems REST handles poorly.

The GraphQL value proposition:

// REST approach: Multiple requests for related data
const fetchUserDashboard = async (userId) => {
  const user = await fetch(`/api/users/${userId}`);
  const posts = await fetch(`/api/users/${userId}/posts`);
  const comments = await fetch(`/api/users/${userId}/comments`);
  const notifications = await fetch(`/api/users/${userId}/notifications`);

  // 4 network requests, potential over-fetching in each
  return { user, posts, comments, notifications };
};

// GraphQL approach: Single request with precise data
const fetchUserDashboard = async (userId) => {
  const query = `
    query UserDashboard($userId: ID!) {
      user(id: $userId) {
        name
        email
        posts(limit: 5) {
          title
          publishedAt
          commentCount
        }
        notifications(unread: true) {
          message
          createdAt
        }
      }
    }
  `;

  const result = await fetch("/graphql", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ query, variables: { userId } }),
  });

  // 1 network request, exactly the data needed
  return result.data;
};

Setting Up GraphQL with Express

npm install apollo-server-express graphql

Basic GraphQL server:

const { ApolloServer } = require("apollo-server-express");
const { gql } = require("apollo-server-express");

// GraphQL schema definition
const typeDefs = gql`
  type User {
    id: ID!
    name: String!
    email: String!
    posts: [Post!]!
    profile: UserProfile
  }

  type Post {
    id: ID!
    title: String!
    content: String!
    author: User!
    comments: [Comment!]!
    publishedAt: String!
  }

  type Comment {
    id: ID!
    content: String!
    author: User!
    post: Post!
    createdAt: String!
  }

  type UserProfile {
    bio: String
    avatar: String
    website: String
  }

  type Query {
    user(id: ID!): User
    users(limit: Int, offset: Int): [User!]!
    post(id: ID!): Post
    posts(authorId: ID, limit: Int): [Post!]!
  }

  type Mutation {
    createUser(input: CreateUserInput!): User!
    createPost(input: CreatePostInput!): Post!
    addComment(input: AddCommentInput!): Comment!
  }

  input CreateUserInput {
    name: String!
    email: String!
    profile: UserProfileInput
  }

  input CreatePostInput {
    title: String!
    content: String!
    authorId: ID!
  }

  input AddCommentInput {
    content: String!
    postId: ID!
    authorId: ID!
  }

  input UserProfileInput {
    bio: String
    avatar: String
    website: String
  }
`;

// Resolvers - functions that fetch the actual data
const resolvers = {
  Query: {
    user: async (parent, { id }) => {
      return await getUserById(id);
    },
    users: async (parent, { limit = 10, offset = 0 }) => {
      return await getUsers({ limit, offset });
    },
    post: async (parent, { id }) => {
      return await getPostById(id);
    },
    posts: async (parent, { authorId, limit = 10 }) => {
      return await getPosts({ authorId, limit });
    },
  },

  Mutation: {
    createUser: async (parent, { input }) => {
      return await createUser(input);
    },
    createPost: async (parent, { input }) => {
      return await createPost(input);
    },
    addComment: async (parent, { input }) => {
      return await addComment(input);
    },
  },

  // Nested resolvers for related data
  User: {
    posts: async (user) => {
      return await getPostsByUserId(user.id);
    },
    profile: async (user) => {
      return await getUserProfile(user.id);
    },
  },

  Post: {
    author: async (post) => {
      return await getUserById(post.authorId);
    },
    comments: async (post) => {
      return await getCommentsByPostId(post.id);
    },
  },

  Comment: {
    author: async (comment) => {
      return await getUserById(comment.authorId);
    },
    post: async (comment) => {
      return await getPostById(comment.postId);
    },
  },
};

// Create Apollo Server
const server = new ApolloServer({
  typeDefs,
  resolvers,
  context: ({ req }) => {
    // Add authentication context
    const token = req.headers.authorization || "";
    const user = verifyToken(token);
    return { user };
  },
});

// Apply to Express app
server.applyMiddleware({ app, path: "/graphql" });

GraphQL in action:

// Client can request exactly what it needs
const DASHBOARD_QUERY = gql`
  query DashboardData($userId: ID!) {
    user(id: $userId) {
      name
      email
      posts(limit: 3) {
        id
        title
        publishedAt
        comments {
          id
          content
          author {
            name
          }
        }
      }
    }
  }
`;

// Different client, different needs - no new endpoint required
const MOBILE_QUERY = gql`
  query MobileUserData($userId: ID!) {
    user(id: $userId) {
      name
      profile {
        avatar
      }
      posts(limit: 1) {
        title
      }
    }
  }
`;

GraphQL vs REST: When to Choose What

Choose GraphQL when:

// ✅ Complex, nested data requirements
// ✅ Different clients need different data subsets
// ✅ Rapid frontend development with changing requirements
// ✅ Mobile applications concerned with bandwidth

const blogDashboard = gql`
  query BlogDashboard {
    currentUser {
      posts(status: PUBLISHED, limit: 5) {
        title
        stats {
          views
          likes
          comments {
            count
            recent(limit: 3) {
              author {
                name
              }
              content
            }
          }
        }
      }
      analytics {
        totalViews
        topPost {
          title
        }
      }
    }
  }
`;
// Single query replaces 10+ REST endpoints

Choose REST when:

// ✅ Simple CRUD operations
// ✅ Caching is critical (HTTP caching works great)
// ✅ File uploads and downloads
// ✅ Teams familiar with REST patterns

app.get("/api/posts/:id", getCachedPost); // HTTP caching works
app.post("/api/upload", handleFileUpload); // File handling is simpler
app.delete("/api/posts/:id", deletePost); // Clear semantics

SOAP: The Enterprise Legacy

SOAP (Simple Object Access Protocol) is mostly legacy but you’ll encounter it in enterprise environments:

<!-- SOAP request example -->
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
  <soap:Body>
    <GetUser xmlns="http://example.com/userservice">
      <UserId>123</UserId>
    </GetUser>
  </soap:Body>
</soap:Envelope>

SOAP characteristics:

Strict standards: WSDL definitions, XML schema validation
Enterprise features: WS-Security, WS-Transaction, WS-ReliableMessaging
Platform agnostic: Works across different systems and languages
Verbose: Significant XML overhead
Declining usage: Most new development uses REST or GraphQL

Real-Time APIs: Beyond Request-Response

WebSockets: Bidirectional Real-Time Communication

WebSockets enable full-duplex communication between client and server, perfect for chat applications, live updates, and collaborative features.

npm install ws socket.io

Basic WebSocket server with Socket.IO:

const { Server } = require("socket.io");
const http = require("http");

const server = http.createServer(app);
const io = new Server(server, {
  cors: {
    origin: "http://localhost:3000",
    methods: ["GET", "POST"],
  },
});

// WebSocket connection handling
io.on("connection", (socket) => {
  console.log("Client connected:", socket.id);

  // Join user to their personal room for notifications
  socket.on("authenticate", async (token) => {
    try {
      const user = verifyToken(token);
      socket.userId = user.id;
      socket.join(`user_${user.id}`);

      socket.emit("authenticated", { success: true });
    } catch (error) {
      socket.emit("auth_error", { message: "Invalid token" });
      socket.disconnect();
    }
  });

  // Chat room functionality
  socket.on("join_chat", (chatId) => {
    socket.join(`chat_${chatId}`);
    socket.emit("joined_chat", { chatId });
  });

  socket.on("send_message", async (data) => {
    const { chatId, message } = data;

    // Save message to database
    const savedMessage = await saveMessage({
      chatId,
      userId: socket.userId,
      message,
      timestamp: new Date(),
    });

    // Broadcast to all users in the chat
    io.to(`chat_${chatId}`).emit("new_message", {
      id: savedMessage.id,
      message: savedMessage.message,
      user: savedMessage.user,
      timestamp: savedMessage.timestamp,
    });
  });

  // Live notifications
  socket.on("subscribe_notifications", () => {
    socket.join(`notifications_${socket.userId}`);
  });

  // Typing indicators
  socket.on("typing_start", (chatId) => {
    socket.to(`chat_${chatId}`).emit("user_typing", {
      userId: socket.userId,
      typing: true,
    });
  });

  socket.on("typing_stop", (chatId) => {
    socket.to(`chat_${chatId}`).emit("user_typing", {
      userId: socket.userId,
      typing: false,
    });
  });

  // Handle disconnection
  socket.on("disconnect", () => {
    console.log("Client disconnected:", socket.id);
  });
});

// Integration with REST API for sending notifications
const sendNotification = (userId, notification) => {
  io.to(`notifications_${userId}`).emit("notification", notification);
};

// Use in REST endpoints
app.post("/api/posts", authenticateUser, async (req, res) => {
  const newPost = await createPost(req.body, req.user.id);

  // Send real-time notification to followers
  const followers = await getUserFollowers(req.user.id);
  followers.forEach((follower) => {
    sendNotification(follower.id, {
      type: "new_post",
      message: `${req.user.name} posted: ${newPost.title}`,
      postId: newPost.id,
    });
  });

  res.status(201).json({ post: newPost });
});

Client-side WebSocket usage:

// Client-side Socket.IO
const socket = io("http://localhost:3000");

// Authentication
const token = localStorage.getItem("authToken");
socket.emit("authenticate", token);

socket.on("authenticated", () => {
  console.log("Connected to WebSocket");

  // Subscribe to notifications
  socket.emit("subscribe_notifications");

  // Join a chat room
  socket.emit("join_chat", "room_123");
});

// Handle real-time messages
socket.on("new_message", (message) => {
  displayMessage(message);
});

// Handle notifications
socket.on("notification", (notification) => {
  showNotification(notification.message);
});

// Send messages
const sendMessage = (chatId, message) => {
  socket.emit("send_message", { chatId, message });
};

// Typing indicators
const startTyping = (chatId) => {
  socket.emit("typing_start", chatId);
};

const stopTyping = (chatId) => {
  socket.emit("typing_stop", chatId);
};

Server-Sent Events (SSE): One-Way Real-Time Updates

Server-Sent Events are perfect for live feeds, notifications, and streaming updates where you only need server-to-client communication.

// SSE endpoint
app.get("/api/events/stream", authenticateUser, (req, res) => {
  // Set SSE headers
  res.writeHead(200, {
    "Content-Type": "text/event-stream",
    "Cache-Control": "no-cache",
    Connection: "keep-alive",
    "Access-Control-Allow-Origin": "*",
    "Access-Control-Allow-Headers": "Cache-Control",
  });

  const userId = req.user.id;

  // Send initial connection confirmation
  res.write(
    `data: ${JSON.stringify({
      type: "connection",
      message: "Connected to event stream",
    })}\n\n`
  );

  // Send periodic heartbeat
  const heartbeat = setInterval(() => {
    res.write(
      `data: ${JSON.stringify({
        type: "heartbeat",
        timestamp: Date.now(),
      })}\n\n`
    );
  }, 30000);

  // Function to send events to this client
  const sendEvent = (event) => {
    res.write(`data: ${JSON.stringify(event)}\n\n`);
  };

  // Store connection for sending notifications
  activeConnections.set(userId, sendEvent);

  // Handle client disconnect
  req.on("close", () => {
    clearInterval(heartbeat);
    activeConnections.delete(userId);
    console.log(`SSE connection closed for user ${userId}`);
  });
});

// Live data feed
app.get("/api/dashboard/live", authenticateUser, (req, res) => {
  res.writeHead(200, {
    "Content-Type": "text/event-stream",
    "Cache-Control": "no-cache",
    Connection: "keep-alive",
  });

  const sendLiveData = async () => {
    const dashboardData = await getDashboardData(req.user.id);
    res.write(`data: ${JSON.stringify(dashboardData)}\n\n`);
  };

  // Send initial data
  sendLiveData();

  // Update every 5 seconds
  const interval = setInterval(sendLiveData, 5000);

  req.on("close", () => {
    clearInterval(interval);
  });
});

// Send notifications via SSE
const sendNotificationSSE = (userId, notification) => {
  const sendEvent = activeConnections.get(userId);
  if (sendEvent) {
    sendEvent({
      type: "notification",
      data: notification,
      timestamp: Date.now(),
    });
  }
};

Client-side SSE usage:

// Connect to SSE stream
const eventSource = new EventSource("/api/events/stream", {
  headers: {
    Authorization: `Bearer ${localStorage.getItem("token")}`,
  },
});

eventSource.onmessage = (event) => {
  const data = JSON.parse(event.data);

  switch (data.type) {
    case "connection":
      console.log("Connected to live updates");
      break;

    case "notification":
      showNotification(data.data);
      break;

    case "heartbeat":
      updateConnectionStatus("connected");
      break;

    default:
      console.log("Unknown event type:", data);
  }
};

eventSource.onerror = (error) => {
  console.error("SSE connection error:", error);
  updateConnectionStatus("disconnected");
};

// Live dashboard updates
const dashboardSource = new EventSource("/api/dashboard/live");

dashboardSource.onmessage = (event) => {
  const dashboardData = JSON.parse(event.data);
  updateDashboard(dashboardData);
};

Rate Limiting and Throttling: Protecting Your Infrastructure

Why Rate Limiting Is Essential

Uncontrolled API access leads to:

Resource exhaustion: CPU, memory, database connections
Service degradation: Slow responses for legitimate users
DDoS attacks: Malicious attempts to overwhelm your service
Cost explosion: Cloud services charging per request/bandwidth
Abuse: Users exceeding fair usage limits

Express Rate Limiting

npm install express-rate-limit express-slow-down

Basic rate limiting implementation:

const rateLimit = require("express-rate-limit");
const slowDown = require("express-slow-down");

// General API rate limiting
const generalLimiter = rateLimit({
  windowMs: 15 * 60 * 1000, // 15 minutes
  max: 100, // Limit each IP to 100 requests per windowMs
  message: {
    error: "Too many requests from this IP",
    retryAfter: "15 minutes",
  },
  standardHeaders: true, // Return rate limit info in headers
  legacyHeaders: false, // Disable X-RateLimit-* headers
  handler: (req, res) => {
    res.status(429).json({
      success: false,
      error: {
        message: "Rate limit exceeded",
        code: "RATE_LIMIT_EXCEEDED",
        retryAfter: Math.round(req.rateLimit.resetTime / 1000),
        limit: req.rateLimit.limit,
        remaining: req.rateLimit.remaining,
      },
    });
  },
});

// Stricter limits for authentication endpoints
const authLimiter = rateLimit({
  windowMs: 15 * 60 * 1000,
  max: 5, // Only 5 attempts per 15 minutes
  skipSuccessfulRequests: true, // Don't count successful requests
  keyGenerator: (req) => {
    // Rate limit by IP + email combination
    return `${req.ip}-${req.body.email || "unknown"}`;
  },
});

// Progressive delays for repeated requests
const speedLimiter = slowDown({
  windowMs: 15 * 60 * 1000,
  delayAfter: 50, // Allow 50 requests at full speed
  delayMs: 500, // Add 500ms delay after 50 requests
  maxDelayMs: 20000, // Maximum delay of 20 seconds
});

// Apply to routes
app.use("/api/", generalLimiter);
app.use("/api/auth/", authLimiter);
app.use("/api/search/", speedLimiter);

Advanced Rate Limiting Strategies

User-based rate limiting:

const userRateLimiter = rateLimit({
  windowMs: 60 * 60 * 1000, // 1 hour
  max: (req) => {
    // Different limits based on user tier
    if (req.user?.tier === "premium") {
      return 10000; // Premium users get 10,000 requests/hour
    } else if (req.user?.tier === "basic") {
      return 1000; // Basic users get 1,000 requests/hour
    } else {
      return 100; // Anonymous users get 100 requests/hour
    }
  },
  keyGenerator: (req) => {
    return req.user?.id || req.ip; // Rate limit by user ID or IP
  },
  skip: (req) => {
    // Skip rate limiting for internal requests
    return req.headers["x-internal-request"] === "true";
  },
});

// Endpoint-specific rate limiting
const createPostLimiter = rateLimit({
  windowMs: 60 * 60 * 1000, // 1 hour
  max: 10, // Max 10 posts per hour
  keyGenerator: (req) => `create_post_${req.user.id}`,
});

app.post(
  "/api/posts",
  authenticateUser,
  createPostLimiter,
  async (req, res) => {
    const newPost = await createPost(req.body, req.user.id);
    res.status(201).json({ post: newPost });
  }
);

Redis-based distributed rate limiting:

npm install redis connect-redis

const redis = require("redis");
const RedisStore = require("rate-limit-redis");

const redisClient = redis.createClient({
  host: process.env.REDIS_HOST,
  port: process.env.REDIS_PORT,
});

// Distributed rate limiting across multiple servers
const distributedLimiter = rateLimit({
  store: new RedisStore({
    client: redisClient,
    prefix: "api_rate_limit:",
  }),
  windowMs: 15 * 60 * 1000,
  max: 100,
});

// Custom rate limiting with Redis
const customRateLimit = async (req, res, next) => {
  const key = `rate_limit:${req.user.id}:${req.route.path}`;
  const limit = getUserRateLimit(req.user);
  const window = 3600; // 1 hour in seconds

  try {
    const current = await redisClient.incr(key);

    if (current === 1) {
      await redisClient.expire(key, window);
    }

    const ttl = await redisClient.ttl(key);

    res.set({
      "X-RateLimit-Limit": limit,
      "X-RateLimit-Remaining": Math.max(0, limit - current),
      "X-RateLimit-Reset": Date.now() + ttl * 1000,
    });

    if (current > limit) {
      return res.status(429).json({
        error: "Rate limit exceeded",
        retryAfter: ttl,
      });
    }

    next();
  } catch (error) {
    console.error("Rate limiting error:", error);
    next(); // Fail open - allow request if Redis is down
  }
};

API Gateways and Proxies: Enterprise-Scale Management

What Are API Gateways?

API Gateways sit between clients and your backend services, providing:

Authentication and authorization across all services
Rate limiting and throttling at the gateway level
Request/response transformation for legacy system integration
Load balancing and service discovery
Monitoring and analytics for all API traffic
Caching for improved performance

Basic proxy implementation with Express:

const httpProxy = require("http-proxy-middleware");

// API Gateway configuration
const apiGateway = express();

// Authentication middleware for all services
const gatewayAuth = async (req, res, next) => {
  try {
    const token = req.headers.authorization?.replace("Bearer ", "");
    if (!token) {
      return res.status(401).json({ error: "Token required" });
    }

    const user = await verifyToken(token);
    req.user = user;

    // Add user context to proxied requests
    req.headers["x-user-id"] = user.id;
    req.headers["x-user-role"] = user.role;

    next();
  } catch (error) {
    res.status(401).json({ error: "Invalid token" });
  }
};

// Service routing with load balancing
const userServiceProxy = httpProxy({
  target: "http://user-service:3001",
  changeOrigin: true,
  pathRewrite: {
    "^/api/users": "/users", // Remove /api prefix for internal service
  },
  onProxyReq: (proxyReq, req) => {
    // Add request ID for tracing
    proxyReq.setHeader("x-request-id", req.id);
  },
  onProxyRes: (proxyRes, req, res) => {
    // Add CORS headers
    proxyRes.headers["Access-Control-Allow-Origin"] = "*";
  },
});

const postServiceProxy = httpProxy({
  target: "http://post-service:3002",
  changeOrigin: true,
  pathRewrite: {
    "^/api/posts": "/posts",
  },
});

// Apply authentication to all routes
apiGateway.use(gatewayAuth);

// Route to appropriate services
apiGateway.use("/api/users", userServiceProxy);
apiGateway.use("/api/posts", postServiceProxy);

// Gateway-level rate limiting
const gatewayLimiter = rateLimit({
  windowMs: 15 * 60 * 1000,
  max: 1000, // Higher limit at gateway level
  keyGenerator: (req) => req.user?.id || req.ip,
});

apiGateway.use(gatewayLimiter);

Service Discovery and Health Checks

Dynamic service registration:

const serviceRegistry = new Map();

// Service health check
const healthCheck = async (serviceUrl) => {
  try {
    const response = await fetch(`${serviceUrl}/health`, {
      timeout: 5000,
    });
    return response.ok;
  } catch (error) {
    return false;
  }
};

// Service discovery middleware
const discoverService = (serviceName) => {
  return async (req, res, next) => {
    const services = serviceRegistry.get(serviceName) || [];
    const healthyServices = [];

    // Check health of all registered services
    for (const service of services) {
      if (await healthCheck(service.url)) {
        healthyServices.push(service);
      }
    }

    if (healthyServices.length === 0) {
      return res.status(503).json({
        error: "Service unavailable",
        service: serviceName,
      });
    }

    // Simple round-robin load balancing
    const selectedService =
      healthyServices[Math.floor(Math.random() * healthyServices.length)];

    req.selectedService = selectedService;
    next();
  };
};

// Register services
serviceRegistry.set("user-service", [
  { id: "user-1", url: "http://user-service-1:3001" },
  { id: "user-2", url: "http://user-service-2:3001" },
]);

serviceRegistry.set("post-service", [
  { id: "post-1", url: "http://post-service-1:3002" },
]);

// Use service discovery in routing
apiGateway.use(
  "/api/users",
  discoverService("user-service"),
  (req, res, next) => {
    const proxy = httpProxy({
      target: req.selectedService.url,
      changeOrigin: true,
    });
    proxy(req, res, next);
  }
);

Webhooks and Event-Driven APIs: Push-Based Communication

Understanding Webhooks

Webhooks flip the traditional API model: instead of clients polling for changes, servers push notifications when events occur.

Traditional polling vs webhooks:

// ❌ Traditional polling - inefficient
const pollForUpdates = async () => {
  setInterval(async () => {
    const response = await fetch("/api/orders/status");
    const orders = await response.json();

    orders.forEach((order) => {
      if (order.status !== previousStatus[order.id]) {
        handleStatusChange(order);
        previousStatus[order.id] = order.status;
      }
    });
  }, 30000); // Poll every 30 seconds - wasteful
};

// ✅ Webhook approach - efficient
const handleOrderStatusChange = (order) => {
  // Webhook endpoint receives immediate notification
  console.log(`Order ${order.id} status changed to ${order.status}`);
  updateUI(order);
  sendNotificationToUser(order.userId, order);
};

Implementing Webhook System

Webhook delivery system:

const webhookSubscriptions = new Map(); // In production, use database

// Webhook subscription management
app.post("/api/webhooks/subscribe", authenticateUser, async (req, res) => {
  const { url, events, secret } = req.body;

  // Validate webhook URL
  try {
    new URL(url); // Throws if invalid URL
  } catch (error) {
    return res.status(400).json({ error: "Invalid webhook URL" });
  }

  // Test webhook endpoint
  try {
    const testPayload = {
      type: "webhook.test",
      timestamp: Date.now(),
      data: { message: "Webhook test successful" },
    };

    await deliverWebhook(url, testPayload, secret);
  } catch (error) {
    return res.status(400).json({
      error: "Webhook endpoint test failed",
      details: error.message,
    });
  }

  const subscription = {
    id: generateId(),
    userId: req.user.id,
    url,
    events: events || ["*"], // Default to all events
    secret,
    createdAt: new Date(),
    active: true,
  };

  // Store subscription (use database in production)
  const userHooks = webhookSubscriptions.get(req.user.id) || [];
  userHooks.push(subscription);
  webhookSubscriptions.set(req.user.id, userHooks);

  res.status(201).json({
    message: "Webhook subscription created",
    webhook: subscription,
  });
});

// Webhook delivery function
const deliverWebhook = async (url, payload, secret, retryCount = 0) => {
  const maxRetries = 3;
  const backoffMs = Math.pow(2, retryCount) * 1000; // Exponential backoff

  try {
    // Create signature for security
    const signature = createSignature(JSON.stringify(payload), secret);

    const response = await fetch(url, {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        "User-Agent": "MyApp-Webhooks/1.0",
        "X-Webhook-Signature": signature,
        "X-Webhook-Timestamp": Date.now().toString(),
      },
      body: JSON.stringify(payload),
      timeout: 10000, // 10 second timeout
    });

    if (!response.ok) {
      throw new Error(`HTTP ${response.status}: ${response.statusText}`);
    }

    console.log(`Webhook delivered successfully to ${url}`);
    return true;
  } catch (error) {
    console.error(
      `Webhook delivery failed (attempt ${retryCount + 1}):`,
      error.message
    );

    if (retryCount < maxRetries) {
      setTimeout(() => {
        deliverWebhook(url, payload, secret, retryCount + 1);
      }, backoffMs);
    } else {
      console.error(`Webhook delivery failed permanently to ${url}`);
      // Store failed webhook for manual retry or disable subscription
    }

    return false;
  }
};

// Security: Create webhook signature
const crypto = require("crypto");

const createSignature = (payload, secret) => {
  return crypto.createHmac("sha256", secret).update(payload).digest("hex");
};

// Trigger webhooks on events
const triggerWebhooks = async (userId, eventType, data) => {
  const userHooks = webhookSubscriptions.get(userId) || [];

  const payload = {
    type: eventType,
    timestamp: Date.now(),
    data,
    userId,
  };

  for (const hook of userHooks) {
    if (!hook.active) continue;

    // Check if webhook subscribes to this event type
    if (hook.events.includes("*") || hook.events.includes(eventType)) {
      deliverWebhook(hook.url, payload, hook.secret);
    }
  }
};

// Use in your business logic
app.post("/api/orders", authenticateUser, async (req, res) => {
  const order = await createOrder(req.body, req.user.id);

  // Trigger webhook
  await triggerWebhooks(req.user.id, "order.created", {
    orderId: order.id,
    total: order.total,
    items: order.items,
    status: order.status,
  });

  res.status(201).json({ order });
});

app.patch("/api/orders/:id/status", authenticateUser, async (req, res) => {
  const order = await updateOrderStatus(req.params.id, req.body.status);

  // Trigger status change webhook
  await triggerWebhooks(order.userId, "order.status_changed", {
    orderId: order.id,
    previousStatus: req.body.previousStatus,
    newStatus: order.status,
    timestamp: order.updatedAt,
  });

  res.json({ order });
});

Webhook security validation (client-side):

// Webhook receiver endpoint (client application)
app.post("/webhooks/myapp", (req, res) => {
  const signature = req.headers["x-webhook-signature"];
  const timestamp = req.headers["x-webhook-timestamp"];
  const payload = JSON.stringify(req.body);

  // Verify timestamp (prevent replay attacks)
  const currentTime = Date.now();
  const webhookTime = parseInt(timestamp);
  if (Math.abs(currentTime - webhookTime) > 300000) {
    // 5 minutes
    return res.status(400).json({ error: "Request too old" });
  }

  // Verify signature
  const expectedSignature = crypto
    .createHmac("sha256", process.env.WEBHOOK_SECRET)
    .update(payload)
    .digest("hex");

  if (signature !== expectedSignature) {
    return res.status(401).json({ error: "Invalid signature" });
  }

  // Process webhook
  const { type, data } = req.body;

  switch (type) {
    case "order.created":
      handleNewOrder(data);
      break;
    case "order.status_changed":
      handleOrderStatusChange(data);
      break;
    default:
      console.log("Unknown webhook type:", type);
  }

  res.status(200).json({ received: true });
});

Key Takeaways

Advanced API patterns extend far beyond traditional REST, enabling real-time communication, efficient data fetching, infrastructure protection, and event-driven architectures. Each pattern solves specific problems that basic CRUD APIs can’t handle effectively.

The pattern selection mindset you need:

REST for CRUD: Perfect for resource-based operations with clear HTTP semantics
GraphQL for complex data needs: When clients need flexible, efficient data fetching
WebSockets for real-time bidirectional: Chat, collaborative editing, live updates
SSE for real-time unidirectional: Live feeds, notifications, streaming data
Webhooks for event-driven: Push notifications instead of wasteful polling

What distinguishes advanced API architecture:

Pattern matching: Choosing the right tool for each specific requirement
Infrastructure protection: Rate limiting and gateway management at scale
Real-time capabilities: Beyond request-response for modern user experiences
Event-driven thinking: Push-based communication for efficient system integration

Series Conclusion

We’ve completed our comprehensive API design and development journey—from REST principles through implementation details to advanced patterns. You now understand how to build APIs that scale from simple CRUD operations to complex, real-time, event-driven systems.

The complete API architect’s toolkit:

Design principles that create intuitive, scalable interfaces
Implementation patterns that handle real-world data securely
Advanced techniques that solve complex integration challenges
Infrastructure management that protects and scales your services

What’s Next

With APIs mastered, we’ll dive into database fundamentals in the next phase. You’ll learn data modeling, SQL operations, NoSQL patterns, and the persistence layer that powers all the APIs we’ve built.

APIs are the contracts between systems. Databases are where the actual business data lives. Master both, and you can build complete backend systems that handle any business requirement at any scale.

You’re no longer just building endpoints—you’re architecting communication systems that enable entire software ecosystems. The foundation is complete. Now we build the data layer that makes it all meaningful.