Third-party Integration - 1/2
The $12 Million Integration Cascade That Destroyed Black Friday Launch
Picture this horror story: November 24th, 11:58 PM EST. One of the most anticipated e-commerce launches of the year is about to go live—a flash sale featuring limited edition products from major brands. The marketing campaign has generated 2.5 million pre-registered users. Everything has been tested. The servers are scaled. The databases are optimized.
Then 12:00 AM hits, and the digital equivalent of a house fire begins.
Within the first 10 minutes, users start experiencing a nightmare:
- Credit card payments failing with cryptic “Service temporarily unavailable” messages
- Users creating accounts but never receiving confirmation emails
- Product images loading as broken placeholders from the CDN
- Social media sharing buttons leading to 404 pages
- Inventory counts wildly inaccurate because the ERP system integration froze
- Customer support chatbot responding with “I don’t understand” to every query
- Push notifications never sending, leaving users in the dark
The frontend was flawless. The backend could handle the load beautifully. But every single third-party integration—the invisible connective tissue that makes modern applications work—had failed spectacularly.
Here’s what went catastrophically wrong:
- Payment processor rate limiting: No exponential backoff, causing payment failures to cascade
- Email service overload: Sending 50,000 confirmation emails simultaneously without batching
- CDN authentication expired: Image assets returning 403 errors at peak traffic
- Social media APIs hitting quota limits: Share buttons breaking when users needed them most
- ERP system timeout: Inventory updates taking 45 seconds, causing overselling
- No fallback strategies: When one integration failed, it brought down related features
By 3 AM, they had:
- $12 million in abandoned carts due to payment failures
- 800,000 users who never received account confirmation emails
- 1.2 million failed social media shares during peak virality window
- Complete breakdown of inventory management causing legal issues with overselling
- Emergency rollback to a version with 70% fewer features
- Customer service overwhelmed with 50,000 support tickets
The brutal truth? They had built a technically perfect application that couldn’t communicate with the outside world when it mattered most.
The Uncomfortable Truth About Third-party Integration
Here’s what separates applications that gracefully orchestrate dozens of external services from those that collapse when a single API hiccups: Third-party integration isn’t just about making HTTP requests—it’s about building resilient, fault-tolerant communication layers that can handle the unpredictable nature of services you don’t control.
Most developers approach third-party integration like this:
- Find an API endpoint and start making requests
- Add basic error handling for 4xx/5xx responses
- Hope that external services will always be available
- Discover during traffic spikes that rate limits exist
- Panic when a third-party service goes down and takes your features with it
But systems that handle real-world integration complexity work differently:
- Design integration architecture with failure as the default assumption
- Implement intelligent retry strategies and circuit breakers from day one
- Build comprehensive monitoring for every external dependency
- Plan graceful degradation when third-party services fail
- Treat external API calls as distributed system challenges requiring sophisticated solutions
The difference isn’t just reliability—it’s the difference between applications that enhance user experience through seamless integrations and applications where third-party failures become user-facing disasters.
Ready to build integrations that work like Shopify’s payment processing instead of that API wrapper that breaks when someone sneezes at the data center? Let’s dive into the patterns that power bulletproof third-party integration.
REST API Consumption: Building Bulletproof Client Libraries
Production-Ready HTTP Client Architecture
// Robust REST API client with comprehensive error handling and resilience
import axios, { AxiosInstance, AxiosRequestConfig, AxiosResponse } from "axios";
import pRetry from "p-retry";
class RobustAPIClient {
private client: AxiosInstance;
private circuitBreaker: CircuitBreaker;
private metrics: MetricsCollector;
private cache: APICache;
private rateLimiter: RateLimiter;
constructor(config: APIClientConfig) {
this.client = this.createAxiosInstance(config);
this.circuitBreaker = new CircuitBreaker(config.circuitBreakerOptions);
this.metrics = new MetricsCollector();
this.cache = new APICache(config.cacheOptions);
this.rateLimiter = new RateLimiter(config.rateLimitOptions);
this.setupInterceptors();
this.setupRequestTracing();
}
private createAxiosInstance(config: APIClientConfig): AxiosInstance {
const instance = axios.create({
baseURL: config.baseURL,
timeout: config.timeout || 30000, // 30 second default
headers: {
"Content-Type": "application/json",
"User-Agent": `${config.serviceName}/${config.version}`,
...config.defaultHeaders,
},
maxRedirects: 3,
validateStatus: (status) => status < 500, // Don't throw on 4xx
});
return instance;
}
private setupInterceptors(): void {
// Request interceptor
this.client.interceptors.request.use(
async (config) => {
const startTime = Date.now();
// Add correlation ID for tracing
config.headers["X-Correlation-ID"] = this.generateCorrelationId();
// Add request timestamp
config.metadata = { startTime };
// Apply rate limiting
await this.rateLimiter.waitForToken();
// Log request
console.log(
`API Request: ${config.method?.toUpperCase()} ${config.url}`
);
return config;
},
(error) => {
console.error("Request interceptor error:", error);
return Promise.reject(error);
}
);
// Response interceptor
this.client.interceptors.response.use(
(response) => {
const duration = Date.now() - response.config.metadata?.startTime;
// Record metrics
this.metrics.recordRequestSuccess(
response.config.url || "",
response.status,
duration
);
// Log successful response
console.log(
`API Response: ${
response.status
} ${response.config.method?.toUpperCase()} ${
response.config.url
} (${duration}ms)`
);
return response;
},
async (error) => {
const duration =
Date.now() - (error.config?.metadata?.startTime || Date.now());
// Record metrics
this.metrics.recordRequestError(
error.config?.url || "",
error.response?.status || 0,
duration,
error.code
);
// Enhanced error logging
console.error(
`API Error: ${
error.response?.status || "UNKNOWN"
} ${error.config?.method?.toUpperCase()} ${
error.config?.url
} (${duration}ms)`,
{
status: error.response?.status,
statusText: error.response?.statusText,
data: error.response?.data,
correlationId: error.config?.headers["X-Correlation-ID"],
}
);
return Promise.reject(this.enhanceError(error));
}
);
}
// Resilient API request with retry logic and circuit breaker
async request<T = any>(config: APIRequestConfig): Promise<APIResponse<T>> {
const cacheKey = this.generateCacheKey(config);
// Try cache first if enabled
if (config.cache?.enabled) {
const cachedResponse = await this.cache.get(cacheKey);
if (cachedResponse) {
console.log(`Cache hit: ${config.method} ${config.url}`);
return cachedResponse;
}
}
// Check circuit breaker
if (this.circuitBreaker.isOpen()) {
throw new APIError(
"Circuit breaker is open",
"CIRCUIT_BREAKER_OPEN",
503
);
}
try {
const response = await this.executeWithRetry(config);
// Cache successful responses
if (config.cache?.enabled && response.status < 400) {
await this.cache.set(
cacheKey,
response,
config.cache.ttlSeconds || 300
);
}
// Record circuit breaker success
this.circuitBreaker.recordSuccess();
return response;
} catch (error) {
// Record circuit breaker failure
this.circuitBreaker.recordFailure();
throw error;
}
}
private async executeWithRetry<T>(
config: APIRequestConfig
): Promise<APIResponse<T>> {
const retryOptions = {
retries: config.retry?.maxAttempts || 3,
factor: config.retry?.backoffFactor || 2,
minTimeout: config.retry?.initialDelayMs || 1000,
maxTimeout: config.retry?.maxDelayMs || 30000,
randomize: true, // Add jitter
onRetry: (error: any, attempt: number) => {
console.warn(
`API retry attempt ${attempt} for ${config.method} ${config.url}:`,
error.message
);
this.metrics.recordRetry(config.url || "", attempt);
},
};
return pRetry(async () => {
const axiosConfig: AxiosRequestConfig = {
method: config.method,
url: config.url,
data: config.data,
params: config.params,
headers: {
...config.headers,
// Add idempotency key for POST/PUT requests
...(config.method !== "GET" && {
"Idempotency-Key": this.generateIdempotencyKey(config),
}),
},
timeout: config.timeout,
};
const response = await this.client.request(axiosConfig);
// Check if response indicates a retryable error
if (this.isRetryableResponse(response)) {
throw new APIError(
"Retryable response received",
"RETRYABLE_RESPONSE",
response.status
);
}
return this.transformResponse<T>(response);
}, retryOptions);
}
private isRetryableResponse(response: AxiosResponse): boolean {
// Retry on server errors and specific client errors
const retryableStatuses = [
429, // Rate limited
500,
502,
503,
504, // Server errors
408, // Request timeout
];
return retryableStatuses.includes(response.status);
}
private transformResponse<T>(response: AxiosResponse): APIResponse<T> {
return {
data: response.data,
status: response.status,
statusText: response.statusText,
headers: response.headers,
correlationId: response.config.headers?.["X-Correlation-ID"],
requestDuration: Date.now() - (response.config.metadata?.startTime || 0),
};
}
private enhanceError(error: any): APIError {
const apiError = new APIError(
error.message || "Unknown API error",
this.getErrorCode(error),
error.response?.status || 0,
{
originalError: error,
correlationId: error.config?.headers?.["X-Correlation-ID"],
url: error.config?.url,
method: error.config?.method,
requestData: error.config?.data,
responseData: error.response?.data,
}
);
return apiError;
}
private getErrorCode(error: any): string {
if (error.code) return error.code;
if (error.response?.status) {
const statusCodes: Record<number, string> = {
400: "BAD_REQUEST",
401: "UNAUTHORIZED",
403: "FORBIDDEN",
404: "NOT_FOUND",
422: "VALIDATION_ERROR",
429: "RATE_LIMITED",
500: "INTERNAL_SERVER_ERROR",
502: "BAD_GATEWAY",
503: "SERVICE_UNAVAILABLE",
504: "GATEWAY_TIMEOUT",
};
return statusCodes[error.response.status] || "HTTP_ERROR";
}
return "UNKNOWN_ERROR";
}
// Convenience methods for common HTTP verbs
async get<T = any>(
url: string,
config?: Omit<APIRequestConfig, "method" | "url">
): Promise<APIResponse<T>> {
return this.request<T>({ ...config, method: "GET", url });
}
async post<T = any>(
url: string,
data?: any,
config?: Omit<APIRequestConfig, "method" | "url" | "data">
): Promise<APIResponse<T>> {
return this.request<T>({ ...config, method: "POST", url, data });
}
async put<T = any>(
url: string,
data?: any,
config?: Omit<APIRequestConfig, "method" | "url" | "data">
): Promise<APIResponse<T>> {
return this.request<T>({ ...config, method: "PUT", url, data });
}
async patch<T = any>(
url: string,
data?: any,
config?: Omit<APIRequestConfig, "method" | "url" | "data">
): Promise<APIResponse<T>> {
return this.request<T>({ ...config, method: "PATCH", url, data });
}
async delete<T = any>(
url: string,
config?: Omit<APIRequestConfig, "method" | "url">
): Promise<APIResponse<T>> {
return this.request<T>({ ...config, method: "DELETE", url });
}
// Bulk operations with concurrency control
async bulkRequest<T = any>(
requests: APIRequestConfig[],
concurrency: number = 5
): Promise<APIResponse<T>[]> {
const results: APIResponse<T>[] = [];
const executing: Promise<any>[] = [];
for (const requestConfig of requests) {
const promise = this.request<T>(requestConfig).then(
(result) => {
results.push(result);
return result;
},
(error) => {
// Store error as result for partial success handling
results.push({
data: null as any,
status: error.status || 0,
statusText: error.message,
headers: {},
error: error,
} as APIResponse<T>);
return error;
}
);
executing.push(promise);
if (executing.length >= concurrency) {
await Promise.race(executing);
executing.splice(
executing.findIndex((p) => p === promise),
1
);
}
}
await Promise.all(executing);
return results;
}
// Health check for the API
async healthCheck(): Promise<HealthCheckResult> {
const startTime = Date.now();
try {
const response = await this.get("/health", {
timeout: 5000,
retry: { maxAttempts: 1 },
});
return {
status: "healthy",
responseTime: Date.now() - startTime,
details: response.data,
};
} catch (error) {
return {
status: "unhealthy",
responseTime: Date.now() - startTime,
error: error.message,
};
}
}
// Get client statistics
getStatistics(): ClientStatistics {
return {
circuitBreakerState: this.circuitBreaker.getState(),
rateLimiterStats: this.rateLimiter.getStats(),
cacheStats: this.cache.getStats(),
requestMetrics: this.metrics.getMetrics(),
};
}
private generateCorrelationId(): string {
return `${Date.now()}-${Math.random().toString(36).substr(2, 9)}`;
}
private generateIdempotencyKey(config: APIRequestConfig): string {
const payload = JSON.stringify({
url: config.url,
method: config.method,
data: config.data,
timestamp: Math.floor(Date.now() / 60000), // 1-minute window
});
return Buffer.from(payload).toString("base64");
}
private generateCacheKey(config: APIRequestConfig): string {
const keyData = {
method: config.method,
url: config.url,
params: config.params,
// Don't include data for GET requests in cache key
...(config.method !== "GET" && { data: config.data }),
};
return Buffer.from(JSON.stringify(keyData)).toString("base64");
}
private setupRequestTracing(): void {
// Add request/response logging and tracing
if (process.env.NODE_ENV !== "production") {
this.client.interceptors.request.use((config) => {
console.debug(`→ ${config.method?.toUpperCase()} ${config.url}`, {
headers: config.headers,
data: config.data,
});
return config;
});
this.client.interceptors.response.use(
(response) => {
console.debug(`← ${response.status} ${response.config.url}`, {
data: response.data,
});
return response;
},
(error) => {
console.debug(`← ERROR ${error.config?.url}`, {
status: error.response?.status,
data: error.response?.data,
});
return Promise.reject(error);
}
);
}
}
}
// Supporting classes
class CircuitBreaker {
private failures = 0;
private lastFailureTime = 0;
private state: "CLOSED" | "OPEN" | "HALF_OPEN" = "CLOSED";
constructor(private options: CircuitBreakerOptions) {}
isOpen(): boolean {
if (this.state === "OPEN") {
if (Date.now() - this.lastFailureTime > this.options.resetTimeoutMs) {
this.state = "HALF_OPEN";
return false;
}
return true;
}
return false;
}
recordSuccess(): void {
this.failures = 0;
this.state = "CLOSED";
}
recordFailure(): void {
this.failures++;
this.lastFailureTime = Date.now();
if (this.failures >= this.options.failureThreshold) {
this.state = "OPEN";
}
}
getState(): string {
return this.state;
}
}
class RateLimiter {
private tokens: number;
private lastRefill: number;
constructor(private options: RateLimitOptions) {
this.tokens = options.maxTokens;
this.lastRefill = Date.now();
}
async waitForToken(): Promise<void> {
this.refillTokens();
if (this.tokens <= 0) {
const waitTime = this.calculateWaitTime();
await new Promise((resolve) => setTimeout(resolve, waitTime));
return this.waitForToken();
}
this.tokens--;
}
private refillTokens(): void {
const now = Date.now();
const timePassed = now - this.lastRefill;
const tokensToAdd =
Math.floor(timePassed / this.options.refillIntervalMs) *
this.options.refillAmount;
this.tokens = Math.min(this.options.maxTokens, this.tokens + tokensToAdd);
this.lastRefill = now;
}
private calculateWaitTime(): number {
return this.options.refillIntervalMs - (Date.now() - this.lastRefill);
}
getStats(): any {
return {
availableTokens: this.tokens,
maxTokens: this.options.maxTokens,
};
}
}
class APICache {
private cache = new Map<string, CachedResponse>();
constructor(private options: CacheOptions) {}
async get(key: string): Promise<any> {
const cached = this.cache.get(key);
if (!cached) return null;
if (Date.now() > cached.expiresAt) {
this.cache.delete(key);
return null;
}
return cached.data;
}
async set(key: string, data: any, ttlSeconds: number): Promise<void> {
this.cache.set(key, {
data,
expiresAt: Date.now() + ttlSeconds * 1000,
});
}
getStats(): any {
return {
size: this.cache.size,
maxSize: this.options.maxEntries || 1000,
};
}
}
class MetricsCollector {
private metrics = {
requestsTotal: 0,
requestsSuccessful: 0,
requestsFailed: 0,
retriesTotal: 0,
averageResponseTime: 0,
};
recordRequestSuccess(url: string, status: number, duration: number): void {
this.metrics.requestsTotal++;
this.metrics.requestsSuccessful++;
this.updateAverageResponseTime(duration);
}
recordRequestError(
url: string,
status: number,
duration: number,
errorCode?: string
): void {
this.metrics.requestsTotal++;
this.metrics.requestsFailed++;
this.updateAverageResponseTime(duration);
}
recordRetry(url: string, attempt: number): void {
this.metrics.retriesTotal++;
}
private updateAverageResponseTime(duration: number): void {
this.metrics.averageResponseTime =
(this.metrics.averageResponseTime * (this.metrics.requestsTotal - 1) +
duration) /
this.metrics.requestsTotal;
}
getMetrics(): any {
return { ...this.metrics };
}
}
// Custom error class for API errors
class APIError extends Error {
constructor(
message: string,
public code: string,
public status: number,
public metadata?: any
) {
super(message);
this.name = "APIError";
}
toJSON(): any {
return {
name: this.name,
message: this.message,
code: this.code,
status: this.status,
metadata: this.metadata,
};
}
}
// Interfaces
interface APIClientConfig {
baseURL: string;
serviceName: string;
version: string;
timeout?: number;
defaultHeaders?: Record<string, string>;
circuitBreakerOptions: CircuitBreakerOptions;
rateLimitOptions: RateLimitOptions;
cacheOptions: CacheOptions;
}
interface CircuitBreakerOptions {
failureThreshold: number;
resetTimeoutMs: number;
}
interface RateLimitOptions {
maxTokens: number;
refillIntervalMs: number;
refillAmount: number;
}
interface CacheOptions {
maxEntries?: number;
defaultTTL?: number;
}
interface APIRequestConfig {
method: "GET" | "POST" | "PUT" | "PATCH" | "DELETE";
url: string;
data?: any;
params?: Record<string, any>;
headers?: Record<string, string>;
timeout?: number;
retry?: {
maxAttempts: number;
initialDelayMs?: number;
maxDelayMs?: number;
backoffFactor?: number;
};
cache?: {
enabled: boolean;
ttlSeconds?: number;
};
}
interface APIResponse<T = any> {
data: T;
status: number;
statusText: string;
headers: any;
correlationId?: string;
requestDuration?: number;
error?: any;
}
interface CachedResponse {
data: any;
expiresAt: number;
}
interface HealthCheckResult {
status: "healthy" | "unhealthy";
responseTime: number;
details?: any;
error?: string;
}
interface ClientStatistics {
circuitBreakerState: string;
rateLimiterStats: any;
cacheStats: any;
requestMetrics: any;
}
Payment Gateway Integration: Handling Money Safely
Secure Payment Processing Architecture
// Comprehensive payment processing with multiple gateways and security
class PaymentGatewayManager {
private gateways: Map<string, PaymentGateway> = new Map();
private primaryGateway: string;
private fallbackGateways: string[];
private paymentLogger: PaymentLogger;
private fraudDetector: FraudDetector;
private webhookValidator: WebhookValidator;
constructor(config: PaymentManagerConfig) {
this.primaryGateway = config.primaryGateway;
this.fallbackGateways = config.fallbackGateways || [];
this.paymentLogger = new PaymentLogger();
this.fraudDetector = new FraudDetector(config.fraudConfig);
this.webhookValidator = new WebhookValidator();
this.initializeGateways(config.gateways);
this.setupHealthChecking();
}
private initializeGateways(gatewayConfigs: GatewayConfig[]): void {
for (const config of gatewayConfigs) {
let gateway: PaymentGateway;
switch (config.provider) {
case "stripe":
gateway = new StripeGateway(config);
break;
case "paypal":
gateway = new PayPalGateway(config);
break;
case "square":
gateway = new SquareGateway(config);
break;
case "braintree":
gateway = new BraintreeGateway(config);
break;
default:
throw new Error(`Unsupported payment provider: ${config.provider}`);
}
this.gateways.set(config.name, gateway);
console.log(`Initialized ${config.provider} gateway: ${config.name}`);
}
}
// Process payment with automatic fallback
async processPayment(request: PaymentRequest): Promise<PaymentResult> {
const transactionId = this.generateTransactionId();
try {
// Log payment attempt
await this.paymentLogger.logPaymentAttempt(transactionId, request);
// Fraud detection
const fraudCheck = await this.fraudDetector.analyzePayment(request);
if (fraudCheck.risk === "HIGH") {
throw new PaymentError(
"Payment blocked due to fraud risk",
"FRAUD_DETECTED",
{ fraudScore: fraudCheck.score }
);
}
// Try primary gateway first
const primaryGateway = this.gateways.get(this.primaryGateway);
if (primaryGateway && (await primaryGateway.isHealthy())) {
try {
const result = await this.executePayment(
primaryGateway,
request,
transactionId
);
await this.paymentLogger.logPaymentSuccess(transactionId, result);
return result;
} catch (error) {
await this.paymentLogger.logPaymentError(
transactionId,
error,
this.primaryGateway
);
// If primary fails, try fallbacks
if (this.shouldFallback(error)) {
console.warn(
`Primary gateway ${this.primaryGateway} failed, trying fallbacks`
);
return await this.tryFallbackGateways(
request,
transactionId,
error
);
}
throw error;
}
}
// Primary gateway unhealthy, go straight to fallbacks
return await this.tryFallbackGateways(
request,
transactionId,
new PaymentError("Primary gateway unavailable", "GATEWAY_UNAVAILABLE")
);
} catch (error) {
await this.paymentLogger.logPaymentFailure(transactionId, error);
throw error;
}
}
private async executePayment(
gateway: PaymentGateway,
request: PaymentRequest,
transactionId: string
): Promise<PaymentResult> {
// Add transaction tracking
const enhancedRequest = {
...request,
transactionId,
metadata: {
...request.metadata,
gateway: gateway.getName(),
timestamp: Date.now(),
userAgent: request.userAgent,
ipAddress: request.ipAddress,
},
};
// Execute payment with timeout
const timeoutPromise = new Promise<never>((_, reject) => {
setTimeout(
() => reject(new PaymentError("Payment timeout", "TIMEOUT")),
30000 // 30 second timeout
);
});
const paymentPromise = gateway.processPayment(enhancedRequest);
const result = await Promise.race([paymentPromise, timeoutPromise]);
// Validate result
this.validatePaymentResult(result, request);
return {
...result,
transactionId,
gateway: gateway.getName(),
processedAt: Date.now(),
};
}
private async tryFallbackGateways(
request: PaymentRequest,
transactionId: string,
primaryError: Error
): Promise<PaymentResult> {
for (const fallbackName of this.fallbackGateways) {
const gateway = this.gateways.get(fallbackName);
if (!gateway) continue;
if (await gateway.isHealthy()) {
try {
console.log(`Trying fallback gateway: ${fallbackName}`);
const result = await this.executePayment(
gateway,
request,
transactionId
);
// Log successful fallback
await this.paymentLogger.logFallbackSuccess(
transactionId,
fallbackName,
primaryError
);
return result;
} catch (fallbackError) {
await this.paymentLogger.logPaymentError(
transactionId,
fallbackError,
fallbackName
);
console.warn(
`Fallback gateway ${fallbackName} failed:`,
fallbackError.message
);
}
}
}
// All gateways failed
throw new PaymentError(
"All payment gateways failed",
"ALL_GATEWAYS_FAILED",
{ primaryError, gatewayCount: this.fallbackGateways.length + 1 }
);
}
// Refund processing with gateway routing
async processRefund(refundRequest: RefundRequest): Promise<RefundResult> {
const originalPayment = await this.getPaymentDetails(
refundRequest.originalTransactionId
);
if (!originalPayment) {
throw new PaymentError("Original payment not found", "PAYMENT_NOT_FOUND");
}
const gateway = this.gateways.get(originalPayment.gateway);
if (!gateway) {
throw new PaymentError(
"Original payment gateway not available",
"GATEWAY_NOT_AVAILABLE"
);
}
const refundId = this.generateRefundId();
try {
const result = await gateway.processRefund({
...refundRequest,
refundId,
originalPayment,
});
await this.paymentLogger.logRefundSuccess(refundId, result);
return result;
} catch (error) {
await this.paymentLogger.logRefundError(refundId, error);
throw error;
}
}
// Webhook processing for payment updates
async processWebhook(
provider: string,
signature: string,
payload: any,
headers: Record<string, string>
): Promise<WebhookResult> {
const gateway = Array.from(this.gateways.values()).find(
(g) => g.getProvider() === provider
);
if (!gateway) {
throw new Error(`No gateway found for provider: ${provider}`);
}
// Validate webhook signature
const isValid = await this.webhookValidator.validate(
provider,
signature,
payload,
headers
);
if (!isValid) {
throw new PaymentError("Invalid webhook signature", "INVALID_SIGNATURE");
}
// Process webhook
try {
const result = await gateway.processWebhook(payload, headers);
// Update payment status based on webhook
if (result.transactionId) {
await this.updatePaymentStatus(
result.transactionId,
result.status,
result.metadata
);
}
return result;
} catch (error) {
console.error(`Webhook processing error for ${provider}:`, error);
throw error;
}
}
// Payment method tokenization for recurring payments
async tokenizePaymentMethod(
tokenizationRequest: TokenizationRequest
): Promise<TokenizationResult> {
const gateway = this.gateways.get(this.primaryGateway);
if (!gateway) {
throw new PaymentError(
"Primary gateway not available",
"GATEWAY_UNAVAILABLE"
);
}
try {
const result = await gateway.tokenizePaymentMethod(tokenizationRequest);
// Store token securely (implementation depends on your security requirements)
await this.storePaymentToken(result.token, {
customerId: tokenizationRequest.customerId,
gateway: gateway.getName(),
paymentMethodType: result.paymentMethodType,
lastFour: result.lastFour,
expiryMonth: result.expiryMonth,
expiryYear: result.expiryYear,
createdAt: Date.now(),
});
return result;
} catch (error) {
console.error("Payment tokenization failed:", error);
throw error;
}
}
// Recurring payment processing
async processRecurringPayment(
recurringRequest: RecurringPaymentRequest
): Promise<PaymentResult> {
const tokenData = await this.getPaymentToken(recurringRequest.tokenId);
if (!tokenData) {
throw new PaymentError("Payment token not found", "TOKEN_NOT_FOUND");
}
const gateway = this.gateways.get(tokenData.gateway);
if (!gateway) {
throw new PaymentError(
"Gateway not available for token",
"GATEWAY_UNAVAILABLE"
);
}
const paymentRequest: PaymentRequest = {
amount: recurringRequest.amount,
currency: recurringRequest.currency,
customerId: recurringRequest.customerId,
paymentMethodToken: recurringRequest.tokenId,
description: `Recurring payment: ${recurringRequest.description}`,
metadata: {
recurring: true,
subscriptionId: recurringRequest.subscriptionId,
...recurringRequest.metadata,
},
};
return await this.processPayment(paymentRequest);
}
// Payment gateway health monitoring
private setupHealthChecking(): void {
setInterval(async () => {
for (const [name, gateway] of this.gateways) {
try {
const isHealthy = await gateway.isHealthy();
if (!isHealthy) {
console.warn(`Gateway ${name} is unhealthy`);
}
} catch (error) {
console.error(`Health check failed for gateway ${name}:`, error);
}
}
}, 60000); // Check every minute
}
// Utility methods
private shouldFallback(error: any): boolean {
// Determine if error warrants trying fallback gateways
const fallbackErrors = [
"GATEWAY_TIMEOUT",
"GATEWAY_UNAVAILABLE",
"RATE_LIMITED",
"TEMPORARY_ERROR",
];
return fallbackErrors.includes(error.code) || error.status >= 500;
}
private validatePaymentResult(
result: PaymentResult,
request: PaymentRequest
): void {
if (!result.transactionId) {
throw new PaymentError(
"Missing transaction ID in result",
"INVALID_RESULT"
);
}
if (result.amount !== request.amount) {
throw new PaymentError("Amount mismatch in result", "AMOUNT_MISMATCH");
}
if (result.currency !== request.currency) {
throw new PaymentError(
"Currency mismatch in result",
"CURRENCY_MISMATCH"
);
}
}
private generateTransactionId(): string {
return `txn_${Date.now()}_${Math.random().toString(36).substr(2, 9)}`;
}
private generateRefundId(): string {
return `ref_${Date.now()}_${Math.random().toString(36).substr(2, 9)}`;
}
// Abstract methods that would be implemented based on your storage solution
private async getPaymentDetails(transactionId: string): Promise<any> {
// Implementation depends on your database
return null;
}
private async updatePaymentStatus(
transactionId: string,
status: string,
metadata?: any
): Promise<void> {
// Implementation depends on your database
}
private async storePaymentToken(
token: string,
tokenData: any
): Promise<void> {
// Implementation depends on your secure storage solution
}
private async getPaymentToken(tokenId: string): Promise<any> {
// Implementation depends on your secure storage solution
return null;
}
}
// Example Stripe gateway implementation
class StripeGateway implements PaymentGateway {
private stripe: any; // Stripe SDK instance
private apiClient: RobustAPIClient;
constructor(private config: GatewayConfig) {
// Initialize Stripe SDK
// this.stripe = require('stripe')(config.secretKey);
this.apiClient = new RobustAPIClient({
baseURL: "https://api.stripe.com",
serviceName: "stripe-gateway",
version: "1.0.0",
defaultHeaders: {
Authorization: `Bearer ${config.secretKey}`,
"Stripe-Version": "2023-10-16",
},
circuitBreakerOptions: {
failureThreshold: 5,
resetTimeoutMs: 60000,
},
rateLimitOptions: {
maxTokens: 100,
refillIntervalMs: 1000,
refillAmount: 10,
},
cacheOptions: {},
});
}
async processPayment(request: PaymentRequest): Promise<PaymentResult> {
try {
const paymentIntentData = {
amount: Math.round(request.amount * 100), // Convert to cents
currency: request.currency.toLowerCase(),
payment_method: request.paymentMethodId,
customer: request.customerId,
description: request.description,
metadata: request.metadata,
confirmation_method: "manual",
confirm: true,
};
const response = await this.apiClient.post(
"/v1/payment_intents",
paymentIntentData
);
return this.transformStripeResult(response.data);
} catch (error) {
throw this.transformStripeError(error);
}
}
async processRefund(
request: RefundRequest & { originalPayment: any }
): Promise<RefundResult> {
try {
const refundData = {
payment_intent: request.originalPayment.gatewayTransactionId,
amount: request.amount ? Math.round(request.amount * 100) : undefined,
reason: request.reason,
metadata: request.metadata,
};
const response = await this.apiClient.post("/v1/refunds", refundData);
return {
refundId: response.data.id,
amount: response.data.amount / 100,
currency: response.data.currency.toUpperCase(),
status: response.data.status,
processedAt: Date.now(),
gatewayRefundId: response.data.id,
};
} catch (error) {
throw this.transformStripeError(error);
}
}
async tokenizePaymentMethod(
request: TokenizationRequest
): Promise<TokenizationResult> {
// Stripe handles tokenization through payment methods
try {
const paymentMethodData = {
type: "card",
card: request.paymentMethodData,
billing_details: request.billingDetails,
};
const response = await this.apiClient.post(
"/v1/payment_methods",
paymentMethodData
);
return {
token: response.data.id,
paymentMethodType: "card",
lastFour: response.data.card.last4,
expiryMonth: response.data.card.exp_month,
expiryYear: response.data.card.exp_year,
brand: response.data.card.brand,
};
} catch (error) {
throw this.transformStripeError(error);
}
}
async processWebhook(
payload: any,
headers: Record<string, string>
): Promise<WebhookResult> {
const event = payload;
switch (event.type) {
case "payment_intent.succeeded":
return {
type: "payment_completed",
transactionId: event.data.object.metadata?.transactionId,
status: "completed",
metadata: event.data.object,
};
case "payment_intent.payment_failed":
return {
type: "payment_failed",
transactionId: event.data.object.metadata?.transactionId,
status: "failed",
metadata: event.data.object,
};
default:
return {
type: "unhandled",
status: "ignored",
metadata: event,
};
}
}
async isHealthy(): Promise<boolean> {
try {
const response = await this.apiClient.get("/v1/account", {
timeout: 5000,
});
return response.status === 200;
} catch (error) {
return false;
}
}
getName(): string {
return this.config.name;
}
getProvider(): string {
return "stripe";
}
private transformStripeResult(stripeResult: any): PaymentResult {
return {
transactionId: stripeResult.metadata?.transactionId || stripeResult.id,
gatewayTransactionId: stripeResult.id,
amount: stripeResult.amount / 100,
currency: stripeResult.currency.toUpperCase(),
status: stripeResult.status === "succeeded" ? "completed" : "pending",
gatewayResponse: stripeResult,
};
}
private transformStripeError(error: any): PaymentError {
const stripeError = error.response?.data?.error || error;
const errorCodeMap: Record<string, string> = {
card_declined: "CARD_DECLINED",
insufficient_funds: "INSUFFICIENT_FUNDS",
invalid_cvc: "INVALID_CVC",
expired_card: "EXPIRED_CARD",
rate_limit: "RATE_LIMITED",
};
return new PaymentError(
stripeError.message || error.message,
errorCodeMap[stripeError.code] || "GATEWAY_ERROR",
error.status,
{ stripeError, originalError: error }
);
}
}
// Supporting classes and interfaces
class PaymentLogger {
async logPaymentAttempt(
transactionId: string,
request: PaymentRequest
): Promise<void> {
console.log(`Payment attempt: ${transactionId}`, {
amount: request.amount,
currency: request.currency,
customerId: request.customerId,
});
}
async logPaymentSuccess(
transactionId: string,
result: PaymentResult
): Promise<void> {
console.log(`Payment success: ${transactionId}`, result);
}
async logPaymentError(
transactionId: string,
error: any,
gateway?: string
): Promise<void> {
console.error(`Payment error: ${transactionId} (${gateway})`, error);
}
async logPaymentFailure(transactionId: string, error: any): Promise<void> {
console.error(`Payment failure: ${transactionId}`, error);
}
async logFallbackSuccess(
transactionId: string,
fallbackGateway: string,
primaryError: Error
): Promise<void> {
console.log(`Fallback success: ${transactionId} via ${fallbackGateway}`, {
primaryError: primaryError.message,
});
}
async logRefundSuccess(
refundId: string,
result: RefundResult
): Promise<void> {
console.log(`Refund success: ${refundId}`, result);
}
async logRefundError(refundId: string, error: any): Promise<void> {
console.error(`Refund error: ${refundId}`, error);
}
}
class FraudDetector {
constructor(private config: FraudConfig) {}
async analyzePayment(request: PaymentRequest): Promise<FraudAnalysis> {
// Implement fraud detection logic
// This is a simplified example
let riskScore = 0;
// Check for suspicious patterns
if (request.amount > 1000) riskScore += 20;
if (request.ipAddress && this.isHighRiskIP(request.ipAddress))
riskScore += 30;
if (request.userAgent && this.isSuspiciousUserAgent(request.userAgent))
riskScore += 25;
const risk = riskScore > 70 ? "HIGH" : riskScore > 40 ? "MEDIUM" : "LOW";
return {
score: riskScore,
risk,
reasons: [], // Would contain specific fraud indicators
};
}
private isHighRiskIP(ip: string): boolean {
// Implement IP risk checking
return false;
}
private isSuspiciousUserAgent(userAgent: string): boolean {
// Implement user agent analysis
return false;
}
}
class WebhookValidator {
async validate(
provider: string,
signature: string,
payload: any,
headers: Record<string, string>
): Promise<boolean> {
// Implement signature validation for different providers
switch (provider) {
case "stripe":
return this.validateStripeSignature(signature, payload, headers);
case "paypal":
return this.validatePayPalSignature(signature, payload, headers);
default:
return false;
}
}
private validateStripeSignature(
signature: string,
payload: any,
headers: Record<string, string>
): boolean {
// Implement Stripe signature validation
return true; // Placeholder
}
private validatePayPalSignature(
signature: string,
payload: any,
headers: Record<string, string>
): boolean {
// Implement PayPal signature validation
return true; // Placeholder
}
}
class PaymentError extends Error {
constructor(
message: string,
public code: string,
public status?: number,
public metadata?: any
) {
super(message);
this.name = "PaymentError";
}
}
// Interfaces
interface PaymentManagerConfig {
primaryGateway: string;
fallbackGateways: string[];
gateways: GatewayConfig[];
fraudConfig: FraudConfig;
}
interface GatewayConfig {
name: string;
provider: string;
secretKey: string;
publicKey?: string;
webhookSecret?: string;
environment: "sandbox" | "production";
}
interface PaymentGateway {
processPayment(request: PaymentRequest): Promise<PaymentResult>;
processRefund(
request: RefundRequest & { originalPayment: any }
): Promise<RefundResult>;
tokenizePaymentMethod(
request: TokenizationRequest
): Promise<TokenizationResult>;
processWebhook(
payload: any,
headers: Record<string, string>
): Promise<WebhookResult>;
isHealthy(): Promise<boolean>;
getName(): string;
getProvider(): string;
}
interface PaymentRequest {
amount: number;
currency: string;
customerId?: string;
paymentMethodId?: string;
paymentMethodToken?: string;
description?: string;
metadata?: Record<string, any>;
ipAddress?: string;
userAgent?: string;
}
interface PaymentResult {
transactionId?: string;
gatewayTransactionId: string;
amount: number;
currency: string;
status: "pending" | "completed" | "failed";
gateway?: string;
processedAt?: number;
gatewayResponse?: any;
}
interface RefundRequest {
originalTransactionId: string;
amount?: number; // Partial refund if specified
reason?: string;
metadata?: Record<string, any>;
}
interface RefundResult {
refundId: string;
amount: number;
currency: string;
status: string;
processedAt: number;
gatewayRefundId: string;
}
interface TokenizationRequest {
customerId: string;
paymentMethodData: any;
billingDetails?: any;
}
interface TokenizationResult {
token: string;
paymentMethodType: string;
lastFour: string;
expiryMonth: number;
expiryYear: number;
brand?: string;
}
interface RecurringPaymentRequest {
tokenId: string;
amount: number;
currency: string;
customerId: string;
subscriptionId?: string;
description: string;
metadata?: Record<string, any>;
}
interface WebhookResult {
type: string;
transactionId?: string;
status: string;
metadata?: any;
}
interface FraudConfig {
enabled: boolean;
highRiskThreshold: number;
mediumRiskThreshold: number;
}
interface FraudAnalysis {
score: number;
risk: "LOW" | "MEDIUM" | "HIGH";
reasons: string[];
}
Email Service Integration: Reliable Communication
Multi-Provider Email System with Failover
// Comprehensive email service with multiple providers and advanced features
class EmailServiceManager {
private providers: Map<string, EmailProvider> = new Map();
private primaryProvider: string;
private fallbackProviders: string[];
private emailQueue: EmailQueue;
private templateEngine: EmailTemplateEngine;
private deliveryTracker: DeliveryTracker;
private unsubscribeManager: UnsubscribeManager;
constructor(config: EmailManagerConfig) {
this.primaryProvider = config.primaryProvider;
this.fallbackProviders = config.fallbackProviders || [];
this.emailQueue = new EmailQueue(config.queueConfig);
this.templateEngine = new EmailTemplateEngine(config.templateConfig);
this.deliveryTracker = new DeliveryTracker();
this.unsubscribeManager = new UnsubscribeManager();
this.initializeProviders(config.providers);
this.setupEmailProcessing();
}
private initializeProviders(providerConfigs: EmailProviderConfig[]): void {
for (const config of providerConfigs) {
let provider: EmailProvider;
switch (config.service) {
case "sendgrid":
provider = new SendGridProvider(config);
break;
case "mailgun":
provider = new MailgunProvider(config);
break;
case "ses":
provider = new SESProvider(config);
break;
case "postmark":
provider = new PostmarkProvider(config);
break;
default:
throw new Error(`Unsupported email provider: ${config.service}`);
}
this.providers.set(config.name, provider);
console.log(`Initialized ${config.service} provider: ${config.name}`);
}
}
// Send email with automatic provider failover
async sendEmail(emailRequest: EmailRequest): Promise<EmailResult> {
const messageId = this.generateMessageId();
try {
// Validate email request
this.validateEmailRequest(emailRequest);
// Check unsubscribe status
if (await this.unsubscribeManager.isUnsubscribed(emailRequest.to)) {
throw new EmailError("Recipient has unsubscribed", "UNSUBSCRIBED", {
recipient: emailRequest.to,
});
}
// Process template if provided
const processedEmail = await this.processEmailTemplate(emailRequest);
// Add tracking and unsubscribe links
const enhancedEmail = await this.enhanceEmailContent(
processedEmail,
messageId
);
// Try to send via primary provider
const primaryProvider = this.providers.get(this.primaryProvider);
if (primaryProvider && (await primaryProvider.isHealthy())) {
try {
const result = await this.sendViaProvider(
primaryProvider,
enhancedEmail,
messageId
);
await this.deliveryTracker.trackSent(messageId, result);
return result;
} catch (error) {
console.warn(
`Primary provider ${this.primaryProvider} failed:`,
error.message
);
// Try fallback providers if the error is retryable
if (this.shouldTryFallback(error)) {
return await this.tryFallbackProviders(
enhancedEmail,
messageId,
error
);
}
throw error;
}
}
// Primary provider unhealthy, try fallbacks immediately
return await this.tryFallbackProviders(
enhancedEmail,
messageId,
new EmailError("Primary provider unavailable", "PROVIDER_UNAVAILABLE")
);
} catch (error) {
await this.deliveryTracker.trackFailed(messageId, error);
throw error;
}
}
// Send bulk email with batching and rate limiting
async sendBulkEmail(bulkRequest: BulkEmailRequest): Promise<BulkEmailResult> {
const batchId = this.generateBatchId();
const batchSize = bulkRequest.batchSize || 100;
const delayMs = bulkRequest.delayBetweenBatches || 1000;
try {
const results: EmailResult[] = [];
const errors: BulkEmailError[] = [];
// Process template once for all recipients
const baseEmail = await this.processEmailTemplate({
template: bulkRequest.template,
templateData: bulkRequest.commonData || {},
} as EmailRequest);
// Split recipients into batches
const batches = this.chunkArray(bulkRequest.recipients, batchSize);
for (let i = 0; i < batches.length; i++) {
const batch = batches[i];
console.log(
`Processing batch ${i + 1}/${batches.length} (${
batch.length
} recipients)`
);
// Process batch concurrently
const batchPromises = batch.map(async (recipient) => {
try {
// Personalize email for this recipient
const personalizedEmail = await this.personalizeEmail(
baseEmail,
recipient
);
const result = await this.sendEmail(personalizedEmail);
return { success: true, result, recipient: recipient.email };
} catch (error) {
return { success: false, error, recipient: recipient.email };
}
});
const batchResults = await Promise.allSettled(batchPromises);
// Process batch results
for (const promiseResult of batchResults) {
if (promiseResult.status === "fulfilled") {
const { success, result, error, recipient } = promiseResult.value;
if (success) {
results.push(result);
} else {
errors.push({ recipient, error });
}
} else {
errors.push({
recipient: "unknown",
error: new EmailError(
promiseResult.reason.message,
"BATCH_PROCESSING_ERROR"
),
});
}
}
// Delay between batches to respect rate limits
if (i < batches.length - 1) {
await new Promise((resolve) => setTimeout(resolve, delayMs));
}
}
return {
batchId,
totalEmails: bulkRequest.recipients.length,
successCount: results.length,
failureCount: errors.length,
results,
errors,
processedAt: Date.now(),
};
} catch (error) {
throw new EmailError("Bulk email processing failed", "BULK_EMAIL_ERROR", {
batchId,
error,
});
}
}
// Queue email for later delivery
async queueEmail(
emailRequest: EmailRequest,
options: EmailQueueOptions = {}
): Promise<string> {
const messageId = this.generateMessageId();
const queuedEmail: QueuedEmail = {
messageId,
emailRequest,
queuedAt: Date.now(),
priority: options.priority || "normal",
sendAt: options.sendAt,
maxAttempts: options.maxAttempts || 3,
attemptCount: 0,
};
await this.emailQueue.enqueue(queuedEmail);
console.log(`Email queued for delivery: ${messageId}`);
return messageId;
}
// Process webhooks from email providers
async processWebhook(
provider: string,
signature: string,
payload: any,
headers: Record<string, string>
): Promise<WebhookResult> {
const emailProvider = Array.from(this.providers.values()).find(
(p) => p.getProviderName() === provider
);
if (!emailProvider) {
throw new Error(`No provider found for: ${provider}`);
}
// Validate webhook signature
const isValid = await emailProvider.validateWebhook(
signature,
payload,
headers
);
if (!isValid) {
throw new EmailError("Invalid webhook signature", "INVALID_SIGNATURE");
}
// Process webhook events
const events = await emailProvider.parseWebhookEvents(payload);
for (const event of events) {
await this.processWebhookEvent(event);
}
return {
provider,
eventsProcessed: events.length,
processedAt: Date.now(),
};
}
private async processWebhookEvent(event: EmailWebhookEvent): Promise<void> {
switch (event.type) {
case "delivered":
await this.deliveryTracker.trackDelivered(
event.messageId,
event.timestamp,
event.metadata
);
break;
case "bounced":
await this.deliveryTracker.trackBounced(
event.messageId,
event.timestamp,
event.bounceType,
event.reason
);
break;
case "complained":
await this.deliveryTracker.trackComplaint(
event.messageId,
event.timestamp,
event.metadata
);
// Automatically unsubscribe spam complainers
if (event.recipient) {
await this.unsubscribeManager.addUnsubscribe(
event.recipient,
"spam_complaint",
{ messageId: event.messageId }
);
}
break;
case "clicked":
await this.deliveryTracker.trackClicked(
event.messageId,
event.timestamp,
event.url,
event.metadata
);
break;
case "opened":
await this.deliveryTracker.trackOpened(
event.messageId,
event.timestamp,
event.metadata
);
break;
case "unsubscribed":
if (event.recipient) {
await this.unsubscribeManager.addUnsubscribe(
event.recipient,
"manual",
{ messageId: event.messageId }
);
}
break;
}
}
// Email analytics and reporting
async getEmailAnalytics(
dateRange: DateRange,
filters?: EmailAnalyticsFilters
): Promise<EmailAnalytics> {
return await this.deliveryTracker.getAnalytics(dateRange, filters);
}
private async tryFallbackProviders(
email: ProcessedEmail,
messageId: string,
primaryError: Error
): Promise<EmailResult> {
for (const fallbackName of this.fallbackProviders) {
const provider = this.providers.get(fallbackName);
if (!provider) continue;
if (await provider.isHealthy()) {
try {
console.log(`Trying fallback provider: ${fallbackName}`);
const result = await this.sendViaProvider(provider, email, messageId);
// Log successful fallback
console.log(
`Email sent via fallback provider ${fallbackName}: ${messageId}`
);
await this.deliveryTracker.trackSent(messageId, result, {
fallbackUsed: true,
fallbackProvider: fallbackName,
primaryError: primaryError.message,
});
return result;
} catch (fallbackError) {
console.warn(
`Fallback provider ${fallbackName} failed:`,
fallbackError.message
);
}
}
}
// All providers failed
throw new EmailError("All email providers failed", "ALL_PROVIDERS_FAILED", {
primaryError,
providerCount: this.fallbackProviders.length + 1,
messageId,
});
}
private async sendViaProvider(
provider: EmailProvider,
email: ProcessedEmail,
messageId: string
): Promise<EmailResult> {
const startTime = Date.now();
try {
const result = await provider.sendEmail({
...email,
messageId,
headers: {
...email.headers,
"X-Message-ID": messageId,
},
});
const duration = Date.now() - startTime;
return {
messageId,
providerMessageId: result.providerMessageId,
provider: provider.getProviderName(),
status: "sent",
sentAt: Date.now(),
duration,
recipient: email.to,
};
} catch (error) {
const duration = Date.now() - startTime;
throw new EmailError(
error.message,
error.code || "PROVIDER_ERROR",
error.status,
{
provider: provider.getProviderName(),
messageId,
duration,
originalError: error,
}
);
}
}
private async processEmailTemplate(
emailRequest: EmailRequest
): Promise<ProcessedEmail> {
if (emailRequest.template) {
return await this.templateEngine.render(
emailRequest.template,
emailRequest.templateData || {}
);
}
return {
to: emailRequest.to,
from: emailRequest.from,
subject: emailRequest.subject!,
html: emailRequest.html,
text: emailRequest.text,
headers: emailRequest.headers || {},
attachments: emailRequest.attachments || [],
};
}
private async enhanceEmailContent(
email: ProcessedEmail,
messageId: string
): Promise<ProcessedEmail> {
let enhancedHtml = email.html || "";
let enhancedText = email.text || "";
// Add tracking pixel for open tracking
if (enhancedHtml) {
const trackingPixel = `<img src="${process.env.TRACKING_URL}/open/${messageId}" width="1" height="1" alt="" style="display:none;" />`;
enhancedHtml += trackingPixel;
}
// Add unsubscribe link
const unsubscribeUrl = `${process.env.UNSUBSCRIBE_URL}/${messageId}`;
const unsubscribeHtml = `<p><a href="${unsubscribeUrl}">Unsubscribe</a></p>`;
const unsubscribeText = `\n\nUnsubscribe: ${unsubscribeUrl}`;
if (enhancedHtml) {
enhancedHtml += unsubscribeHtml;
}
if (enhancedText) {
enhancedText += unsubscribeText;
}
return {
...email,
html: enhancedHtml,
text: enhancedText,
headers: {
...email.headers,
"List-Unsubscribe": `<${unsubscribeUrl}>`,
"List-Unsubscribe-Post": "List-Unsubscribe=One-Click",
},
};
}
private async personalizeEmail(
baseEmail: ProcessedEmail,
recipient: EmailRecipient
): Promise<EmailRequest> {
return {
to: recipient.email,
from: baseEmail.from,
subject: this.personalizeString(baseEmail.subject, recipient.data),
html: this.personalizeString(baseEmail.html || "", recipient.data),
text: this.personalizeString(baseEmail.text || "", recipient.data),
headers: baseEmail.headers,
attachments: baseEmail.attachments,
};
}
private personalizeString(
template: string,
data: Record<string, any>
): string {
let result = template;
// Simple template variable replacement
for (const [key, value] of Object.entries(data)) {
const pattern = new RegExp(`{{\\s*${key}\\s*}}`, "g");
result = result.replace(pattern, String(value));
}
return result;
}
private setupEmailProcessing(): void {
// Process queued emails
setInterval(async () => {
try {
await this.processQueuedEmails();
} catch (error) {
console.error("Error processing queued emails:", error);
}
}, 10000); // Every 10 seconds
// Health check providers
setInterval(async () => {
for (const [name, provider] of this.providers) {
try {
const isHealthy = await provider.isHealthy();
if (!isHealthy) {
console.warn(`Email provider ${name} is unhealthy`);
}
} catch (error) {
console.error(`Health check failed for provider ${name}:`, error);
}
}
}, 60000); // Every minute
}
private async processQueuedEmails(): Promise<void> {
const emails = await this.emailQueue.dequeueReady();
for (const queuedEmail of emails) {
try {
await this.sendEmail(queuedEmail.emailRequest);
await this.emailQueue.markCompleted(queuedEmail.messageId);
} catch (error) {
queuedEmail.attemptCount++;
if (queuedEmail.attemptCount >= queuedEmail.maxAttempts) {
await this.emailQueue.markFailed(queuedEmail.messageId, error);
} else {
// Requeue with exponential backoff
const delay = Math.pow(2, queuedEmail.attemptCount) * 60000; // Minutes
await this.emailQueue.requeueWithDelay(queuedEmail.messageId, delay);
}
}
}
}
// Utility methods
private shouldTryFallback(error: any): boolean {
const fallbackErrors = [
"RATE_LIMITED",
"PROVIDER_UNAVAILABLE",
"TIMEOUT",
"TEMPORARY_ERROR",
];
return fallbackErrors.includes(error.code) || error.status >= 500;
}
private validateEmailRequest(request: EmailRequest): void {
if (!request.to) {
throw new EmailError("Missing recipient email", "MISSING_RECIPIENT");
}
if (!this.isValidEmail(request.to)) {
throw new EmailError("Invalid recipient email", "INVALID_EMAIL");
}
if (!request.subject && !request.template) {
throw new EmailError("Missing subject or template", "MISSING_SUBJECT");
}
if (!request.html && !request.text && !request.template) {
throw new EmailError("Missing email content", "MISSING_CONTENT");
}
}
private isValidEmail(email: string): boolean {
const emailRegex = /^[^\s@]+@[^\s@]+\.[^\s@]+$/;
return emailRegex.test(email);
}
private chunkArray<T>(array: T[], size: number): T[][] {
const chunks = [];
for (let i = 0; i < array.length; i += size) {
chunks.push(array.slice(i, i + size));
}
return chunks;
}
private generateMessageId(): string {
return `msg_${Date.now()}_${Math.random().toString(36).substr(2, 9)}`;
}
private generateBatchId(): string {
return `batch_${Date.now()}_${Math.random().toString(36).substr(2, 9)}`;
}
}
// Example SendGrid provider implementation
class SendGridProvider implements EmailProvider {
private apiClient: RobustAPIClient;
constructor(private config: EmailProviderConfig) {
this.apiClient = new RobustAPIClient({
baseURL: "https://api.sendgrid.com",
serviceName: "sendgrid-email",
version: "1.0.0",
defaultHeaders: {
Authorization: `Bearer ${config.apiKey}`,
"Content-Type": "application/json",
},
circuitBreakerOptions: {
failureThreshold: 5,
resetTimeoutMs: 60000,
},
rateLimitOptions: {
maxTokens: 100,
refillIntervalMs: 1000,
refillAmount: 10,
},
cacheOptions: {},
});
}
async sendEmail(
email: ProcessedEmail & { messageId: string }
): Promise<ProviderEmailResult> {
try {
const sendGridPayload = {
personalizations: [
{
to: [{ email: email.to }],
subject: email.subject,
},
],
from: { email: email.from },
content: [
...(email.text ? [{ type: "text/plain", value: email.text }] : []),
...(email.html ? [{ type: "text/html", value: email.html }] : []),
],
custom_args: {
message_id: email.messageId,
},
tracking_settings: {
click_tracking: { enable: true },
open_tracking: { enable: true },
},
};
const response = await this.apiClient.post(
"/v3/mail/send",
sendGridPayload
);
return {
providerMessageId: response.headers["x-message-id"] || email.messageId,
status: "accepted",
};
} catch (error) {
throw this.transformSendGridError(error);
}
}
async validateWebhook(
signature: string,
payload: any,
headers: Record<string, string>
): Promise<boolean> {
// Implement SendGrid webhook signature validation
return true; // Placeholder
}
async parseWebhookEvents(payload: any): Promise<EmailWebhookEvent[]> {
const events: EmailWebhookEvent[] = [];
for (const event of payload) {
events.push({
type: this.mapSendGridEventType(event.event),
messageId: event.sg_message_id,
recipient: event.email,
timestamp: event.timestamp,
url: event.url,
reason: event.reason,
bounceType: event.type,
metadata: event,
});
}
return events;
}
async isHealthy(): Promise<boolean> {
try {
const response = await this.apiClient.get("/v3/user/account", {
timeout: 5000,
});
return response.status === 200;
} catch (error) {
return false;
}
}
getProviderName(): string {
return "sendgrid";
}
private mapSendGridEventType(eventType: string): string {
const eventMap: Record<string, string> = {
delivered: "delivered",
bounce: "bounced",
dropped: "bounced",
spamreport: "complained",
click: "clicked",
open: "opened",
unsubscribe: "unsubscribed",
};
return eventMap[eventType] || eventType;
}
private transformSendGridError(error: any): EmailError {
const sgError = error.response?.data?.errors?.[0] || {};
return new EmailError(
sgError.message || error.message,
sgError.field === "from.email" ? "INVALID_FROM_EMAIL" : "SENDGRID_ERROR",
error.status,
{ sendgridError: sgError, originalError: error }
);
}
}
// Supporting classes would be implemented similarly...
// Interfaces for email system
interface EmailManagerConfig {
primaryProvider: string;
fallbackProviders: string[];
providers: EmailProviderConfig[];
queueConfig: any;
templateConfig: any;
}
interface EmailProviderConfig {
name: string;
service: string;
apiKey: string;
webhook_url?: string;
sandbox?: boolean;
}
interface EmailProvider {
sendEmail(
email: ProcessedEmail & { messageId: string }
): Promise<ProviderEmailResult>;
validateWebhook(
signature: string,
payload: any,
headers: Record<string, string>
): Promise<boolean>;
parseWebhookEvents(payload: any): Promise<EmailWebhookEvent[]>;
isHealthy(): Promise<boolean>;
getProviderName(): string;
}
interface EmailRequest {
to: string;
from: string;
subject?: string;
html?: string;
text?: string;
template?: string;
templateData?: Record<string, any>;
headers?: Record<string, string>;
attachments?: EmailAttachment[];
}
interface ProcessedEmail {
to: string;
from: string;
subject: string;
html?: string;
text?: string;
headers: Record<string, string>;
attachments: EmailAttachment[];
}
interface EmailResult {
messageId: string;
providerMessageId: string;
provider: string;
status: string;
sentAt: number;
duration: number;
recipient: string;
}
interface BulkEmailRequest {
template: string;
commonData?: Record<string, any>;
recipients: EmailRecipient[];
batchSize?: number;
delayBetweenBatches?: number;
}
interface EmailRecipient {
email: string;
data: Record<string, any>;
}
interface BulkEmailResult {
batchId: string;
totalEmails: number;
successCount: number;
failureCount: number;
results: EmailResult[];
errors: BulkEmailError[];
processedAt: number;
}
interface BulkEmailError {
recipient: string;
error: Error;
}
interface EmailQueueOptions {
priority?: "high" | "normal" | "low";
sendAt?: number;
maxAttempts?: number;
}
interface QueuedEmail {
messageId: string;
emailRequest: EmailRequest;
queuedAt: number;
priority: string;
sendAt?: number;
maxAttempts: number;
attemptCount: number;
}
interface EmailWebhookEvent {
type: string;
messageId: string;
recipient?: string;
timestamp: number;
url?: string;
reason?: string;
bounceType?: string;
metadata?: any;
}
interface ProviderEmailResult {
providerMessageId: string;
status: string;
}
interface EmailAttachment {
filename: string;
content: string;
type: string;
disposition?: string;
}
interface DateRange {
startDate: Date;
endDate: Date;
}
interface EmailAnalyticsFilters {
provider?: string;
template?: string;
recipient?: string;
}
interface EmailAnalytics {
totalSent: number;
delivered: number;
bounced: number;
opened: number;
clicked: number;
complained: number;
unsubscribed: number;
deliveryRate: number;
openRate: number;
clickRate: number;
complaintRate: number;
}
interface WebhookResult {
provider: string;
eventsProcessed: number;
processedAt: number;
}
class EmailError extends Error {
constructor(
message: string,
public code: string,
public status?: number,
public metadata?: any
) {
super(message);
this.name = "EmailError";
}
}
// Placeholder classes that would need full implementation
class EmailQueue {
constructor(config: any) {}
async enqueue(email: QueuedEmail): Promise<void> {}
async dequeueReady(): Promise<QueuedEmail[]> {
return [];
}
async markCompleted(messageId: string): Promise<void> {}
async markFailed(messageId: string, error: Error): Promise<void> {}
async requeueWithDelay(messageId: string, delayMs: number): Promise<void> {}
}
class EmailTemplateEngine {
constructor(config: any) {}
async render(
templateName: string,
data: Record<string, any>
): Promise<ProcessedEmail> {
return {} as ProcessedEmail;
}
}
class DeliveryTracker {
async trackSent(
messageId: string,
result: EmailResult,
metadata?: any
): Promise<void> {}
async trackDelivered(
messageId: string,
timestamp: number,
metadata?: any
): Promise<void> {}
async trackBounced(
messageId: string,
timestamp: number,
bounceType?: string,
reason?: string
): Promise<void> {}
async trackComplaint(
messageId: string,
timestamp: number,
metadata?: any
): Promise<void> {}
async trackClicked(
messageId: string,
timestamp: number,
url?: string,
metadata?: any
): Promise<void> {}
async trackOpened(
messageId: string,
timestamp: number,
metadata?: any
): Promise<void> {}
async trackFailed(messageId: string, error: Error): Promise<void> {}
async getAnalytics(
dateRange: DateRange,
filters?: EmailAnalyticsFilters
): Promise<EmailAnalytics> {
return {} as EmailAnalytics;
}
}
class UnsubscribeManager {
async isUnsubscribed(email: string): Promise<boolean> {
return false;
}
async addUnsubscribe(
email: string,
reason: string,
metadata?: any
): Promise<void> {}
}
Key Takeaways
Third-party integration isn’t just about making HTTP requests—it’s about building resilient communication layers that can handle the unpredictable nature of services you don’t control.
Essential integration patterns:
- Robust API clients with circuit breakers, retry logic, and comprehensive error handling
- Payment processing with multi-gateway fallback and fraud detection
- Email services with provider failover and delivery tracking
- Webhook processing with signature validation and event handling
- Health monitoring for all external dependencies
The production-ready integration framework:
- Use comprehensive error handling that distinguishes between retryable and permanent failures
- Implement intelligent fallback strategies when primary services fail
- Build rate limiting and circuit breakers to protect both your system and external APIs
- Design idempotent operations that can be safely retried
- Plan graceful degradation when third-party services are unavailable
Integration best practices:
- Always expect failures from external services and design accordingly
- Monitor third-party service health as part of your system monitoring
- Implement comprehensive logging for troubleshooting integration issues
- Use correlation IDs to trace requests across service boundaries
- Cache responses when appropriate to reduce external API calls
The architecture decision framework:
- Use direct API integration for real-time, low-latency requirements
- Use queue-based integration for high-volume, eventually consistent operations
- Use webhook processing for event-driven updates from external services
- Use batch processing for bulk operations and data synchronization
- Use fallback services for critical functionality that must always work
What’s Next?
In the next blog, we’ll complete our third-party integration journey by diving into webhook handling and processing, API rate limiting and retry strategies, service reliability and fallbacks, integration testing strategies, and monitoring external dependencies.
We’ll explore the operational challenges of maintaining integrations at scale—from handling webhook floods during traffic spikes to debugging why a critical API integration started failing at 3 AM.
Because building integrations that work in development is straightforward. Making them bulletproof enough to handle the chaos of production traffic while maintaining your sanity—that’s where third-party integration becomes an art form.