
A collection of hands-on lessons based upon the authors' considerable experience in enterprise integration, the 65 patterns included with this guide show how to use message-oriented middleware to connect enterprise applications.
Growing up as an integration engineer, I built my career religiously reading the Enterprise Integration Patterns book from Gregor Hohpe. Content-Based Router, Dead Letter Channel, Claim Check these weren't just patterns, they were how I thought about systems. That book shaped an entire generation of us who built middleware, ESBs, and messaging infrastructure.
Two decades later, I found myself staring at a eerily familiar set of problems but with a completely different consumer. Not applications. Not services. AI agents.
It's been an exciting and one-of-its-kind experience working with the Arcade tool engineering team to extract implementation patterns from 7,000+ production tools. Especially Evan Tahler Renato Byrro & Francisco Liberal
54 patterns across 10 categories, so everyone building tools and MCP servers can ship higher quality tools for their agents.
A couple things that make this practical:
Every pattern downloads as a flash card. Drop them into your architecture decks, design reviews, team workshops.
Every pattern exports as markdown / llms.txt for your IDE.
The workflow: point your coding agent at one or more patterns and say "build an MCP server for this API."
The pattern becomes your implementation spec.
Blog: https://blog.arcade.dev/mcp-tool-patterns
Full catalog: https://arcade.dev/patterns
Another one I found very helpful is Designing Data-Intensive Applications by Martin Kleppmann.
We support streaming to Kafka and NATS. If you don’t have a messaging system, you can use our serverless hosted one. To get your data at rest, you can stream to databases like Postgres.
I (Anthony) joined IFTTT in the early 2010s. I worked on a lot of integrations there. At the time, it was clear the landscape was a mess. Yet, more than a decade later, nothing’s changed.
A common misconception is that integrating is about making API requests. Or running background jobs that paginate endpoints. But that's just API extraction. After extraction is when the real integration starts. The integration is all the custom business logic that determines how your system should use that data or respond to it changing. [1]
Big companies have to integrate hundreds of systems. They use message buses. That’s because message buses are the best way to integrate. [2]
But what happens if the system is a 3rd-party API? How do you get data into your message bus? You need to extract.
Unfortunately, most APIs stop at REST. But REST sucks for extraction (have you tried detecting deletes?)
ETL tools like Fivetran and Airbyte help with extraction. But they're built for data engineers, not for developers. This is why ETL tools are not real-time, have poor support for messaging systems, are not programmable, and don’t support writes.
Integration platforms like Zapier and Tray.io can help you wire up external systems. But this means writing business logic in their WYSWIGs or DSLs. Developers want to maintain business logic in their code and tools (e.g. git).
The vision is to get developers to stop writing undifferentiated and tedious extraction code. We want every developer to be able to integrate with message buses – no matter the project. When a messaging system is your backbone, integrations throughout your system and across many external APIs all feel the same. Most of the differences between APIs are in frivolous details. The differences between ServiceNow and NetSuite's query parameters should not matter to you. When the API is just data, these differences disappear.
While APIs haven’t evolved much since the early 2000s, message systems have. Companies like Upstash, WarpStream, and Synadia offer serverless message systems. They’re cheap to get started with and easy to scale.
Along with partnering with these providers, we offer our own serverless hosted message bus [3]. This uses familiar message queue semantics.
Sequin uses a number of strategies for each API to extract data in real-time. You don't need to stub out APIs in your tests or setup ngrok to receive webhooks. Both test and dev just need to read messages off a queue or records in a database. And we offer a uniform interface for writes. Writes are committed to both the API and your message stream simultaneously, so you can “read your writes.”
Sequin's early, so it’s not perfect. We don't have any onboarding in our product. We want to add a lot more sources. We need better graphs and insights for observability and monitoring. But we pride ourselves on speed and consistency. We know that missing even a single event erodes trust in our event streams, so we work hard to eliminate that possibility.
We'd love to hear your thoughts and questions!
[1] https://martinfowler.com/articles/cant-buy-integration.html
[2] Using message buses to integrate goes all the way back to 2003's classic "Enterprise Integration Patterns" (Hohpe and Woolf)!
[3] https://sequin.io/docs/stream-apis/consume/overview - Uses NATS JetStream under the hood
You would have an idempotent queue + a circuit breaker pattern. If the queue depth is too large - you break the circuit.
But if your profit per request is higher than your infracost per request - why not autoscale till the cows come home? Within reason of course.
both predate microservices, and both are still very useful today.
The first two books I consider almost a must read. Actually it's not about reading them but learning the patterns. More about this here: https://www.industriallogic.com/papers/learning.html