From Microservices to Serverless: How to avoid converting “Distributed monolith” microservices into “Serverless monoliths”
Learning from the past: converting a monolith into… a worse monolith
When microservices became mainstream, a lot of companies started to migrate their monolithic systems to a distributed microservice architecture. The advantages were clear:
- Increased scalability
- Flexibility when deploying
- The ability to choose the right technology (or language) for a specific use case
- Isolation of higher priority services from lower priority ones to prevent cascading failures across the system
- Etc.
But as usually happens when you work with a technology for the first time and are dragged in by the hype, many of the early microservice implementations were not done properly.
The same people who created optimal architectures for monolithic systems tried to reuse that knowledge and experience to create microservice architectures. They broke the existing code into logical pieces without thinking about dependencies and replaced direct method calls with HTTP requests. This led to what is often called a “distributed monolith”:
- Instead of having an application with several modules coupled together with method calls made through the memory space of a computer…
- … we now have an application with several microservices coupled together with HTTP calls made through a network.
As you’ve probably guessed, in both cases we have practically the same architecture, but our new microservice version is even worse: it is more fragile because we are using the network as the “glue” to stick our modules together, which is way less reliable than direct method calls in the same machine.
Does this mean that microservices architectures are a lie and they are even worse than monolithic architectures?
No, microservices are just a specific tool to solve specific problems and, as with any tool, you need to understand how to use it correctly. Using a screwdriver to pound a nail could work for an emergency, but you’ll probably end up worse than you started: you’ll break the screwdriver and the nail won’t be properly pounded in. Is the screwdriver a useless tool? Of course not.
In order to use the microservices “tool” properly, you need to create the right architecture for it, which is quite different from architectures in monoliths. These right architectures usually have:
- Microservices that can actually work in isolation because they make sense by themselves as smaller systems
- Minimal direct dependencies
- Denormalized data (done right)
- Service discoverability
- Fault tolerance
- Event-driven (reactive) communication
Stumble over the same stone twice
Imagine we are one of those companies that migrated to a “distributed monolith” and we are suffering its consequences. We now know it was an error we made in the past, but we’ve learned and we won’t make the same mistake again.
Now all the hype is moving to Serverless (which could be seen as an extreme form of microservices), but this time it’s real! Serverless is a much more flexible tool and is even more advantageous than microservices:
- You can forget about handling the infrastructure. Serverless = No server management!
- Infinite scalability: Lambdas (or functions as a service, FaaS) are fully parallelizable and are created by the provider on the fly when needed
- Cost savings: you don’t pay for resources you don’t use (idle time is not charged)
- Fine-grained control over use cases as each one has its own isolated FaaS
- It embraces reusability of third-party services for typical use cases (Auth0, DynamoDB, S3, Kinesis, etc.)
OK, time to move our current “distributed monolith” architecture to Serverless using Amazon Web Services. So we do something like the following:
- Create an AWS account
- Split the microservices into Lambdas. There are several approaches to do this:
- One Lambda = one microservice
- One Lambda = one endpoint with all its verbs (GET, POST, PUT, DELETE…)
- One Lambda = one verb of one endpoint - Migrate the logging to CloudWatch
- Set up alerts
- Configure security and IAM roles properly
- Migrate the data layer to one of the Amazon solutions (S3, DynamoDB…)
And that’s it! We now have a full-fledged Serverless application instead of a crappy “distributed monolith.” Let’s publish an article about how our company has embraced Serverless and how easy it is to migrate to Serverless from a microservices architecture.
This time it was easier than we expected!
Welcome to the new “Serverless monolith” era
As you might have guessed, we have made the same mistakes again because we used the same mindset and design patterns from the old monolithic or microservice architecture to create a Serverless architecture.
What happened to the coupling between our microservices? It’s still there! We have just replaced “microservice” with “Lambda.” Coupling “lives” in the way you communicate between different parts of your system, not in the technology that underlies it.
Monolithic Application ≈ Distributed Monolith ≈ Serverless Monolith
OMG, we screwed up again. Why? Because we didn’t know that we didn’t know how to use the new tool that we got.
We learned that screwdrivers were not the proper tool to pound nails, so we bought a drill and smashed the nail with it. Did we meet our objective? Kind of, but not really.
What we keep missing is that, although we can use a hammer, a screwdriver, and a drill to join two pieces of wood together, we can’t use a nail with all of them.
- A hammer requires a nail. You bang it.
- A screwdriver requires a bolt. You screw it.
- A drill requires a whole new process to approach the task.
In the same way, Serverless requires a complete rethinking of our architecture so that the benefits outweigh the downsides (because, yeah! Serverless has downsides too).
Ideally, serverless architectures are married to event-driven architectures. This means that our Lambda functions won’t depend directly on others. Instead, they will emit an event to some kind of “event-bus” (Amazon SNS, Kinesis, RabbitMQ, etc.) and then they are done. Another Lambda can react to that event and do further processing, but they are not coupled anymore. If this latter Lambda fails, the first one doesn’t even care.
This is how we get fully isolated components, high fault tolerance, and amazing scalability.
To conclude
- Study and know your tools before using them.
- The fact that an architecture worked very well with one tool doesn’t mean that it will work well with other tools.
- Serverless is a very powerful tool. Learning to use it well requires a change of mindset. On the other hand, Serverless is a very complex tool. Take special care to decide whether you really need it for the problem you want to solve.
I would like to include a special mention of my teammates and colleagues Javier Toledo, Dana Peele, José Manuel Martínez, and Abraham Romero for their help reviewing this article and providing suggestions. Thank you very much!
