Is Platform Engineering the New DevOps or SRE?
What is platform engineering and why might it be key to good developer experiences?
Almost every day we hear about another organization building an internal developer platform or developer control plane. We’re not alone in observing this trend in platform engineering; when Humanitec polled 1,850 engineering organizations in 2021, they found that the majority are already building, or plan to build, their own internal developer platforms.
And why not? Reducing complexity while enabling developer self-service is always a popular move, particularly when it works. While most developer platforms and control planes are built with these goals at their core, getting the balance right for the majority of developers can prove to be challenging.
Platform Engineering and Developer Control Planes: If You Build it, Pave the Path
The modern developer platform idea crystallized and moved from concept to reality (for me) thanks to these kinds of industry observations, a wealth of anecdotal evidence from consulting work I was doing back in ~2015, and most of all, a Netflix OSS team talk on providing a developer-focused “paved path” to shipping and running apps quickly and safely.
Netflix led the way in creating an internal developer platform and supporting toolchain, ensuring that developers enjoyed a low-friction, ops-supported platform and processes. This kickstarted a kind of DevOps evolution, largely led by the Netflix motion toward “full cycle developers”. In this new paradigm, developers became responsible for the full code, ship, and run life cycle of their applications, that is, a “you build it, you run it” model. While this exact approach has not been adopted wholesale, some variation on developer ownership of the full life cycle (or at least a deep understanding of it) has gained traction across the industry.
A range of “paths” do exist. It can be a platform engineering team and a site reliability engineering team working to empower developer teams. Or organizations can implement a Google-like approach in which application developers are enabled to focus on their core competencies without worrying too much about deploying and running their software. A core part of the paved platform engineering path is helping developers to have some understanding of the infrastructure and the roles of SREs and DevOps without necessarily making them do those jobs in whole themselves.
Taking the full landscape into consideration, I compiled a range of resources about platform engineering, what it is, where trends are heading, and why it matters to cloud-native engineering organizations.
The result went viral and confirmed my suspicions: software engineering organizations are all grappling with and aiming to reduce complexity and lessen cognitive load for their developers. A great deal of crossover is happening between previously siloed teams, with platform engineering, site reliability engineering (SRE), and developer experience/enablement (DX) talking to each other, united by a common mission (as outlined in the Google SRE book): eliminating toil for developers to enable faster, safer shipping and running of software.
The Evolution of Today’s Platform Engineering and Developer Control Plane
Taking a look back, it’s clear that the developer control plane and platform engineering are not new concepts. It has just evolved with technology. We’ve been building various types of applications and web platforms for years. The lineage goes back to on-premise, bare-metal, ticket-driven platforms, extends to first-generation one-size-fits-all-style platform-as-a-service (PaaS) models, and is evolving into a next-generation custom PaaS experience — self-service (facilitated by platform and ops teams), container-based, with fast feedback loops, and above all, an emphasis on providing a good developer experience.
The tricky part of this evolution is identifying what exactly constitutes a good developer experience and being able to pave the developer path in a flexible and customized way. Some organizations are prepared to give developers the reins to drive the full software life cycle; others aren’t and want some guardrails and guidelines. Both models are perfectly okay. After all, “the developer experience” is no more monolithic than the microservices architectures these platforms now aim to simplify.
A more important consideration in the platform engineering space is to establish understanding. First understand what your organization is trying to achieve to help shape the abstractions your platform includes. But before building anything, talk to your developers to understand them and find out what they need.
Understand Platform Engineering Goals and Your Developers: The Brass Tacks of Paving the Path
If the platform engineer’s role is to “examine the entire software development lifecycle from source to production and build a workflow that enables application developers to rapidly code and ship software”, the platform must be built first to accommodate the actual needs of the organization. Secondly, it must be built to accommodate the vast majority of developers, that is, the 99% who build real-world applications.
Most organizations are not Netflix, Google, or aspiring unicorns, and in building a developer platform, shouldn’t strive to be. Instead they should shift focus away from what “developer influencers” talk about and focus on building platforms and tooling for the daily reality of developers in their own organizations. The 99% work on the software that powers everything around us: banking, retail, insurance, healthcare, and more. It would be neither realistic nor wise to disrupt most of the legacy software practices that underpin these fundamental services without stable innovations and good reasons to do so.
In the real world, we’ve seen many good examples of how companies have successfully taken the right steps for their own organizations and developers. An organization invests in a platform because they expect a certain value as ROI. And the way the platform is built should be driven by what the ultimate business goals are. If it’s about developer productivity and achieving cost savings, the platform will be built one way, but if the goal is innovation, it will be structured in another way. Teams, their roles, and organizational and cultural decisions follow.
Business goals, needs and budgetary constraints, in addition to legacy infrastructure, inform decision-making in these business-critical situations in which the 99% are dominant. For example, Bo Daley, as DevOps platform engineer from Zipcar, described his experience of evolving the organization’s thinking on delivering developer support. It moved from a more traditional DevOps approach toward building a centralized platform that would give developers a paved path from coding locally to production more efficiently.
Another example is Alan Barr, Internal Developer and Security Platform Product Owner at Veterans United Home Loans, who challenged the status quo in the relatively staid mortgage business to secure buy-in for a centralized developer control plane. His path to success was not paved by selling in innovative technology but by building a business case around unlocking developer efficiency, focusing on customer needs, controlling costs, automating processes, and ensuring security.
The Delicate Top-down, Bottom-up Balance of Custom Developer Platforms
You can build it, but they might not come.
You may have eschewed the shiny future proposed by developer influencers and focused squarely on the business case. You may have secured all the top-down internal buy-in needed. You may have gone all-in on platform engineering as a way to get the efficiency you want for your developers. You may in fact have built a platform that looks, from the outside, to be the magic developer platform that can help engineers self-serve everything they need to code, ship and run their applications.
But if the motion is 100% top-down, and developers have had no input into the way a custom developer platform is built, they cannot reasonably be expected to like or use it. This approach is a bad idea, and a recipe for things going wrong. What is the point of having the flexibility to customize a platform for developers if they are not involved in what you build? It is not surprising that developers don’t adopt a platform that has not included their feedback and preferred workflows.
How does a platform team tackle the bottom-up motion and capture the developer perspective and needs in the platform experience? Always listen to the platform customer (i.e., developers) and recognize that the only way to succeed with platform adoption today is via a bottom-up, developer-led approach. Focus on the developer experience.
Value the Human, Reap the Rewards
We’re coming full circle now to the concept of investing in different forms of understanding to create effective developer platforms.
Listen to your audience (developers) and understand their needs. For example, if you’re building a platform to reduce toil, increase productivity and lessen cognitive load for your developers, it’s probably the 99% of developers you need to build for. Take a practical and hands-on approach.
Create a two-way understanding: You deliver a platform that meets developer needs and provides a good developer experience; the developer agrees that regardless of how much of the full life cycle they need to own, they need to understand it fully. This requires mechanical sympathy and a willingness to be responsible for their code and its potential downstream effects of what they have put into the software.
Be human, encourage empathy: Kelsey Hightower has spoken extensively about empathy for the end user and elevating it as a part of the software development process. This requires developing soft skills, like communication and putting oneself into the user’s shoes.
Remember that developers are human: Extending Kelsey Hightower’s empathetic engineering concept, Alan Barr also highlighted Twilio’s “hospitality” focus as an example of creating a developer experience (and platform) that a developer would like to use. That is, remove all the cruft, unblock the road, and remove challenges that stand in the way of developers doing their best work.
Conclusion: Platform Engineering for a Flexible Developer Control Plane: Paved Path, Middle Ground
What does a developer control plane look like? For the 99% of developers who just want to do their jobs but don’t want to be pushed into doing things and using tools they don’t like, the developer control plane doesn’t have to be that different from the platform for full lifecycle code-run-ship developers. The whole point is that there is no one-size-fits-all approach. Instead there is a flexible middle ground, and the eventual platforms created can look different in different implementations.
Ultimately the middle ground is a control plane that enables some top-down centralization, tooling options, visibility and opinions (a paved path) that provides one way of shipping software with speed and safety. But at the same time, the same platform won’t clip another developer’s wings and can enable developer ownership of their own tooling choices, as long as they take responsibility for the outcome.
And the outcome is usually whether or not end users adopt and use your software. Making the right platform engineering choices on a developer platform can help or hinder this.
