Guide to Platform Engineering: Everyone’s Doing It, Should You Be Too?

Platform engineering isn’t brand new, but it has been getting a lot of airtime recently. It made the Gartner® Hype Cycle™ for Software Engineering in 2022 and has been suggested as the evolution of DevOps but, much like DevOps, the industry has yet to converge on a definition of platform engineering. The lack of consensus can make it tricky for organizations to develop their own platform engineering practice. Where do you start when the playbook is still being written?

In this post we’ll take a look at what platform engineering is, the benefits it can bring, and how it differs from DevOps and site reliability engineering (despite these terms often being used interchangeably). We’ll also explore how to decide if your organization should invest in an internal developer platform, and if so, how to go about building your platform engineering team. 

What is platform engineering?

The common theme of platform engineering is around integrating the tools and designing and creating the workflows that enable self-service capabilities for software developers, often focusing on cloud computing services (Kubernetes) and Infrastructure as Code tools such as Terraform. While your development teams work on building and improving your product, to the platform engineering team, your internal developer platform is the product.

"A platform is the sum of all tech and tools that a platform engineering team binds together to pave a golden path or paths. Developers leverage this path to self-serve with low cognitive load."—Kaspar von Grünberg, CEO of Humanitec

Low cognitive load is a key concept here: one of the common gripes with DevOps is that developers don’t want to do operations work in addition to their existing jobs. They’re reluctant to learn the intricacies of infrastructure provisioning, continuous integration pipelines, and container orchestration. 

In small companies, team members are often expected (and expect) to straddle a number of roles, but as an organization grows this isn’t scalable. It’s just not efficient for one team member to be writing code and business logic, as well as provisioning servers and architecting deployment strategies. A platform engineering team can set up and maintain the elements that make up your build and software delivery processes so that developers don’t have to become experts in these to self-serve them. 

With a robust internal developer platform, your development teams can focus on application development, thanks to the automation of repetitive tasks and streamlining of common workflows. 

Note: internal developer platforms should not be confused with Platform as a Service (PaaS) products. These are provided by third-party vendors and therefore aren’t built as a custom internal developer platform for your organization’s specific needs. 

How is platform engineering different from DevOps?

DevOps emerged over a decade ago and so is more established than platform engineering. DevOps is a set of practices that aim to eliminate the twin silos of development and operations in software delivery by improving cross-functional collaboration and automating manual tasks. Many see platform engineering as an extension or natural evolution of DevOps. The outcomes of successful DevOps implementation and platform engineering are largely the same—less time spent on toil and a smoother path to software delivery—but the methods to get there look slightly different. While DevOps can improve coordination between development and operations, an effective internal developer platform reduces the need for coordination in the first place.

How is platform engineering different from site reliability engineering? 

These two concepts are also related: both are about creating and maintaining systems, but while platform engineers are responsible for building and evolving the internal developer platform, site reliability engineers (SREs) are focused on the systems that run applications, automatically and reliably. You could say that platform engineering serves developers, while SRE serves operations teams.  

The benefits of platform engineering

With an internal developer platform, software engineers can take more ownership of their applications beyond just writing the code, without having to learn everything about continuous integration or container orchestration—these operations are automated via the platform. And these self-service capabilities have a number of effects:

Faster, more frequent releases

The platform paves the way for continuous delivery, which automates the entire software delivery process. Developers can move more quickly since they’re able to focus on writing code. They’re not spending time learning how to do operations tasks or relying on a dedicated operations team to enable them. 

Standardized workflows

Platform engineering can help to bring consistency to your organization’s toolchain and workflows. For example, Spotify shares how they reduce fragmentation in their software ecosystem though “golden paths,” their recommended (although not mandated), supported methods for achieving something, while still leaving room for edge cases. 

There can also be drawbacks to a prescriptive, internal developer platform. Platform engineering efforts can backfire if you try to force everyone to use the platform. 

“Golden paths should be the easiest path, the best one, the one that covers most of your cases. But it's almost impossible to avoid exceptions. And if developers don't have those exit routes to do some exceptions, they will be stuck on your golden path. And it's no longer a golden path; it's a golden cage or a golden tunnel.”— Mathieu Frenette, Director of DevOps, Nesto 

Those exceptions and the exit routes are useful signals: you may be able to build and integrate the solutions for them into the overall platform.

Better developer experience

We touched on the idea of “cognitive load” above, and Puppet’s 2021 State of DevOps report also notes that “highly evolved teams tend to do a good job of limiting extraneous cognitive load on delivery teams (through good practices, automation, and support from other teams).” Reducing painful, manual tasks and the resulting mental costs is critical for developers to spend more time in flow state, resulting in happier team members and better retention.

Does your org need a platform engineering team?

The State of DevOps report cites platform engineering as “key to success at scale.” Not every engineering organization needs or will benefit from a platform engineering team … yet. If yours is an early-stage startup or small-scale company, you may not need a dedicated internal developer platform. With no legacy systems to maintain or complex migrations to manage, your engineering time is probably better spent on building and shipping new features and functionality for your customers. 

Your lean team may already be comfortable with cloud services and other new technology, so they’re able to self-serve without the support of a platform. With a small number of engineers working closely together, you probably don’t have the enterprise-scale challenge of people duplicating efforts to solve common problems over and over. 

As you scale though, the pressure to deliver new features increases and technical debt accumulates, because your team members can’t afford to spend time on the underlying platform. At the point where you have multiple teams working concurrently—building processes, services, and solutions that may be useful to other teams—it’s time to consider building your internal developer platform. 

“If you can take the service and give it to another product team, or even give it to another company, and they can use it right away, then that belongs in the platform.”—George Hantzaras, Director, Cloud Platform Engineering at Citrix Systems, Platform Engineering: DevOps evolution or a fancy rename?

By standardizing the solutions to complex, recurring problems, you can free up your developers to move faster and enable new hires to onboard and start contributing more quickly. 

How to build a platform engineering team

When trying to motivate your organization to build a platform engineering practice from within, the initial challenges are more likely to be around getting buy-in from other groups and leadership. Then, you'll need to manage people’s expectations when you’re getting things off the ground.

The key to getting support for an internal platform is to treat your platform as a product, and connect it to the business’s overall goals. You wouldn’t expect the business to invest in a product that customers won’t buy. As a platform engineer, you are a service provider for your internal customers. You need to show the value proposition of your platform and demonstrate how you will measure its success: what are the metrics you will pay attention to? Some of these might include measuring the time it takes for a code change to make it to production, deployment frequency, and cost reduction from reducing resources allocated to unsupported workflows. 

Selling the platform to both leadership and the team members who you expect to use it is equally important: imposing platform adoption unilaterally is rarely effective.

Managing expectations

As you’re gathering support for your platform, you might find yourself overwhelmed by requests for custom functionality from individual teams. There’s a fine line to walk between catering to your internal customers’ needs and keeping the focus on building something realistic that works for most, without getting slowed down by niche requests. 

“The idea is to prioritize low effort and high impact requests to provide the best value for our customers. In platform engineering, your customers are other engineering teams and the end-users, so you have to balance the impact on each side … It’s up to your team to examine all streams of customer and team feedback to figure out what’s going to have the highest ROI.”— Zoe Sobin, Senior Engineering Manager, Platform Engineering 101: Takeaways from Building Internal Tools at HubSpot

A big part of a platform engineer’s role is advocating for the platform and teaching others how to engage with the team effectively: ideally people should be coming to the platform team with the problems that they want to solve instead of requesting the solutions they think they need. Maybe there’s a way to address a recurring problem across multiple teams with a slightly different solution than the one that works for one team. “This allows you to partner up to find the best solution together,” explained Zoe. 

What skills and experience does a platform engineer need?

Qualities

• Empathy: Platform engineers must develop a deep understanding of the needs of developers and the points of friction they experience in getting their code deployed successfully. Engineers who only want to write code aren’t a good fit for a platform role.
• Strong communication and collaboration skills: These help platform engineers understand the problems that the platform needs to solve for other teams. 
• Willingness to teach: Helping others to adopt the platform and golden paths is a core part of the role.

Experience and expertise

“Platform engineers need to have an interest and knowledge of the interface between the software and the machine,” says Camille Fournier, Managing Director at Two Sigma. Below are some of the common technologies platform engineers need exposure to and experience in.

• Kubernetes and container orchestration
• Cloud native technologies
• Microservices
• API management
• Continuous integration and pipeline management, continuous delivery
• Infrastructure as Code (Terraform, AWS Cloudformation)
• Observability, monitoring, and alerting 
• Experience on an SRE or DevOps team (advantageous but not essential)

The role of the platform engineer will vary depending on your company, product, and stack, and will likely continue to evolve as the practice climbs towards Gartner’s Peak of Inflated Expectations. Maintaining a laser focus on serving your development teams’ and customers’ needs will help keep you on track.