Podcast Takeaways: Cloud Developer Experience, Staging Environments, and Continuous Delivery
Learning from the Ambassador Livin’ on Edge podcast experts
The first batch of the Ambassador Labs Livin’ on Edge podcasts has seen four superb guests sharing their wisdom and experiences on creating cloud platforms and building effective developer experiences. Here are our top four takeaways from the conversations:
- The cloud developer experience can be “complicated”
- A cloud platform must be managed as a product
- The old “staging environment” model needs updating
- Don’t underestimate the value of continuous delivery
The remainder of this article explores each of these takeaways in more detail.
You can find the Soundcloud links to the recent episodes below, and you can also subscribe on other podcast platforms: Google Podcasts, Apple Podcasts, Spotify, Overcast.fm, and via RSS Feed
Cloud developer experience: “It’s complicated”
Several of our guests thought that adopting cloud technologies offers many benefits, but the current developer experience should ideally be much more effective. Both Nic Jackson, developer advocate at HashiCorp, and Paris Apostolopoulos, Java Champion, agreed that there are currently too many things a developer working with cloud platforms has to learn in order to push code to production.
Matt Klein, software engineer at Lyft and creator of the Envoy proxy, discussed how systems that synchronize local and remote Kubernetes-based development environments, such as Telepresence, Garden, and Tilt, are interesting. However, Matt argued that engineers should take care to ensure their usage of these provides net positive return on investment across their team; the tooling should support an existing effective workflow.
Creating a fast development loop is essential for engineering productivity. Sometimes engineers may have to “bend the rules” in order to make this work for them…
All engineers will be responsible for debugging an issue running in a live environment at some point. Alla Babkina, head of engineering at Headstart, point out that observability must be designed and built into all of the applications and also the platform, upfront. Engineers must create metrics appropriately for alerting, and also use logging effectively in order to provide enough “breadcrumbs” to locate where the issue is.
Building effective monitoring dashboards is an important component of the developer experience.
Treat a cloud platform as a product
Paris stated that Kubernetes was the “missing part of the container puzzle” that enabled deployment at scale and provided a foundation on which to build a cloud platform. However, there is a big jump from using Kubernetes for a personal project to running this in production.
Alla argued that If an engineering team decides to build an application platform, this must be treated like any other product within the organization. Requirements should be gathered from customers (e.g. developers), and the delivery and maintenance of the platform should be explicitly managed. Appropriate people, time, and resources should also be provided.
When a team is considering building a cloud platform and improving their development workflows, Matt stated that they should always start by identifying their most impactful/blocking problems.
In the near-term future, organizations will be dealing with a lot of core/vintage systems (that make money), and so the immediate goals of many platform teams are to bridge the gap between the core and cloud native technologies.
When applying solutions, teams should aim to “keep it simple”.
The old “staging environment” model needs updating
Matt also cautioned that the old model of the “staging environment”, where teams attempt to recreate a copy of the production environment to allow a gated or “approved” release candidate, often provides poor value to developers and the organization.
It is very challenging to make a development environment like production, particularly when working with a large service-based system or dealing with large amounts of data
The return on investment can often be greatest by creating an effective local development experience and providing the ability to test in production.
When working locally, the judicious use of mocks and virtual services can provide effective component and integration testing. A service-based system should make strong use of APIs and contracts (e.g. interface definition languages, such as Protobuf), which can be used to automatically generate skeletons for mock components.
Several guests discussed the value of testing in production using techniques such as canary releasing. Nic argued that being able to canary release functionality, both via an edge (API) gateway and via a service mesh, provides a controlled method of testing and verifying new functionality. The HashiCorp Consul team have invested heavily in service-to-service Layer 7 Traffic Management for just this purpose.
The value of continuous integration and continuous delivery
Paris began his podcast by stating that when he first saw continuous delivery implemented correctly it was an “epiphany” for him. The ability to increase the speed and fidelity of customer feedback for engineers provided a massive competitive advantage for the organization.
Developers need self-service access to the platform and continuous delivery tool, Nic argued, which allows them to understand both the operational impact and business impact of their changes.
A continuous delivery pipeline should codify all application quality and safety requirements. The execution of the pipeline should be fast and consistent (e.g. minimize flaky tests).
Alla discussed the benefits of engineers becoming operationally aware, but stated that not every engineer wants to be “fullstack”. Platform/operations support should be provided to engineering teams as appropriate, and self-service continuous delivery pipelines should make it easy to do the right thing.
Subscribe for the next batch of podcasts
We’ve recently recorded podcasts with Dana Lawson, Kelsey Hightower, and Charles Pretzer, and so subscribe to the Ambassador Labs Livin’ on the Edge Podcast in order to ensure you don’t miss the publication of these!
You can find the Ambassador Labs podcast on all good podcasting platforms. Please rate and review us!:
You can find links to all of the podcasts and the show notes at the Ambassador Labs Podcast Resource Library.
