Last week MongoDB CEO Dev Ittycheria likened present-day AI to the “dial-up phase of the internet era.” He’s not wrong. I recently suggested we’re in the “awkward toddler phase” of generative AI (genAI). It’s full of excitement (“her first step!”), but straightforward tasks are clumsy, like basic math. Yet in some areas, the future of generative AI is happening right now. One area is coding assistants such as Amazon CodeWhisperer.
Sure, you’ve heard anecdotal stories that someone built a complex banking application for their Fortune 100 bank in two minutes with a command prompt, but that kind of hype helps no one. What are real developers doing today to use genAI to improve their productivity?
That’s the question I asked Doug Seven, general manager of Amazon CodeWhisperer and director of software development for Amazon Q, at AWS. As Seven tells it, developers at any stage in their career can (and most definitely do) make good use of generative AI today. The key for AWS is letting developers focus more on their code and less on all the buzzwords infecting the AI landscape.
The Amazon effect
First, what exactly do tools like CodeWhisperer and Amazon Q (a genAI-powered chatbot assistant) do? At its most basic, CodeWhisperer assists the software development process by offering contextual coding suggestions. As a developer adds comments to code, CodeWhisperer infers from those comments what the developer is doing and gives code suggestions. CodeWhisperer can also scan code to detect security vulnerabilities, inform you about them, and then fix them.
Over the past year since AWS GA’d CodeWhisperer, the company has added the ability to fine-tune those coding suggestions with customizations. For example, a company can point CodeWhisperer at its internal frameworks, private libraries, or even just coding conventions, and CodeWhisperer will reflect those corporate conventions and preferences. “You get code more like your code instead of more like general-purpose code,” as Seven explains.
Amazon Q, for its part, acts as an assistant. Need some code explained to you? Ask Q. Need that same code refactored or debugged? Ask Q. With Q, a developer can keep coding while Q runs off to do “errands.”
All fine and interesting. In fact, “everyone’s doing it” these days: GitHub Copilot, Google Codey, ChatGPT, and more. But something feels different about AWS’s approach—something very Amazonian.
Not just for AWS services
From its inception, AWS has always strived to alleviate the “undifferentiated heavy lifting” that developers must do to stand up servers, manage databases, and more. The company’s approach to generative AI is much the same. The market is hypersaturated with AI hype, making it hard to separate signal from noise, as I’ve recently written. I asked Seven how AWS aims to help with this problem. He responded, “One of the benefits of CodeWhisperer is we abstract all of that generative AI science.”
In other words, developers don’t need to be stressed about large language models (LLMs), retrieval-augmented generation (RAG), or whatever is being released today. CodeWhisperer helps them write code more productively without worrying about the models running behind the scenes. This is great news for developers. They may, for example, realize that they need to refine the training of their coding assistant for better output, yet not have any idea how to achieve this through other means. Seven says AWS wants CodeWhisperer and associated tools to make this “relatively turnkey for you. So you can just say ‘here’s my repos, do your thing,’ and then [AWS will] use a variety of techniques behind the scenes.”
Nor is AWS fixating on its own services for CodeWhisperer, though of course the company aims for a first-class experience with its first-party services such as Aurora for MySQL. As Seven puts it, “One of the … principles [for] CodeWhisperer is that we have to be able to help developers do whatever they’re doing.” This means helping developers who may run services across different clouds (or on premises). “We don’t want to restrict CodeWhisperer to being good at only AWS [services]. We want to make it good at everything that a software engineer is trying to do,” he declares.
Will GitHub Copilot or other third-party coding assistants end up working better with AWS services than CodeWhisperer? Not if AWS can help it. “We do a lot of work to make sure that we’re better at doing AWS things than anybody else,” Seven says.
Coding with CodeWhisperer
How should a developer get started? Despite all AWS’s efforts to “abstract away the data science,” to use Seven’s words, “sometimes there can be a learning curve in terms of how you express yourself” to get CodeWhisperer (or any of these genAI tools) to yield the results you want. Still, “it’s really fast to get started, and you learn as you go,” he continues.
This holds true whether the developer using CodeWhisperer is experienced or new to the field.
Seven says the company ran an internal productivity challenge, and participants who used CodeWhisperer were 27% more likely to complete a set of tasks successfully. Even better, they did so 57% faster than those who didn’t use CodeWhisperer. This was true regardless of experience level.
Why? A developer might be unfamiliar with a particular SDK and CodeWhisperer’s code suggestions helped guide the developer past the hurdle without having to slow to read documentation. For experienced developers who already know what they’re doing, CodeWhisperer helps smooth out speed bumps like this to work faster, while also enabling them to plow through boilerplate code much more quickly. For the less-experienced developer, CodeWhisperer prompts them with code suggestions that keep them from getting stuck. As Seven puts it, “They’re able to get suggestions that help them figure out where they’re going and they end up having to interrupt other people [e.g., to ask for help] less often.” Everybody wins.
Moving beyond the tool age of AI
Seven says he sees both bottom-up approaches (a developer or team has success and spreads the word) and top-down approaches (executive mandate) to adoption. What he’s not seeing is any sort of slowdown to generative AI innovation.
Today we use things like CodeWhisperer almost as tools—like a calculator, he suggests. But a few years from now, he continues, we’ll see more of “a partnership between a software engineering team and the AI that is integrated at all parts of the software development life cycle.” In this near future, “Humans start to shift into more of a [director’s] role…, providing the ideas and the direction to go do things and the oversight to make sure that what’s coming back to us is what we expected or what we wanted.”
As exciting as that future promises to be for developers, the present is pretty darn good, too. Developers of any level of experience can benefit from tools like Amazon CodeWhisperer. How developers use them will vary based on their level of experience, but whether they should use them is a settled question, and the answer is yes.