As a new product owner, application programming interfaces (APIs) and technical stories were my weak spots. I struggled for a long time to know where to start, what to include, how to communicate the necessary things to the development team, and how detailed to be.
Our team learned these lessons the hard way, but you don't have to. While every development team, environment and product owner is different, these basics should help you get started out on the right foot.
Do your initial research
Getting started with APIs/technical stories can be daunting. There is so much complexity, so many unknowns, and so many things to consider that it's easy to see why folks balk at creating them. But a little research and prep work can alleviate many of the pain points.
Not all APIs are the same, and they certainly aren't created equally. Focus your research on finding out these three things.
What type of API are you dealing with?
There are three different types of web APIs: Simple Object Access Protocol (SOAP), Representational State Transfer (REST), and Remote Procedural Call (RPC).
To know which one you're dealing with, pull in your technical team or technical lead. They will be able to fill in that blank for you easily. Creating a good relationship with your technical team while working on APIs or technical stories will save a lot of headaches in the long run.
What data payload are you dealing with?
These APIs can usually send data in either XML or JSON, or both, but that is not always the case. Data payload is another question that your technical team can answer; they may need it in one format or the other.
The answer can also depend on the source consuming this data. Another good relationship to cultivate in this process is with the people behind the consuming system. In some instances, it will be within your team's full control, while in others it will be another team or system that is dependent on you.
Do you have the appropriate documentation?
If you are integrating with an existing API, the documentation should be robust and come complete with a Web Services Definition Language (WSDL) for SOAP applications or a Web Application Description Language (WADL) for REST applications.
That said, robust documentation is often a pipe dream. I find it best to lean on the technical team again via spike stories to research the API in order to help fill in knowledge gaps.
If you are creating a brand-new API, documentation will let future developers know how to format their requests and what data is available. Without proper documentation, an API integration is quite difficult.
Very few developers really enjoy documentation. It's best to lean hard into the test-driven development (TDD) so the tests document the work or to write the stories to put the developers in the shoes of the next person.
Both methods will get you to the end result of good documentation for the next team to attempt to integrate with your work.
Start your data flow planning
After you have done the appropriate research, it is time to plan the data flows—a critical part of the API integration process. Data flows are the objects or data you wish to transfer from one application to the other, and there are two types of data flow diagrams that could meet your needs.
Logical data flows
Logical data flows focus on what happens. These are handy in the early stages, since they focus on what information is being sent, who is receiving it, and which processes are occurring.
Logical data flows tend to lean heavily on the process and away from the technical, so anyone can understand them. They are a great visual tool when collaborating with nontechnical or business folk.
Physical data flows
Generally, these are a collaborative effort with your technical team, but they are also a good thing to be a part of so you fully understand what is going where and why. Physical data flows lean heavily on the technical details such as the specific software, hardware, and files.
Some other things to consider for data flow
You should consider a few technical topics and discuss them with your technical team as you work through the story writing process. Some of these may be within your control, and some may be dictated by the industry you're in or by compliance needs.
Authentication
Some APIs—and most private web APIs—are protected by some form of security. You cannot just create a URL send it across the web, and expect the data to come back unless you have met the required security measures.
Popular security protocols include OAuth and Security Assertion Markup Language (SAML).
Also consider how long you need to keep the authentication instance running. Will these data flows need to open continually or intermittently?
Timing, volume, and speed
When does data need to be transferred, and how often and how much? An integration will only make sense if the data can get to the other system in a timely manner. Balance the volume of data that needs to be transferred with a rate of data transfer and the time that the data needs to be in place.
For example, if an organization would like to have all of its sales orders integrated by the end of each business day, then the number of orders and the rate of data transfer will determine if this goal is possible.
Data discrepancies
When transferring data across applications, make sure it is in a format that is acceptable to the other endpoint. This usually involves things such as date/time formatting and field size limitation. Although translating this data might seem trivial, neglecting to do so will result in a failed integration.
Create, read, update, delete (CRUD)
The intent of this data should be considered as well. When sending information to a new system, that system needs to know what you want to do with the information. It can either create a new record, read or select a record, update a record, or delete a record.
Map your data (if you need to)
At this point in the research and prep process, you might have enough information to begin story writing. Keep in mind that the goal is not to create processes and documentation for the sake of processes and documentation, since that would be very anti-agile. The goal is to create enough information to be able to communicate fully about what is being built with the business/nontechnical folk and your technical team.
If the previous efforts have not generated enough of an understanding to proceed, the next step is to create a data map. This will make your life easier when it comes time for development.
The data map is also a handy way to help you identify the source of truth for your data. In most cases, you want to get as close to that source of truth as possible. While that is not always possible, be sure to know the specifics about any delays or changes that are made to the data between the source of truth and where you are ultimately pulling it from.
Here's why this is important: If you are working with something that needs the most current data and you are pulling from a source that is updated from that source of truth with a delay of 15 minutes, you could potentially have a problem. This exercise helps you identify those issues and communicate them properly.
Elements of a data map
The data map will have the starting (source) application on one side, with all the objects you wish to transfer and all of the relevant information about those objects next to it. On the other (target) side, you will list all the objects that the starting objects will become once they are in the target system. Here is a sample data map.
Define key performance indicators
Once you have finished the prep work and have enough information to communicate what needs to be built, it’s time to start thinking about what "good" looks like by defining some KPIs. These will demonstrate how effectively your API is achieving its business objectives.
In some environments, these will show up as nonfunctional requirements, and in others, you create them from scratch.
Common KPIs include:
Audience journey metrics
Business goal metrics
Application API metrics
Requests per minute
Average and max latency
Audience journey metrics
These are characterized by measuring the various steps it takes for your end user to reach and consume your API. These are used if the API is the only product being developed for sale or use by customers. Here's a classic example of audience journey KPIs for a traditional API model.
Business goal metrics
These metrics are tied more closely to your API's purpose. Here are a few examples of business-centric KPIs.
Application API metrics
For APIs specifically, there are other metrics that should be considered. They tend to be important when serving internal applications or websites and revolve mostly around the speed and performance of the API.
Requests per minute
RPM is a performance metric often used when comparing HTTP or database servers. Usually, your end-to-end RPM will be much lower than an advertised RPM, which serves more as an upper bound for a simple “Hello World” API. A server will not consider latency incurred for I/O operations to databases, third-party services, etc.
While some like to brag about their high RPM, an engineering team's goal should be efficiency, and the work should be an attempt to drive RPM down. Certain business functions that require many API calls can be combined into a fewer number of API calls. Common patterns, such as batching multiple requests into a single request, can be useful, along with ensuring you have a flexible pagination scheme.
Your RPM may vary depending on day of week or even hour per day, especially if your API is geared for other businesses that exhibit lower usage during nights and weekends. Metrics related to RPM include requests per second (RPS) and queries per second (QPS).
Average and max latency
One of the most important metrics to track customer experience is API latency, or elapsed time. While an increase in infrastructure-level metrics such as CPU usage may not actually correspond to a drop in user-perceived responsiveness, API latency definitely will.
Tracking latency by itself may not give you a full understanding of why an increase occurred. It's important to track any change of your API, such as new API versions released, new endpoints added, schema changes, and more to get to a root cause of why latency increased.
Writing the user stories
Finally, after you have done the research and thought about what "good" looks like, it’s time to start writing the stories. No one said API/technical stories were easy. There is still more to think about as you start writing. Here is where you start considering the who, the what, and the why.
Think about the who
First, pinpoint your persona. During the discovery phase, it is generally a good idea to detail the persona as a character that identifies the end user of the product you want to implement.
Let's say your customer, Randi's Computing Central—a company that sells online PC components—wants you to build a set of APIs to integrate its website and the product database.
One of the user stories in your product backlog must cover the build of an API to show the available products on the website. Who is the actual persona that will benefit from the development that the story delivers?
In this scenario, the temptation is to write the user story as follows: "As an API, I need to read the products database so that the spare parts available appear on the website."
Although this sentence describes what the API should do, it does not give the development team the bigger-picture goal. We do not know the value that this development is bringing to the solution and to the end user. This lack of context makes it difficult for the development team to understand why this user story is important for the project.
Now step back and think about who your persona can be:
Zach, an online buyer
Randi, products database manager for Randi's Computing Central
Rethink your requirement and the impact that it has on your personas. That might change the user story to:
- "As Zach, I want to browse the online products list page so that I know what is available."
- "As Randi, I want to update the products database so that the products that are available appear on the website."
The sentences above give us more context and tell us the value that this piece of the project wants to deliver. By associating the persona to the product functionality, you immediately have a broad picture of what the goal is.
This will help your development team better understand what is needed and help you keep the bigger picture in mind as you write. You never know; considering the viewpoint of Zach or Randi might inspire you to come up with additional things they need.
Exceptions
As with most things in life, there are exceptions to every rule. In some circumstances, you might be dealing with multiple systems and the work may be very abstracted from the end user. In those cases, personas are not always the best way to communicate the need to the development team, and might even confuse them more.
It is possible to have a system persona. You can think of a system persona as another user with needs and wants, but one that is abstracted from the actual end user.
If we go back to the Randi's Computing Central example, you might use:
Products API
Products database
In these cases, in the user story you will name the system that benefits from the work or the one that is going to perform the required action. Keep in mind what the bigger picture goal is and fight hard to make this an exception, so you do not forget that no matter what, a user ultimately needs to be able to gain something from this.
Think about the what
The second element you define when writing an API/technical user story is the action that your persona needs or wants to complete.
This element appears in the second part of the user story template: "As a [persona], I want [to do something], so that [something else happens]."
There are many ways this can be approached, from using HTTP verbs to CRUD operations to by state.
HTTP verbs
In integration projects, this activity is probably easier than identifying the persona. Generally, a great place to start is knowing the main HTTP verbs when building APIs:
POST
GET
PUT
PATCH
DELETE
You can easily translate these verbs into a less technical language:
Create
Read
Update/Replace
Update/Modify
Delete
So, going back to the Randi's Computing Central project, if the requirement is to show the new products added in the database on the website, your user story is not:
"As an API, I have to POST new products on the website so that I complete my job."
But:
"As Ruth, I need to create new products in the database so that they appear on the website."
CRUD operations
Another option is to think about the CRUD operations for APIs. These do not need a translator.
Create
Read
Update
Delete
Going back to the Randi's Computing Central project, if the requirement is to create an account to show the user order history, the user story would not be:
"As an API, I want to create a new user account so that we can display order history by user."
But:
"As Zach, I want to create an account so I can view my order history."
By state
There are few times when you will want just a single option. Alternatively, think about the state a persona could find themselves in. They could be:
Waiting on a response
Getting an error
Succeeding
Hitting a limit
Providing default values
Receiving an empty value
If you were to write stories for Randi's Computing Central project, and the requirement is to submit an order, the default thinking could be to write your story to look something like this:
"As an API, I want to send a success message when the order is submitted."
Try this instead:
"As Zach, I want an order confirmation so that I know my order was received."
Consider these states; you will find things you missed. For example, what will happen to the user if the data is corrupted and there is nothing in the required fields? Worse, what happens to the user if the request errors out?
Think about the why
In the last part of the agile user story, you answer the question of why the persona wants whatever was defined in the "what." This is often called the business value. Put yourself in the persona's shoes and think about a clear WIIFM (what's in it for me).
This is where you can really drive home to your development team the importance of what they are doing. It is also where the development team might come back and tell you that if your stated "why" is what you are trying to achieve, there might be a better way to get there.
Do not skimp on the why. It's time-consuming to think about why something might be needed, but it is the linchpin to the story.
Pitfalls in API/technical stories
There are several ways developers can be tripped up. Here are the major causes of problems.
Sometimes a story is not necessary
There are situations when a story really is not the best way to communicate complex technical requirements. There are times when I drop the user story in favor of a technical spike or a diagram.
In these situations, remember that the intent of the user story is to clearly communicate what is needed, for whom, and why. If there is a better way to communicate that to your developers, don't be afraid to drop the story.
Story fatigue
Ever been writing stories (API/technical or otherwise) and find yourself bored or out of ideas? It happens to the best of us. You end up with stories such as this:
"As a content manager, I want a text editor so I can edit text."
Yes, it is technically following the format, but not really telling the development team anything useful. Save yourself and take a break or ask for help from someone else to see if what you are writing is remotely useful.
Horizontal slicing
Think of story writing like a layer cake. A vertical slice gets you a little of everything and is delicious to the "user." A horizontal slice will get you maybe a bit of cake with no frosting, filling, etc. In some instances, that is fine, but for the most part, everyone wants the full, yummy cake experience.
But it's not always feasible to get that vertical slice with all the goodness. Horizontal slicing happens so easily it's almost second nature to teams that are used to not owning the full code base or that have environments with lots of complexity. It can fluctuate based on the team structure, the expertise on the team, and many other factors.
Most development teams will default to splitting stories into the "front end" versus the "back end." It can be hard to fight that momentum, but try, if possible, to keep bringing the stories back to that valuable horizontal slice. This ensures that the development team is focused on the thing that makes what they are doing usable to the end user, not just to the front or back ends.
Right-sizing your vertical slices
Once you get the hang of vertically slicing stories, fine-tuning to a manageable slice of cake is the tricky part. Many find themselves including layer after layer until it's not feasible or is the entire cake.
When trying to find the right size for your stories, it helps to think about what is useful, not what is release-able. You might go through eight stories before getting to something deemed release-able, but each story is slightly closer to your goal. You might start with a single field on a screen, then keep at it until you have all the functionality you want.
Most times, if you can get your stories down to something that the development team can tackle in a day or two, you have hit the sweet spot. This is another area where having your development team involved is invaluable. It can help you get down to a reasonable size.
Removing the creativity
Your user stories should not remove creativity from the development team. In fact, that's the exact opposite of why you create user stories. This often directly correlates with adding too much detail to your stories.
The entire idea of using these stories is to inspire unique, outside-the-box thinking among the development team members. Do not limit them so much that they can't come up with their own solution to the problem you laid out for them. The development team will hate you, and you are pretty much guaranteed to come out with a lesser product for it.
Difficulties communicating value
When writing APIs/technical stories, it can be difficult to get people to see the value of building a thing no user will actually see. This is when it is key to make sure your "why" in the user story really explains the benefit to those users, even if they never actually see the API.
Sometimes a user may never see it, but a system will—for instance, adding alerting if traffic on an API spikes. The user benefits by having a stable system, and the system benefits with an early-warning system.
APIs/technical stories often lay the foundations for great things to come. Make sure that value is clear so any technical or business staffer can read your story and get a good idea of why you are building this thing.
Don't forget quality assurance
Anytime you forget to consider QA, you are in for a world of trouble. In the case of technical stories, it can really bite you, especially if you are only delivering an API for another front-end consumer.
Making sure everything meets the contract expectations, changing up the expected data types, and throwing in null or empty values into the data are often the kinds of things your QA team will come up with and you might miss if you have not considered that when writing the story.
It is generally best to get QA involved when you are first writing the stories. This way, they will keep testing when you are writing these stories.
Your mileage will vary
In the end these are just guidelines. The entire point of story writing is to give your development team enough to start a conversation. By "enough," I mean whom they are doing this for, what they are asking for, and why. The rest are details that can be worked out during the conversation.
So get some stories written, invite your development team into a conversation, and see where you end up.
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.
