Experience Report

Using a chatbot to do DevOpsCust

About this Publication

How do you re-invent great customer experience in a 230-year-old compliance regulated conservative company -whose products are the least sexy you can buy, and whose customers identify your brand with an old man in tweed? This is a story about combining courage, girl power, experimenting with agile methodologies and conversational AI and how we succeeded in 2 months.

1.      INTRODUCTION – What in the world is Alm Brand?

The year is 1792. The Danish King, King Christian the 7th signs a royal decree to establish the insurance company Alm. Brand with the purpose to offer fire insurance for properties outside of the Copenhagen area. Today we are still an insurance company – and one of the biggest in Denmark.

The insurance business has not changed significantly during those 230 years – and we are no exception. Our customers consider us the traditional, the stable, and the conservative ones – and we are proud of that.

If we fast forward to 2022, then the perception of us as the traditional and conservative company has now started to change as we are becoming more and more innovative on how we develop our products and offer services to our customers.

2.      Background

It is not a secret that we in Alm. Brand for 230 years have not been first movers or the most innovative when it comes to how we interact with our customers. We are still visiting our clients in their homes, and they are always able to reach us by telephone – unlike many other companies who are no longer offering these services, home visits and phone calls, as they are simply too time consuming and thereby considered too costly. These services are a victim to cost cutting in a competitive market where the demand is limited. It’s not like you wake up some morning and suddenly feel like buying an additional home insurance.

But a few years back, EVP – Kristian Hjort-Madsen – started our digital transformation. And that was not just a digital transformation, but also an agile transformation that means we shifted gears in the way we develop and deliver digital solutions.

What does that mean? It means, that we went from a more traditional waterfall development approach, trying to predict how our  customers could use our solutions via long requirement specifications – we have now turned everything upside down.

The goal is always to put the customers first in everything we do. But instead of only talking about it – we walked the talk and went “all in” on changing how we work, to incorporate customer-focused agility everywhere.

3.      Our Story – More Accessible Customer Service

During 2020 we identified several business cases and wrote one-pagers for more accessible and cost-effective customer service. Those business cases would also start the customer service transformation. Conversational AI was the answer for launching chatbots internal as well as external. A conversational AI team was organized, and the work began.

The traditional analysis and design phases started, but it went too slow without much progress or results. We realized that helping our customers with conversational AI would require an MVP approach. MVP is ‘minimum viable product’ and means that you release a version with just enough features to be usable by customers who then provide feedback for future iterations. With the old traditional development approach, it would take a long time before we would be able to launch a chatbot, because we would need to try to predict what insurance events for a full year to add content for.

3.1      Meet Albotta

Before continuing I would like you to meet our chatbot Albotta. She was born summer 2021. She is animated and uses emojis and speaks the language of the customer.

In the beginning she could be a little ‘rude’ – unintentionally though – (customer: thanks for a crappy service. Albotta: you’re welcome, anytime!) but we have now taught her to behave nice even when angry customers curse and swear at her (that’s something I never thought I’d be doing at work – write a bunch of data containing combinations of cursing and swearing – all with the purpose to improve our customer service).

3.2      Faster, stronger, agile

So now we wanted to do things differently. And that’s what we did. June 2021, we went live with Albotta. She was launched with very little content – on our corporate main website which is visited 1500 times a day. Why is that genius? The more visits a day you get, the more you know what the customer needs.

The interactions our customers had with Albotta showed us what the customers wanted. In effect, the customers wrote the requirements.

Albotta allowed us to be close to our customers, and even deploy what they wanted several times a day. Working in an agile way is perfect for this with the ability to ‘turn on a dime, for a dime’

The R&D Department was already working with 2-week iterations followed be a demo for stakeholders. The chatbot team consisted of a combination of AI Trainers and subject matter experts. The chatbot is built on ML technology built on boost.ai.

Every day, the team assessed all chats and put them in the following categories:

The categories were used to get an overview of the performance as well as to assess which content were missing or should be improved.

In addition, the team is going through all conversations that has an either ‘unknow prediction’, negative comment or negative feedback.

Especially in the beginning it was important to monitor trends from the chat very closely to see what the clients were searching for and how they were using the chatbot and navigating. Since we were following all chats so closely, we could easily respond and deploy new content immediately.

Through these data, we also get insights about our customers’ current needs. And we get them in real time. A good example is how Covid-19 and the use of face masks caused a lot of broken glasses. With knowledge like this we can be proactive, and advise targeted customers about risks and safety measures, so they can, hopefully, avoid those damages.

3.3      Results

So how did it go? Well, after 2 months Albotta was able to answer more than 80% of all the questions. Today she has had more than 90.000 conversations and keeps the success rate above 87% which is high in a market where 70-80% is considered very good. Based on these results, Albotta was even labelled as one of the fastest implemented chatbots in Denmark at that point.

Albotta has around 1200 topics and she can answer everything from opening hours, digital harassment,

GDPR, extreme sports, Covid-19 and veteran cars. In the case where Albotta is not able to understand the customer, she is still designed to try to help the customer in the best possible way. For example, she will try to suggest going to another topic or to refer to customer service. Sensitive topics are always referred directly to customer service.

The graph below shows the messages per hour. Approximate 30% of the conversations happen outside our normal opening hours.

In addition, our data tells us that 1/3 of the chats are done from a smart phone. Albotta is also designed to be easy to use via a smartphone in order to make her accessible for the customers whenever or wherever you need help.

The value of the initial business case was hard to achieve on a short term. Instead, in my opinion we achieved a lot of qualitative rather than quantitative results that are difficult to measure the monetary value of.

For example, we now have a much better collaboration around the variety of possibilities we offer our customers to get in touch with us. We need to stay aligned when we experiment and keep a system thinking approach to avoid local optimizations in teams.

Even though Albotta has a very high success rate, a chatbot is never ‘done’. We can also help other parts in

the organization. If there’s an extraordinary queue on the phone, customers will write to Albotta – and we can then use this info and ask other departments if we can add some additional intents on a specific topic that can help them to ease the pressure on the phones.

In February we had a big storm in Denmark. And that happened in a weekend where our customer service is closed. But fortunately, Albotta answered more than 2000 questions during that weekend. That was a solid proof she was there for our customers when they needed a quick answer.

4.      What We Learned – experiment, experiment, then experiment some more

It turned out, that an agile approach with 2-week sprints, did not really fit the situation where we needed to act faster and deploy new content every day, which also made sense since we have the situation where we get daily feedback from our customers. We started out by reading the chats together to ensure common understanding. That turned out to be too time consuming.

Then the team members read the chats on their own. Both the daily stand-ups and a busy ongoing team chat proved very valuable to stay aligned on what content to create or correct.

Everything was added to a Kanban board, and everyone were free to pull what they wanted. That turned out to have too much focus on day-to-day tasks only, and in effect we lost focus on more long-terms tasks that were also in the sprint.

Both were important. We needed to respond to customer concerns, and at the same time we needed the long-term strategic direction. We started to get more and more feedback. Both from the chats, and also from colleagues working on other solutions that didn’t always support the goals we were trying to achieve. And that we needed to align on too.

So to ensure a better balance in the focus of our work, we started to work in pairs and swap between day-to-day tasks and strategic tasks. Stand-ups would become mini plannings sometimes as everyone would present if they had new input that we needed to take a decision what has highest priority.

I really like this approach, that inside a team, inside a sprint, everyone can ‘pitch’ ideas/tasks based on what they have observed, mixed with the short-term/day-to-day tasks that also need to be performed.

Team members then self-organize around those tasks and implement them during the next couple of days before they again meet and pitch ideas. When the sprint is over, a mix of both short term and long term tasks were implemented. Learning and alignment in the team are ensured as team members switch tasks and pairs. Planning is continuously during the sprint, and not only done on 2-week intervals.

We cannot train our AI models alone, so the chatbot is a great tool to get the customers to help do so with their domain knowledge, so to speak. At the same time, the data generated from the chat is not enough for the AI technology to learn from, so by combining with the AI trainer’s knowledge, we have a really great set-up for building a good chatbot.

It’s hard to prove a business case with technology only. And technology only doesn’t provide good customer service. It’s about deciding and considering the entire customer journey to align the expectations between the technology and the user.  And Albotta should not be used for everything either, for example sensitive subjects should always be handled by real people. So, the intention is not that Albotta should take over everything. But she’s a great supplement for both our customers and ourselves.

5.      Finishing/Conclusion

With Albotta, our customers are now able to access a service that is always there for them. She is an online, living service based on the costumers needs, what they ask for, worry about, or prioritize and focus on. Not what we as a company think.

The customers that choose to use Albotta gets an answer right away, whenever they need it and wherever they need it. Time of the day or different time zones are not a problem – if you need help while travelling abroad, Albotta is still accessible from your smartphone and with few clicks you can access SOS international for example.

5.1      What the organization learned and benefited from

It takes courage to turn things upside down. It took courage from the management team to give their ‘go’ to launch a chatbot with very little content, as it also meant providing a less good solution for the first customers who used the Chatbot. But imagine we had spent years on adding thousands of intents based on our guesses of what is most relevant for the customers. Instead, Albotta is built with agile principles that ensure we only build what is valuable for our clients, and then we expand the solutions as we go. And the best part is, that it is all driven based on the direct customer feedback through the chatbot.

For companies who wants a competitive advantage in a world that changes faster than ever, business agility becomes more and more important – the world as we know it the last 2-3 years has proved that disruption is the norm rather than exception.

Getting a lot of real-time data directly from the customers as input for designing customer service is essential in our effort to obtain business agility.

One thing is the direct benefits we can offer our customers with Albotta. But another thing is the advantages we gain from the data generated by the chatbot as well. Albotta is also a steppingstone for us. We plan to offer AI driven voicebots to our customers, so we can handle requests by telephone outside opening hours.

Not only do we want to be there for our customers, when accidents occur, and they need us the most – as we have done it for 230 years now – but we want to help them avoid accidents in the first place. This is a big shift in approach, and it takes extra effort to put the customer first. Because for every new technology we introduce, there is new behavior we need to account for, in order to make the customers want to use our new solutions.

We are in the process of moving the boundaries of what our customers can expect from us. And we are not there yet. But with the mix of courage and experimenting with agile methodologies that has enabled us to turn things upside down and learn and co-create our solutions in-directly with the customers, it will enable us to give the best customer experience while exploring our further journey into conversational AI.

5.2      What the team learned and benefited from

Being an all-girls-development team in a very technical world, the team quickly realized, that with the right technology, everyone can build a chatbot, but only few can build a great. So, it was a brave team, which realized we do not need to be technical experts before we start. We are experts on our fields already, and that is more than sufficient.

So together we helped each other along the way. Complemented each other and not afraid of trying things out – we knew we would get the customer feedback immediately to which we could adapt. The team was not afraid to say out loud, that everything does not have to be perfect. And why would they?

Albotta is in fact build on everything she cannot do. She’s built on the negative feedback. Failures and success should not be regarded as opposite components, but instead both are components that builds the best achievements together.

In addition, this unique team spirit brought the team many bottles of champagne when celebrating Albotta’s successes, as well as a trip to Websummit in Lisbon with ‘Women in Tech’.

5.3      What did I learn and benefit from

It is my dream to work in an environment where the above teamwork takes place, but in an even bigger scale. Where people have the empowerment to self-organize – and with management trusting employees to be able to do so. Where ‘failure’ and ‘failing fast’ is a natural part of your success. And where you can see the direct impact of your work immediately, and where you also at the same time get the direct feedback from the customers right away.

It might take courage from management to start this, but it also took courage from myself and the team to collaborate and achieve the benefits from real teamwork. It’s been a privilege, and I am proud to have been part of this journey.

6.      Acknowledgements

In connection to writing this report, I would like to send a few thanks to some of the people who has been involved:
First of all, I’d like to thank the fantastic all girl development team for which I was the scrum master initially and who introduced me to the wonderful world of conversational AI.

To Frank Olsen who encouraged to share this story. And to Johanna Rothman for shepherding. It’s been a privilege to receive your feedback as well as tips and tricks and advice on how to write.

Add to Bookmarks Remove Bookmark
Add to Bookmarks Remove from Bookmarks
Add to Bookmarks Remove from Bookmarks
XP 2022

Your Bookmarks

No favorites to display. You must have cookies enabled to add bookmarks.

Have a comment? Join the conversation

Related Agile Experience Reports

Discover the many benefits of membership

Your membership enables Agile Alliance to offer a wealth of first-rate resources, present renowned international events, support global community groups, and more — all geared toward helping Agile practitioners reach their full potential and deliver innovative, Agile solutions.

Not yet a member? Sign up now