Kentik Engineering: An Introduction

Over the years, many people have asked us to author an engineering blog series at Kentik. It’s taken us a while to get our act together, but we’re finally here.

As I contemplated how to kick this series off over the past couple of months, I reflected a lot on our journey as a team and a company. I recall so many memories and see so much potential, so much excitement within our engineering team. I wonder what people imagine when they think about Kentik engineering, who we are and what we do.

Who do you think we are? Network nerds? Ok, guilty. Data dreamers? Definitely guilty. Really cool, fun people that enjoy building software? Guilty, guilty, guilty. But, let me pull back the curtains a bit, and you will see so much more.

As I look around me, I see a team of people I’ve gained so much respect for as we’ve built a crazy scalable SaaS platform and products together, fought back production fires together, created fun easter eggs and hackathon projects together, and enjoyed team events together, including blacksmithing/MIG welding, escape rooms, and swamp tours in the Louisiana Bayou.

I see people I consider friends and treat like family. I see people I would want to work with again and again.

I see hard workers, communicators, fun-lovers and parents (and so many future Kentikians entering the world!). I see many different and intriguing hobbies with people spread around the world (we were built as a remote team before 2020).

But most importantly, I see a team. A team that cares deeply about each other and our mission to unlock the mysteries that only network data can solve. A team that has demonstrated expertise in distributed systems, data processing, and modern user experience. I see one of the most talented teams, top to bottom, I’ve ever been a part of.

I have been at Kentik for just over six years. We’re about to hit our seventh birthday as a company. It’s both awe inspiring and mind boggling to me as I think about all the things we’ve done together in this time.

I remember starting with two “simple” engineering goals: store every data record we receive, and be able to query this data at subsecond speeds… all while being able to scale horizontally.

Today, our U.S. SaaS deployment currently receives almost 8 million records per second. To save you from the math — that’s 691 billion records per day, or 252 trillion records per year. Think about that for a minute while we process another 480 million records.

When users want to query this data, they can ask any question they want across hundreds of millions to billions of records (I take care to say records instead of rows because we don’t store the data in row format), and we will typically answer it within seconds. This year, we expect to run somewhere around 1.5 billion queries for our users. I think we’ve knocked those goals out of the park.

And it didn’t stop there. We have thousands of BGP peers and hundreds of customers that have leveraged what we’ve built to make DDoS attacks an afterthought. We’ve launched several major products in the last two years, and are always looking at how to paint an even more complete picture to network observers around the world, all within a state-of-the-art UI.

Did I mention that we have done all of this with under 30 engineers to date? I’m so proud of our team and what we’ve accomplished with so few hands on keyboards. Today, we’re growing, and growing fast. We plan to almost double in headcount this year and don’t plan to stop there. There are so many new and exciting challenges ahead of us. Check out our job openings!

As we grow, this blog series will grow with us, and I’m very excited to take you on our journey. Our team will share where we’ve been, the decisions we’ve made, the problems we’ve solved, and fun facts about Kentik and our platform. We will talk about everything engineering, from technical solutions, to organizational decisions, to team culture, and how all of the tradeoffs we’ve made have paid off (or not) over time. Our hope is that sharing our experiences will help others feel the joy we feel as they build out their teams and products.

For now, I’ll leave you with a little present from the ghosts of Kentik past.

V1

V2

V3

V4

Until next time — stay healthy out there!