I’m not sure I felt it at the time, but before I had kids I had life figured out. I could do what I wanted, when I wanted and I could generally stay on top of life. Finances were easy (comparatively), and my schedule was my own to determine once I’d finished work. Checking in with my partner was easy and we’d move on. Then one day I had four kids.

Well technically it took a lot longer than one day, but I did have four kids in two years so it feels like overnight in my memory. I remember walking through a busy town and counting my kids to keep an eye on them, “one, okay… now there’s two… where’s three? there he is, and four… now, where did four go?” and by the time I found four it was right back to one. Staying on top of life was astonishingly more difficult. Just walking somewhere now required constant monitoring. Something was always happening, someone was always unhappy, life was messy.

Microservices in Kubernetes is an incredible new world

This picture is very similar to the complexities of moving from a monolith to a microservices architecture in Kubernetes. One day you can stay on top of everything—you have one app, maybe one database, and everything just works and then the next you have a sprawling world that is scaling up and down in ways you’re not always aware of. In a Kubernetes environment, by the time you’re sure three services are up, the fourth may be down, and then you have to start counting again.

Don’t get me wrong. Kubernetes is amazing at what it does—it provides defaults for cloud-native architecture in a way we all would have killed for in 2014 when we were defining it ourselves. But in simplifying the management of sprawling architectures, it also spits off a tremendous amount of data. Suddenly you have to check memory and cpu resourcing, you want to make sure your bins are being packed correctly. You’ll find yourself checking on every node, pod, namespace, and cluster——and just when you start to wrap your head around it, your organization will be spinning up new clusters and tearing down others.

The promise of the cloud is amazing—you can get off and running building a SaaS application without the need to provide all the overhead for the hardware. Likewise the promise of microservices is amazing—you can assign individual services to teams or individuals to maintain and update and own all the way through to production without them needing to touch every other part of production.

I love having children—but both kids and microservice cloud-based architectures come with trade-offs—you’ll need to stay on top of much more than ever before.

Monitoring that complexity doesn’t have to be terrible

There are ways to reduce the cognitive overload of walking my kids through traffic. The easiest is to hold everyone’s hand. Then I get constant signal about whether they’re with me or not without having to stare at them (two people may be needed for the four kids’ hands). Similarly, with systems in place you can get constant signal about the state of complex microservices in Kubernetes with ease. A great place to start is to implement tracing throughout your codebase. By leveraging a standard like Open Telemetry you’ll immediately be at something like 90% of all the work you need done to keep an eye on your systems.

You’ll still need to write logs because there’s always something to log. And you’ll need to collect metrics and set up alerts and do all the other things necessary for true observability/monitoring in your stack. And tracing is a great place to get started today—because too much out the gate can be overwhelming. With the right software you’ll be able to get dashboards out of the box to keep an eye on things. You may no longer be able to walk by the server racks on your way to your cubicle and see the blinking lights and know everything is working, but we have tooling that provides the constant feedback you’ll need to stay on top of things.

There is so. much. more. data.

Finally, it’s worth noting that the sprawl isn’t just how many more machines there are—but also the organizational changes about who owns those machines and services. Modern service ownership means your entire organization is building, deploying, monitoring, configuring, and securing their workloads. Without standards, best practices, and unified tooling you’ll have no hope of navigating the sprawl.

The proliferation of event data due to technological and organizational complexity in the cloud native world is astounding—but it doesn’t have to cause you to drown. 

Thankfully modern problems come with modern tooling built to handle the deluge of data and we don’t need to live in fear of it being too much—modern solutions allow for a cost effectiveness that no longer requires sampling, filtering or deleting the data you’re not sure you’ll need. Just like I’ve managed to stay on top of four kids, you too can stay on top of your cloud native microservice architecture—even in Kubernetes. 

Join us at KubeCon + CloudNativeCon North America this November 6 – 9 in Chicago for more on Kubernetes and the cloud native ecosystem.