Operations is going through a fundamental shift as infrastructure itself shifts from hardware to software. The “software-defined” future has arrived and it’s something enterprises must embrace if they want to deliver Internet-scale applications and pivot as quickly as today’s business environment requires.
Software-defined architecture uses a virtualization layer to optimize the use of resources. It also speeds provisioning and the ability to adapt to changes. The resulting flexibility is necessary in light of some of today’s biggest trends, including Big Data analytics, IoT, mobile, and social.
The software-defined trend is consistent with application development trends including Agile, DevOps, continuous delivery and continuous deployment. Delivery speed is important, as is the ability to pivot as necessary in in light of company objectives, customer demands and disruptive innovation.
However, speed isn’t the only requirement. Customers expect software to be of high quality, which isn’t just a matter of coding, it’s about performance, security and other things that fall into the realm of IT Ops, such as container management.
Increasingly, developers and IT Operations have to work closer together to deliver better quality products faster despite rapid technology changes, shifting business priorities and increasingly fickle customers. DevOps and continuous delivery efforts underscore the need for developers and operations to work together as a team.
Collaborating is one thing, understanding what the other side does is another.
How Much Should Developers Know About IT Ops?
Developers should understand something about operations, and operations should understand something about development, but how is a matter of debate. Some say developers just need to understand the broad brush while others believe developers should understand what operations does in considerable detail, including the technology stack that comprises the production environment. In that view, if developers understand the environment in which their software will run, then they’re in a better position to deliver software that performs more reliably in production.
Robert Stroud, a principal analyst at Forrester Research, believes developers need to understand the supported operating systems and environments though not in detail. In his view, developers only need to be aware of what comprises the environment and understand the security parameters.
“What operations really wants to know conversely from developers is that developers have completed appropriate testing so the code is of a relative quality level so it won’t fail in production,” said Stroud.
Given the growing trend in which everything hardware is being virtualized in software, one might assume that no knowledge of systems or infrastructure is required, which isn’t exactly the case.
“Despite all the trends, software still needs hardware and hardware still needs software,” said Greg Schulz, founder of Storage.io. “If you’re deploying software-defined networks, then you better have a fundamental understanding of networks and if you’re deploying software-defined storage, you need to have knowledge of storage. You need to understand the fundamentals.”
“Software-defined” everything is of importance to enterprises because of the scalability, flexibility, and ROI it provides
In a software-defined context, one does not have to configure hardware, although configuration is still necessary. There are also infrastructure-related metrics of which developers may not be aware.
“There are still things you need to consider from an operations point of view, even if you’re in the cloud, including scalability, durability, availability, mean time to recovery, and mean time between failures,” said independent infrastructure engineer Tom Hall. “There are all these service-level metrics that an infrastructure administrator has to be concerned with that developers are not concerned with.”
Understanding what the other side does and why they do it helps the organization as a whole, because there’s less finger-pointing based on misunderstandings.
“It’s crucial that engineers understand that operations is not separate, but essential to being an engineer and creating good products,” said Benjamin Forgan, founder and CEO of IoT Stack provider Hologram.io. “As a new entity, we think of DevOps as the platform on which you do continuous deployment. It’s important to understand [operations] because if you engineer something but you don’t have an infrastructure that lets you scalably deploy or it doesn’t consider enterprise requirements, you’re in trouble.”
Infrastructure as Code Helps
The virtualization of computers, storage, and networks has given rise to the infrastructure as code movement, which is becoming increasingly necessary. Almost every organization wants to deliver software faster because it’s necessary from a competitive standpoint. Infrastructure as code helps expedite software delivery, because developers no longer have to wait weeks or months to get access to physical equipment. Virtual resources can be provisioned almost instantaneously, except in organizations that still require ticket filing and a string of approvals.
Related Content: Infrastructure terminology you should know
Another reason infrastructure as code is important is because it helps bridge the gap between development and operations. Historically, developers have not understood infrastructure. Conversely, operations has not understood programming. They understand configuration scripts, which differs from creating applications.
The point about scripts is important in an infrastructure-as-code context because it requires more than just automation scripts. Infrastructure as code is code and so it has to source-controlled, and peer-reviewed. Infrastructure as code also requires testing and validation.
“Developers have to understand that this not is a normal way for operations people to think and work, and that they’re going to need help,” said Hall. “If they’re not doing things in a predictable, repeatable way, it’s not because they don’t understand the reality of that; it’s just not the domain they live in, and not the environment they grew up in.”
When WWT Asynchrony Labs started doing infrastructure as code, the operations team was using Chef. It didn’t take long before Ops realized it needed help with the code part of infrastructure as code.
“The tech lead came to us and told us the code wasn’t very good. He said, ‘Do you want me to refactor this?’ so I said sure,” said Matthew Perry, WWT Asynchrony Labs director, IT Ops. “Developers can look at infrastructure as code and understand it.”
Now, the WWT Asynchrony Labs operations team is able to provide self-service environments so developers can configure their own environments without filing a ticket.
Whiteboarding also helps break down the walls that have separated development and operations because it allows operations to explain what comprises the infrastructure, why the pieces are important, and how all of that impacts developers.
“It helps to have a common medium that both parties can talk through and understand the details,” said Matt Baxter, VP of Engineering at Jibestream, an indoor navigation and mapping engine provider. “When you really want to have a conversation about this stuff, you need to whiteboard the architecture of the system, draw boxes and lines to show where things are and how they’re communicating.”
Automation and Its Effects
Delivering software faster means automating more of the DevOps pipeline, but it needs to be done carefully and mindfully.
“We’re starting to see this new trend where Ops is starting to develop what the code pipeline looks like. I go from development to operations to testing to production, or some derivation, and that pipeline is totally automated allowing a developer to push a piece of code out using Jenkins or something like that,” said Forrester’s Stroud. “Because everything is modeled as a release and treated as a release, it automatically transitions itself through the various stages and upon success into production.”
Forrester is seeing end-to-end pipeline automation in only 29 percent of companies at the present time. The idea is that operations can help developers dramatically by delivering a consistent pipeline that developers can just push their code to without worrying about operational requirements.
“The real challenge, which is why developers have been building everything in DevOps, is because operations teams have not been giving developers these types of environments that they can just lift, push code to, and have a true representation now in production,” said Stroud.
The pushback from some operations personnel is the fear of being automated out of a job. Stroud thinks automation is actually a positive for operations team members because it allows them to focus on managing pipelines rather than coming in and fixing everything that’s broken.
Containers Are Also Driving Convergence
Many of today’s developers are very interested in containers because of the flexibility they provide, but who should be in charge of container orchestration? After all, container orchestration platforms are being promoted to developers and operations, although operations is more likely to make the purchase and take on the responsibility.
The use of containers requires the participation of developers and operations, with developers deciding what goes into the containers and operations handing the orchestration platform.
“Right now, most container management systems developed with somebody coding around Kubernetes, which most developers don’t have the skills to do,” said Forrester’s Stroud. “Secondly, they may choose to use a platform like OpenShift or Cloud Foundry or one of the could providers, or they’ll try to twist existing technologies to manage containers. So, we’re seeing three camps at the moment.”
Developers like the fact they can write an application in a container once and have it run on any Docker server or cloud service that supports Docker.
“For containers to run in production, operations must understand how to use containers,” said Hall. “I see a lot of developers really excited about containers and honestly, I don’t think most operations people I’ve met are still trying to understand configuration management.”
Storage.io’s Schulz has a different observation. He’s also noticed developers’ interest in containers and he’s also seen considerable interest from the infrastructure side.
“Containers are the shiny new thing. We have this great solution so let’s go find problems for it,” he said. “That said, containers are great because you can develop a freestanding piece of software that’s part of something bigger.”
When some of the teams at WWT Asynchrony Labs started using containers, they were installing Docker on a single virtual machine and deploying to that. Then they realized they had a single point of failure.
“You get to that point and you realize you need an orchestration engine that will allow you to do different types of deployments, such as Canary deployments. That way, you don’t have any down time,” said WWT Asynchrony Labs’ Perry. “We have quite a few teams that have asked how to handle downtime and we’re like, ‘you don’t have to worry about that now. There are tools and orchestration engines that can run things in a way that you don’t have to worry about downtime at all.”
Security and IoT
Should security be built into code or should operations ensure security? The answer isn’t either/or, it’s both. Cloud platforms provide encryption by default, although organizations continue to lose control of IP, electronic health records, and other kinds of data. Nothing is inherently secure, which is why DevOps teams need to be more vigilant about security on a number of levels.
“Part of a developer’s job, in addition to writing code, is also making it secure,” said Storage.io’s Schulz. “Someone has to make it easy for them or tell them, let these people in, put these safeguards in so your application and data are safe.”
To some extent, the IoT is driving the need for developers and operations to work closer together. More developers are getting pulled into IoT software development regardless of what industry they’re in. At the same time, operations and IT need to worry about a larger infrastructure footprint and the scalability to support it. Moreover, a lot of IoT devices aren’t being designed with security in mind which can wreak havoc on infrastructure.
“DevOps and IoT go together,” said Storage.io’s Schulz. “You’ve got a particular item, maybe it’s a sensor or a switch or a monitor of some sort that has the ability for somebody to write a piece of code for managing, monitoring, and configuring it and plugging it in somewhere. That’s ideal for creating a module, which you might develop on your desktop or in the cloud, but it’s going to be deployed somewhere else.”
The Bottom Line
Modern software development and software-defined architectures are both necessary for organizations to achieve the level of ability they seek. To do that, developers and operations have to overcome the points of friction that still exist between them. Doing that requires bilateral understanding at several levels including understanding something about what the other side does. For developers that means understanding what operations people do day-to-day, how it impacts software delivery, and what the production environment looks like.
Working software is everyone’s problem, particularly in this modern era in which businesses compete on software.
