Serverless infrastructure calls for updated IT ops expertise

Serverless infrastructure requires refocused ops skills

In a world of serverless infrastructure, IT ops must think about systems-level engineering rather than server-level tasks.

"The idea of running Nagios or updates for OpenSSL will fade away, because those problems don't exist anymore," said Matt Weagle, director of infrastructure and operations at ShiftLeft.io, a security software startup in Santa Clara, Calif., and the creator of Sparta, a framework that transforms apps into AWS Lambda-based microservices.

Instead, the concern for IT ops pros might be how to optimize serverless infrastructure to reduce bottlenecks, such as by sharding traditional databases, or to convert to more modern database systems that better support serverless traffic patterns, Weagle said.

Systems-level architecture isn't the only new focus for IT ops pros in a serverless world. Infrastructure automation skills, especially the ability to write infrastructure as code, will also be mandatory.

"The serverless side is all about infrastructure as code, whether you're using [AWS] CloudFormation or something like [HashiCorp] Terraform," said Dave Townsend, principal software engineer at Matson Inc., a cargo shipping company in Honolulu. "We're using CloudFormation for everything."

The company's ops team must deepen its knowledge of CloudFormation to help Townsend's architecture and innovation team secure and troubleshoot its serverless mobile app. Townsend's team found that its app needs a web application firewall to block malicious traffic, and wants the ops team to learn how to deploy that with CloudFormation, as well as add it to the build pipeline for other apps.

"Observability and monitoring are also going to be key," Townsend said. Matson uses a tool by IOpipe to monitor serverless infrastructure metrics, which the ops team can access but has yet to exploit.  

The Matson ops team also runs the company's Splunk environment for log monitoring, which Townsend would like them to bring into the serverless fold.

"I'm looking to them to get [Amazon] CloudWatch logs into a Splunk dashboard so we have a nice view of the serverless world and all the health of all the components, and they seem pretty excited about that," Townsend said.

Serverless infrastructure shifts troubleshooting approach

The ability to observe and monitor any infrastructure is only as useful as its contributions to meaningful incident response, and serverless infrastructures require IT ops to shift its approach there as well.

Serverless infrastructure isn't as transparent as other cloud infrastructures, where users have visibility into the underlying container or host, but it lends itself to rapid deployment. Thus a grasp of immutable infrastructure concepts, in which systems are continually redeployed rather than updated to make changes, will come in handy for IT ops pros who work with serverless platforms.

"If you're having a problem and your logging is terrible, you can push out a new function with better logging in a matter of minutes, rather than having to go through a much larger, heavier process for a monolithic app," said Ernest Mueller, director of engineering operations at AlienVault, an IT security firm based in San Mateo, Calif. "To a degree, the speed of change allows us to compensate for the lack of introspection that serverless has."

Ops pros who can code may do that redeployment, or they may develop a set of overall logging strategy guidelines and let developers do the implementation, Mueller said, which puts ops into a more consultative role than it's had historically.

"I could scale my team up with our growth, but I don't think that's responsible nowadays," Mueller said. "I'd much rather equip developers to write their own infrastructure code, their own monitoring as code, and the expert consulting and tooling to make that work."