DevOps news and trends to watch in 2021
Join veteran IT journalists in a conversation about the top development, DevOps, low-code and CI/CD news in 2020, and where these trends will take us in 2021.
IT organizations and the vendors they rely on did not stop moving this year. The pace of change won't slow in 2021, so developers and DevOps adopters best be prepared.
To make sense of this year's software development and DevOps news, SearchSoftwareQuality's editors spoke to TechTarget senior news writer Beth Pariseau and news director Chris Kanaracus.
This wide-ranging discussion touched on the following topics:
- DevOps remote work. Remote work has made prominent aspects of DevOps that developers did not previously emphasize.
- Low-code trends. The search for customer success stories continues, as the big cloud vendors race to enter the low-code space.
- CI/CD tooling trends. The CI/CD market has seen many acquisitions and consolidations; CI/CD toolchains are becoming easier to buy.
- On-premises versus cloud tool shifts. Vendors like Atlassian got serious about cloud-hosted tooling in 2020.
- Enterprise open source software trends. IT enterprises must be responsible patrons of open source code and still maximize their control over intellectual property.
Editor's note: To see more news and analysis about cloud, DevOps, low-code and more, visit the author pages for Beth Pariseau and Chris Kanaracus.
So, welcome Chris and Beth, thank you for joining us for this episode of Test & Release. And also special shout out to Meredith, for joining us for the first time.
Beth Pariseau: Thanks!
Chris Kanaracus: Great to be here, yeah, it's a pleasure.
So, Beth, one of the topics you wrote about this year -- and I don't think it'll be a surprise to people -- is the switch to remote work on DevOps teams [and] IT organizations in general. So, could you talk about what the switch to remote work and the implications unique to DevOps have been, and what you've heard in your coverage?
Pariseau: What I've seen about the switch to remote work isn't so much its effect on DevOps as DevOps's effect on it. It seems like the further along companies were in their digital transformation -- including DevOps, continuous delivery, cross-functional collaboration, small iterative changes, documenting changes, measuring results -- the better off they were, when everyone had to go remote. In fact, it enforced some DevOps best practices that maybe hadn't been closely followed before.
The idea, ultimately, with Agile and DevOps is that you document and you measure your work. But when people were in person, they could stop in the hallway, or go over to somebody's desk, and they can have these side conversations. So, there was still a lot of tribal knowledge that was happening. That wasn't going to fly with remote work. So, in a way, [working from home] also made some people adhere more to DevOps best practices. And the thing that's really hard is that if you hadn't started your digital transformation -- you hadn't moved to cloud, you hadn't put together a centralized CI/CD pipeline for app releases -- [then] the pandemic is not only that much harder, but the pandemic makes it that much harder to move to those things to try and survive.
But in the story that I wrote about this [linked above], there were some interesting presentations by SRE experts about how SRE approaches and DevOps approaches of iterative improvement could be applied to focus on one incremental improvement at a time ... because just about anything is better than the old way of doing things while everyone is remote.
When you say 'SRE' you mean site reliability engineers?
Pariseau: Yeah. It's a term that is becoming more popular. There's still -- like DevOps -- disagreement about what it actually means. But generally, the idea of DevOps is that developers operate their own applications and troubleshoot them. So, that leaves IT ops people to mostly become either … some call themselves SREs, others call themselves platform engineers; they create the infrastructure where developers deploy their apps, support it, try to make it more reliable. And if there are systemic issues with that platform, they troubleshoot that.
To pivot ... to a slightly different topic, I also want to pick both of your brains about low-code because low-code continue[d] to be a trending topic this year. I recall earlier in the year, Google ventured into low-code, I believe they bought AppSheet in January. And I think the stated reasoning, in the reporting there, was to bolster their cloud offering.
So, this might be a good question to get Chris's insight on. [I'm] curious in terms of the low-code news you've seen throughout the year: Is that a phenomenon you've seen reoccur where cloud vendors pair their low-code and cloud offerings in tandem? Or, if that's not the case, I'm curious what other observations regarding low-code you've both seen.
Kanaracus: What's telling about the Google AppSheet acquisition was that they already had something called App Maker, and they killed it after they bought App Sheet. Because App Maker, it was a flop. It came out in 2016 and no one used it. And the low-code idea has been around for decades, maybe three decades; looking at Microsoft Visual Basic [as a] type of lower-code environment. It's still around and people have written thousands and thousands, if not millions of corporate applications with it over the years that still run in companies today.
In terms of the cloud vendors: Yeah, then AWS came out with Honeycode in the middle of the year, using a spreadsheet type of interface to provide a low-code editor. I guess the bottom line is that this stuff has been swirling around for decades. There are some bigger independent vendors like Mendix that are making real money. But they come out of BPM [business process management], which is a much more complicated previous generation of technology. And no one's really cracked the code, to make a terrible pun, I think a lot of the stuff we've seen from the big cloud vendors is just to get a prototype type of product out there so they can say, 'We have low-code; we have a flag in the ground.'
If it sounds like I'm skeptical, I am; I just don't see it as something that is sustainable, long term. ... All these things are walled gardens. ... It's not like a traditional enterprise application, which was written in common languages and written in an IDE [integrated development environment] and that can be updated by other people after they leave the company and can be ported to different types of hardware. And especially with the cloud vendors, it's all on their own sites, it's very locked in. I guess I'd say there's lots of hype, but ... true success has alluded this space for decades. So, I don't know, we'll see. Next year will be interesting.
Chris, can I ask what you expect next year to bring with low-code? Will it be more of these large vendors acquiring low-code IP? Or, do you see vendors developing low-code products in-house? Will the big cloud players have a move into low-code?
Kanaracus: I think you'll see both, but what I would like to see more is examples of real-world customer success; not just, 'Hey my power Excel user can write some widget with this thing.' But how did that translate to business value? And for the company: How did they make the company money? How did it help a company enter a new market? How did it help the company optimize its workforce? And things like that. [I] want to see that on the trade shows next year from these big vendors: customers on stage really telling their story. Then, I'll be less skeptical.
I will piggyback on that idea of hype and vendor changes and actual users, to talk about a trend that I think is more [in] the mainstream, but has certainly gone through the same struggles, which is CI/CD tools.
Beth, I know you've written countless articles about the various upgrades, acquisitions, capabilities and integrations between all the different tools that you would use for a DevOps or CI/CD pipeline. Was there anything that stood out to you in 2020, and anything that you see continuing into 2021?
Pariseau: I think the major trend was unsurprising, but pretty consistent, which is that CI/CD pipelines have gone from being these bespoke artisan creations, cobbled together from raw open source code by ninjas and rock stars, to something that mainstream enterprises want to use, but don't want to build from scratch. They want support, they want the integration ... they're not in the business of building CI/CD pipelines. They're in the business of delivering whatever their business logic is in their apps.
So, you're starting to see vendors, especially big incumbent, IT vendors, like Red Hat, CloudBees -- which is the commercial backer of [open source] Jenkins, which is still a widely used tool in that space -- and Atlassian putting things together that are more end-to-end, to use a loathsome marketing term. And Red Hat has also begun to integrate CI/CD more deeply -- including event-driven CI/CD -- with OpenShift. So, they're trying to connect the CI/CD pipeline with that platform that developers are deploying to and SREs or platform engineers are supporting. So, when you see things like CI/CD go into this prebuilt, packaged set of products, it's going mainstream. Basically, this isn't a bleeding-edge thing anymore, where you have to be an O.G., [or an] open source programmer to put one together.
So, it's also not the most thrilling and exciting trend, but there's [a] majority. I've been on the beat since 2016, that overlaps pretty nicely with the mainstreaming of things like DevOps and CI/CD. So when I started, there was a lot of talk about, 'What are the best tools to put together?' and, 'How do you put them together?' Now, there's more talk about, pragmatically, 'Which of your incumbent vendors are you going to buy this whole package from?' And … competitively there's consolidation in the industry and attrition. Or, there will be because, ultimately, there's a pretty heated battle between vendors that had been the rulers of different domains. And now everything is so cross domain, that everybody's trying to do everything for everyone. And users end up having to pick who their main supply of DevOps software is going to be.
And this sounds like this consolidation might be helpful if you are in the early stages of CI/CD, and you're looking at more comprehensive pipeline packages. A lot of our readers and listeners, I think, have some stage of CI/CD maturity and might already have 10 to 15 tools, maybe even different pipelines across different parts of their companies. So, they might be facing a more challenging decision to make about whether to bet on one company or continue stringing together these pipelines that they've built.
Pariseau: Well, best of breed versus buy is a time-honored debate. And [as for] what's in fashion, the pendulum swings back and forth. One thing that I think is good about this current wave of change in IT is that most of it at heart is built on open source. So, at least theoretically, the core code of something like say Kubernetes or Jenkins -- or even observability tools, like Prometheus and Grafana -- doesn't belong to your vendor. It's not quite the level of lock-in people used to have with proprietary software vendors 10, 15, 20 years ago. But nobody really wants to be managing 10 to 15 tools at the end of the day. And at the end of the day, most companies are really not in the business of putting together a CI/CD platform. They want to ship their code faster; that's what this is about. And they want to link their business more tightly with what their tech folks are doing. And so those are the real goals that people are starting to focus on, with CI/CD more as a means to an end.
So, to try to stay along the lines of tooling options available to developers and, [what's] shifted in the past year ... Chris, can you just give us a brief overview of essentially, on-premises tools versus cloud-side tools?
Kanaracus: The big thing there in 2020 -- and really starting late 2019 and on -- we saw a shift toward cloud-based code editors, coming from the big vendors -- your big names like Google and Microsoft. And at [Microsoft] Build 2019, late 2019, they demo'd a few of these newer things. And you know that something's resonating with a developer crowd when people stand up and cheer basically. And I wouldn't say they were throwing confetti, but people were psyched at what they saw. You can do stuff like pair programming in real time, in the cloud with another developer. In this age of 2020. To be able to do that remotely was really huge.
Google's also really been leading the way with Google Cloud, a lot of extensions to existing IDEs, Visual Studio and others, so they're trying to keep one foot in what remains now probably 80% of programming is done in on-premises workstations locally, but trying to bring more functionality into the cloud, where all that work is done. So, I think that will be pretty hot in 2021. A lot of people have kicked the tires on these newer [code] editors, and we're going to see a lot more of that in 2021.
Thank you, Chris. And you, of course, brought up a number of vendors like Google and Microsoft. But if my understanding is right, Atlassian is another vendor that exemplifies this tension between on-premises and cloud. And, Beth, you've written about Atlassian a fair bit. I was wondering, could you talk a bit about them and what's happening with them?
Pariseau: Just as a backdrop, tying it back to the pandemic and remote work like Chris touched on: Companies that were still relying on VPN access to on-premises data centers in the pandemic also struggled much more than companies that had already switched to cloud-based and SaaS services. Because all people need when they work at home, if you're using a cloud-based software product, is their internet connection, and ideally some type of authentication or user identity and access management system. Again, if you were far along on your cloud and digital transformation transition, you would have [those cloud systems] in place already. And so that also separated the haves from the have-nots as the pandemic came down.
And on the other hand, there are some very large enterprise organizations that still have security requirements that they cannot put certain applications or data in the cloud. Cloud has become so much more mainstream [that] it's not even noteworthy to say it's mainstream anymore. Big businesses -- including banks -- do trust cloud infrastructure and application services. Amazon has the [AWS] GovCloud that the CIA uses and DoD. It is doable, but there are some -- especially financial -- institutions that just do not want to move certain apps to the cloud. So, that's where people start to talk about hybrid cloud. And you have a mix of on-premises and cloud-based resources.
And Atlassian is interesting, because they started as an on-premises licensed software vendor. And initially their cloud offerings were not very strong in terms of their reliability. But a couple years ago they migrated their back end to AWS instead of their own self-managed data centers and started offering versions of their products on the cloud. And gradually those versions started to become their clear preference in terms of what they developed features for first, what they shipped first, what they touted the most. And they started to try to gently push and incentivize users to switch from their on-premises products to cloud versions [with] things like discounts and migration help and free migration assessment tools and that [sort of] thing. And people could see what was coming earlier this year, but there was still some surprise and consternation when they ultimately went from the carrot, so to speak, to the stick in late October and discontinued their Server line, which was their entry-level license for on-premises products.
[Atlassian also] drastically hiked the licensing for data center versions of their products -- the fully featured enterprise versions of their software. And [the company] as much as said that they're cloud-first now, and that's how it's gonna be. They handled that transition as carefully as I think anybody could and they were very responsive to customers' concerns. But at the end of the day, companies that are large enough -- with enough highly regulated data to deal with, like banks -- are going to pay what it takes to buy those data center licenses. But it was the first time I've seen a company on my beat really starkly enforce that transition. Maybe because a lot of [vendors in this space] didn't start from on-premises in the first place. But I think it's very clear that people with on-premises investments are in the minority. As apps modernize, as digital transformation happens, and most of the major concerns about cloud security and reliability have been put to rest for most mainstream enterprise companies, I think we can say we're finally there.
Kanaracus: Hey Beth, in the SaaS years -- in the run up to SaaS, in that era -- the pitch was always that, '[Cloud] is better than on-prem, because we can update you four times a year, we'll manage everything, obviously. You'll get rapid iteration on the software, not just [inaudible] but cascade a stream of features instead of like you having to install a new version once a year, once every two years.'
Is that the same case? In Atlassian, with Jira, do people want that rapid iteration that the cloud can give?
Pariseau: Well, so [Atlassian] also overhauled Jira Service Desk recently in that manner. And you have to be careful, because there are some companies that have a big appetite for change, but there are others that are going to freak out if you shift the ground under them too often and too rapidly and too drastically. So, Atlassian, at least with the Jira Service Desk, it's now Jira Service Management. When they made that transition, they automatically gave everyone the new features, but they didn't take any of the existing ones away yet. And so I think that's an appeal to some companies, but when you're selling to mainstream enterprises, that is not going to apply to everybody.
Something else that I think might be a source of tension for some IT enterprises [is] something else you, Beth, have wrote a wrote about this year: company's dealings and sometimes fraught dealings with open source contributions. And I use the word fraught, because I think a focus for your articles.. you talk about conservative legal departments [that] have issues with, like I said, open source contributions. But yeah, not to make it too much of a leading question, but can you lay out essentially, the landscape of companies' dealings with open source contributions and what the current nature is?
Pariseau: Sure. Open source … it's gone from a fringe thing, to mainstream, to just a requirement now. I think the appeal initially was, 'Hey, it's free.' But as they say in open source circles: 'There's free as in beer and there's free as in a puppy.' And open source is definitely free as in a puppy. You get the thing for free but you got to take care of it, and you got to raise it and you got to train it, and you need someplace to turn [to] for support. And it's not just as simple as, 'Hey, install this license, it's free.'
And so another one of the appeals for open source software is that you can modify it as you need to. But not only does that take skills, it also takes a new approach to corporate IP, and what do we own? What is our value-add as a business? What's proprietary to us that we need to keep in-house? And what can we contribute back to the community? And that butts up against corporate culture first. So you have developers -- and everybody wants to hire whiz-kid developers -- but the thing is [that] whiz-kid developers tend to be pretty staunch advocates for openness and open source in what they're creating. And if you want to recruit the top developers in the world, you're going to have to let them use open source software and create it from within your company. A company's open source reputation is most valuable to it as a recruiting tool, for IT people in the midst of a skill shortage.
So, companies now have to face this issue. And there are still some things that are not settled about … there's no legal precedent, really. Google versus Oracle -- in terms of APIs and Java -- is the closest thing to a possible coming precedent, but even that might not necessarily apply in all cases. And it's uncharted territory from a legal standpoint. And it's also a pretty widespread issue where you have upper management and people like risk management and legal teams that are not deeply technical, versus developers that are deeply technical, but not well-versed legal experts. And you have to bring those two sides together. It's not just about collaboration between different IT factions, it's also about integrating the different functions of the business around what's being done technically. So, that's a big challenge for people culturally. People thought making devs and ops work together was hard. Try putting a corporate lawyer in a room with an open source software developer -- they're speaking different languages.
And then the other thing that becomes its own whole rabbit hole is licensing. And companies like, for example, Bloomberg, who's in the rare position of having an executive in the position of managing: What do we license [and] how do we license open source things that we create within our company, and making that coordinated decision about what's kept in house and what's not, and working with these open source foundations. There's just certain things -- practices around legal contracts, documents, copyright agreements, [that] has not caught up to modern software. And basically, companies are starting to take an interest in being the owners of the code for purposes of open sourcing it instead of the individual developer. And in part that ties back to that recruiting tool, ties back to visibility -- they want to be seen as good corporate citizens -- and frankly, they don't want this developer's work to belong to their employee; they want it to belong to their company, if it's done using company resources and time. So that's its own whole issue. For example, Kevin Fleming, who's the person I talked with at Bloomberg, had to work with multiple software companies to have Bloomberg itself, the corporate entity, be the copyright owner for purposes of open sourcing some code just late last year/early this year, and those companies had never actually done that process that way before.
So, it's still a frontier in all this, but it's going to start coming up for more and more companies, because you just have to use and contribute to open source. And it also reflects the market maturity where: 'Okay, we will use this stuff, okay, now, we're getting really good at using this stuff, except there's this one thing, we need it to do that it doesn't, so okay, we'll make that.' to, 'Well, we really need to share this with the community, because we got all this stuff based on other people's work and donation to the community. And if we want to be part of this whole movement, we really need to give back,' or else you get, you get a bad reputation. And then it becomes harder for you to get community support around using the software. But then it runs into all of these legal questions that are, in some cases, centuries old and that still need to be updated.
You brought up DevOps in terms of bringing developers [and] operations together. And, of course, the tack we always take to talk about with DevOps is finding how to collaborate. But it sounds like with the dynamics you're describing, there might [have to] be concessions on one side or the other, as opposed to being collaboration.
Pariseau: In some cases.
Yeah. But [it] also reminds me that people in technical fields -- whether they're DevOps engineers, or developers, or QA professionals -- never a bad idea to brush up on soft skills, [to] be able to communicate [and] work with people from outside of their department.
Pariseau: There's just no way around it. You just can't not deal with people anymore. Are you doing technical work? [Then] you've got to collaborate with your security and your ops guys, a developer and vice versa. Or, it's communicating with everybody from the upper management of your business to the legal and risk folks. Even the systems that people are using are converging. For example, there's a software category called enterprise service management that's emerging, different companies like ServiceNow -- Atlassian, actually, are getting more into this field. They're starting to expand their products like Jira Service Desk to accommodate legal and HR teams. And they have low-code or no-code interfaces for those constituencies into products like Jira Service Desk, so they can offer things like corporate intranet types of services, so it's not just about IT help desk anymore. [Say] I need to onboard or decommission a new employee, not just in terms of getting their laptop set up, but in terms of their status in our HR system, and you're going to use something like Jira for that.
It's easy to forget this in all the technical weeds, but the initial goal of Agile and DevOps was to improve business; [it] was to deliver business logic faster and more competently. As the world becomes ruled by software, every company now at least needs a mobile app and a website. And so, every company now has to have some amount of software delivery of whatever it is their product is. And that means that you no longer have this separation between the technical propellerheads down in the dungeon doing whatever alchemy it is they do and the business people out front doing what it is they do. You really have to bring those things together. It's not just a nice to have, it's not just an idea. It's necessary if you want to survive in the 21st century.
Beth, Chris, I want to thank you both again for joining us. And also for the news coverage you [both] wrote [that] very much informed this conversation, your news coverage in the past year.
Top 10 software development and test tips of 2020