Atlassian Rovo AI platform grows to 3 million monthly users

Transcript - Atlassian Rovo AI platform grows to 3 million monthly users

Beth Pariseau: From Informa TechTarget, I'm Beth Pariseau, and this is IT Ops Query.

This podcast distills the signal from the noise about enterprise software development and platform engineering. Each week, we'll talk to expert guests about the latest tech industry news and trends that engineering and IT leaders need to know.

Don't forget to subscribe to IT Ops Query for more conversations on AI and the future of the enterprise digital workspace.

Jamil Valliani is vice president and head of AI product at Atlassian, which he joined in 2023. Before that, he spent 20 years at Microsoft, where he built and led product teams for Microsoft's Cortana and Bing.

Since Valliani joined the company, Atlassian has expanded its AI offerings from an AI assistant for Jira and Confluence to the core of its cloud-based enterprise service management platform under the brand Rovo. It has also officially started phasing out its on-premises Data Center edition, confirming in September that most of that product line will reach end of life in 2029.

This week, Atlassian held its Team '25 Europe conference in Barcelona, where it continued repackaging its software products and AI agents into Collections, a strategy first introduced at Team '25 U.S. with Teamwork and Strategy collections. The new Software Collection launched this week includes existing tools like Bitbucket, Bitbucket Pipelines and Compass, along with the newly available Rovo Dev agent and DX, an engineering intelligence company Atlassian acquired in September.

Welcome, Jamil.

Jamil Valliani: Thank you, Beth, for having me here.

Pariseau: Thanks for joining us. I know you're busy with Team '25 Europe coming up next week, so I appreciate you taking the time.

Valliani: Absolutely. It's wonderful to have a chance to talk to your audience and tell you a bit about what we're doing with Rovo and AI around the company.

Pariseau: Sure. So, just to take a step back for a moment, at Team '25 U.S. in April, there was a lot of discussion about the industry's growing pains with AI usage, especially ROI. How do things look six months later, just in general?

Valliani: So, we're very proud of our progress and what we've been able to deliver to customers just in the past six months. When we talked and when we kind of heard that buzz about six months ago about how we get the ROI, one of the first realizations we had is that we need to get these tools into the hands of substantially all of our customers.

And so we announced this pretty significant change in how we're going to offer Rovo by integrating it into the platform and giving it to, pretty much, to everybody who is a paying subscriber, and really focusing on getting them to learn the tools, learn how they can transform their workplaces and their processes with those tools. And that would be the way, sort of, forward. Right? I don't think this is something you just buy out of the box and just say, 'Ah, now I'm AI-enabled.'

So, just in that six months since we said we would do it, we did it. So, we now have over 3 million users every month using Rovo capabilities across the product line. The Rovo Search product grew over 100 times in its usage. In that time, as people, like, really got to see what enterprise search could really be like and feel like -- something so much more relevant than what they're used to in their offices -- Rovo Chat is getting 50 times more usage than it did six months ago. And that's again, people are just saying, 'Oh, I do have an AI teammate that does know what's going on.' And they're starting to use it more and more, and they're understanding that the capability is there.

And I think the most exciting is that we now have, just this last month alone, over 800,000 business workflows of all types of flavors have been accelerated through agents. And that's like a 50% month-over-month increase for us.

Pariseau: Wow.

Valliani: And that was about, 9,000 agents were created just last month alone by our various customers. And that just represents so much real-world savings.

And we've talked to customers. At Team Europe, we have Mercedes-Benz who'll talk to us about what they've been doing with agents, trying to eliminate that manual duplicate ticket reviews, and automated reporting will help them all focus more on doing what they do best, delivering great cars. Royal Caribbean has been using unified search now and Rovo to search across their Atlassian SharePoint and OneDrive content. And Tempo.io has actually accelerated a lot of the work processes from their teams across the board, in some cases, by as much as three times.

So, it's really, really exciting early benefits that we're seeing from these customers just in the six months since we started this rollout. So, it's been exciting times.

Pariseau: Was folding the AI into the subscription, as you mentioned, the real catalyst for that? Or was there something about the tools that have rolled out that helped there, too?

Valliani: Yeah, I think there's a bit of both. So, for sure, the ability just to, for everyone, to start trying, especially Search and Chat, without having to think about, 'Oh, I need to add and sync my subscription and set everything up.' For them, just to see it start working one day, I think, was a huge catalyst, and just giving awareness and comfort with the new tools.

We've also been investing a ton in improvements. So, for example, search relevance has improved by over 20%, performance has now improved by 30% there. We continue to shift, like, just more and more improvements into Chat that make it feel like a consumer experience, because we know enterprise customers never get that benefit of that high-grade consumer experience, and we thought that was important to bring. And we see it -- we do that, the engagement just goes up a ton.

So, we've been doing all that in parallel. Where I think we've made the most visible progress is in Studio. Six months ago, we had a fairly simple experience around creating agents, for example. Now with Studio, and what we're announcing at Team and bringing into beta, is this ability to have a single interface where you create, through natural language, everything from new agents to automations to even entire Forge apps.

And as we've sort of seen customers trying out these capabilities, that improvement that I mentioned around workflow, automation and improvement -- that 800,000 number in just the last month alone -- like, that we think is just gonna skyrocket.

So, it's been a combination of both. But certainly, I do believe just giving it to everyone, in their hands, and helping them think about their own application at work has been a substantial move and benefit for them.

Pariseau: Well, great. A lot has changed in a very short period of time.

Valliani: Fun times.

Pariseau: So, speaking of ROI, I know that's part of what DX has been surfacing for customers and its tooling and what will be part of the Software Collection with DX. But now that that product is part of Atlassian, how do you expect that to expand?

Valliani: I think that the trend you'll see is when we look at our software teams -- our original audience, who we still care deeply and serve deeply -- everything we're doing in that Software Collection is about accelerating every step of that development cycle.

And I think what DX gives us and our customers is visibility into that whole cycle. You know, where are the stacks? I think a typical developer only spends a portion of their workday actually writing code. They're often doing other things, like managing their backlogs, reading spec documents, trying to work on designs, communicate with colleagues.

These are all things that you'll have optimization opportunity, but without that insight into saying, 'Ah, here's where we see you setting more cycles than necessary. What if we can optimize that with AI?' We think that DX gives us that insight, gives our customers that insight. And then you can see us, I think, you'll see us starting to actually bring tools and agents to market that optimize some of the most common pain points in that entire workflow.

Pariseau: Cool, data-driven decision-making.

Valliani: Absolutely, yeah. We think that's where a lot of companies are going to be heading in the next year or two, as they really think about AI transformation in the workplace. They're going to look at these old processes they've had for so many years and really just systematically break them down and say, 'Okay, where can we simplify and accelerate with the tools that are coming to market?'

Pariseau: And in the meantime, what does Rovo Dev bring to the table at GA? Does GA mean there's a service-level agreement, or is there anything different about it than in preview earlier this year?

Valliani: In the preview, we were really having a certain number of our customers just get early their hands on it and giving us feedback. In GA, now you'll actually see the ability for Rovo Dev to go and actually use it the same way Atlassians use it, which is actually at scale, every customer can now go and have Rovo Dev look at the issues in their backlog and actually get pull request recommendations written up by Rovo Dev proactively.

And then there'll be a model where, for every request that is successful and then review accepted, like, we'll have a chargeback for that, right? Which is where we feel like is the most fair thing, where people, as they see benefit, they pay a credit, essentially. We think that's like a fair way for folks to safely experiment with this. And as they're getting more value out of it, we're able to go and generate more value for ourselves. We think that's the right model there, and that's what we're bringing to market with Rovo Dev going to GA.

I think we'll also see, our customers will also experience that the quality of code generated by Rovo Dev should be substantially higher than perhaps what they're used to with other tools. And that's because Rovo Dev has so much context. It's actually pulling data from the Atlassian properties that the customer has, like it has the Jira issue, and the entire history of comments and other similar issues that that may have been solved, and the pull requests behind them, and the other knowledge base articles that map to it, that direct us to Confluence, to GitHub repos.

All that context really helps us, in measurable ways, show Rovo Dev actually has a higher success rate than any other tool out there that, we think, will show the value to our customers.

Pariseau: Great. Okay, and just another question on DX: Is there anything that they bring in that will give people visibility into the conversations people are having with AI agents -- what were the prompts, what were the responses -- and kind of evaluate, trace back those conversations with the code that resulted?

Valliani: In DX, there's certainly capabilities there that we'll be exploring. We actually generalize that ability. So, when people build agents in the studios, one of the things we'll talk about at Team is actually the ability for any agent, for the creator, to actually go and look at evals. For example, they run evals and say, 'Hey, I have test cases. Let me go make sure that they're actually being run successfully and that my agent is getting better.'

There are also ways for them to look at real examples and real data that customers have submitted and say, 'Hey, was this successful? Was it not?' So I can learn from that. And then there's a bunch of auditability, so getting insights, for example -- was my agent run successfully a lot? Was it run by a lot of people or by a few people? Those are all tools we're actually bringing to the platform so that anyone who creates agents and runs them on our platform can actually see that and use that to hopefully make their agents better.

Pariseau: And will they be able to roll that information up to managers -- say, software engineering managers -- just to see how the team is using these tools and how they're performing?

Valliani: Yeah, we're certainly working on those capabilities. We're introducing, like, a lot of those basic controls first. So, a lot of customers, for example, will say, 'Well, I want my agents to only work for a certain group of people' -- for example -- 'or just be visible only to me until get to a point where I'm comfortable with the quality to raise them.' So, we're introducing those controls first, and then as we get more and more signal back from our customers, we'll certainly expand in the directions that they lead us to.

Pariseau: Okay. With things changing so fast and furious, what do you think is the biggest remaining challenge for enterprise AI agent adoption?

Valliani: Oh boy.

Pariseau: What are people up against right now?

Valliani: It's hard to boil it down to one single thing. Maybe a few areas that I think a lot about. The first is, still, for a lot of customers, getting the tools deployed in the hands of their end users.

We made, certainly, a lot of progress from our side. We said, 'Look, let's just make sure that the packaging and all that's not in the way, everyone should have access to it.' But for a lot of companies, they're working to figure out what their standards are, what their practices are, which teams will have access first. And our goal with them is to walk with them on that journey, make sure that they have all of the, sort of, security and control mechanisms that they need to make sure they can roll this out in a way that they feel comfortable with.

So, for example, we've started offering an Atlassian-hosted LLM, so there are some folks who are not comfortable -- or can't, for regulatory reasons -- use an external LLM provider. So we said, 'Okay, we'll host one ourselves,' and that should hopefully help eliminate a barrier. Some folks need role-based access controls because they can have certain departments adopt first before other departments. So, we've been trying to adopt that.

So, we're trying to meet customers and the admin community where they're at on that front by saying, 'Let's give you all the controls so that you can manage your risk and kind of ease in there.' But that is still, like, an area we partner a lot with, just with our bigger customers on.

The second is really, I kind of call it the etiquette, right? I think we're introducing a new capability, like 'AI as your teammate,' and that's a pretty novel concept, right? People are seeing these agents come in, even if you're on an online meeting these days, you'll see, oh there's a recording agent in my meeting. And it's like, oh, do I have to behave differently? And what's being recorded? And what's it gonna say about me later? It's kind of a novel thing.

And you can imagine that same thing coming into other tools, right? If you see an agent starting to work on your behalf, and you're like, oh, did I want you to do that, did I not? And so what we're doing a lot with the advances we've made is saying, hey, like, how do we go and give people all these tools and the best practices so they know how to engage the agents, how to make it part of their team, how to think of them as accountable team members, and things that can be controlled at the right point in time but also then eventually trusted to do more and more. Something simple, like being able to, in a Jira issue, @mention an agent, right? And say, 'Hey, can I assign this to you?' Or 'Can I mention you and trust you to go and take a specific action, but then not take it over the top?'

We're going to show that, we think we've made actually substantial progress, and we're excited about sharing, for example, the ability to assign even a third-party agent a task and seeing that work out really well.

So, I do think, though, that as customers bring these capabilities to their offices and their workplaces, there's going to be a bit of an adaptation period where the people say, 'Ah, this is what the agent should do. This is how we interact with it. This is the, sort of, norms.'

A simple example could be today, if you're going on holiday for a couple of weeks, there's a practice saying, 'Hey, like, why don't you write down the steps that you take for certain tasks so that somebody else can go through that.' Right? It's like, well, we want to think the same way about an agent, right? Maybe there's certain areas where, hey, like, I'm the single point of failure, or this is just taking too much time. What's the best way for me to capture this, push it to an agent, make sure that the agent works well enough, and then share that with the team, and then describe it to the team, right? That might be the new way of doing a task like that.

But all early muscles that are getting built right now, and it's been actually a lot of fun for us to go work with so many customers to say, 'Ah, let's see how you're doing and what's working for you,' and then slowly build those best practices.

But yeah, I'd say those two things are what we work a lot on.

Pariseau: So, how have you seen humans start to adapt? How have you seen job functions start to change as a result of using these tools?

Valliani: The starting point usually is, like, just tell me your pain, right? Everyone has things in their job that they just, they wish they could do less of, and if they could do less of it, they'd do something else more.

And so, as we go to customer sites and we say this, 'Tell us about your day, tell us what some of the tasks you do are that just seem so frustrating.' And oftentimes in there, we find an opportunity. Right? Say, 'Hey, let's go find one thing that in that flow -- it doesn't even have to even be the whole flow, could be one part of it that's really painful -- and let's start figuring out how we can go and work with you to build the right agent or introduce the right search or track capabilities to make that a bit better.'

And we'll work together, we'll have a proof of concept, we'll try it out together, and they'll say, 'Oh, that really is better.' Even if it's 20%, 30% better, that's still better than before. And we know that we can work on it. So, as we get signal back, as people give us all different cases, we can say, 'Okay, let's go and give more instructions to the agent.' Let's perhaps create multiple scenarios for the agent to handle, which is the capability we have now. Let's create an automation that weaves several agents together that might solve more and more of these cases.

So, we can kind of expand that way once we get our first foot in there together and solve more and more problems.

But at the end of the day, when a customer feels like, hey, this first agent, or this first problem has a set of agents working to almost automatically to make it a bit better, that usually is a good way for people to start thinking, 'Oh, what are the five or 10 other things that can happen?'

I find that's also paired with, like, leadership. At Atlassian, we have our founder, Mike, we have our president, Anu, all the C-suite are often building their own agents or running searches, and they are setting up channels in Slack to talk to their teams about, 'Hey, I tried out this agent, or I just built this thing, to make this better. What are you all doing?' And their teams seeing their own leaders transforming their patterns and their work really inspires others.

And they've actually, that's actually taken hold in a really magical way. Like, now we have a case where every department has, actually, a set of agents that are helping transform their work, and those leaders are evangelizing and promoting the use of those agents, which has, I think, also helped a ton kind of reinforce that behavior, that transformative behavior.

Pariseau: Are there still trust issues? Are people still worried that they'll be replaced by agents?

Valliani: I actually hear it a lot more in, like, non-employee conversations and non-company conversations. I hear more from, like, you're talking to folks in the investor and media and that kind of community.

And because, I think when people actually start using them and they apply them to their work, they kind of realize, 'Oh, this is just taking the stuff I don't want to do off my plate.' Like, nobody was hired to write a status report. That's not, like, the thing that you were hired and promoted based on; it was a thing you gotta do, right? To move your job forward, to help your team work together. Nobody was hired to go and say, 'Hey, I need to go and clean up the backlog of Jira items.' That's not the thing that you were hired to do, but you got to do it to keep your team organized, right? But gosh, if you took those things off my plate, I could do a lot more of what I was hired to do.

And I do think, just taking any historical perspective in our industry over the last 30, 40 years, the introduction of the PC, introduction of mobile and internet -- you know, they have always created new opportunities, and dramatically new opportunities, because people, again, they're focused now on the thing they can be good at, or being more creative, being more innovative. And I think this will be a similar, if not even bigger, opportunity than those transformations.

Pariseau: Have you seen that start to happen within customers? Or is it still too early yet?

Valliani: We certainly see examples of that happening. You know, that's one of the examples I mentioned earlier, where Tempo is saying, hey, now instead of having every team member have to go and do spend time writing, like, a status report, or writing my roadmap or whatever, the agents are helping them get that started. That means that they're focused a lot more actually on doing the work.

I've got this great example. They have an IT department that has to go and make sure that various equipment cabinets, like with wires and cables and, you know, different electronic equipment and different installations, are following certain standards. And they sent us a note saying, 'Gosh, this is, it's kind of nuts, but we actually have a Confluence base that has all the standards that can be followed.' And there's, like, a bunch of documents that outline the standards.

And we built an agent that basically just takes a photo of an equipment cabinet and analyzes the photo and says, 'Hey, compared to all the standards that I know in the space, what might not be being met, what is, what looks good, what doesn't look good.' And without even much prompting, it not only actually gave a pretty reasonable analysis, it started saying, 'Well, here's how I'd remediate, or what I think we should remediate first, second, third,' which, again, was based on, like, the policies that were in place, and then actually said, 'Here's an email draft that you could actually send to that site -- right? -- that outlines those things.'

And that was all with, like, very, very minimal prompting. And this just blew their minds, because the feedback was 'This just changes the whole way we can do audits.' Like, now we can actually spend a lot more time hoping those folks remediate and make it better. You know? Instead of just trying to go and send people out there to say, 'Okay, here are the five things that are wrong.' And I think that's -- that was a customer without us even there. They just conceived that scenario on their own and did it and sent us the feedback.

And so, I think we're seeing, like, folks who are kind of having these light-up moments, and then they're -- again, their feedback to us is not like, 'Oh, how can I go and do less work? Now I don't need as many people.' It's more like, 'Now how many people do the things that actually are important?' So, it gives me a lot of promise that this is the start of some really awesome new opportunities for folks.

I've had a long career in the industry, was at Microsoft for many years before I began at Atlassian, working in their consumer enterprise. And I've just been so amazed by the speed of transformation that Atlassian is bringing to the market. If you think about it, a year ago, we made Rovo generally available. Six months ago, we said we're gonna give it to everyone. We've executed on all that, and now we have over 3 million people using Rovo Search on a regular basis. That makes us, like, one of the biggest enterprise search players on the market. We have these high usage stats on Chat and Agents. And I think that if you really -- I know a lot of companies out there are promising transformation -- but I think we're really delivering it, and we have real examples to show with our customers, and we just can't wait to partner with more customers to deliver these transformations. And I think I'm just really impressed by what our customers are doing and pleased with how much we're able to go and support them in that transformation. That's just exciting.

Pariseau: Certainly never a dull moment right now with the way things are changing.

Valliani: Absolutely.

Pariseau: Yeah. Well, again, thank you, Jamil.

Valliani: Thanks for having me here, and always happy to catch up more.

Pariseau: Thank you for tuning in to IT Ops Query. To learn more about enterprise software development and platform engineering, explore our content on Informa TechTarget sites. Find us on YouTube at our channel, Eye on Tech. Subscribe to our podcast to receive the latest episodes as they drop. And if you liked what you heard today, give us a rating and review on Apple, Spotify or wherever you're listening. Thank you for joining us.