Ai2's agentic open source AI platform aids scientific research

Helping scientists

The Asta platform has three functionalities. One is the agentic research assistants called Asta agents. Another is AstaBench, a suite that provides researchers and developers with an evidence-based way to evaluate and compare agents. It includes 16 leaderboards that examine agentic performance across all benchmark categories. Finally, Asta Resources is a developer toolkit with open source agents, APIs, post-trained language models for science and an MCP extension of Ai2's Semantic Scholar API infrastructure.

The agentic platform is currently used by 194 institutions, including the University of Chicago and the University of Washington, Ai2 said.

The new agentic platform seeks to help scientists facing serious pressure to publish, according to Bradley Shimmin, an analyst at Futurum Group.

"So much of the challenge of that is tied up in processing a lot of information," Shimmin said, adding that it's like law firms specializing in contract or patent research. "It's an area where deep neural nets and transformer models excel."

He added that Asta is similar to OpenAI's deep research feature in ChatGPT. The difference is that it's not searching the web and instead has a unique data set it's searching.

A specific domain

Asta demonstrates the value of domain-specific agents.

"When we talk about agents...it's hard to, maybe not even advisable, to do general-purpose agents," said Chirag Shah, a professor in the Information School at the University of Washington. "Instead, focus on a specific domain and task, where there's a good chance of it being successful."

While Asta will help the scientific community, it's also a way Ai2 uses its existing resources, one of which is Semantic Scholar.

Semantic Scholar is Ai2's free AI-powered research tool for scientific literature, which includes 200 million papers. While the research tool was already valuable, Asta boosts its usefulness.

"The way I see it is not so much developing agents to meet the needs of a scientific community, but I see this more as expanding Semantic Scholars and the tools that they were building, and now it's done using this agent," Shah said.

Hallucinations and other considerations

While Asta also contributes to the open source community, users must carefully use it for specific tasks, such as  data analysis. Although data analysis is not yet a part of Asta, it is something Ai2 is working on. The feature will allow users to upload their own datasets and explore them using their natural language, ask sophisticated questions and receive rigorous, explainable answers, AI2 said.

"Those things will need a lot more rigorous testing, in case of hallucination," Shah said.

Another challenge with hallucinations is that the stakes may be higher if something is wrong with scientific research and discovery. A 1% hallucination in scientific research could cause compounding problems because one discovery tends to lead to the next.

"The subsequent science that happens is based on this science that happens now," Shah said. "It could create a cascade effect where things get propagated in a bad way. And so, who's going to take responsibility for it?"

Ai2 deals with the problem of hallucination in two ways, according to Daniel Weld, general manager and chief scientist for Semantic Scholar.

"The specific algorithms that we use for generating answers to questions and reading the literature, we've optimized those to be per factuality," Weld said in an interview. "The other thing we do is make it really easy in the answers and reports that Asta generates to verify the answers very quickly."

If scientists rely too much on agentic AI tools and platforms like Asta, it can also be dangerous, Shah said.

"If we have overreliance on something like this, it takes over our ability to learn, adapt and discover things on our own," he said.

Despite all these concerns, Shimmin said it's clear Ai2 cares about the AI community.

"What they've done here is they've innovated and shown as an example just what you can do with what we have in our hands right now," Shimmin said.

Esther Shittu is an Informa TechTarget news writer and podcast host covering artificial intelligence software and systems.