Every paper cites previous and related work, but finding these sources in the sea of science is not easy. And some, like systematic reviews, cite or use data from thousands.
For one study, Moritz recalled, “The authors had to look at 3,500 scientific publications, and a lot of them ended up not being relevant. It’s a ton of time spent extracting a tiny amount of useful information — this felt like something that really ought to be automated by AI.”
They knew that modern language models could do it: one experiment put ChatGPT on the task and found that it was able to extract data with an 11% error rate. Like many things LLMs can do, it’s impressive but nothing like what people actually need.