Every researcher knows the quiet thrill of a citation. Someone read your paper, found it useful, and built on it. But an academic citation only tells you that other academics noticed. It says nothing about whether your work reached the places where real decisions get made: a finance ministry’s budget, a global development strategy, a public health guideline, a think tank brief that lands on a minister’s desk.
We wanted that second signal. So we spent several weeks putting a tool called Overton through its paces, trying to trace our own research out of the journals and into policy. This is the story of that exploration. What we hoped to find, where the tool delighted us, where it fought back, and the workarounds we invented along the way.
What Overton actually is
Think of Overton as a search engine, except that instead of indexing web pages it indexes policy documents. Tens of millions of them: government reports, strategies from international organizations, think tank papers, NGO position papers, across many countries and many languages. Crucially, it also reads the reference list at the back of each document. That is the magic. If a government report cites an academic paper, Overton can connect the two.
That connection was exactly what we were after. We publish research. Overton knows which policy documents point back to it. Put those together and you have a map from “what we wrote” to “where it landed.”
Surprise number one: there is no single button
We came in imagining a simple request: “Overton, show us everything that mentions us.” No such button exists, and honestly, that turned out to be a feature rather than a bug. Impact is not one thing, so Overton makes you approach it from several angles. We ended up using five, and they are worth meeting, because each answers a different question.
- Does a policy document list one of our papers in its references? We call this “cites,” and it works by looking up a paper’s DOI, the permanent identifier every published paper is assigned.
- Does a document name one of our institutes in its text, even without a formal citation? We call this “mentions,” and it is a plain full-text search.
- Which documents did we ourselves publish that the database catalogs as policy? That is the “source” lens.
- Which policy documents cite our institution’s work, according to Overton’s own records? That is the “affiliation” lens, which keys off a standardized institution ID called a ROR.
- And the most fun one: which policy documents cite the policy documents that cite us? That is the “cascade.”
The first four are front doors. Each takes something we already know about ourselves and hands back the policy documents connected to it. The fifth, the cascade, is different, and we will come back to it, because it lets you follow influence as it ripples outward.
The bridge, and the cracks in it
To use the first door, we needed a list of everything we have ever published. For that we leaned on OpenAlex, a free and open catalog of the world’s research. OpenAlex tells us who we are and what we have written; Overton tells us where that work traveled. A tidy bridge between two databases.
Then we found the cracks. OpenAlex does not know about all of us. Several of our institutes have no entry in it at all, and others are only partly mapped. If we had trusted OpenAlex to define “everything we published,” we would have quietly undercounted, and some units would have been invisible from the very start.
So we widened the on-ramp. Alongside OpenAlex we pulled in records from Crossref, the registry publishers use to mint DOIs, and ran extra searches by institution name. The lesson stuck with us: never let one database silently set the ceiling on another. When we measured the gap, it was real. Broadening our starting list recovered thousands of additional policy documents that OpenAlex alone would have missed, and it rescued several institutes from invisibility.
There was a nicer surprise waiting in the same corner. Even an institute that has no ROR, and therefore no way onto the affiliation lens, is not lost. That is where the mention lens earns its keep. A mention is simply the institute’s name turning up in the body text of a policy document, whether or not that document formally cites a specific paper of ours. Overton searches the full text of everything it has indexed, so as long as a report names us somewhere in its pages, we can find it, with no identifier required at all. A mention is a softer signal than a citation in the reference list, and we are careful about that distinction later on, but it means the very units that the ID-based databases cannot see still leave a trace we can follow.
Where the API pushed back
Before describing the limitations we encountered, it is worth noting that our evaluation was conducted using an Overton trial account. Some of the behaviors we observed may reflect trial-account restrictions rather than the capabilities available under a full subscription.
No good exploration is without walls, and Overton has a few.
The first wall was a ceiling on results. Ask Overton to list the documents that cite a paper and it hands you at most about 150 of them, three pages of fifty. Ask for a fourth page and you get a flat error. For a paper cited by only a handful of documents, that ceiling never comes into play, so you would never know it was there. For a flagship paper cited by hundreds, it is a real limit: you cannot list them all in a single query.
The saving grace, and the single most important thing we learned about Overton, is this: the counts are always exact even when the list is capped. Alongside the capped list, Overton returns totals it calls facets, telling you how many citing documents came from each country, each type of organization, and each source. Those totals are complete and precise, drawn from the full set rather than the capped sample. So we could always state exactly how many documents cited a paper and where they came from, even when we could not name every last one. We repeated that line to ourselves constantly: counts are exact, only the list is capped.
The second wall was speed. Overton limits you to about one request per second. That rate limit is considerate and entirely reasonable, but it means sweeping a large body of work is a job measured in hours, not seconds.
Then came the quirks. The “mentions” search is plain text matching, which is wonderful until a short acronym starts matching things you never meant. And not every filter combines with every search, so a query we expected to narrow by year or country simply ignored those filters.
And then there was the trap. One of the parameters we tried looked like the obvious way to ask “which documents cite our institution.” We sent the request and got back an enormous pile of results. Not hundreds. Not thousands. Essentially the entire database, some twenty-seven million documents. The parameter we had reached for was not actually real, and rather than returning an error, the API had shrugged and handed us everything. The genuine filter had a different, less obvious name. That afternoon taught us to treat a suspiciously huge result as a red flag rather than a jackpot.
Turning walls into puzzles
This was the most satisfying stretch of the whole project, because every wall became a puzzle with a solution.
Beating the 150 ceiling. If you can only see 150 documents through the doorway, ask a narrower question. We re-ran each capped query once per source, once for a given development bank, once for a given ministry, and so on, so that each slice came in under 150 and could be read in full. Then we merged the slices and removed duplicates. Picture a crowd too big to count at the door, so you ask everyone to line up by the city they came from and count each line instead.
Reaching the stragglers. Source partitioning still left a few documents hiding past the cap. So we re-asked the same question in a different sort order. A different order surfaces a different window of 150, and merging those windows shook loose the last stragglers. We only marked a paper “fully listed” once a fresh ordering turned up nothing new.
Surviving the slow sweep. At that one-request-per-second pace, a full sweep of a large body of work takes hours over a sometimes fragile network connection, so we saved every response to a local cache and made the whole job resumable. If the connection dropped, the run stopped cleanly and picked up exactly where it left off next time, without re-fetching what it already had. In principle, a later run would only fetch what had genuinely changed.
Building a cascade that could go deep without looping forever. The cascade is the one lens built to recurse: in principle it can follow citations of citations of citations, as far out as you let it run. That ambition is exactly why it needs a brake, because policy documents cite each other in loops. Report A cites report B, which cites report A, and heavily used reports get cited from every direction at once. Let a deep walk wander that web naively and it never stops. So we gave it a simple memory of every document it had already expanded and refused to let it expand the same one twice, the difference between a walk that finishes and one that spins forever. In our evaluation we went just a single hop outward, the secondary layer. That walk was still in progress by the end of the evaluation period; going deeper would require more time and API budget.
The cascade payoff
The cascade turned out to be the most striking part of the picture. It is one thing to know that tens of thousands of policy documents cite our research directly. It is another to follow the ripples one step further and ask who cites those documents in turn. That single hop, the secondary layer, is as far as we have taken it so far, and even then the map bloomed outward to many times that initial count. Influence, it turns out, does not stop at the first handshake.
An honest word about what a citation means
Here is the caveat we kept front and center, because these numbers are easy to over-read. A citation in a policy document’s reference list means the authors knew about the work and chose to point to it. That is genuinely meaningful. It is not the same as proving the work changed the decision. A mention in the body text is weaker still: it signals visibility, not endorsement.
So we treat all of this as reach and uptake, strong as an aggregate signal and modest for any single document. The honest claim is that our research is turning up in policy venues at a certain scale and in certain places, not that it caused those policies. Keeping that distinction clear is not just good manners. It is what keeps the whole exercise credible.
The story over time
One more finding was almost accidental. Because we stamped every connection with a date, the data formed a time series rather than a snapshot. It became possible to see when a body of work first entered the policy conversation, estimate how long the trip from publication to policy actually took, and compare any two points in time. For a general reader, those time views often tell a clearer story than any single headline number.
Importantly, that dated record does not inflate the headline counts. The current-state picture is always drawn from the connections that hold at the time of the snapshot. If a citation later drops away, it stops counting toward the present total even though the record of its existence is preserved. The timeline comes for free without double counting, and any point-in-time reach figure reflects that moment, not an accumulation of everything ever seen.
What you can expect to find
With the caveats in place, here is a sense of the picture that emerges. Every count comes from those exact facet totals, so it holds even where the list was capped.
At the direct layer, the policy documents that formally cite your papers or name your institution, even a mid-sized research organization can expect to find tens of thousands of citations if you cast the net wide enough. Most of them will go undetected if you query naively; the five lenses together recover substantially more than any one of them alone.
Follow the ripples one hop further, to the secondary layer, and the totals grow by a further order of magnitude, because policy documents heavily cross-cite one another. The secondary reach is typically many times the direct reach, which is why the cascade is worth the engineering effort.
And how much did we have to leave on the table because of that 150-result ceiling? After all the workarounds, the portion we could not individually enumerate was a small fraction of the total, and every case is flagged rather than hidden. The exact counts, as always, stayed exact.
Our verdict, and a small wishlist
This approach proved workable for a question that academic citations cannot answer: not who read our work, but whether it surfaced where decisions get made, in which countries, from which kinds of institutions, and how quickly. The key discipline is respecting the data’s grain: counts are exact and trustworthy, recovering every individually listed document at the high end requires some engineering, and the honest framing is reach rather than proof. As noted, we worked from a trial account, so full subscribers may not encounter all of the friction points described here.
A few API additions would significantly ease this kind of work for anyone following a similar path. A way to retrieve the citations of a document in batches would make the cascade far faster. Filters that combined freely, especially by year and country, would remove the need for our cleverest workaround. And slightly deeper facet totals, past the top fifty sources, would close the last small gaps.
The bigger lesson is the one we started with. Ask the right question, in the right five ways, and you can watch research leave the library and walk into the room where decisions are made. That is a view worth the climb.
This post describes our experience using one policy citation tracking tool during an exploratory trial. Other products are available in this space. Nothing here should be read as an endorsement of any particular tool or its commercial offerings.