NISO Altmetrics Working Group C “Data Quality” Code of Conduct Self-Reporting Table
| Description | Supports CoC Recommendation | Aggregator / Provider Submission |
| List all available data and metrics (providers and aggregators) and altmetric data providers from which data are collected (aggregators). | T1 | Overton collects and processes documents from policy sources around the world such as governments, think tanks and IGOs. We link various entities to the policy documents we track using third party data sources (Crossref, OpenAlex, Wikipedia) |
| Provide a clear definition of each metric. | A1 | Overton currently does not provide any metrics or indicators. |
| Describe the method(s) by which data are generated or collected and how data are maintained over time. | T1, T2, R1 | Data are collected in various ways including web scraping and third party APIs. New data points are created using our own AI/machine learning systems. We have checks in place for broken sources that are investigated by the team as soon as possible. More information is available on our support pages. |
| Describe all known limitations of the data. | A3 | We do not claim to track every policy source in the world and we add more sources all the time, which may lead to slightly different results depending on when a search of the database has been run.We believe our coverage of the sources we do track is broadly complete back to 2015, before which the coverage may become more patchy. This is due to source links changing over time and the fact that many organisations started publishing their documents online relatively recently.There may be errors in the affiliation and funding data which exist on the third party data sources we use, over which we have limited control. Funding data in particular is geographically patchy because of reporting different norms between countries, with more reliable information in the UK and US.Although we do have global coverage, there is a known bias in global policy document availability towards knowledge economies, which tend to have a wider range of organisations engaged in the policy process and which have norms of publishing policy documents publicly and in a digital format. |
| Provide a documented audit trail of how and when data generation and collection methods change over time and list all known effects of these changes. Documentation should note whether changes were applied historically or only from change date forward. | R1, R2, R3 | None to date |
| Describe how data are aggregated. | T2 | We enable searches by DOI, ORCID, PMID, ISBN, researcher name, policy organisation/source, topic, SDG, funder, publisher and journal. |
| Detail how often data are updated. | T3 | Depending on the data point, we refresh data daily, weekly or monthly. More information is available on our support pages. |
| Describe how data can be accessed. | T4 | Overton provides access via the web app, API and data snapshots. |
| Confirm that data provided to different data aggregators and users at the same time are identical and, if not, how and why they differ. | R4 | Yes the data provided is identical. |
| Confirm that all retrieval methods lead to the same data and, if not, how and why they differ. | R4 | Yes all methods lead to the same data. |
| Describe the data-quality monitoring process. | T5, A2 | We monitor the database for sources that seem inactive and check them for broken links. For institutions that are duplicated, we have a method for merging records when these are discovered. |
| Provide a process by which data can be independently verified. | R5 | Our data is accessible via the web app, API and data snapshots, so users are able to see the underlying data. |
| Provide a process for reporting and correcting data or metrics that are suspected to be inaccurate. | A2 | Any suspected issues or inaccuracies within the data can be reported via support@overton.ioClients have the opportunity to request that we add additional policy sources |
For more information about the NISO Altmetrics standards, visit niso.org/publications/rp-25-2016-altmetrics