Evidence synthesis methods adapted to discovery and evaluation of research datasets.
Presented to the Discovery Cluster on May 21, 2026 by Lindsay Barbieri
This presentation explores an ESIP-related project focused on understanding how evidence synthesis methods can be adapted to discover and evaluate research datasets. We applyed systematic review methods to dataset discovery for climate-smart agriculture. We welcome discussion, feedback, and most importantly: we are seeking collaboration!
Research Abstract:
Most data included in evidence-based reviews is reported in journal articles, study reports, and other text-based, narrative documentation. A key component of the systematic review process for evidence-based reviews is data extraction, a manual process to identify and collect data for synthesis across studies. Yet the release of datasets as research products is increasing. If relevant datasets, rather than documents, could be systematically identified using standardized review processes, the manual extraction process could be reduced or eliminated. Moreover, the ability to search for and synthesize datasets directly to answer scientific questions could enable evidence generation for broader and more complex topics. This approach could also improve the identification of data gaps more efficiently than traditional document-based review approaches. In this study, we demonstrate how evidence-based review methodologies, typically used to synthesize findings from publications, can be applied to the discovery of datasets to answer a scientifically-motivated research question. In this case, we use the question “what are the characteristics of the evidence (e.g. the data that has been collected) to substantiate synergies between climate change adaptation and mitigation from various agricultural practices?” to identify climate-smart agricultural datasets. We employ a standardized process to identify relevant data repositories that aggregate or host agriculture-related climate datasets. For each repository we identified, we evaluate characteristics such as search infrastructure and metadata practices, to determine their support for systematic search and retrieval. We then assess how systematic review methodologies can be applied following the dataset discovery phase and identify key gaps in current data repository infrastructures that limit their usefulness in supporting evidence synthesis workflows. Finally, we share lessons learned from applying these methods in a real-world context and outline the implications for the scientific community and repository developers, highlighting opportunities to improve data discovery and reuse for evidence generation in climate-smart agriculture and beyond.
