Collaborative Approach to Defining Data Sharing Metrics Named Finalist in NIH S-index Challenge
The National Institutes of Health (NIH) Data Sharing Index (S-index) Challenge aims to incentivize and reward data sharing excellence, promoting a new metric for assessing how effectively researchers share valuable data, driving a culture of openness in science. The $1 million prize competition is led by the National Eye Institute with contributions from multiple NIH Institutes, Centers, and Offices. A collaborative multidisciplinary team that includes Northwestern’s Kristi Holmes, PhD has been named a finalist for their innovative approach to measuring data sharing impact.
Led by Griffin Weber, MD, PhD (Beth Israel Deaconess Medical Center), members include Katy Börner, PhD (Indiana University), Amy Brand, PhD (The MIT Press), Kristi Holmes, PhD (Northwestern University), Daniel Hook, MBA, PhD (Digital Science), John Inglis, PhD (openRxiv and Cold Spring Harbor Laboratory), Chonnettia Jones, PhD (Addgene), Arjun Krishnan, PhD (University of Colorado Anschutz Medical Campus), and Steinn Sigurdsson, PhD (arXiv and Penn State). This team unites experts from academia, publishing, libraries, research materials and data sharing, research information management systems, identity management, evaluation and impact, scientific funding, and policy making — and includes providers of widely used quantitative metrics and data visualizations related to scientific productivity and impact.
Our model reflects a strong commitment to building a data ecosystem that supports transparent, responsible, and impactful research.”
“We’re thrilled to be named a finalist in the NIH S-Index Challenge, reinforcing the importance of collaborative, user-driven approaches to recognizing and rewarding data sharing. Our model reflects a strong commitment to building a data ecosystem that supports transparent, responsible, and impactful research,” says Holmes, director of the Galter Health Sciences Library and director of informatics and data science at the Northwestern University Clinical and Translational Sciences (NUCATS) Institute. “Data is not just a byproduct of research; it is an engine for discovery.”
The seven finalist teams announced September 16, 2025, will each receive $15,000 to further develop their concept in Phase 2 of the Challenge, and will showcase their final solutions at the NIH S-index Innovation Event, to be held in Bethesda, Maryland. Prizes will be awarded to the top three teams, and the winner of Phase 2 will receive $500,000.
“Developing an S-Index metric will promote culture change through incentivizing excellence in data sharing — in terms of quality as well as quantity,” says Michael F. Chiang, director of the National Eye Institute, which administers the challenge on behalf of NIH. “The ideas that have emerged in Phase 1 demonstrate enormous creativity and reflect a strong commitment to making research data findable, accessible, interoperable and reusable, according to the FAIR principles.”
The NIH S-Index Challenge was launched to generate new approaches for measuring and rewarding data sharing across the biomedical research community. Currently, researchers who invest significant time in creating high-quality, well-documented datasets receive little professional recognition for this work. The S-Index Challenge aims to address this problem by strengthening openness, transparency, and reuse in ways that ultimately increase the impact of research. By incentivizing strong data-sharing practices and creating practical, implementable metrics, the Challenge seeks to shift norms in biomedical research. The competition is structured to balance innovation with rigorous evaluation (including testing and validation) while ensuring feasibility for broad adoption by institutions, journals, funders, and researchers.
The team’s proposed model uses a participatory approach that engages stakeholders in the co-development and implementation of the S-Index to ensure that it is understandable, usable, and meaningful. Close collaboration with stakeholders has been essential to the widespread adoption of open science innovations and the S-Index is no different. Trust is a critical component of the S-index’s success. As with any metric, there is a potential for unintended negative consequences. Researchers must be assured that the S-Index won’t favor certain disciplinary perspectives, negatively impact their promotion and tenure, or encourage competitors to use data without proper credit. Without foundational trust, the S-Index cannot achieve its intended purpose, drive meaningful impact, or foster the cultural shift it aims to achieve. Research is nuanced, making it essential to reveal the critical but often overlooked role datasets play in driving knowledge, discovery, and ultimately, societal impact.
“The S-Index offers an opportunity to evolve current attitudes towards data sharing, with the expectation that the metric will lead to cultural change,” says project team captain Griffin Weber, MD, PhD, associate professor of Medicine and Biomedical Informatics at Beth Israel Deaconess Medical Center. “This is a monumental task, and we’ve presented a conceptual framework that captures different components of data sharing, quantified as a composite metric. Our central idea maintains that key stakeholders must be involved in defining this new metric to ensure its long-term impact, scalability, and sustainability.”
Written By Roger Anderson
