Use the CRF Critical Appraisal Tool
In the evolving landscape of biomedical and public health research, the question of how to appropriately use race and ethnicity has sparked intense debate. Two pivotal documents, *The Critical Race Framework Study* (CRF) by Christopher Williams (2024) and *Rethinking Race and Ethnicity in Biomedical Research* (RRE) by the National Academies of Sciences, Engineering, and Medicine (2025), tackle this issue head-on, each offering distinct perspectives on how researchers should navigate the complex terrain of race in scientific inquiry. As I delved into these works, I sought to answer two critical questions: which document provides the best guidance for using race in research, and which makes the more compelling argument? Through a careful examination of their objectives, methodologies, recommendations, and argumentative rigor, a clear narrative emerged, revealing the strengths and limitations of each.
The Critical Race Framework Study: A Focused Tool for Public Health
The CRF, a dissertation by Christopher Williams, emerges from a deep concern about the misuse of race in public health research. Williams argues that race, as an anachronistic social construct rooted in historical injustices like slavery, lacks the scientific rigor needed for high-quality research. His mission is to develop and test a tool—the Critical Race Framework—that enables researchers to critically appraise studies using racial taxonomy, focusing on four pillars of scientific evaluation: reliability, validity, internal validity, and external validity. The CRF is a call to action, urging researchers to treat race with the same skepticism as any other variable, exposing its flaws to elevate research standards.
Williams’ approach is hands-on and empirical. He conducted a three-phase study to build and refine his tool. In Phase I, he piloted the CRF with public health faculty and students, testing its fit but uncovering poor reliability that demanded improvements in training and design. Phase II expanded to a national survey of experts, assessing the revised tool’s acceptability, appropriateness, feasibility, and validity, achieving promising results in content validity but faltering with inconclusive interrater reliability. Finally, in Phase III, three raters used the CRF to evaluate 20 health disparities and behavioral health studies, finding that these studies often lacked quality or discussion regarding race’s use. These findings bolstered Williams’ argument: race’s poor conceptual clarity and operational definitions weaken research quality.
The CRF’s guidance is practical, centered on its tool. It recommends that researchers adopt the CRF for critical appraisal, train to evaluate race rigorously, and explore barriers to its adoption. Williams envisions a future where alternative categorizations replace race, though he stops short of specifying what these might be, leaving that for future research. The tool’s strength lies in its specificity, offering public health researchers a structured way to scrutinize race’s use. However, its guidance is constrained by its preliminary nature—some results were inconclusive, and the tool needs further refinement. Its focus on public health limits its applicability to broader biomedical contexts, and it offers little on inclusive practices, such as handling multiracial data or engaging communities.
Williams’ argument is clear and focused: race undermines research quality, and a tool like the CRF can expose these flaws. Supported by his empirical findings and literature critiquing race’s scientific validity, the argument is compelling within its scope. Yet, it falters in addressing counterarguments, such as race’s role in tracking health disparities, and its lack of specific alternatives leaves it feeling incomplete. The inconclusive reliability results also temper its persuasiveness, suggesting the argument needs more robust evidence to fully convince the scientific community.
Rethinking Race and Ethnicity in Biomedical Research: A Comprehensive Blueprint
In contrast, the RRE, a consensus report from the National Academies, takes a broader, more inclusive approach. Tasked by the Doris Duke Foundation and Burroughs Wellcome Fund, a multidisciplinary committee of experts set out to provide guidance on whether, when, and how to use race and ethnicity across all biomedical research—from clinical trials to medical devices and algorithms. The RRE acknowledges race and ethnicity as social constructs that shape lived experiences but lack biological grounding, often misused in ways that perpetuate health inequities. Its goal is to chart a path toward ethical, scientifically valid use, promoting equity and reducing harm.
Unlike the CRF, the RRE does not conduct original research. Instead, it synthesizes a vast body of peer-reviewed literature, drawing on studies of race correction, health disparities, and biomedical applications. The committee enriched its analysis with expert consultations, public sessions, and community engagement, ensuring diverse perspectives. This rigorous, peer-reviewed process culminated in findings that resonate across contexts: race and ethnicity are misused as biological proxies (e.g., in clinical calculators), U.S. Office of Management and Budget (OMB) categories are sociopolitical and often inappropriate for analysis, and concepts like structural racism and social determinants of health better explain disparities. The report also highlights the exclusion of multiracial and minoritized groups, calling for inclusive methodologies.
The RRE’s guidance is comprehensive and actionable, structured around six recommendations:
1. Scrutinize race and ethnicity’s use at every research stage, justifying inclusion or exclusion.
2. Provide operational definitions and describe data provenance (e.g., self-identified vs. assigned).
3. Assess technology performance across racial/ethnic groups.
4. Use alternative measures (e.g., social determinants, biomarkers) when appropriate, moving beyond OMB categories.
5. Justify all category inclusions/exclusions, avoiding vague labels like “Other.”
6. Include multiracial/multiethnic participants with clear classification schemes.
These recommendations are practical, offering specific strategies for diverse contexts— from defining race in grant applications to engaging communities like American Indian Tribes. The RRE’s emphasis on inclusivity, ethical use, and alternatives like structural racism makes it adaptable across biomedical fields. However, its lack of original testing means its recommendations are untested in practice, and its broad scope may feel less tailored to specific disciplines like public health.
The RRE’s argument is nuanced and robust: race and ethnicity, while socially significant, are misused in ways that harm research and equity. Thoughtful use, guided by alternative measures and inclusive practices, can rectify this. Supported by a comprehensive evidence synthesis, the argument engages counterarguments (e.g., race’s utility in disparities tracking) and proposes solutions, such as using social determinants or genetic ancestry. The peer-reviewed consensus and National Academies’ authority lend it significant credibility. Its only weakness is the absence of original testing, which might reduce its empirical weight for some audiences.
A Tale of Two Approaches
Comparing these documents is like weighing a precision instrument against a comprehensive map. The CRF offers a specialized tool, meticulously tested in public health, to dissect race’s flaws. Its empirical approach—collecting original data to evaluate its tool—grounds its guidance in direct evidence, making it appealing for researchers seeking a practical mechanism. Yet, its guidance is limited by its preliminary status, inconclusive results, and narrow focus. It lacks the breadth to address diverse biomedical contexts or inclusive practices like multiracial data handling, and its recommendations feel more aspirational than immediately actionable.
The RRE, by contrast, is a panoramic guide, charting a course for all biomedical research. Its recommendations are detailed, covering every research stage and offering strategies like operational definitions and alternative measures. By synthesizing a vast evidence base, it ensures applicability across contexts—from algorithms to clinical trials—while prioritizing inclusivity and ethics. Though it lacks original testing, its peer-reviewed process and expert consensus make its guidance robust and authoritative. It addresses the nuances of race’s social role, proposing solutions that balance critique with practicality.
When it comes to their arguments, the CRF makes a focused case: race undermines research, and a tool can fix this. Its empirical findings lend weight, but inconclusive results and limited counterargument engagement weaken its persuasiveness. The RRE’s argument is broader and more compelling, arguing for a reimagined approach to race with a rich evidence base. It tackles counterarguments head-on, proposing alternatives like structural racism, and its consensus-driven process ensures credibility. The lack of original testing is a minor drawback compared to its comprehensive scope.
The Verdict
In the quest to guide how race should be used in research, *Rethinking Race and Ethnicity in Biomedical Research* emerges as the stronger guide. Its comprehensive recommendations, actionable strategies, and inclusive approach make it a versatile blueprint for researchers across biomedical fields. Whether defining race in a grant application, assessing algorithm performance, or engaging communities, the RRE provides clear, evidence-based direction that the CRF’s narrower, preliminary tool cannot match.
Similarly, the RRE presents the better argument. Its nuanced, evidence-rich case—supported by a broad synthesis and expert consensus—outshines the CRF’s focused but less conclusive argument. By addressing counterarguments and proposing alternatives, the RRE convinces more effectively, offering a vision that balances critique with solutions.
While the CRF’s tool is a valuable contribution for public health, its limited scope and empirical gaps pale against the RRE’s authoritative, inclusive guidance. For researchers navigating the complex terrain of race in science, the RRE is the brighter beacon, illuminating a path toward rigor, equity, and ethical practice.
Certainly. Here's a narrative evaluation comparing the Critical Race Framework Study by Christopher Williams and the National Academies’ report, Rethinking Race and Ethnicity in Biomedical Research, focusing on which best guides the use of race in research and which presents the stronger argument.
When it comes to guiding how race should be used in research, Christopher Williams’ Critical Race Framework Study stands out as the more practical and immediately applicable document. Where many reports remain in the realm of recommendation, Williams offers a fully operationalized solution: a structured tool—the Critical Race Framework 2.0—accompanied by web-based training and empirical testing. This tool isn’t merely theoretical; it was vetted across multiple phases of implementation and refined through participant feedback. The framework evaluates how race is used in health research with a laser focus on four critical domains: reliability, validity, internal validity, and external validity. Researchers and reviewers alike can apply it to assess whether studies using race as a variable do so with the necessary scientific rigor. As such, Williams’ work fills a major gap in public health and disparities research—a space where calls for better use of race have long outpaced tangible methods.
In contrast, the National Academies’ report offers broader, more generalized guidance. It thoroughly maps the complexity of using race and ethnicity in biomedical research, spanning genetics, clinical decision-making, data analysis, and AI. Its value lies in its breadth and credibility. As a consensus document, it carries institutional weight. It cautions against the misuse of Office of Management and Budget (OMB) racial categories, affirms the non-biological basis of race, and provides high-level questions and principles for researchers to consider throughout the research lifecycle. However, it falls short in offering the kind of detailed methodological support that researchers can implement without further development. While it excels at diagnosing the problem and illuminating its nuances, it does not offer a cure.
If the question is who best guides action, Williams wins. His framework is specific, replicable, and backed by field data. It can be applied to individual studies today, not just future systems.
The question of who makes the better argument is more complex.
The National Academies’ argument is institutionally cautious and broadly inclusive. It recognizes the historic and ongoing harm caused by racial categories, but also defends their continued use as socially meaningful constructs, especially for tracking health inequities. It encourages researchers not to ignore race, but to interrogate it carefully and consistently. This middle-ground stance is likely to appeal to a wide range of stakeholders—from funders to journal editors—because it avoids radical claims while still advocating reform.
Williams, by contrast, makes a more philosophically and methodologically rigorous argument. He challenges the very foundations of race as a scientific variable. In his view, race is not just a problematic construct—it is too poorly defined, too inconsistently applied, and too conceptually weak to retain scientific legitimacy in health research. While acknowledging the influence of Critical Race Theory (CRT), Williams also critiques its limitations, including its failure to develop a unified theory or practical methodology. His position is bold: that race, as currently used, obstructs rather than advances science. His framework is not an attempt to reform the use of race—it is an attempt to fundamentally raise the bar for its justification.
In this sense, Williams presents the cleaner, stronger argument. It is internally consistent, principled, and aligned with a long-standing demand for scientific integrity. Yet it is also more likely to provoke institutional resistance, precisely because it asks for a departure from standard norms.
In summary, for researchers who want to change how race is used in actual studies, Williams offers the superior guide. For those navigating institutional change and consensus-building, the National Academies provide the more strategic framing. But in terms of argumentative strength—clarity, coherence, and conviction—Williams makes the more powerful case. His work does not merely recommend rethinking race in research; it shows us how.