Critical Analysis of Neighborhood Eviction Trajectories and Psychological Distress: Addressing Dr. Williams’ Concerns

View Dr. Williams' letter to editors of the American Journal of Epidemiology.

Introduction

The study by Sealy-Jefferson et al. (2024), published in the American Journal of Epidemiology, investigates the association between neighborhood eviction trajectories and the odds of moderate psychological distress (MPD) and serious psychological distress (SPD) during pregnancy among African American women. Grounded in a reproductive justice framework, the study leverages data from the Life-course Influences on Fetal Environments (LIFE) Study and Eviction Lab data to examine how eviction filing and judgment rates over time impact maternal mental health. Despite its novel contribution, Dr. Christopher Williams, in a letter to the journal editors, raises significant concerns about the study’s methodology, data reliability, and policy implications, questioning how it passed peer review. This essay critically evaluates these concerns, focusing on the reliability of Census block data, the peer review process, and the practical significance of the findings, particularly the counterintuitive results for high-eviction neighborhoods. By synthesizing these analyses, the essay underscores the challenges of epidemiological research and the need for rigorous scrutiny to ensure valid, actionable public health insights.

Reliability of Census Block Data

Overview of Census Data in the Study

The study uses Census block group-level data to calculate eviction filing and judgment rates, defined as the ratio of eviction cases or judgments to renter-occupied households (Sealy-Jefferson et al., 2024, p. 969). These rates, sourced from the Eviction Lab at Princeton University and standardized to Census 2010 boundaries, were linked to LIFE Study participants’ addresses at preconception (2007–2009) and during-pregnancy (2009–2011) periods. Williams critiques the reliability of Census block data, arguing that small-area estimates are prone to instability due to large margins of error (MOEs), introducing noise that threatens internal validity (Williams, Point 2).

Granularity and Margins of Error

Census block groups, encompassing 600–3,000 people, are more stable than block-level data but still subject to variability, particularly in the American Community Survey (ACS) 5-year estimates used for housing data (e.g., 2007–2011). Small sample sizes at the block group level can lead to wide MOEs, especially for renter-occupied household counts, which serve as the denominator in eviction rate calculations. For instance, a block group with few renter-occupied homes could produce inflated or unstable rates if a single eviction occurs (e.g., one eviction in five homes yields a 20% rate). The study’s wide rate ranges (0–132.0 filings per 100 homes, Table 4) suggest potential noise, yet the authors do not address MOEs or conduct sensitivity analyses to assess their impact. This omission, as Williams notes, risks misclassification and biased effect estimates, undermining the reliability of associations with MPD and SPD.

Validation and Methodological Efforts

The study mitigates some concerns by using block groups rather than blocks and validating eviction rates at individual and aggregate levels (p. 969). The Eviction Lab’s standardized data is generally reliable, but the accuracy depends on court record completeness and address matching, details the study lacks. The assumption of stable renter-occupied household counts from Census 2010 across 2007–2011 may introduce measurement error, particularly in Detroit’s dynamic housing market. Williams’ concern about spatial dependence—where eviction rates in adjacent block groups may be correlated due to shared socioeconomic factors (Point 3)—is also valid. The study’s use of multinomial logistic regression without hierarchical modeling (due to low block-group variation, p. 970) overlooks potential clustering, which could distort trajectory effects.

Implications for Study Findings

Unreliable Census data could misrepresent eviction trajectories, affecting the validity of reported odds ratios (e.g., 2- to 4-fold increases for medium/medium trajectories, Tables 5 and 6). Noisy denominators might attenuate or inflate effects, particularly for the high/high trajectory, which unexpectedly showed lower odds of distress (p. 972). Future studies should report MOEs, use ACS 5-year estimates explicitly, and employ spatial regression to account for clustering, enhancing data reliability.

Peer Review Process: How Did the Study Pass?

Strengths That Appealed to Reviewers

Despite methodological concerns, the study’s strengths likely facilitated its acceptance. Its focus on eviction and maternal mental health among African American women addresses a critical public health issue, aligning with calls for research on macrosocial determinants and reproductive justice (p. 974). The life-course approach, examining trajectories across two time points, is innovative, improving on single-time-point studies (p. 974). The within-group analysis avoids problematic racial comparisons, responding to critiques of racial inequity research (p. 969). The use of a sizeable cohort (n=808), validated eviction data, and the Kessler Psychological Distress Scale (K6, reliability = 0.72) likely appeared rigorous (p. 970). These factors, combined with funding from NIH and the Robert Wood Johnson Foundation (p. 974), may have lent credibility, overshadowing limitations.

Limitations of Peer Review

Peer review is not infallible, and several factors may explain why Williams’ concerns were overlooked:

• Variable Reviewer Expertise: Reviewers may have lacked expertise in small-area data or spatial epidemiology to flag Census data instability or spatial dependence (Williams, Points 2 and 3). The study’s complex data linkage (LIFE Study, Eviction Lab, Census) might have overwhelmed reviewers, who may have assumed the Eviction Lab’s validation was sufficient (p. 969).

• Focus on Broader Impact: Reviewers likely prioritized the study’s conceptual contribution—addressing housing instability and maternal health disparities—over technical details. The significant findings for medium/medium and low/medium trajectories (Tables 5 and 6) aligned with the hypothesis, potentially diverting attention from the high/high trajectory’s unexpected results (p. 972).

• Missed Methodological Gaps: The sample reduction from 1,410 to 808 (Williams, Point 1) and 0–10% covariate missingness without imputation (Point 4) were disclosed (p. 970), but reviewers may have deemed these acceptable given the study’s transparency. Over-controlling for age, income, education, residence duration, and a neighborhood disadvantage index (Point 5) was supported by one citation (p. 970), which reviewers may not have questioned. The absence of model fit statistics was a gap, but not all journals require them.

• Publication Bias: Journals favor significant results, and the study’s 2- to 4-fold odds ratios (Tables 5 and 6) likely appealed to editors. The counterintuitive high/high findings were not emphasized in the discussion (p. 972), reducing scrutiny.

Editorial and Journal Factors

The American Journal of Epidemiology is a high-impact journal with rigorous review, typically involving 2–3 reviewers and editorial oversight by experts like Drs. Lesko and Schisterman. However, time constraints and the pressure to publish timely research on maternal health disparities may have led to leniency. The study’s alignment with reproductive justice and its funding credibility likely bolstered its case, even if reviewers missed issues like practical significance (Williams, Point 7).

Practical Significance and Policy Implications

Williams’ Critique of the High/High Trajectory

Williams’ most pressing concern is the study’s practical significance, particularly the lower odds of MPD and SPD for the high/high eviction rate trajectory (e.g., aOR = 1.22 for MPD, 1.95 for SPD, Tables 5 and 6). This finding suggests that women in consistently high-eviction neighborhoods experience less distress than those in medium/medium or low/medium trajectories, contradicting the hypothesis that eviction exposure is detrimental (p. 969). Williams argues this could imply that staying in high-eviction neighborhoods is protective or that moving to lower-eviction areas (high/low trajectory) increases distress risk (Point 7). Such an interpretation is problematic, as high-eviction neighborhoods are often under-resourced and tied to structural racism, harming health (p. 969, References 12–18).

Why the High/High Finding Is Counterintuitive

Theoretically, sustained exposure to high eviction rates should exacerbate stress due to anticipatory fear, vicarious racism (e.g., seeing neighbors evicted), or housing instability (p. 969). The lower odds for the high/high group could reflect:

• Coping Mechanisms: Women in persistently high-eviction areas might develop resilience or rely on community support, mitigating distress.

• Desensitization: Chronic exposure might normalize eviction threats, reducing acute distress as measured by the K6.

• Methodological Artifacts: Noisy Census data (Williams, Point 2), over-controlling (Point 5), or sample reduction (Point 1) could attenuate effects. For example, the high/high group may include fewer participants from Detroit’s most distressed areas due to geocoding exclusions (p. 969). Spatial dependence (Point 3) might also mask effects if high-eviction areas are clustered.

• Unmeasured Confounders: Factors like social cohesion or access to mental health resources, not captured in the model, could influence distress.

The study’s discussion (p. 972) focuses on significant findings without addressing the high/high trajectory’s deviation, a critical oversight that reviewers should have flagged.

Policy Implications and Risks

The study’s reproductive justice framework emphasizes safe, healthy environments (p. 969). Misinterpreting the high/high trajectory as protective could lead to harmful policies, such as:

• Perpetuating Inequity: Discouraging housing mobility programs that relocate women to lower-eviction neighborhoods, trapping them in high-risk areas.

• Misguided Interventions: Prioritizing medium-eviction neighborhoods for mental health support while neglecting high-eviction areas, where structural racism is most pronounced.

• Undermining Reproductive Justice: Failing to address eviction as a modifiable risk factor, contradicting the study’s call for macrosocial interventions (p. 974).

Williams’ concern that publication could harm public health policy (Point 7) is valid, as policymakers might misinterpret the findings without clear author guidance cautioning against such conclusions.

Why Peer Review Missed This

Reviewers likely overlooked the practical significance issue due to:

• Focus on Significant Results: The significant odds ratios for medium/medium and low/medium trajectories (Tables 5 and 6) aligned with expectations, overshadowing the high/high anomaly.

• Limited Policy Scrutiny: Reviewers may not have evaluated how findings could be misused, focusing instead on statistical rigor and theoretical alignment.

• Expertise Gaps: Lack of expertise in housing policy or critical race theory (Williams, Point 3) may have prevented reviewers from recognizing the implications of the high/high trajectory.

• Incomplete Discussion: The authors’ failure to explain the high/high results or address their policy implications (p. 972) reduced the likelihood of reviewers probing further.

Recommendations for Future Research and Peer Review

Strengthening Research

To address Williams’ concerns, future studies should:

• Enhance Data Reliability: Report MOEs for Census data, use ACS 5-year estimates, and employ spatial regression to account for clustering.

• Address Sample Reduction: Conduct analyses comparing included and excluded participants to assess bias from geocoding restrictions.

• Justify Covariate Selection: Provide robust citations for covariate choices and report model fit statistics to avoid over-controlling.

• Explore Unexpected Findings: Conduct sensitivity analyses for counterintuitive results (e.g., high/high trajectory) and discuss potential explanations, such as coping mechanisms or data artifacts.

• Clarify Policy Implications: Explicitly caution against misinterpretations that could harm policy, emphasizing eviction prevention and housing stability.

Improving Peer Review

Peer review processes can be enhanced by:

• Diverse Expertise: Including reviewers with expertise in small-area data, spatial epidemiology, and housing policy to catch issues like data instability and practical significance.

• Mandating Robustness Checks: Requiring sensitivity analyses for key assumptions (e.g., missing data, noisy variables) and theoretical reconciliation of unexpected findings.

• Evaluating Policy Impact: Assessing whether findings could lead to harmful interpretations, aligning with journals’ public health missions.

• Transparency Standards: Mandating detailed reporting of validation methods, MOEs, and model diagnostics to ensure methodological rigor.

Conclusion

The study by Sealy-Jefferson et al. (2024) offers a valuable contribution to understanding eviction’s impact on maternal mental health, but Dr. Williams’ critiques highlight significant methodological and interpretive flaws. The reliability of Census block group data is compromised by potential instability and unaddressed spatial dependence, threatening the validity of eviction trajectories. The study’s passage THROUGH peer review reflects its novelty, policy relevance, and alignment with reproductive justice, but reviewers likely missed issues due to expertise gaps, focus on significant results, and incomplete author discussion. Most critically, Williams’ concern about practical significance—where the high/high trajectory’s lower odds could imply staying in high-eviction neighborhoods is protective—underscores a risk of harmful policy misinterpretation. This case highlights the challenges of balancing rigor and impact in epidemiological research and the need for robust peer review to ensure findings are valid and actionable. By addressing these concerns, future studies can better advance reproductive justice and maternal health equity.