Publications

Agent-based models (ABMs) are widely used to estimate causal treatment effects via paired counterfactual simulation. A standard variance reduction technique is common random numbers (CRNs), which couples replicates across intervention scenarios by sharing the same random inputs. In practice, CRNs are implemented by reusing the same base seed, but this relies on a critical assumption: that the same draw index corresponds to the same modeled event across scenarios. Stateful pseudorandom number generators (PRNGs) violate this assumption whenever interventions alter the simulation’s execution path, because any change in control flow shifts the draw index used for all downstream events. We argue that this execution-path-dependent draw indexing is not only a variance-reduction nuisance, but represents a fundamental mismatch between the scientific causal structure ABMs are intended to encode and the program-level causal structure induced by stateful PRNG implementations. Formalizing this through the lens of structural causal models (SCMs), we show that standard PRNG practices yield causally incoherent paired counterfactual comparisons even when the mechanistic specification is otherwise sound. We show that a remedy is to combine counter-based random number generators (e.g., Philox/Threefry) with event identifiers. This decouples random number generation from simulation execution order by making random draws explicit functions of the particular modeled event that called them, restoring the stable event-indexed exogenous structure assumed by SCMs.

Uncertainty Quantification (UQ) is vital to safety-critical model-based analyses, but the widespread adoption of sophisticated UQ methods is limited by technical complexity. In this paper, we introduce UM-Bridge (the UQ and Modeling Bridge), a high-level abstraction and software protocol that facilitates universal interoperability of UQ software with simulation codes. It breaks down the technical complexity of advanced UQ applications and enables separation of concerns between experts. UM-Bridge democratizes UQ by allowing effective interdisciplinary collaboration, accelerating the development of advanced UQ methods, and making it easy to perform UQ analyses from prototype to High Performance Computing (HPC) scale. In addition, we present a library of ready-to-run UQ benchmark problems, all easily accessible through UM-Bridge. These benchmarks support UQ methodology research, enabling reproducible performance comparisons. We demonstrate UM-Bridge with several scientific applications, harnessing HPC resources even using UQ codes not designed with HPC support.

Random numbers are at the heart of every agent-based model (ABM) of health and disease. By representing each individual in a synthetic population, agent-based models enable detailed analysis of intervention impact and parameter sensitivity. Yet agent-based modeling has a fundamental signal-to-noise problem, in which small changes between simulations cannot be reliably differentiated from stochastic noise resulting from misaligned random number realizations. We introduce a novel methodology that eliminates noise due to misaligned random numbers, a first for agent-based modeling. Our approach enables meaningful individual-level analysis between ABM scenarios because all differences are driven by mechanistic effects rather than random number noise. We demonstrate the benefits of our approach on three disparate examples. Results consistently show reductions in the number of simulations required to achieve a given standard error with levels exceeding 10-fold for some applications.

Central to the ongoing debate surrounding rights-based family planning measures is the need for a woman-centered paradigm. Despite this debate, in the 30 years since the 1994 International Conference on Population and Development, widely used family planning measures have evolved relatively slowly. In this paper, we describe the utilization of an understudied family planning indicator—women’s expressed intention to use (ITU) contraceptives—and explore its implications for developing metrics, tracking program impact, and mechanistic understanding. We leverage Performance Monitoring for Action program data in ten geographies and assess, (1) cross-sectionally, the extent to which ITU captures demand and concords with ‘unmet need’ and (2) longitudinally, the extent to which women actualize their ITU over time. We demonstrate that the utilization of ITU more accurately identifies women who have demand and who are (un) able to actualize their ITU. We also discuss the limitations of the current ITU metric, making recommendations for data efforts to improve and include ITU as routinely reported family planning indicators.

Since the coining of the term phylodynamics, the use of phylogenies to understand infectious disease dynamics has steadily increased. As methods for phylodynamics and genomic epidemiology have proliferated and grown more computationally expensive, the epidemiological information they extract has also evolved to better complement what can be learned through traditional epidemiological data. However, for genomic epidemiology to continue to grow, and for the accumulating number of pathogen genetic sequences to fulfill their potential widespread utility, the extraction of epidemiological information from phylogenies needs to be simpler and more efficient. Summary statistics provide a straightforward way of extracting information from a phylogenetic tree, but the relationship between these statistics and epidemiological quantities needs to be better understood. In this work we address this need via simulation. Using two different benchmark scenarios, we evaluate 74 tree summary statistics and their relationship to epidemiological quantities. In addition to evaluating the epidemiological information that can be inferred from each summary statistic, we also assess the computational cost of each statistic. This helps us optimize the selection of summary statistics for specific applications. Our study offers guidelines on essential considerations for designing or choosing summary statistics. The evaluated set of summary statistics, along with additional helpful functions for phylogenetic analysis, is accessible through an open-source Python library. Our research not only illuminates the main characteristics of many tree summary statistics but also provides valuable computational tools for real-world epidemiological analyses. These contributions aim to enhance our understanding of disease spread dynamics and advance the broader utilization of genomic epidemiology in public health efforts.

Somalia is a complex and fragile setting with a demonstrated potential for disruptive, high-burden measles outbreaks. In response, since 2018, Somalian authorities have partnered with UNICEF and the WHO to implement measles vaccination campaigns across the country. In this paper, we create a Somalia-specific model of measles transmission based on a comprehensive epidemiological dataset including case-based surveillance, vaccine registries, and serological surveys. We use this model to assess the impact of these campaign interventions on Somalian’s measles susceptibility, showing, for example, that across the roughly 10 million doses delivered, 1 of every 5 immunized a susceptible child. Finally, we use the model to explore a counter-factual epidemiology without the 2019–2020 campaigns, and we estimate that those interventions prevented over 10,000 deaths.

Sciris aims to streamline the development of scientific software by making it easier to perform
common tasks. Sciris provides classes and functions that simplify access to frequently used
low-level functionality in the core libraries of the scientific Python ecosystem (such as numpy
for math and matplotlib for plotting), as well as in libraries of broader scope (such as
multiprocess for parallelization and pickle for saving and loading objects). While low-level
functionality is valuable for developing robust software applications, it can divert focus from
the scientific problems being solved. Some of Sciris’ key features include: ensuring consistent
dictionary, list, and array types (e.g., enabling users to provide inputs as either lists or arrays);
enabling ordered dictionary elements to be referenced by index; simplifying datetime arithmetic
by allowing date input in multiple formats, including strings; simplifying the saving and loading
of files and complex objects; and simplifying the parallel execution of code. With Sciris, users
can often achieve the same functionality with fewer lines of code, avoid reinventing the wheel,
and spend less time looking up recipes on Stack Overflow. This can make writing scientific code
in Python faster, more pleasant, and more accessible, especially for people without extensive
training in software development.

Background
Digital health interventions (DHI) have the potential to improve the management and utilization of health information to optimize health care worker performance and provision of care. Despite the proliferation of DHI projects in low-and middle-income countries, few have been evaluated in an effort to understand their impact on health systems and health-related outcomes. Although more evidence is needed on their impact and effectiveness, the use of DHIs among immunization programs has become more widespread and shows promise for improving vaccination uptake and adherence to immunization schedules.

Methods
Our aim was to assess the impact of an electronic immunization registry (EIR) using an interrupted time-series analysis to analyze the effect on proportion of on-time vaccinations following introduction of an EIR in Tanzania. We hypothesized that the introduction of the EIR would lead to statistically significant changes in vaccination timeliness at 3, 6, and > 6 months post-introduction.

Results
For our primary analysis, we observed a decrease in the proportion of on-time vaccinations following EIR introduction. In contrast, our sensitivity analysis estimated improvements in timeliness among those children with complete vaccination records. However, we must emphasize caution interpreting these findings as they are likely affected by implementation challenges.

Conclusions
This study highlights the complexities of using digitized individual-level routine health information system data for evaluation and research purposes. EIRs have the potential to improve vaccination timeliness, but analyses using EIR data can be complicated by data quality issues and inconsistent data entry leading to difficulties interpreting findings.

The main idea in this paper is that the age associated with reported measles cases can be used to estimate the number of undetected measles infections. Somewhat surprisingly, even with age only to the nearest year, estimates of underreporting can be generated at the much faster, 2 week time-scale associated with measles transmission. I describe this idea by focusing on the well-studied, 60 city United Kingdom data set, which covers the transition to universal healthcare in 1948, and is, as a result, an interesting case study in infectious disease surveillance. Finally, at the end of the paper, I comment briefly on how the approach can be modified for application to modern contexts.

Understanding the complex interplay between human behavior, disease transmission and non-pharmaceutical interventions during the COVID-19 pandemic could provide valuable insights with which to focus future public health efforts. Cell phone mobility data offer a modern measurement instrument to investigate human mobility and behavior at an unprecedented scale. We investigate aggregated and anonymized mobility data, which measure how populations at the census-block-group geographic scale stayed at home in California, Georgia, Texas and Washington from the beginning of the pandemic. Using manifold learning techniques, we show that a low-dimensional embedding enables the identification of patterns of mobility behavior that align with stay-at-home orders, correlate with socioeconomic factors, cluster geographically, reveal subpopulations that probably migrated out of urban areas and, importantly, link to COVID-19 case counts. The analysis and approach provide local epidemiologists a framework for interpreting mobility data and behavior to inform policy makers’ decision-making aimed at curbing the spread of COVID-19.