Sometimes, scientific progress hinges not just on new discoveries, but on how we build and share the tools that help us make sense of complex data. This is a story about AFNI and SPM, two of the most widely used software packages in neuroimaging, and how their differing approaches to transparency helped shape one of the most significant methodological corrections in the field’s recent history. AFNI SPM I got chatting with a few folks about Open Source Science this week, and how we can measure the value of it. For me, I think it’s a silly question: I’ve seen the value of Open Source in science first hand. This isn’t a general call for Open Science, I’ve lived it and I believe in it. This is a record of what I saw happen first hand, and why I chose to work in Open Source Software. I was in a dual PhD program between National Institutes of Health and University College London, the respective homes of where our main statistical characters both lived: AFNI and SPM. I was one of the few who saw the impacts of this in person, at both institutions, and it changed my view on software development forever. SPM vs AFNI: Same Challenge, Different Response Since the 1990s, both SPM (Statistical Parametric Mapping) and AFNI (Analysis of Functional NeuroImages) have been essential for analyzing fMRI data. Each has enabled researchers to translate noisy, high-dimensional brain data into interpretable results. But beneath their shared purpose lies a fundamental difference in how they were developed and maintained. SPM (Statistical Parametric Mapping) AFNI (Analysis of Functional NeuroImages) SPM, developed by Karl Friston and colleagues, has long been distributed freely for academic use and grounded in rigorous theoretical work. It gained widespread adoption, particularly in university environments where MATLAB—its underlying platform—was already in use. For much of its history, however, SPM development remained fairly centralized. Updates were released periodically, but the inner workings of the tool were largely handled within a core team. AFNI, developed by Robert W. Cox and maintained at the NIH, took a different approach. He had his resignation letter to the NIH framed by the door of this office which explained that if they did not commit to funding the software development, he would leave. NIH did fund the software development, and this letter remained in that frame for another two decades until his retirement. Cox released AFNI under an Open Source license, and it has been community-facing from the beginning. Its codebase has always been accessible, with a public changelog that documents updates, fixes, and changes in detail. Feedback from users often led to direct updates, and many improvements came about through an ongoing dialogue between developers and the broader research community. These differing philosophies - one more academic and centralized, the other open and participatory, would come into sharp relief during what became known as the fMRI clustering controversy. The Clustering Controversy In 2016, Eklund et al. published a paper revealing that several major fMRI software packages, including AFNI, SPM, and FSL, were producing inflated false-positive rates when using standard cluster-based inference methods. The problem stemmed from assumptions about the spatial properties of brain noise - specifically, that it followed a Gaussian distribution, which turned out not to hold in practice. FSL SPM’s implementation of these methods was technically correct in terms of executing the models it was designed to run. But those models rested on statistical assumptions that hadn’t been scrutinized closely enough. Although the SPM team acknowledged these limitations, changes to defaults and workflows came more gradually, and public discussion largely followed the publication of the paper. AFNI, by contrast, had a specific bug in its 3dClustSim function—an error in how it simulated null distributions used for determining statistical significance. Crucially, this bug had already been identified and fixed by the time the Eklund paper was published. The fix came about in part because AFNI’s code was open: external researchers were able to examine the software, spot the problem, and work with the maintainers to resolve it. "We never would have been able to identify the bug in 3dClustSim were AFNI not open source software." — Eklund et al., 2016 "We never would have been able to identify the bug in 3dClustSim were AFNI not open source software." This moment served as a powerful reminder of what openness can enable - not just in terms of collaboration, but in safeguarding the integrity of scientific results. It’s a lesson in software, because it is software. is software In a 2016 response to this controversy, Karl Friston emphasized that SPM was never intended to be a black box. "SPM is not a black box. It’s an open set of methods that we encourage people to understand—and challenge," he said. The SPM team has long prioritized clarity of theory, and this moment sparked a broader conversation about the importance of making that clarity extend into code and implementation. Understanding that how you build software will have an impact on the people who use it, so make it easy for those most impacted by it to have a way in the ways it is meaningfully improved. Create critical feedback loops, and be responsive to them. Transparency as a Catalyst The lessons here extend beyond fMRI. Scientific reproducibility depends not only on data and methods, but on how accessible and transparent our tools are. In AFNI’s case, transparency allowed for real-time auditing, faster fixes, and a culture of shared accountability. Updates were frequent and traceable, and methodological changes were documented in ways that researchers could follow and build upon. SPM, in the years since, has made meaningful strides toward greater openness. In 2023, its codebase was migrated to a public GitHub repository. The team has expanded documentation, rethought statistical defaults, and clarified key model assumptions. These steps reflect a shift in how the SPM ecosystem engages with the broader community—a shift that many warmly welcomed. Software Is Scientific Infrastructure This story isn’t about assigning blame, and in the decade it’s been running around in my brain, it isn’t even about science. It’s really about acknowledging the complexity of modern software and the evolving role of using the community wisely when shaping our conclusions on what, and how, to build. Scientific tools are no longer just utilities that support research; they are themselves embedded in the research process. As such, they need to be subject to the same standards of openness, peer review, and collective improvement that we apply to our studies. The clustering controversy demonstrated that even mature, widely trusted tools can harbor critical issues: some due to bugs, others to underlying assumptions that deserve a second look. What made the difference in how quickly those issues were found and fixed was transparency. And that’s something all software projects can learn from. Moving Forward Reproducibility in science is not just about re-running code. It's about being able to inspect the logic behind results, question defaults, and collaborate openly to improve methods. Open source doesn't automatically make software better, but it creates the conditions in which better becomes more likely. Years later, what stays with me most isn’t the bug or the paper. It’s how one tool invited its community to fix what was broken, while the other learned how valuable that invitation can be. We can’t always predict the future of science, or software, or cybersecurity - but we can choose to build it more transparently (if transparency in cybersecurity seems strange to you, watch my talk on zero trust, it is, in fact, where transparency matters most). Open source doesn't automatically make software better, but it creates the conditions in which better becomes more likely. talk on zero trust Open Source isn’t just a philosophy; it is a principled approach. Transparency is a pillar because the entire foundation falls down without it. Open Source isn’t just a philosophy; it is a principled approach. Transparency is a pillar because the entire foundation falls down without it. We don’t get to choose every outcome in science. We do get to choose the systems we build, the openness we allow, and the feedback we invite. When we choose transparency, we’re choosing resilience. And in the long run, that’s what science depends on. For developers building tools for researchers, the takeaway is clear: make your work accessible. Document your assumptions. Welcome scrutiny. Real progress doesn’t just come from publishing results: it comes from building systems that allow others to question them. This is the total value that you get from open-source software in your scientific studies, and there’s no metric that can quite quantify it. If you do have opinions about the best way to measure academic repositories, then I wrote this because I would love to talk to you. There’s a team at CHAOSS working on this problem now, and I will put you directly in touch. CHAOSS Thank you for reading my little story from my brain days. It’s one that I hope Open Source doesn’t forget when the history books are written, so this is a written record to keep it alive. Stay tuned for my more typical topics on Open Source and Cybersecurity updates, more to come soon!

This story contains new, firsthand information uncovered by the writer.

GitHub

Building Trustworthy Agents: Why Secure Context Matters in a Tool-Augmented World

The Collapse of CVE: How a Funding Failure Threatens Global Cybersecurity

Supporting developers with best practices in automated security and reliability.

The Bug, The Black Box, and the Brain Map: What AFNI vs. SPM Taught Us About Open Source & Science

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

A Tale of Two LLMs: Open Source vs the US Military's LLM Trials

A Beginner's Guide to Neuromarketing

A Catalog of Eternal Moments

Branding and the brain: the neuroscience behind consumer decision making

Can Androids Experience Qualia?

Cognitive Reappraisal or Using Reframing to Cope with Unpleasant Feelings

A Tale of Two LLMs: Open Source vs the US Military's LLM Trials

A Beginner's Guide to Neuromarketing

A Catalog of Eternal Moments

Branding and the brain: the neuroscience behind consumer decision making

Can Androids Experience Qualia?

Cognitive Reappraisal or Using Reframing to Cope with Unpleasant Feelings

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps