In the dim light of early morning, Dr. Donatello Tartaruga sits in his cluttered office, a lukewarm cup of untouched coffee beside his keyboard. His screen illuminates a newly opened email, the latest in a series of exchanges with his colleague, Dr. Peter Hare. With a deep sigh, Donatello reads this latest letter and considers his response.
Exposome Perspectives Blog by Robert O. Wright, MD, MPH
“We have never arrived. We are in a constant state of becoming.” Bob Dylan
Epistolary novels are written in the form of letters to and from characters that reference an ongoing story. One of the earliest examples is Letters from a Peruvian Woman which documents the plight of an Inca Princess kidnapped by French pirates and taken to Europe. Written in 1747 by Françoise de Graffigny, perhaps the most famous female writer of her time, the novel exposes the inequalities of 18th century European society with a feminist perspective- practically unheard of in the 18th century. The letters to and from the Princess to her fiancé Aza, an Incan Royal, also include warnings of the danger he faces from the Spanish Conquistadors who are not the friends they pretend to be. More recently there is The Martian by Andy Weir, about an astronaut stranded on Mars following a dust storm that forces his colleagues to evacuate. It’s written as serial diary entries (i.e. letters to himself, or to the colleagues he hopes one day discover his body) as the protagonist struggles alone to recycle water and grow food on a barren, dry planet with minimal resources other than his engineering skills. You may have seen the movie with the actor Matt Damon. Finally, Aesop, as you may remember, was a star of the old “Rocky and Bullwinkle Show” my favorite cartoon as a child. I think he may have been a writer too. Bringing it all back home, below is a parable on scientific replication.
From: Hare, Peter <peter.hare@college.edu>
Sent: Sunday, February 11, 2024 4:30:50 PM
To: Tartaruga, Donatello<Tartaruga_mutantem@university.edu>
Subject: Mutant Ninja technology
Dear Donatello,
Happy New Year! I know I am quite late with this email, but with the holidays, there is so much to do and so little time. I saw you were at the recent invitation only “Big Data” conference, I apologize for not saying “hello” but I was tied up in a small group meeting with the Program Director. As you might have heard during the opening plenary, she is interested in finding the next “Game Changer” in Biomedical Research, and she invited me to join a small group of disruptors for further discussion. She had seen my recent high impact Exposome publication with your former colleague, Dr Hamato Yoshi, we used our newly developed biotroid technology—as I wrote earlier—a vast improvement on mass spectrometry. I now hold a patent for it! At dinner, everyone was enthralled with my work. I wish you could have been there when I told them how it will revolutionize healthcare!
Yours truly,
Peter
P.S. I heard about your question to the Director—that you believe that the sum of all current knowledge suggests there are only small or modest effects for any single risk factor, be it genetic or environmental, and that we need to factor reality into our designs and focus on creating infrastructure for rigorous replication studies. Do you realize how much time that would take? Anyway, my paper is in the first week of the December issue, starting on page 3109. Be sure to check out the 3-D images.
From: Tartaruga, Donatello<Tartaruga_mutantem@university.edu>
Sent: Sunday, February 18, 2024 1:35:02 PM
To: Hare, Peter <peter.hare@college.edu>
Subject: Re: Mutant Ninja technology
Dear Peter,
I confess I didn’t see your paper but congratulations on your special invitation. I thought that the meeting was interesting, albeit similar to the other big science meetings I’ve attended. I fear my question was misunderstood. I wasn’t suggesting that measuring the exposome is not important, rather, I was just trying to emphasize the need to create exposome research infrastructure so we can conduct rigorous, reproducible science. We can’t afford to skip steps, as there won’t be a quick fix that avoids doing the fundamental work. Simply measuring “everything” won’t give us all the answers. We need infrastructure—ontologies, search engines, shared libraries, shared methods, etc—to understand how multiple risk factors interact over time and across different life stages. Big data “discoveries” are not the final answer; they never are. We should know better than to suggest they are. These discoveries have to be replicated and tested repeatedly and rigorously to prove their validity. To be honest, I am not sure what constitutes a “game changer” in biomedical research. Aside from quitting smoking, exercise, and a healthy diet, there seems to be little else. Genomics is not a risk factor, nor is the exposome. These fields represent billions of risk factors, the majority of which are noise. I fear we’ve come to think just measuring the most “noise” is advancing science. The important issue is separating signal from noise, i.e. replication.
We need to focus on the fundamentals rather than a shotgun measurement approach. Nothing can replace good study design, rigorous data collection, and most importantly, replication and functional testing. We have to go through all the necessary steps. We need a longitudinal study across life stages, potentially incorporating an accelerated longitudinal design. This approach would allow us to comprehensively measure as many environmental factors as possible across time and space, while factoring in health disparities, cultural effects, geographic differences, etc. We will need to make connections with local communities and try to understand the key issues they face. Science is far too complex to think a single assay can explain all disease (even if that assay can measure a million or billion “things”).
Please send my best to Beatrix.
Donatello
From: Hare, Peter <peter.hare@college.edu>
Sent: Sunday, February 18, 2024 10:05:04 PM
To: Tartaruga, Donatello<Tartaruga_mutantem@university.edu>
Subject: Re: Mutant Ninja technology
Dear Donatello,
Good luck with all that, we may see each other at the finish line, but I think we both know who will get there first. Meanwhile, I got another patent! I’m also scheduled for an appearance on National News this Tuesday! And I’ve convinced a tech company to fund my idea on using flopsyoids (an extension of Hamato’s biotroid based assay). Flopsyoids bind to exogenous chemicals in tissues but not to endogenous chemicals. They can be visualized using chromedones—a concept I confess that was Hamato’s idea. However, we hold the patent for that method together. We were able to measure it in cases and controls among patients with renal failure, revealing huge disparities. We have figured out health disparities!!
BTW, saw your recent paper. Are you still using questionnaires and mass spectrometry in life course research? Seriously? That will take years before you get answers. To each his own I suppose, but the world is passing you by, my friend. Meanwhile, I’ve submitted a paper with my new technology to Nature! They have asked for a revision, along with some minor details about study design, exclusion criteria, QA/QC and reproducibility—nothing major.
Peter
From: Tartaruga, Donatello<Tartaruga_mutantem@university.edu>
Sent: Sunday, February 25, 2024 9:35:36 PM
To: Hare, Peter <peter.hare@college.edu>
Subject: Re: Mutant Ninja technology
Dear Peter,
Congratulations on your analytic breakthroughs. I caught your appearance on Network News where you discussed your discovery of the “cause” of both Alzheimer’s and Autism. However, it looks like the research group out of England cannot replicate your findings, nor can the group in Japan. I admit I’m struggling to understand the design of your study—was it a case-control study? The methods section didn’t clearly describe how you enrolled participants, only that they were selected from electronic health records. I worry about the quality of data that weren’t collected specifically for research.
Alzheimer’s and Autism are complex conditions. There may be differences in age, socioeconomic status, air pollutants and social stressors that were not addressed in your analysis. You may want to account for environment in pregnancy, as a significant body of scientific literature suggests early life environment plays a crucial role in the development of diseases later in life. I don’t believe we have the infrastructure yet to apply your method rigorously or to understand the past exposome. We need a large longitudinal study, we need to be able to interpret results, and we must be able to systematically replicate findings across independent populations. We need data repositories, capacity for federated data analysis, search engines for data, and detailed participant information to identify any potential biases from enrollment or data collection. Science doesn’t just suddenly arrive, it has to meticulously become.
Donatello
From: Hare, Peter <peter.hare@college.edu>
Sent: Sunday, June 25, 2024 12:35:34 PM
To: Tartaruga, Donatello<Tartaruga_mutantem@university.edu>
Subject: Re: Mutant Ninja technology
Dear Donatello,
I am leaving replication to others, I focus on “novel” research—not “repetitive” research. Why should I just repeat what’s been done? So I will tell you exactly what I plan to do—measure more!!! I have successfully extended the assay to 10 million measures! That’s right! I just measured 10 million chemicals. This is big! Big, Big data! Replication is not the problem- the problem is that we are clearly missing the important chemicals—so we have to measure more of them!
Peter
(P.S. I have started a company! And I am going to buy a boat!!)
From: Tartaruga, Donatello<Tartaruga_mutantem@university.edu>
Sent: Sunday, September 22, 2024 10:35:02 PM
To: Hare, Peter <peter.hare@college.edu>
Subject: Re: Mutant Ninja technology
Dear Peter,
I see another study you published failed to replicate. Beyond the issue of multiple comparisons, have you considered the possibility that the missing element in your research might not be chemical? Do you genuinely believe there are 10 million important chemicals? Why not focus on what is already known and measurable within the social, physical and infectious exposomes as well? We don’t yet know how these risk factors might combine at different life stages in mixtures.
Why is there such a heavy emphasis on discovery and so little effort on replication? This approach contributes to the public’s growing skepticism towards science. Time and again, we proclaim major breakthroughs with press releases that later turn out to be false. We can’t keep doing this. It’s eroding public trust in our work.
Donatello
From: Hare, Peter <peter.hare@college.edu>
Sent: Sunday, October 6, 2024 10:23:09 AM
To: Tartaruga, Donatello<Tartaruga_mutantem@university.edu>
Subject: Re: Mutant Ninja technology
Dear Donatello,
Great points, meanwhile, back to my problem. Yes, it’s true that another research team was not able to reproduce our findings. They even suggested that our work may be a false positive and that my research lacks rigor. How ridiculous is that? This is the 3rd study that didn’t replicate (maybe 4th depending on how you count) but I bet they did something wrong. I have no control over that. This is the evolution of science in the 21st century. The real work is done by the innovators and disruptors using big data for predictions.
I admit my study yielded about 500,000 significant results but I only reported on the top 3 because they had the smallest P values. I’m considering publishing the remaining results. What are your thoughts on that? Unfortunately, my new grant was just triaged—they want details on the design and quality control. Maybe you could read it over and share your insights on the design?
Yours humbly,
Peter
From: Hare, Peter <peter.hare@college.edu>
Sent: Sunday, December 8, 2024 6:05:11 PM
To: Tartaruga, Donatello<Tartaruga_mutantem@university.edu>
Subject: Re: Mutant Ninja technology
Dear Donatello,
Just checking as you didn’t respond to my last few emails…
Peter
From: Hare, Peter <peter.hare@college.edu>
Sent: Sunday, December 15, 2024 1:24:02 AM
To: Tartaruga, Donatello<Tartaruga_mutantem@university.edu>
Subject: Re: Mutant Ninja technology
Dear Donatello,
Perhaps the holidays have you tied up and you haven’t been able to get to your computer? Maybe you will have some free time soon? If not, could you ask Michael, Raphael or Leonardo to reach out to me? I really need some help. Another team at Virginia couldn’t replicate my analysis. So it’s not just the grant anymore, the journal wants a retraction of my paper and I am not sure what to tell them….
Peter