Embarrassing typos reveal the dangers of the lonely data analyst

A silly, but actually very serious, error in the supplementary material of a recent paper in Organometallics is causing a stir on the internets (I saw it on Andrew G.'s blog). The error in question is a comment in the supplementary material of the paper:

Emma, please insert NMR data here! where are they? and for this compound, just make up an elemental analysis . . .

As has been pointed out on the chemistry blogs, this is actually potentially a pretty serious problem. Apparently, the type of analysis in question is relatively easy to make up or at minimum, there are a lot of researcher degrees of freedom.

This error reminds me of another slip-up, this one from a paper in BMC Bioinformatics. Here is the key bit, from the abstract:

In this study, we have used (insert statistical method here) to compile unique DNA methylation signatures from normal human heart, lung, and kidney using the

These slip-ups seem pretty embarrassing/funny at first pass. I will also admit that in some ways, I'm pretty sympathetic as a person who advises students and analysts. The comments on intermediate drafts of papers frequently say things like, "put this analysis here" or "fill in details here". I think if one slipped through the cracks and ended up in the abstract or supplement of a paper I was publishing, I'd look pretty silly to.

But there are some more important issues here that relate to the issue of analysts/bioinformaticians/computing experts being directed by scientists. In some cases the scientists might not understand statistics, which has its own set of problems. But often the scientists know exactly what they are talking about; the analyst and their advisor/boss just need to communicate about what is acceptable and what isn't acceptable in practice. This is beautifully covered in this post on advice for lonely bioinformaticians. I would extend that to all students/lonely analysts in any field. Finally, in the era of open science and collaboration, it is pretty clear that it is important to make sure that statements made in the margins of drafts can't be misinterpreted and to check for typos in final submitted drafts of papers. Always double check for typos. 

This entry was posted in Uncategorized. Bookmark the permalink.
  • Rafa

    We should wait until all the facts are in before jumping to judgement on this one. It could easily be an innocent mistake made by a non-native english speaker. For example, by "make up an elemental analysis" they might mean "quickly prepare and run a simple analysis". . I do agree with the general sentiment of your post though.

    • jtleek

      For sure, but even if it is a mistake like that, you'd probably agree it is pretty embarrassing.

      Maybe I was confusing in the original post, my points are:

      (1) It is hard being a lonely analyst and can lead to weird situations like the ones involved in these two typos.
      (2) I'm sympathetic because intermediate comments out of context could be embarrassing.
      (3) We should still be really careful about making sure data analysts are supported/understand the decisions they are making.

      • rafa

        You were not confusing. But if you follow the link you included, you will see they practically accuse the authors of fraud.

  • Bill Raynor

    Actually, that is the way I was taught to plan research. Start with a summary of what we expected to see, then outline the table and trends we wished to learn about, then plan and execute the study to meet those goals. My "team" commonly involved an epidemiologist, a biostatistician (me), and other scientists as needed.

    So this just looks like poor editing. (We had secretaries to catch stuff like that, and they did.)

  • hadley

    "silly to" should be "silly too". I recommend always double checking for tpyos ;)

    • jtleek

      For real?

      • orizon

        I reckon that was a joke. This was'nt though.

  • utdiscant

    While not fixing the actual cause of the main problem, the LaTeX package fixme can help a bit with not leaving author comments in the paper (http://www.lrde.epita.fr/~didier/software/fixme.pdf)