The case of the missing Editorial Blog Post (and journal team)

Here’s a wee puzzle: A mature Open Data focused journal (“Journal A”), owned and launched by an company or Institute (“Institute B”), developed into the flagship of an Academic Publisher (“Publisher C”), runs their own properly archived and citable blog with DOIs etc (“Blog D”). If a briefly published editorial Blog Post (“Editorial E”) disappears from their Blog, could it be an accident, or something else?

An accidental deletion of a blog post by a publisher suggests might be taken as incompetence, but if the post isn’t returned it starts to look deliberate to me. Normally I would have expected a formal retraction, but this feels like self-censorship in an effort to control public perception? Neither is a good look for a custodian of the Scientific Record.

Can you tell what I’m talking about yet? What if I hint the missing “Editorial E” blog post was by the out-going Editor In Chief of “Journal A”, celebrating many years of innovation, and noting the decision of “Institute B” to terminate the jobs of a large part of the team at “Publisher C”, and that this post on “Blog D” is/was the only announcement of those changes? Answer below…

What have you done to your keyboard?! Hands Down Promethium on a MacBook

There were a few stickers on the lid, but why does my laptop look like this now? In short, I'm learning to touchtype a non-qwerty layout.

My Japanese MacBook keyboard has 34 stickers for a custom layout

FASTQ uploads to ENA FTP site with rclone

I've recently been working what I considered to be a large scale FASTQ upload to the European Nucleotide Archive (ENA), from where it will be mirrored to the NCBS Short or Sequence Read Archive (SRA). Although the total size was "only" 37GB, this was about 3500 pairs of Illumina MiSeq FASTQ files - more than enough to make me worry about the job being interrupted and needing to resume without repeating uploads.

BLAST max-target-seq meets metabarcoding

This is my first blog post in years - primarily down to a second child who is now a toddler. And what better topic to return to than a mainstay of past content, NCBI BLAST? This time with a motivating example from my recent work, metabarcoding. This is term used for sequencing a diagnostic region of DNA using specific primers for a group of organisms of interest, and then matching that amplicon to a database of known species. Human interpretation of a BLAST search can generally put a good guess as the organism - weighing hits and annotated taxonomy (e.g. ignoring the odd suspicious uncultured "fungal" match).

This post is about how sometimes BLAST on the NCBI website can miss 100% identical (albeit not full length) matches, returning instead lots of very good but longer matches. Basically the online defaults don't suit this use-case.

An overly aggressive optimization in BLASTN and MegaBLAST

All my recent blog posts have been looking at issues raised by the recent letter Shah et al. (2018) "Misunderstood parameter of NCBI BLAST impacts the correctness of bioinformatics workflows", and the associated test case which they ought to have included with the letter itself. Thus far I have focussed on:

• "What BLAST's max-target-sequences doesn't do" (2015 blog post, with links to follow up posts in 2018), an issue Suaji Kumar and I called a scary BLAST+ bug to do with alignment limits, which the NCBI BLAST team viewed as expected behaviour of a heuristic setting.
• "BLAST tie breaking by database order" (2018 blog post, with links to relevant older posts)

Over the Christmas break, there were two notable public developments. The BLAST team's reply was published, Madden et al. (2018) "Reply to the paper: Misunderstood parameters of NCBI BLAST impacts the correctness of bioinformatics workflows". Quoting from this:

"we examined the new example and it became clear that the demonstrated behavior was a bug, resulting from an overly aggressive optimization, introduced in 2012 for BLASTN and MegaBLAST (DNADNA alignments). This bug has been fixed in the BLAST+ 2.8.1 release, due out in December 2018. The aberrant behavior seems to occur only in alignments with an extremely large number of gaps, which is the case in the example provided by Shah and collaborators."

And BLAST+ v2.8.1 was released. Quoting the release notes,

"Disabled an overly aggressive optimization that caused problems mentioned by Shah et al."

So, what was this aggressive optimisation in BLASTN and MegaBLAST, and how did it combine with the internal candidate alignment limit and database order to produce counter intuitive results? It turns out to explain the details I'd not yet followed up from blog post part three.