essay, Uncategorized

Engineering bioinformatics in seconds, not hours

Cross posted on the Color blog.

10,000 Year Clock, Long Now

It was winter 2014. Pharrell had just dropped Happy, the Rosetta probe landed on a comet, President Obama was opening diplomatic relations with Cuba

…and here at Color, the bioinformatics team had a problem. Our pipeline — the data processing system that crunches raw DNA data from our lab into the variants we report to patients — was slow. 12 to 24 hours slow.

This wasn’t a problem in and of itself — bioinformatics pipelines routinely run for hours or even days — but it was a royal pain for development. We’d write new pipeline code, start it running, go home, and return the next morning to find it had crashed halfway through because we’d missed a semicolon. Argh. Or worse, since we hadn’t launched yet, our live pipeline would hit similar bugs in production R&D samples, which would delay them until we could debug, test, and deploy the fix. No good. Continue reading

Standard
repost

⏲️ Bioinformatics pipelines can take days, but software engineering edit/compile/test loops should happen in minutes or seconds. Here’s how we moved from twiddling our thumbs to iterating in real time: blog.color.com/engineering-bi…

Reposted https://twitter.com/Color/status/1105516986900013056.

retweeted this.

Standard
like

Help me keep home.mcom.com online. Dear Lazyweb, tell me who inside of Oath / Verizon has the ability to edit the DNS records of the “mcom.com” domain. Or, preferably, just transfer the domain to me. Longer version: For eleven… jwz.org/b/yjLK

Likes https://twitter.com/jwz/status/1105321474729103365.

Standard