Pessimistic induction

One of my favorite ideas from recent memory is pessimistic induction. As usual, a quick search finds plenty of smarter people who thought of it before me and easily refuted it. Even so, it’s oddly compelling, a gem of a misconception.

Looking back at history, most of our ideas, even the best ones, have turned out to be wrong. Very few have stood the test of time. Newtonian physics, the rational economic actor, and fat vs sugar are just a few famous examples.

It’s easy to think that we’re at the final culmination of our entire historical arc of science, art, and civilization. We may have been wrong in the past, but we’ve rooted out our mistakes, corrected them, and we now have everything figured out. It’s such a common misconception that it has its own name: the end of history illusion.

That name is apt, though: it really is just an illusion. The present day may feel special, but it’s usually just like every day before it. Many things are good, some things are bad, and our currently accepted scientific ideas are almost certainly wrong, to one degree or another. This is the pessimistic induction.

It’s chillingly elegant, but it has a fatal flaw. Yes, today’s best science may likely be wrong, but right and wrong are rarely black and white. Modern physics is famously incomplete, but working physicists would still say that the standard model and string theory and holographic universe are better ideas than Newtonian physics. We may not be perfectly right about everything, or maybe even anything, but we’re probably more right than we used to be.

That’s a comforting thought. The universe may be cold and indifferent, but we can admit our flaws and still hew ever closer to understanding. Onward.


What I work on

I had a conversation with a good friend recently that crystallized something I’d always felt strongly, at a gut level, but never thought through: how I choose what to work on.

When I look for a new job, I think about project, people, compensation, role, company, commute, etc. I’ve tried focusing on different factors over time, and I’ve found that for me, project is often the most important. I’ll suffer with low pay, long train rides, or a role I’m overqualified for if I’m working on something I care about and believe in.

I prefer tools over products. Systems over tools. Protocols over systems. Problems over users. Wicked over tame. Research over application. Many of these are stereotypical engineer cliches, but they boil down to an interesting theme: I prefer to work in areas where the goals and incentives don’t change much over time.

I don’t know where I developed this tendency toward the long term, but it’s a big personal motivation. The time scales I’m thinking about are centuries and millenia, not years or decades. I could just as well replace time with generations. I’m fine with not shipping code often, or not making any progress for longer stretches, if I know the problem will still be around and my work will still apply down the road.

What does this mean? Well, scratch most products – consumer, enterprise, or other. Some of them last centuries, but not many. Scratch applications and services in general. I’m happy to do work that’s used in a product or service, but usually only if there’s an underlying problem with a longer lifespan.

The two main areas that fit are research and infrastructure. Academic departments and conferences rise and fall, but the central goal of research has stayed the same forever: pursuit of truth, knowledge, and understanding. That won’t change anytime soon.

Infrastructure, on the other hand, is worlds removed. Construction workers in hard hats on building sites don’t overlap much with tweedy professors in ivory towers. They do have one thing in common, though: their goals are consistent over time. If you want to cross a river today, you build a bridge, just like a thousand years ago. We still need roads to get from one place to another. Plumbing to carry water and sewage. Electricity and communication grids may be newer, but we’ll need energy and communication in a thousand years just like we do now.

When I look at the projects I’ve enjoyed most in my career, they fit the bill. Sharding databases and later Paxos etc: classic infrastructure. Networking: infrastructure and applied research (OpenFlow). Color Genomics: applied research. App Engine: infrastructure as a product. Even the side projects I’m looking into now fit: climate change, p-hacking, the reproducibility crisis.

Why do I care if goals change over time? I’m not sure. Some of it may be the natural human desire to leave a legacy. If I work on big, long standing problems, I’m more likely to be remembered after I die. I don’t spend much time thinking about legacy, but it could still be lurking in my subconscious.

A modern variation is “changing the world.” It’s a well worn phrase here in Startupland, but for me personally, it’s always seemed hopelessly ambitious. I have no illusions that I’m personally going to change the world in any significant way. Maybe a little, if I’m lucky, but not a lot.

Another Silicon Valley buzzword is “impact.” Everyone wants to work on something impactful. Most people use it to mean a bold new product, or a big user base, or innovating and disrupting an industry. I want to have impact, sure, but I want to do it by moving the needle on a big, important, long term problem. Growth hacking and TechCrunch coverage aren’t part of my personal equation.

Research and infrastructure aren’t unique. There are plenty of other areas where goals and incentives stay the same over time. Art, clearly. Philanthropy, education, entertainment, health care, public policy…the list goes on. I’d get restless if I was a teacher or actor or nurse and didn’t do anything new, but there’s plenty of opportunity to push on big problems in those fields on the front lines. I’d hate being a campaign manager, but I could easily do a stint as a policy wonk at a think tank.

This may not mean much to you, or even to me. After all, it was guiding my career decisions long before I thought it through and wrote it up. Still, now I know…and knowing is half the battle!

Don’t get the wrong idea, I’m still loving it at Color Genomics! I’m not going anywhere. On the contrary, we’re actively looking for good people. If you want to work on something meaningful and challenging, drop me a line!

Also, scratching my own itches is one big exception to this rule. If software is the tool of the knowledge worker, I’m lucky to be a toolsmith. I’ve written and modified lots of software over the years to solve my own problems. Some took significant time and effort, like granary and P4, and some have real user bases, like Bridgy and huffduff-video.

Even so, these tools have always felt practical, utilitarian, even a bit disposable. I don’t consider them a big part of my career or life’s work. I won’t need them forever, and they’ll all grow old and die eventually. That’s OK.


A medium is born

Artistic mediums are a small, rarefied lot. The spectrum from War and Peace to The Da Vinci Code is huge, but it’s all a single medium: books. Citizen Kane and Jackass may not have much in common, but they’re both movies. Art, music, theater, radio, games, maybe graphic novels…that’s pretty much it. We can haggle over subtypes like radio drama or journalism or stand-up comedy, but still. Thousands of years of civilization, and we can count the main artistic mediums on just two hands, with room to spare.

…except now. VR is here, and it’s a truly new medium. You and I have the awesome privilege of witnessing its birth firsthand. That doesn’t happen often, and it’s pretty damn cool.

Mediums don’t spring into existence fully formed. Music has come a long way since Gregorian chant. Modern movies may descend from Méliès and the Lumières, but they’re so different as to be almost unrecognizable.

For example, film buffs celebrate movies with extended single-take scenes like Children of Men and Birdman, but the first movies were all single takes. Early filmmakers thought cuts would confuse audiences or lose them entirely. Reality doesn’t have cuts, right? Moving cameras, close-ups, subtitles, and establishing shots all have similar origin stories. They didn’t happen automatically. Early filmmakers discovered them through slow, tedious experimentation. And for every technique that lasted, ten others were tried and discarded.

We’re already seeing the same thing happen with VR. Just to get mainstream devices out the door, developers had to find the right resolution, framerate, field of view, latency, and inter-pixel distance (aka screen door effect) to make the experience viable.

Once creators got their hands on dev kits, they faced an entirely new set of challenges. VR infamously makes some people nauseous, especially when they’re moving in the virtual world but not in the real world. Creators have tried all sorts of things to prevent this, and many are now settling on two that work: teleporting and tunnel vision. Similarly, filmmakers aim their cameras, but 360° films let viewers look anywhere, so VR filmmakers are learning to frame shots with sets and lighting instead of cameras.

Good artists borrow; great artists steal, as the saying goes, and artistic techniques are no exception. Video games have faced many of these problems for decades, and developed their own solutions. Valve’s Half Life 2 famously pioneered set direction and framing to get players with freedom of movement to look where they wanted. Likewise, user interface conventions for Nintendo Wii and Microsoft Kinect motion controllers laid the groundwork for many VR UIs.

I find all this fascinating. Technology is constantly evolving, so we often think about how new technology enables and shapes art, whether it sticks (like 35mm film) or not (like multimedia). But art also shapes technology. Set framing and teleporting in VR may be intangible, but they’re arguably still technology just like 35mm film.

Even when we do think about art shaping technology, it’s much harder to actually see it happen in real time. Evolution is an apt analogy here. When life first hits a new environment, there’s an explosion of differentiation as evolution finds all the species that can survive. Hence “Cambrian explosion.” Progress soon slows down to incremental improvement, though, as more and more niches in the ecosystem are filled in.

Artistic mediums are the same way. At the beginning, no one knows the rules – there aren’t any! – so creators try anything and everything to see what sticks. This experimentation slows down as winners emerge. Novels have been around for millenia, for example, so we don’t often see truly new structures or techniques. We may gush about unreliable narrators or experimental forms these days, but even those are rooted decades or centuries ago.

OK, so there aren’t many different artistic mediums, VR is a new one, and we get to witness its birth. So what? Should we watch for anything in particular? Should we nudge it in some specific direction? Are there unique opportunities opening up that we may not see again?

Probably not. VR has plenty of attention and press coverage, and a small army of developers and creators pushing it in every direction. We’ll be learning new things for decades to come, and each step along the way will be heavily documented. More importantly, as with all technology, we don’t have much control over how it evolves anyway.

VR isn’t even the first new medium in recent history. Art, music, theater, and the written word may predate the 20th century, but radio, television, film, graphic novels, and video games were all born in just the last 2% of recorded history. We watched creators grapple with them, documented their progress, and adopted them into the mainstream just like we will with VR.

So, my only call to action is to pay attention. There’s plenty of precedent, sure, but the birth of a wholly new artistic medium still feels unique and new to me. It’s the first time I’ve ever really seen it happen. I don’t know where creators will take VR next, but I can’t wait to find out.


Drawbridge up, drawbridge down

When populations are homogenous, people see that other people are mostly like them – ethnically, culturally, socioeconomically, etc. This boosts trust broadly, which makes everyone more open to progressive social policies and safety nets.

When populations are diverse, people are more visibly different, at least on the surface, which leads to othering, dampens trust, and leads to more protectionist, socially conservative policies.

This is an oversimplification, and happens only in our subconscious, but there may still be a nugget of truth to it. It might be one reason that Scandinavian countries have long been so open, progressive, and even socialist: their populations are extremely homogenous. North America and western Europe are historically diverse, on the other hand, now more than ever, which has coincided with populist waves of nationalism, isolationism, and xenophobia.

Stephan Shakespeare, co-founder of pollster YouGov, has a famously evocative metaphor:

We are either “drawbridge up” or “drawbridge down.” Are you someone who feels your life is being encroached upon by criminals, gypsies, spongers, asylum-seekers, Brussels bureaucrats? Do you think the bad things will all go away if we lock the doors? Or do you think it’s a big beautiful world out there, full of good people, if only we could all open our arms and embrace each other?

Whichever you feel personally, this begs the question: why do you feel that way? Homogeneity or diversity in your immediate surroundings could be one answer.

Subtle, powerful point that I hadn’t fully appreciated until now. Thanks to Jonathan Haidt.


Channeling the hacker way

Also posted on the Color Genomics blog.

We have big dreams and ambitious plans. We want to push the state of the art in health and genetics, and we need broad, crazy moonshot product ideas to get us there. How do we find those ideas?

We’ve always been inspired by 20% time at Google and 3M and hack weeks at Twitter, so we decided to do our own hack week. We invited everyone to put normal work on hold for a full week to try out new ideas, no matter how crazy or tangential. We didn’t know how many people would participate, or whether we could ship any of the results, but it was worth a shot!


Our first task was to pick a theme. We chose engagement: how can we help people continue to engage with their genetics and health over time? We dragged everyone into a few big rooms and brainstormed project ideas. No judgment, no constraints, no requirements, no analysis or pro/con lists. Just ideas. By the end of these sessions, we had over 50 candidates to explore in the week to come.

We kicked off hack week proper on Monday morning with an example project: reincarnation detection. Could we identify who you’d been in past lives? With tongue firmly in cheek, we described how to recruit teammates with good karma, work with researchers to identify genetic markers across lives, partner with Buddhist monks, and design a “Talk to the Other Side” messaging UX.

We set everyone loose, and they hit the ground running. People wrote up pitches, formed teams with great names – Pink Duck was a personal favorite – and jumped on company mailing lists to recruit team members. Many people joined two or three teams each.

We emphasized that we wanted everyone to participate, across job roles. Design mockups and prototype code were nice, but not required! We encouraged people to start broad and high level, then find a vertical slice that they could flesh out in a single week. We put up screens around the office that rotated between team Slack channels, project docs, and mockups and prototypes as they came together.

The week culminated in an all hands Demo Day where each team got a few minutes to present their project. We saw live demos, skits, theme songs, and even a custom video. We voted on yearbook-style awards like Best Dressed, Most Likely to Succeed, and Crazy like a Fox. Everyone had a blast.

After the excitement wound down and the dust settled, we surveyed the results. Over half the company participated, working on 11 projects that all made significant progress toward proofs of concept.

Hack week infused us with new ideas to help our clients understand and act on their genetic data. We look forward to implementing some of the best ideas, and we hope they’ll help our clients lead longer, healthier lives. That’s the real prize.

We’re always looking for talented engineers. Join us!


Wanted: climate change project

I’m in the market for a new side project. I’m looking for something related to climate change. I don’t know if I’ll end up farming biochar, spraying aerosols into the sky, hacking solar panel trackers, or something else entirely, but those are the kinds of thing I’m thinking about.

I’m leaning toward something technical. I’m open to coding, but ideally that won’t be my primary contribution. I’ve had a blast hacking on open source projects, and I’m sure I’ll do more, but right now I’m thinking about something different.

So far, I’ve just been reading a lot and learning as much as I can. There’s lots of great stuff going on, and I’ve found a few open communities here and there, but most substantial projects are either companies or academic research labs. Those are both great, but they’re hard to join part time and contribute a handful of hours a week.

The next step is to talk to people who know more, or know other people who do. If you know the space and have an interesting problem I might be able to help with, or if you know someone who might, please drop me a line!


College is more than a job ticket

It’s become fashionable recently to second guess college. The ROI no longer works: it’s too expensive and doesn’t guarantee you a good job. It’s elitist and out of touch with reality. Student debt is predatory and out of control. Anyway, MOOCs’ unbundling model is the future of higher ed, so we might as well get on board, right?

That all may be true, but I think it’s too narrow. There’s a corollary to “you can’t optimize what you don’t measure”: measurement can give you tunnel vision. You can collect X University grads’ incomes, divide by tuition, and compare to College Y, but that doesn’t mean you can reduce either one to a simple financial investment you optimize to get the best salary.

College is an experience. It’s one of the most critical periods of your life: when you become an adult. You learn what to eat, what to drink, when to go to sleep and when to wake up. You manage your time (or not), juggle priorities (or not), make commitments and break them. You find substances, wonderful horrible tempting substances. You make friends and significant others, some more significant and some…um…less.

Most of us do this by making mistakes. No matter how mature we were already, every one of us slept through the History 101 final and flunked, or jumped into bed with someone we knew would break our heart, or woke up in a bush with no pants and the mother of all hangovers.

You can do this without college, of course, but the real world is harsh, and college has training wheels. Advisors, RAs, dorms, cafeterias, and built in health care make for a forgiving place to learn to “adult.” Drunk bicycling is a lot less dangerous than drunk driving. Classes are the perfect practice for jobs. Hated one? Failed the midterm? Start fresh next semester, older and wiser and still in the same dorm and meal plan.

Sadly, one big flaw with these safety nets is that they’re unequal. On campus housing, student advisors, and extracurricular activities all cost money. They may be standard at expensive top tier schools, but not at smaller state schools and community colleges. Maybe we should vote for Bernie next time.

I think a lot about how to prepare my daughter for the real world. I catch her if she’s about to fall off the bed, but I also show her the edge, let her look down, and say, “See? If you fall, it’ll hurt!” Sometimes I even let her fall a bit – not far, just enough to notice.

My college gave me the same kind of real world training wheels. The degree helped me get a job, sure, and the classical education made me a better person and citizen, but I treasure the safe space it gave me to grow. College is more than a financial investment. It’s a critical transition from childhood to adulthood. Don’t give that up.