Optimising my own frustration

In movies science is mostly about what is known as the eureka moment. A moment where you have an amazing idea, stare off into the middle distance and say something like “but of course – the cheese!” And then jump cut to an amazing cheese-based new form of physics solving the problem.

Now these moments do actually happen. I had one where I was playing with my nephew and doing the static trick sticking balloons to the ceiling and realised that was the answer to my thin film experiment. I’m a fun uncle.

But these moments are just moments. The vast majority of time in science is spent doing something fan more delicate and slow processed called optimising. 

Optimisation is a simple process – you take something that just about works and make it work better. Often by trying to find the conditions and setup that best achieves the results through lots of small changes. The process may ‘work’ in a wide range of conditions but there’s likely a sweet spot where it works best (or at least best enough to collect data that you can show to another person).

Discovering that sweet spot unfortunately takes time and patience because unless you’re the Fonz you can’t hit the experiment go ‘eheyyyy’ and it magically be right. You have to slowly adjust the settings one small iterative step at a time from the start

Normally starting optimisation is simple. You take the system as it works now and see how it works if you change it a bit. Now choosing ‘a bit’ is a difficult thing and basically why experienced scientists are payed the slightly-better-than-terrible bucks. But you change it a bit one way and it might get better or worse. Then you change it the same amount of a bit the other way and again see if it gets better or worse.

That tells you where you ‘sweet’ spot might be. And from there you do the same by guessing where it might be based on how much better or worse it is every time you look a bit more or a bit less than. Each time making the ‘a bit’ a smaller amount and you narrow in on your sweet spot.

If you write that down it sounds simple and pretty linear. The more you fiddle the better it gets. But that description omits one key problem – it’s being done by humans.

Humans are brilliant at lots of things. Our mastery of communication and fire are unmatched and we’re the the only species that has developed doughnut technology. But us humans are also impatient and think we know better almost all of the time.

When optimising something these are terrible qualities.

The temptation to try and ‘hit’ that sweet spot by doing larger-than-small changes is strong. Instead of doing lots of increasingly small changes the temptation to ‘guess’ how much further to go and hit the spot without all those intervening steps is strong.

So most optimisation consists of a short run of increasingly small changes followed by a big change, followed by swearing, followed by a longer run of small changes to a sweet spot.

In a perfect world that would then be where you stop. But inevitably most people continue to change the setup in the hope of hitting a magical sweeter spot. This sweeter spot never exists.

The end point of all this optimisation is normally getting frustrated and settling on a point a few changes away from the place you were before you tried to find the sweeter spot and angry denials that it could ever be better.

If in reading this article you think that this process sounds utterly infuriating that you’d be would wrong. This process is beyond infuriating, it is a whole new level of infuriating only described as “ARRRRRRRRRGH”. The worst part is that it’s completely all my fault for being human.