You can’t iterate your way out of a bad design: Multivariate testing and the problem of local maximum
At the most recent Fantastic Tavern we debated what the recent emergence of multivariate testing means for ‘designed’ experiences and design as a discipline. Taken to the extreme, will we see experiences that essentially evolve themselves into some optimal form that no designer could have ever conceived through a Darwinist, survival of the fittest process? Will the designers of today be replaced by some great crowdsourced process that runs on cold hard data?
Absolutely not. Why? The problem of the local maximum.
This is a concept that I came across as part of my Information Architecture Masters course and although it is usually talked about in mathematics circles, it has some big implications on design – particularly iterative design. In a design context, local maximum describes the fact that an ‘optimised’ design is fundamentally anchored to its original concept. To put it as bluntly as possible: You can’t iterate a bad design into a good one.
I will dig up more visuals to illustrate this clearly but indulge me in this thought exercise in the meantime. Imagine a mountainous landscape with some molehills, some hills, and some mountains. Now imagine that these all represent alternative designs – and the bigger the mountain, the better the design. Suppose that you were parachuted in with the goal of scaling the highest mountain. To do this you would obviously need to land somewhere on that mountain and climb from there. If you landed on the smaller mountain next door and just started climbing, it doesn’t matter how long and hard you climb, you can’t climb higher than its peak (the local maximum!). Although this might represent quite a climb, you will never reach the height of the bigger mountains around you. You are fundamentally constrained by its lesser height.
Hopefully the leap from this metaphor across to what I’m trying to explain for design isn’t too far. When you put a concept together you are defining a starting point. That starting point has a local maximum. Whether you reach that local maximum depends on how long and how well you iterate that concept, but regardless of how much time, money and technology you may have – you will never exceed it. How do you get past the local maximum? By coming up with a different concept, a different starting point that has a higher local maximum.
Tools like multvariate testing are giving us better quality data and insights that allow us to increase the speed and progress that we can make towards optimal designs. But if you try to iterate a bad design into a good one you are ultimately polishing the proverbial turd. Thanks to the constraints of iteration, we designers aren’t going anywhere soon.