Set the mould before you pour.

We love a single answer

Market research has a love for buying models; we do love a good framework. And we also want there, somewhere out there, to be a single algorithm or technique clever enough to answer the whole brief in one swoop. Find the right model, ask the right question, and our clients walk away with that single golden answer. Rinse and repeat… and nice work if you can get it.

Most of the problems worth paying to solve, however, need their own framework, some custom-designed scaffolding before data can light the way to the right answers. Engagement. Loyalty. Brand equity. Or anything you can bolt the word “experience” to the front of calls for more than button pushing. Wishing otherwise is like pouring concrete on the ground and hoping for a house.

Most problems are systems

Underneath, these problems are systems: a dozen variables leaning on each other, feedback loops, relationships that bend or plateau rather than run in a straight line, and most of it not the tidy continuous numbers a regression wants, but five-point agreement scales and tick-box brand attributes, whatever shape they actually arrive in. And it’s usually chaos! But it can also be a network!

Final_Model_v8.xls

I learnt this the hard way in 2011, on one of my first real data modelling projects, when I was brought in as a bright-eyed junior statistician on an engagement tracker for an insurance firm. The client wanted one set of numbers to look at: ‘What makes my people stay.’ Run a regression, bang it into PowerPoint. Boom. (Also in the days when being able to put a background in your slides made you advanced in PowerPoint.)

So I loaded the data into SPSS, hovered down the menus and clicked tick boxes to run the regression as asked. I checked it out, nodded in myself, then exported it and named it Final_Model_v8.xls; and yes, it did exactly what a driver model is supposed to: quantified the influence of each factor on retention intent, complete with an R-squared in each market.

But on closer inspection it didn’t make any sense, and my regression-based smugness evaporated.

What such a model won’t stomach is a dozen Likert-scale predictors that all move together: pay satisfaction, perceived fairness, stuff about line-management and tenure, so tightly correlated that the coefficients flipped sign depending on which three I happened to leave in.

Multicollinearity, in the end, not really a puzzle to solve so much as a clue that one equation had been asked to do a system’s job. I had built a good answer to the question he’d asked me for; not the one he had.

And because my model didn’t even reflect the reality of what the employees were telling me about how they thought. Nor did I even consider how the business thought their employees thought.

On so many grounds, my model was useless.

This is not an argument against regression

I’m not having a pop at regression; almost everything in market research data is nicely handled by one form of regression analysis or another; most market research questions are perfectly handled with a fixed list of relatable drivers. You don’t need anything more than that.

Trouble only starts when you try to model the entire customer relationship itself (in my case, employee experience); where there is a legitimate hierarchy, levels of drivers, feeding into each other, where no matter how hard one tries, it won’t sit in one equation. Either it does what Final_Model_v8 did, or there is no signal in the data to model at all.

What a belief network does instead

That’s okay, because modern marketers don’t like to throw all drivers of ROI into the same bucket. On any day of the week, the same problem can be architected in different ways. One day a driver is the chicken; the next it’s the egg, depending on who’s speaking.

This is exactly the shape of a problem a belief network is built for.

Not because it’s any smarter than a regression, but because it seeks to understand the bigger picture first, before divvying the ‘system’ into smaller problems, more easily handled by traditional regression approaches. To do this, instead of one equation built around a single dependent variable, it holds every variable in the room at once: sketched out as what statisticians call a directed acyclic graph, each variable a point, each hypothesised relationship an arrow.

While this might strike us as more complex, at its heart, it is merely a series of crosstabs, which not only are much easier to explain but also mean we’re able to treat the data as nonparametric, so a five-point agreement scale or a tick-box brand attribute goes in exactly as collected, threshold effects and genuine nonlinearities included, with nobody guessing the right transformation.

A model that looks unfinished

There’s something refreshing about this directness. The ability to show a network of interlaced things as crosstabs.

Network modelling lets us work with those big ideas that would ordinarily have to remain on a whiteboard can be tested against the data without statistical assumptions getting in the way.

Some links are supported by the data; others won’t be.

While a smaller model looks finished and can be handed to the boardroom.

A network also looks like what it is.

Unfinished. Where smaller models might be one wall of a building, one could argue that at least the paint is dry. Network models are never done.

That brings us to the thing I like about working with networks. Their open-ended nature also means they’re always up for debate and very much hands-on. Draw those arrows with the people who run the business, and the drawing is usually the first time anyone has had to say their assumptions out loud, in front of each other, in a room. The result is a sharable, inspectable model of how their own market behaves, one they helped build and can watch move as the next wave comes in. Commercially, that’s harder, but worth the sweat. Ask a real boardroom which they’d rather have: a coefficient they can trust completely, or a model they’re allowed to argue with, even when their hypotheses don’t survive daylight.

The model can never be at fault because the graph isn’t set in stone. And, where the joints and surfaces don’t run plumb, you can see it, and you can go and check. And for some, that’s better than a model that’s wrong everywhere at once in a way nobody can see.

Earning trust wave after wave

A network can also earn its trust the way a good tracker survey does: wave after wave, checked at every joint, not delivered once and believed forever. When the market changes, so can our underlying assumptions. There’s no need to pour concrete on it. And it lets the business keep changing its mind, without being locked into one way of thinking.

So, for now, there is a model to love. And it’s always the one the business wants.

Moreover, this model loves us back because it reflects how modern market research wants to think of itself. As part of an open-ended conversation, rather than a model of unimpeachable truth. A model that is big enough to hold the business up, that can weather storms, and that’s honest about its foundations.