The rhythms of research with MindCanvas

Since launching MindCanvas, just over six months ago, my life has kind of been overtaken by that (typical with startups I am told). You learn to live from crisis to crisis (sometimes the server crashes, sometimes a customer wants a major change in the system right now! You move heaven and earth within your little organization to do it, and present it nonchalantly the next day.) Personally, I have also started thinking of myself as an entrepreneur as much as a researcher and designer. Its been an exciting ride, and inspired by a posting by the founder of Riya describing the last few months at his startup, I am going to start writing more about the last 180 days with MindCanvas.

06_june_treesort.gif
MindCanvas TreeSort Interface

To begin, apart from the excitement of life in a startup, what I am enjoying the most is having a ringside view of how research happens in the real world. Let me confess, I am a research methods junkie. What fascinates me are ways to understand people. Doing good design is partly about getting the right picture of users. What are good ways of doing that? What methods are used at what time in the design cycle. With MindCanvas, I have learnt a lot about the rhythms of research – what type of research people do, at what stage of design.

The first thing you notice is what methods are popular during what part of the design cycle. Currently, MindCanvas offers four main methods: OpenSort, TreeSort, Divide-the-dollar, and FreeListing (with three more methods almost ready to be introduced). My observations are based on that, though I do have a sense of what other methods are being used at that stage).

6 months ago, I had made a guess as to what methods we most expected to be popular. This is what I had predicted: OpenSort, followed by Divide-the-dollar.

Of course, I was completely wrong…

And this is why every entrepreneur needs to launch as early as possible. Because you can sit in your office and speculate. But the market will give you the answer.

This is what the real picture is
FreeListing>TreeSorting>OpenSorting>Divide-the-dollar

FreeListing is by far the most popular. But that’s primarily because its very easy to slip in multiple FreeList questions as follow ups to the other methods. So, its not as if these are studies centered around FreeListing.

No, the real surprise is that the bulk of the studies are around TreeSorting exercises. Its not just a little more popular than OpenSort. Its about 7 or 8 times more popular. Why?

Some reasons are obvious. Maybe people are using other software for doing OpenSorting. There are more alternatives to OpenSorting than ClosedSorting – to the best of my knowledge. Its an obvious reason, but its not the whole (or even the main story). I know for a fact that in case of the most of the TreeSort studies, there was no earlier OpenSort study (especially using any software).

I often help design MindCanvas studies, and ask customers “Why not do an OpenSort”. The answer is almost always the same. “We already have a pretty good sense of what the structure should be. There are enough constraints that we completely change it based on user feedback. The broad direction is based on many constraints. We want to improve and refine it, alter it as needed. OpenSorting might lead us in a totally different direction, which we might not be able to accommodate… TreeSort allows us to explore within our broad vision.”

(these words are mine. But this is the jist of what I have heard MANY times).

Another reason I have sensed: TreeSort is done at the end of the design cycle – when you starting to think of the launch. This is a time when the people are starting to bit their nails and wonder: Will users like this? Will they get it? Is it too different than what we currently have? Companies are willing to spend money to make sure they are doing the right thing. It is the last chance to get it right before the rubber meets the road.

This is the time when people want larger sample sizes and different user segments. MindCanvas makes this easy.

This is also the time when designers might have multiple designs that they need to decide between. They really need to compare both in an A/B fashion: find out which one is more effective. You cannot make such a comparison in an OpenSort study.

Another observation: TreeSorting is often used at around the same time as when wireframes are being developed.

More MindCanvas musings to follow…