Game-like elicitation methods: A new approach to user research (or MindCanvas is live!)

For the past year, if someone asked me what I am upto – I told them consulting, which is the truth, but only half the truth. The other half, which has kept me up at night (both literally and figuratively) is MindCanvas. No, don’t click the url yet – first hear the story.

We started working on it a year ago, but I have been dreaming of it for a long time. It probably started with seeing the boredom in the eyes of my research subjects (sorry participants) in graduate school. And continued with consulting, with every computer-based study I ever did. We do qualitative research as well, but truth be told – everyone resorts to surveys, at some time or other. The idea of basing product decisions on bored respondents robotically checking HTML boxes and choosing dropdowns really bothers me.

So, what is the solution? A better survey application with some AJAX peppered in? No, we wanted to go far beyond that. Our goal was to reimagine what online research can be. MindCanvas was the answer (a side-note about the name: it satisfies my geeky desire for it to be something about the mind. The words “brain” & “neuro” were summarily rejected by the rest of the team!)

mindcanvas logo MindCanvas is a research service to help companies gather insights about customers’ thoughts & feelings. We use Game-like Elicitation Methods (GEMs) to let online users participate in answering the complex questions that you face in designing a product or service.

MindCanvas is not Yet Another Survey Research Service. In creating MindCanvas, we dreamt up Game-like elicitation methods. And, of course, we needed an acronym – GEM.

Games users play (or what’s game-like about GEMs)

the mind Prior to MindCanvas, I did not play computer games and felt perplexed at how much time people spend playing. When I was introduced to games, I was amazed by the complexity games get away with. Step by step, games can elicit pretty complex behaviors. We started looking into the psychological aspects of game design. The way that attention is controlled, the feedback schedules, the positive reinforcement, the step-wise challenges and the attract mode.

We looked to games to solve the complexity problem with GEMs. How do you engage users in pretty darn complex tasks? How do you gain and keep their attention? How do you get across what they can do next? How do you make sure they have “fun” doing this? Games helped us with this and much more. I should add that our entire team got hooked on various games in the process of creating MindCanvas!

Focus on the mind (how people think)

Many market research methods focus on understanding what people want. The typical method is to ask people directly – through surveys / or focus groups. Asking people directly has its uses, but it’s not a good way to understand people. Additionally much of human thought lies below conscious awareness and its important to use methods that tap into the subconscious.

the mind In his book, How Customers think, Gerald Zaltman makes a powerful case for understanding how people think for marketing.

“Though called by other names, many consumer-research topics directly involve cognitive structures, including product perceptions, brand attitudes, brandattribute, beliefs, brand personality, and consumer expertise. As consumers acquire new knowledge and interrelate it with existing knowledge in memory, they are assumed to form cognitive structures in memory. These cognitive structures or mental models represent the interpreted meanings of a product or a brand.”

What he says applies not just to marketing, but to user experience as well. Zaltman focuses on metaphor elicitation, we are interested in methods for understanding how people think on a broader level. When we started looking into methods for understanding user thinking – sorting comes up again and again (in psychology, cognitive anthropology, market research and most of all creativity techniques). The technique is known by different names: card-sorting in the UX world, pile-sorting in cognitive anthropology, simply sorting/clustering in Psychology. Many techniques like K-J technique, mind-mapping have as their basis sorting items by groups of people. We took sorting as our own starting point for methods to understand people’s concepts and languages. We layered on various types of statistical analysis and a visualization engine.

Zaltman also emphasizes going beyond words – using images to tap into the unconscious. With MindCanvas, we made sure that you can use images as easily as words. You can easily use pictures to show a concept and get people’s reaction to it.

How GEMs fit into the research methods landscape

We think that GEMs offer a new paradigm for online research and fundamentally change the research landscape (as below diagram shows). We have just at the beginning – wait as we translate more methods into GEMs.

The X axis shows the richness of data collected. On one end are open-ended methods such as interviews, observation. On the other end are survey methods, where you prepare both questions & response options, and simply ask people to choose. Its a very structured method of data gathering.

research landscape

On the other Y axis, we see several correlated dimensions. “Reach” is about number & types of user groups you can reach with a method. With qualitative methods, geography, convenience, availability, all factors lead to in-depth study of few people (sample sizes of 4-5 are common). On the other end, are survey methods (and the recently popularized Wisdom of Crowd methods – for creating markets) that have a wide reach (especially online). Reaching large samples of representative users and collecting data is inexpensive (relatively). This dimension is also correlated with ease of analysis. Its easier to find patterns across people with survey methods. In contrast, its difficult to analyze and aggregate data from open-ended methods.

So, your choice is between rich, qualitative, small-sample research and shallow, structured large-sample research. With MindCanvas, we aim to add another option – rich data-collection methods for remote, large-sample studies. We took methods that are semi-structured, fun to begin with (such as card-sorting, free-listing, divide-the-dollar). By translating these mehtods to online systems without loosing the fun factor, we enhance their reach, and ease of data collection/analysis.

Where MindCanvas is now

We are in a Beta stage currently. MindCanvas has been used by some design teams and anyone who used it has wanted more of it. We think it fundamentally changes the design research process. It is not suitable for every design research problem, but there are a few we think we solve particularly well. Apart from GEMs there are many other innovations with MindCanvas (for example, interactive deliverables-to-go). I will be writing more about that in the next few days (both here and on the MindCanvas blog).

The concept of GEM is larger than MindCanvas. Read the GEM Manifesto to understand our thinking while creating GEM. And wait as we translate more methods into GEMs. Or check out the demos, follow the MindCanvas blog as we write about developments and inspiration.

Are there other methods you can imagine as GEMs? Tell us about it. And in case you are curious to take a direct look, call/email us for a demo. We love showing it off!

11 thoughts on “Game-like elicitation methods: A new approach to user research (or MindCanvas is live!)

  1. Hey Saad,

    Thanks – yes its been in private Beta for a while now. The website took a while to put together – we knew what we wanted for it. Just took a while to get the design right.

    -rashmi

  2. Hi Rashmi,

    Nice work! I came across this site while searching for HCI on google. I’m a PhD student trying to get undergrad “subjects” to participate in my research and its frustrating when they do it just for the credit. MindCanvas looks really interesting. Will look into it more after my candidacy tomorrow.

    Sampada.

  3. Congratulations again. This solves many small problems… primarily of providing an incentive for participating in surveys.

    Just a tiny suggestion, I think Gerald Zaltman’s name should be used with a bit of caution here because MindCanvas is quite distinct from how Zaltman thinks of user research. You don’t want people to think this is Zaltman’s methodology.

    Btw, it’s interesting to see the trend of UX consultants/ designers moving into applications/ services sphere. 37Signals, Adaptive Path, Creative Good and now Uzanto. It’s a good thing actually, because it means lots of wonderful applications for the rest of the world. All the best!

  4. Thanks Manu,

    Your note regarding incentives: Actually I dont think game-like methods should mean that incentives should not be provided. People are giving up their time to help you out – its right that incentives be provided. I see it more as a problem of bad data from respondents who are not engaged in the task.

    For the studies we have done so far, we have made it point to suggest to the client that incentives be provided (even if a one in a 100 / 200 chance of winning a small gift).

    But this idea of not providing an incentive might work in a game never-ending scenario. The research input would be in much more bite-sized chunks (3-4 minutes) and the game-like aspects would be enhanced. Also, it would be always on, so researchers would not have this deadline for data gathering.

    Yes, the rise of the UX entrepreneurs is definitely an interesting trend.

  5. Rashmi, the demos are very good, expecially the top ones.

    They’re attractive and clean, and that’s important. I would look a little more at the text view shrinkage on the treesort of travel opportunities – may be better for example to design in a clean short text alternate for that part of the display.

    A very strong shot is that the canvasee gets to define their own categories – that’s a great direction.

    Kind regards…

  6. Rashmi, I am very impressed by your work on UX research and specially MindCanvas.

    I am now thinking on the long term impact MindCanvas is going to have on the industry post Web 2.0.

    So much excited that I have posted my two nuggets about you, Uzanto and MindCanvas on my blog http://harsh-thefirstattempt.blogspot.com/.

    Have a look.

    Toast to you !
    Cheers

    Harsh

Comments are closed.