As far as exploratory statistics, I think your methods are converging with traditional marketing techniques — although all too often marketing research focus on features, done right it first focuses on consumer needs. You might be interested in “Counterintuitive Marketing” by Kevin Clancy and Peter Krieg http://www.amazon.com/exec/obidos/ASIN/0684855550/interactionby-20/002-7723352-5434432 which discusses this sort of research at length — and also offers a sharp critique of what’s wrong with a most current marketing departments. Of course, they’re hard-core quantative guys, so their solution to everything requires six months and six figures worth of research, so I’m not convinced their solutions are entirely viable either.

Interestingly, one of my marketing books — it may be Counterintuitive Marketing — essentially duplicates the concept of personas (they’ve got a nice term for it: “indi _visuallizing_ your customers), which they use to put a human face onto a mound of demographic data. As far as I can tell, they seemed to have evolved this indepedently of any knowledge of personas in the UI world. My best guess is it developed out what’s called “account planning” in the advertising world, which has techniques that are often similar to “our” user research.

(It’s really interesting to see the convergences among different fields. For example, I first ran across Kano questions as part of James and Suzanne Robertson’s excellent Volere specifications template http://www.systemsguild.com/GuildSite/Robs/Template.html in the software engineering field — although they didn’t appear to know about Kano, since they didn’t suggest doing any of the analysis that’s part of Kano. The Robertson’s requirements development method, build upon on Gerald Weinberg’s excellent work, is heavy on understanding users — albeit from a traditional IT perspective.)

Anyway, it makes sense for us to ally ourselves with others, such as marketing, when it makes sense and particularly when we can piggyback on some the work they’re already doing. The biggest difference is one of focus. Marketing is focused on getting people to a “buy decision.” I like to think that we’re focused on getting people to a “buy again decision” — that’s to say they’re so satisified with the site/software/product that they want to use it and buy it again.

Obviously doing more quantative research adds time and cost to the development process — and can easily be misused and abused — so in large part I see the need to do this (as well as prototyping, testing, etc.) as related to the risk involved. The bigger the risk, the more you probably want to consider using methods that research/validate against a wider pool, assuming of course you’ve got time and money to do so.

What Mello suggests in terms of verifying results of qualitative technuques makes sense as well, but I am using survey in a slightly different way (for exploration rather than verification).

Let me explain: The fact that I am using exploratory statistics is more important than the fact that I am using a survey. Exploratory statistics is most similar to visualization techniques. Allows you take a chunk of data and see patterns. A low-cost way of identifying if there are patterns that fall out your analysis. (Also I think that exploratory statistics are far more interesting to UCD than “inferential statistics” (means, t-tests etc.) which are more commonly used. Exploratory statistics make suggestions rather than lay down laws. Unfortuantely they are rarely used in UCD. Card sorting with cluster analysis is probably the best example of the power of exploratory statistics.

Having said that I do think Mello’w use of surveys for verification is equally valid and possible in UCD as well. The survey would need to be slightly differently designed. Let me get hold of that book before I make more comments.

Also I am in complete with your below statement: Finally, just to point out the obvious, like any other testing, it needs to be balanced against the designers judgement…

The persona creation technique I describe only suggests a few groupings of user needs. How the designed uses that information to form the personas is entirely up to them. An example: In the writeup I describe three personas. Actually the analysis gave me four user need groupings. I exercised my judgment as a designer to decide that only three of the groupings were useful to me. Thats the beauty of exploratory statistics, it shows you patterns. Rest is upto you.

I should shut about exploratory statistics before I put everyone to sleep, and also show myself as a total geek :-)

I’ve been working on adapting a nice list that Larry Constantine and Lucy Lockwood talked about in “Software for Use” http://www.amazon.com/exec/obidos/ASIN/0201924781/interactionby-20/002-7723352-5434432

The checklist focused characteristics of the interaction (frequency, continuity, intensity, etc.) as well as of the information involved (flow direction (to/from user), volume, complexity, etc.) as well as some other factors.

Since Larry’s a software guy, it’s oriented towards ID, but I bet the librarians in the crowd could help come up with a list of “tactical” information characteristics that would be useful for personas on IA-focused projects.

Rashmi does highlight one issue that’s been nagging me for a while — validing that you’ve got the right personas and scenarios. With software-ish sorts of things this has always seemed easier for me, for a couple reasons. Usually it’s been automating an existing task, so while you may end up doing some process re-engineering, you’re starting with a group of known users, as well as tasks you can analyze. (I haven’t done general audience software products, so I realize it may be a harder to nail down in that situation.) With the content-oriented web sites, I’ve worked on, it seems like identifying the users and scenarios was more difficult, but again this could be due to the projects, which were sites with broader audiences — and these sites arguably did a poor job in developing a marketing focus that would’ve targeted particular users.

In general, I think I’ve done a good job of getting it right by intuition — and arguably that’s part of the skills expected of good UX practitioners. But while I think survey research shouldn’t be required to develop personas and scenarios, I think it’s potentially quite useful in validating them (assuming the client is willing to spend the time and money, and depending on the amount risk involved in the development costs). I’ve run across some interesting techniques in Sheila Mello’s “Customer-Centric Product Definition” http://www.amazon.com/exec/obidos/ASIN/0814406688/interactionby-20/002-7723352-5434432 which comes from the traditional product development field. (In her field, I think it makes sense given the high risks involved: the tooling of manufacturing plants, the costs of a flopped products stemming from wasted marketing and distribution efforts, unsold inventory, etc.)

She uses similar methods to UX research (task analysis, observation, etc.) to develop the equivalent of personas and scenarios. But what’s interesting to me is that once you’ve identified a number of the key task/goal scenarios, she then uses a survey to check both prioritization of which things people feel are important, as well as a Kano survey (which asks about the satisfaction/dissatisfaction related to whether the purposed product would/wouldn’t enable the user to accomplish a particular task/goal. If I remember one her examples from designing a golf club bag, the Kano question pairs look something like: How would you feel if the bag helped you find the club quickly? How would you feel if the bag didn’t help you find the club quickly. (The idea is that some things people won’t give you credit if it’s implemented but will be upset if it’s not there, and others that they won’t feel deprived is it’s missing, but will be happy if it’s there.) It’s important to point out Mello is testing scenarios and not features.

Mello argues that advantage of surveying is that you check your ideas against a wider against, which guards against your ideas having been skewed by the small sample size that’s inherent with qualitative methods. My take, is that it’s also got the advantage of coming up with some numbers to keep quantitative types (i.e. most business execs) happy.

Given that online survey’s are relatively easy to set up these days, I think it’s worth being open to seeing how quantative techniques can complement our intuitive/qualitative ones. Of course there’s still the time and expensive of recruiting survey subjects, but the methods Mello describes seem like developing the survey questions can be done quickly, since they’re based around the scenarios already developed. And if you’ve got an online survey, you can capture the responses directly to a database, which eliminates much of the grunt work of compiling things (of course you’ve still got to do the analysis, but Mello’s analysis are pretty straight-forward).

Finally, just to point out the obvious, like any other testing, it needs to be balanced against the designers judgement, since part of their skill should be seeing solutions the users may not see.

As for the book party, i havent’ figured out the where and when… working too much I guess. But soon, and I’ll annouce on ye old blog,…

You got it right: there is a possible tool which could do part of this process.

I followed the Grudin paper and got an error. Is this the link referenced above?

http://www.research.microsoft.com/research/coet/Grudin/Personas/Pruitt-Grudin.pdf

Secondly I think information rich domains are different than the examples you offered. One needs to understand types of user needs, and mental models, not just the “needy” user.

Also I think of this technique as another alternative in the toolkit of designers rather than a replacement. For certain design problems this might be more appropriate, for others it might not be.

Finally regarding the goal for this: Think of the way designers use card sorting to understand user mental models in a quick and dirty fashion. Well if you trace card sorting back to its academic roots, its a long ponderous technique (like most academic techniques). But designers have adapted it their own needs in a very cool way. My goal is similar: to reduce this method its essence into something that can be carried out by every designer who wants a quick way to understand their users better. Currently this write-up just describes the jist of the idea. Its not there yet (by far), but I am working on it :-)

I notice your book is out. Congratulations! When is the book launch party? Or did I miss it.

The classic example is the stewardess and the rolling case. Stewardesses were often small women, traveling all over the place, never knowing where they are going next, possibility of luggage loss– they had a set of highly specific and distinct needs. Voila, the rolling case was born and is now carried by millions of travellers. I think while statistically defining the representative user might be useful for developing a persona used for evaluation through cognitive walkthroughs, I don’t think that persona would be equally useful for designing a system.

Cooper used to have a case study on their site that i can’t find now, that showed off the design of a inflight entertainment system. While the typical flyer could use a complicated navigation system, they chose (if memeory serves me right) an older gentleman who wasn’t tech-savvy and a child as personas, leading to the design of a very simple system with a navigation tool that was easy to use for someone with low manual dexterity. This system could be used by the typical and the challenged user. Thus the design met more users’ needs.