If its ethnography, its gotta be right: reflections on DUX 05

This thought came back to me again and again during the DUX conference that I just got back from. Many speakers told us about the “ethnographic research” they conducted. Sometimes they shared some video of their observations – of children playing, or people in their homes, sitting on a chair, or watching TV. And the audience would watch delightedly – look at that, its people! People playing, laughing, sitting, walking… It all seemed very rosy – “we observed some people, maybe for a few hours, maybe we lived with them for a week or two – they still send us postcards – the dears. And at the end of it, we had the Aha moment, when it all fell into place. And the product was born.” And everyone lived happily ever after.

Ok, I am exaggerating. But it did strike me that no one (I did miss /get distracted during a few presentations) ever talks about when ethnographic research lead them astray, or a product concept that does not generalize, or ideas that do not validate. If its ethnography, its gotta be right!

First of all, I doubt that most people are even doing ethnography in the real sense of the word. Call it user/customer research, observation / qualitative interviews / design research. Sometimes when talking to clients, they ask us if we do “ethnography” – I always say, “well kind of”, feeling guilty about calling the type of qualitative research that one has time for – ethnography.

Lets assume for a moment that its fairly easy to do the qualitative research (its not, but lets assume, shall we). Even so, there are many challenges remaining – how do you make sure that the insights in the observer’s head reach other members of the product team (too often, the researchers learns a lot, but only a very small portion of the distilled insight is transferred). How do you synthesize those insights? How do you go from that synthesis to the product concept? And how do you validate those product concepts – make sure they generalize beyond the few people you were able to observe?

Those are just some of the questions that need answers, and it would be nice to see papers that tackle those issues on a deeper level.

(As a note, the conference was very enjoyable overall – the organizers did a great job creating a very well choreographed experience, with great attention to detail – I do not want to come off just being critical – so more on what I enjoyed later.)

17 thoughts on “If its ethnography, its gotta be right: reflections on DUX 05

  1. Although if we are to believe the panel discussion all we need to do is see that someone else brushes their teeth and realize they are just like a person just like us since we brush our teeth and therefore we can empathy for them and then whatever we design for them (websites or toothbrushes or beyond) will so much easier.

    This was VERY disheartening and I don’t think I’m stretching the point that was made.

    Not to mention the closing plenary speaker, Edward Tenner, mentioning a number of times that although he knew nothing about user research he didn’t think it was a good idea and that intuition was a better tool.

  2. I agree, it seems that ethnography is a big part of user-centered design while there is so little ethographic about the research done. It seems ethnography is on its way of becoming another “web2.0” buzz-word.

  3. If you’re not at risk of “going native”, it’s not ethnography.
    But it’s a cool word anyway.
    And even small doses of qualitative research have proven to be healthy to development projects.

    Best
    Gunnar

  4. And even small doses of qualitative research have proven to be healthy to development projects.

    Gunnar, there is no doubting that. I guess I would like to move beyond the self-congratulatory “we did ethnography, it was great” at conferences.

    Steve, yes Edward Tenner’s comments about user research were weird given his central thesis about unintended consequences. I have not read his book, but one would think that one way to prepare for unintended consequences (make them predictable even if not intended) would be user research. I did not take Tenner’s words about user research that seriously. He has an interesting thesis, but his viewpoint is that of an observer. He does not roll up his sleeves and do design. He looks from a distance and reflects. His role is valuable (and I loved his talk), but I would not look to him for advice about design research.

  5. As an anthropologist, I follow the discussions about ethnography in the business, marketing, design worlds with real fascination. For anthropologists, ethnography is anything but a buzzword — we’ve been doing it for well over 100 years. It is the cornerstone of our methods in many ways, and it forms something of a philosophical outlook on the study of ‘others.’

    The question of ‘if it’s ethnography, it’s gotta be right’ is neat too. Anthropological debates frequently revolve around the interpretations people make about those that they study … particularly if two studies are done in different eras. We accept the subjectivity of ethnography and the small sample sizes that are common. We don’t accept that ethnographers always get it right … rather, we might ask ‘right for whom’ or ‘right in what contexts.’

    Still, the idea of writing about another culture without actually practicing methods of ethnography like long terms fieldwork (graduate anthropology at my school requires a minimum of a year away conducting ethnographic research) would be unimaginable.

    Is there more to say about what designers mean by ethnography?

  6. Tad,

    Thanks for the thoughtful comments. Designers use ethnography like they use any other method – in a pragmatic manner – do what is absolutely necessary. This is the way it should be. Also, I think subjectivity and small samples are core parts of the ethnographic method. If you don’t like those, then you there are other methods you should be thinking of.

    Follow-up on your comment about being “right for whom” and “right for what contexts”. The context here is design. Now I am wondering – what is it about design, that makes such rapid ethnography useful. Or is it a case of – its always better than no qualitative research.

    As a methods junkie, I like a critical evluation of methods. And currently, we have unbridled enthusiasm, which makes for broing conferences…

    Enjoyed finding your blog, by the way.

  7. Rashmi … I’m glad too to have found your site. When you say to ‘do what is absolutely necessary’ I presume you mean use any method necessary for finding out the design of objects impacts peoples’ lives. In anthropology we talk about ‘participant observation’ as a method for inquiring ethnographically — that is participating and observing in the lives of others in heightened and conscious ways — and as I tell my students, my participation in and observation of an aboriginal culture might actually mean going fishing. That’s what I love about ethnographic research.

    This idea (label?) of ‘rapid ethnography’ is new to me, however, and it strikes me as somewhat oxymoronic.

    Thanks for the dialogue …

  8. Rashmi, you are quite right about the rosie case studies we always hear at conferences. It surprises me just how many of the issues explored also didn’t need an ethnographic approach at all – just common sense. Finally, the sheer number of existing and potential clients I meet who have had their fingers badly burnt by expensive ethno studies is too many to count. I find I become a better ambassador by talking about potential dangers than successes.

  9. I felt the same way about the other types of research people were discussing. Usability tests, expert reviews, and surveys were all discussed at the conference, apparently all yielding perfect results every time. Nothing confusing, nothing contradictory. Just pure, powerful insight, every time. Wow!

  10. I have to say, the deployment of “ethnography” as part of the design practice is problematic in many regards outlined here. Most of it is bargain-basement participant observation, at best.

    In Douglas Rushkoff’s insightful “Cool Hunters” piece for Frontline several years ago, he talked to a fellow from MTV (where I once worked) who claimed to be doing ethnography when arrived at a “subjects” home, in a Lincoln Town Car, with a camera crew and asked the youth to talk about his clothes and the objects in his room.

    If Clifford Geertz or Jim Clifford had seen that moment, they would’ve never stopped throwing up.

  11. Great discussion. Yes, Geertz, Durkheim, Malinowski, and Howard Becker would have all thrown up. I’ve got a BA in anthropology and a Ph.D. in sociology, emphasizing field methods. I was genuinely puzzled by the in-home interview that is labeled “ethnography”.

    In the time frame given to a market research project, meaningful ethnography is impossible. I’ve found that what it ends up being is cultural tourism, where the comfortably middle class travel to the working class world like Jacques Costeau with his underwater cameras track exotic sea life. The observers don’t understand what they see, really, but the novelty of the experience seems to satisfy them. The real information comes from the in-home interview.

    It wouldn’t be a bad idea to insist on a little field training for marketing and ad execs who are planning on being part of “ethno teams”. The QRCA could raise awareness of the issue by requiring moderators to have at least a one semester field methods class in anthropology or sociology.

    For the moment, however, I would feel better if they simply called these activities what they are: in-home interviews.

  12. The fact is, Chelsea, that in most cases client exitement turns to sheer boredom as they wait to see something ‘interesting’ when hanging out with a household. Added to which we ask them not to ask questions relevant to our explorations (co-discoveries using the video as stimulus happen later) and clients have returned after a 2 hour field visit annoyed at the waste of THEIR time.

    As a rule we don’t even take clients in-home any more unless they insist and we have trained them.

    The unfortunate thing is that commercial researchers who do little more than qualitative interviews with video camers have hijacked the word, ethnography, as their own. In fact we all have (and our people spend days in-home but still not strictly ethnography). My first contact with a client, therefore usually involves un-teaching them what they think ethnographic research is.

  13. Great discussion. Really enjoying it.

    The shorter time frame of qualitative research (in design/market research) is inevitable, IMO.. But it would be nice to have some discussion of what is the minimum you need to do to call it “ethnography” and not just at-home-interviews, or qualitative research.

    Any thoughts on specific criteria for judging if something is ethnography or not?

  14. the relentless stream of positive results is a well-known result of the “scientific” publication process – any program committee or editorial board member knows that you have to justify what “the reader” (the “user”!) will take away from a paper, and that’s a lot easier if the results are positive rather than negative or mixed.

    here’s a question: if you did two years of fieldwork in aceh, then came back a bit later to look at a specific question about (say) teenage media consumption, would you again spend two years because you were not current with the teen culture? what i’m suggesting is that specific criteria for time or methods is not all that helpful – perhaps it depends more on the question and the depth to which you need to explore it.

    if anything, in studies of western middle class consumers, do we need to worry most about analysts’ (in)ability to reflect on their own interpretations? if one cuts too many corners on training and analysis (as well as data), does one just get back preconceptions and some supporting videos?

    (i hope you are all going to ken and tracey’s party this week, as the questions here ought to be central.)

  15. Critical analysis of the effectiveness of our methods is rare and mostly unwanted. It’s hard to do, extremely hard for for non-researchers and those without a solid research education.

    I wouldn’t compare this to the problems with scientific publishing, but rather psuedoscientific publishing: critical assessment is rare and usually unwanted. Of course, this is behavioural science, so the difference between science and psuedoscience is pretty vague to begin with.

  16. Rashmi, I read this post earlier and came back now after reading Tad’s post on this :)
    I can understand that anthropologists feel chagrined at the way the term ethnography is being used to describe any observation method.

    and ouch to rapid ethnography. but some thoughts on your question on the ‘minimum’ – I have found that in case of the ‘rapid’ method it helps, in fact, it is essential to do more than one short / rapid visits. one rapid home visit is certainly not ethnography. the researcher can make up lack of time spent during one visit by stretching it over two or three?
    and it helps to use audio-visual aids seriously to document the data and observations rather than rely only on the researcher’s notes and observations.

    I think in design research, it makes sense for the design / product person t be part of the team – the research insights are not just passed on second hand but are experienced by the designer him/herself.
    (sorry about this long comment but I wonder about this myself al the time!)

  17. I’ve heard of rapid assessment used for some applied anthropology projects. Usually it’s when knowing something has a particular urgency. The only specific example I can think of right now is for medical anthropology, where medical anthros conduct short and intensive studies of newly set up refugee camps to identify health problems and head them off before they become full-blown disasters (you know, like water purity problems, weak points in food distribution networks, and so on). Usually rapid assessment is done by a multi-disciplinary team, so you’ll also have doctors, psychologists, epidemiologists, etc.

Comments are closed.