Marissa Mayer director of Consumer Web Products at Google, spoke at BayCHI on Tuesday evening. Some interesting points she made:
(a) When she first got involved in User Interface Design at Google, she was asked not to spend more than one day a week working on the interface! Also, she was asked not to give her opinions, but to provide data. So right from the beginning, design at Google has been very data driven.
(b) They do a lot of usability testing (mostly discount tests), both task-based and think aloud. From what I understood they do look at statistical significance of such test results.
(c) The method she talked about the most was fairly sophisticated log analysis (seems to be Google’s staple method). They make small changes, present it to sections of user population and watch the results (both the log based metrics, and emailed customer feedback). Also, they use the logs to understand any user experience problems on the site. Based on such tests, they decide on design direction.
An example she described was the placement of the “Do you mean “X”” at the bottom of the page. Looking at the logs they realized that people would misspell a word, get results that were not relevant, and immediately hit the “dissatisfied” link at the bottom of the page. This was in spite of a “Do you mean” link available at the top that could correct their spelling. In general, Google has realized that people are very focused on the first result, and rarely notice anything above that. They type a query and immediately go the first result. Based on this knowledge, Google added in a “Do you mean” link at the bottom of the page.
(d) Marissa also described how they launch a product early and use customer feedback to refine it while still in Beta phase (yes, the infamous Google beta). She described another design dilemma – when they struggled for days deciding whether to allow sorting by location or by date for Google News. Finally, they decided to launch Google News without sorting. Within three hours, they had hundreds of emails asking for sorting by date, but only a few asking for sorting by location.
(e) Marissa also addressed whether User Experience is a sustainable competitive advantage. Although analysts such as Gartner believe that User Experience is not a sustainable competitive advantage (because it can be copied easily) Google has observed that their competitors have not been able to catch up. Marissa thinks that since Google started from such a different point (a very bare interface), other companies have had a difficult time reaching that point (since they started from such a busy interface, and have so much paring down to do).
(f) Marissa does not seem to think much of personas (seemed like she preferred not thinking about deeper user motivations – who they are, what they want out of life!). Instead she prefers thinking of different interaction styles for user types. For example, with Gmail, they observed 6 different interaction styles, and tried to make sure that Gmail could accommodate all of them. (I would be curious to know if this disdain of personas is a commonly held conception at Google.)
(g) She mentioned that they found card-sorting very useful and have been using it to decide how to group items on pages. I was glad to hear this, as I am partial to card-sorting myself, and think the method is underused.
(h) Google does use focus groups as well to make product decisions. For example, their general rule is that something should be on the main page only if 20% of people use it, and in preferences, only if 5% of people use it. The “I am Feeling Lucky” button is not used much. But their focus group participants always tell them they like it, and ask Google not to remove it (even though these participants have never used that button, nor do they ever intend to use it!). So its not all rationality and data driven design at Google!
The question from a practitioner perspective is: What lessons does the UX method and approach of Google offer UX practitioners. Clearly Google search has a great interface, and they seem to really value UX issues at the company. While I do think that Google has some lessons to offer UX practitioners, I doubt that many practitioners are going to model their work based on Google. Here is why.
For one, Google’s forte is their search interface (specifically its sparseness, and the well-done contextual help). Google’s other interfaces might be good, even excellent, but they are not head and shoulders above other similar interfaces. For example, Google News, Froogle, Orkut.
Many design domains would not be served well by the Google zen search box paradigm. The minimalist design philosophy is inspirational, but not universal (as evident by Google’s more busy interfaces such as Froogle and Google News). For many domains, there is no dealing with the complexity through a sophisticated backend, a simple search box, and all other options hidden behind “More”. The complexity needs to be dealt with upfront (e.g., the eBay home page could not be like the Google home page).
Secondly, Google’s reliance on log analysis simply would not work at most companies for three reasons.
(1) Most companies are not setup to do live AB testing. I have suggested AB testing to several companies we worked with. When they actually started looking into this, we found out they were not technically capable of launching such A/B versions simultaneously.
(2) Most companies do not have big user populations who give them immediate feedback. One of the essential ingredients in the Google equation is quick feedback. Very few companies would be able to get millions of responses in a day.
(3) Log analysis is not the forte of most designers. Log analysis requires some sophistication with data analysis, or working with people who can deal with numbers and garner the type of insights designers would benefit from. Given a choice in the means of getting user feedback, most designers would opt for usability testing or some hands method such as observation over log analysis.
So while Google definitely seems have its act together on the UX front, I am not sure how much of an impact its UX methods and processes will have on the UX community.
Still I found Marissa’s pragmatic, data-driven approach towards design and usability a refreshing change.