On this week’s podcast, we are joined by Molly Stevens from Uber who shares how a modern digital-first company approaches improving the experience of users.
- More on Balsamiq.
- More about Fullstory.
- My book on building user-centric culture.
- My card deck of techniques for building a user-centric culture.
- My Complete Guide: How to ensure user experience is a priority for your company.
Paul Boag: On this week's podcast, we're joined by Molly Stevens from Uber who shares how a modern digital-versed company approaches improving the user's experience.
Paul Boag: This week's podcast is sponsored by Fullstory and Balsamiq.
Paul Boag: Hello, and welcome to the Boagworld Show Podcast about all aspects of digital design, development and strategy. My name is Paul Boag, and joining me on this week's show is Marcus Lillington. Hello Marcus.
Marcus Lillington: Hello Paul. There, there, Paul.
Paul Boag: Ahhh.
Marcus Lillington: Paul.
Paul Boag: Ahhh.
Marcus Lillington: You keep talking about moving into your house on every show.
Paul Boag: Well-
Marcus Lillington: You just get more and more kind of like downhearted every time I speak to you.
Paul Boag: It's bloody exhausting, this moving house business. I'm not … it's the worst. It'd be alright if it was just moving house, but of course we're renovating our house, moving back into it, and it's not done. And it is … you know what they say about these kinds of projects, that they are money pits, and they really, really, really are.
Marcus Lillington: Yeah. You just found this new architectural treasure underneath your floorboards, and we'll have to bring in, I don't know, a museum to do that-
Paul Boag: Yeah.
Marcus Lillington: … and it'll cost you a million pounds.
Paul Boag: Yup. Yeah. Or in our case, we've just found that most of your building … most of your house is structurally unsound, and we need to rip it down and underpin it, and yeah. Oh, and the whole of the front of the house has to come off as well, and so it goes on.
Paul Boag: So yes, it's joy! But I'll tell you what, there is definite … It so annoys me that I do this, but I can't help but go through experiences like this and go, “Well, there's lessons to be learned here from a user experience perspective.” That's my … Ah, for crying out loud, Paul, get a grip. But just about communication and … It made me think a lot about how we work with clients and customer … our customers, our clients, and how scary it all must feel to them because they have the equivalent experience that I'm having with the builder where I don't know anything about it. I don't understand what's going. And especially as well, because the way that we've done it, it's not a fixed price project, you know? It's agile. So you don't actually know what you're getting for your money.
Paul Boag: And all of that kinda stuff … and you can understand why clients freak out because you do feel totally out of control of it. And of course to us as web designers, it all feels so natural and … We've just been down to the house talking to the guy who's doing it, and well, he's so calm and, "Well, you know, yes, it's all under control and it's all going to be fine," and while we're freaking the hell out. And I'm thinking, that's what it feels like to be a client on a web design project.
Marcus Lillington: Yeah, it's about understanding the processes, isn't it?
Paul Boag: Yeah.
Marcus Lillington: And even, no matter how much we rattle on about it at the beginning, we expect people to just understand what we do.
Paul Boag: And remember all of that and to feel … To be confident in that. A lot of it's about confidence, isn't it? You've got to be confident in your supplier and believe that they're going to do what they say they do, do it when they say it, and it only takes one little thing and you can lose that confidence and it kind of knocks the relationship. And I'll tell you the other big thing that's really come across to me is how important regular and clear communication is, you know? We found it quite hard to get hold of the lead building guy, the manager who's managing all this, and immediately, it undermines your whole attitude towards the process. Because, you know, of course if you can't … You know, your supplier isn't communicating with you, you feel all the more out of control and that makes you more panicky and all of the rest of it, doesn't it? So-
Marcus Lillington: Yeah.
Paul Boag: It's really … It's an interesting … There are really good lessons in it if it wasn't so horribly painful.
Marcus Lillington: The problem is from our side is, I don't know, it's nobody likes to be the bearer of bad news.
Paul Boag: Yeah.
Marcus Lillington: So often, I guess suppliers in any field will think, right, "We'll just knuckle down for an extra week cause we lost those few days. We'll just get it back in." But they don't communicate it cause they don't want to sort of make it look like they're not promising what they said they would or anything like that. So yeah, it's a real toughie, isn't it? That kind of continually … It's easy to keep up to date if everything's going well and everything's on track. It's just if things … If things are going wrong or you get, obviously you got some of that in house renovation. You will find surprises.
Paul Boag: Yeah.
Marcus Lillington: You just don't know were there. Web design is less likely, but you do come across hurdles that you haven't planned for. If somebody's ill or something like that. I never want to sort of say to a client, "Well, we've lost a bit of time because somebody was ill," even though it's a perfectly legitimate thing. You just sort of feel it's a bit naggy and a bit kind of maybe not quite what you want to be saying to them. But yeah, you're right, you need to just constantly communicate whether it's good or bad news.
Paul Boag: And of course, the thing is that you're all the time … from the client's perspective, if you're all the time juggling this thing, whether it be a house build or a website or whatever, it doesn't really matter, with lots of other different criteria that all got to kind of fit in with it. So in my case it's like the removals, people, you know? And when we give up our rent on our house, those kind of things. And in a web design, it might tying it up with a marketing campaign or [inaudible 00:06:04] or whatever else. So if someone isn't giving you a long runway and heads up potential problems, then it could be too late by the time you find out about them.
Paul Boag: So we could have rented this place for another month, and it would have been kind of fine. We could have dealt with that. But we'd handed in the notice by that point, so you committed to it. So it's all so … I think this is the biggest project I've ever undertaken where I've been on the client's side of it. And I know it's not a web design project, but it certainly … as someone's that's worked my entire career on the agency side, it's made me a lot more sympathetic towards clients.
Marcus Lillington: Yeah. Great, isn't it? Will you be in your new chair at your new desk when we do this show next week?
Paul Boag: I will be somewhere other than here, that's all I can commit to. I might be coming to you live from a garbage bin somewhere. I might be homeless.
Marcus Lillington: [crosstalk 00:07:08] office.
Paul Boag: Yes. No, I should be. I should be and whether we've gotten internet access, who knows? At the moment, we don't even have a phone socket, so, you know, that's a bit of an issue. But my desk is arriving. That's arriving so. I've got nowhere to put it, but I've got a desk. Paul coming live from a [inaudible 00:07:30] somewhere.
Marcus Lillington: Yeah. In the pub garden.
Paul Boag: Yeah.
Marcus Lillington: On his new table.
Paul Boag: Exactly. The table will be great. It's just everything else will be shit. Anyway, I've kind of done my thought for the day-
Marcus Lillington: Yes.
Paul Boag: … which is more of a rant for the day. But what about yours? What do you got for us?
Marcus Lillington: What I've got today … and this is something I will admit I've kind of spoken a bit about before, but it's this idea, taking a slightly different viewpoint on it, of recognizing that clients are different and that if you worked in a particular sector, on a charity sector, there is potentially a chance that we all might just think that, well, every client's going to be the same as the last one. And then, the same as a the last one. And that kind of thing. So what we're going to talk about is kind of recognizing, being awake to differences, uniqueness if you like, and how to deal with that if you like.
Marcus Lillington: This came about really because we work with … I mentioned before we work with US law firms. Well, one of the firms we work with, still are working with, but we did a kind of major project with them last year, are … basically they're kind of one of America's premier litigation firms. All they do is courtroom, you know, the kind of tele-type stuff and they … very high profile people hire them to defend them and all this kind of stuff. So … and we were aware of this reputation before we started working with them. So we imagined this kind of ultra-aggressive adversarial company culture, where it would be the kind of … bit of kind of culture that would accompany a reputation like that. But what we found was really quite different.
Marcus Lillington: So when we interviewed a number of their attorneys, we were trying to … obviously trying to understand the firm's objections, for its site, who its users are, accounts or in personality, and all the usual things you're trying to do when you're doing stakeholder interview. But we started noticing trends in what people were telling us, and significantly, they all said that they wanted the site to reflect how different the firm was. So that maybe kind of taken literally was that they wanted the site to look manifestedly different to other law firms. And I've heard that before. You've heard that before.
Paul Boag: Yeah, yeah.
Marcus Lillington: Many-
Paul Boag: People say it, and then they don't want it when it comes to it. Yeah.
Marcus Lillington: Exactly. Yeah, we're a quirky bunch of guys here kind of thing, and then you give them a quirky design and they go, "Oh, no, no. Our clients won't like that sort of thing."
Paul Boag: Yeah.
Marcus Lillington: What they actually want is a kind of better version of their competitor's site. So it was … with the case of the law firm, it was a case of going back to them and quite simply, but we didn't bother to do this, but basically saying is that really what you mean? The key question was, do people hire you because of your differentness? Because of your uniqueness? And the short answer was yes. So then that gave us the platform or gave us the kind of … the green light to ensure that their design was different to your average law firm. So that's kind of it really today on my thought, but just sort of pay attention to the fact that people might be telling you something different to what you think you're hearing.
Paul Boag: Yeah, and that happens so often, you know? Communication between people is massively unreliable. That's why I love things like prototyping and mood boards and wireframes, because it's sort of tangible, you know, that you can actually see. Because people are really bad … well I don't know whether people are really bad at expressing or we're really bad at hearing or probably-
Marcus Lillington: Yeah.
Paul Boag: … a combination of both, really.
Marcus Lillington: Yeah. That … and it does just underline the fact that all of these kind of maybe sometimes feel a bit silly exercises that we do in workshops are actually really useful. Cause I think that … I've known people to roll their eyes before internally about we're going to do some mood boarding with the client.
Paul Boag: Yeah.
Marcus Lillington: So divide up into teams. But that exercise with this particular client I'm talking about was massively helpful. We actually took some of the elements from one of the mood boards that they did, made it all the way into the final design.
Paul Boag: Yeah.
Marcus Lillington: So often it's just a … they … those kinds of exercises are just to get people thinking about the whole process and understanding our processes. But in that case, it really did inform the design so well worth it.
Paul Boag: And the trouble is as well is that we can use exactly the same words but kind of have different meanings to them, you know? What one person considers edgy or radical, another person wouldn't, you know? And so actually, it's a bit like isn't it with pain, you know? It must be horrible for doctors to have to go, you know, "How much pain are you in?" Well my-
Marcus Lillington: [crosstalk 00:12:21].
Paul Boag: … Sense of pain is totally different than somebody else's sense of pain, isn't it? You know?
Marcus Lillington: Yeah. That's why they use the 1 to 10 thing. You're still thinking, well, I don't know, let's be brave and say four?
Paul Boag: Yeah, exactly.
Marcus Lillington: Actually, just one more final point on that. Another thing we do … We do this with most clients now is exactly that idea of what does an open design mean as opposed to a formal design. So what we've started doing is saying, "This is what we mean by … This is a kind of a mood board or a bunch of images, kind of different styles that mean-
Paul Boag: Yeah.
Marcus Lillington: … formal or informal. And which way do you lean?"
Paul Boag: Yeah.
Marcus Lillington: So we don't have to do that kind of guessing what people think different words mean.
Paul Boag: And also-
Marcus Lillington: Anyway, I'll stop now.
Paul Boag: No, no. It's good. I know also sometimes I think it … to do the opposite is good, you know? Okay, so say you want a casual design, well what doesn't that mean? What wouldn't be included in that? Cause sometimes thinking about the opposites, that helps, or it does me anyway. Anyway.
Paul Boag: Let's talk about our first sponsor which is the wonderful Balsamiq, my favoritest brands in the world. I've so enjoyed working with them. Actually talking about different kind of character types, I'm right in two things at the moment. I'm putting this course together with Balsamiq people, and I'm doing another piece of work with another client that I won't talk about. And could not be more different experience between those two. You basically do a very similar thing, but just totally different approaches and attitudes, and it's fascinating. I have to say, I'm more inclined to the Balsamiq laid back approach. They're a really lovely team. Anyway.
Paul Boag: So if you don't know, Balsamiq is an easy-to-use wireframing tool that's got an incredibly low barrier to entry. Anybody can pick it up and basically use it straight away. So pretty much every person we've spoken to on this show has said that they've used Balsamiq at some occasion. But it's also great for not professionals, too. So people that aren't designers, you know? It's aimed at actually marketers, managers, those kinds of people. As designers, we've got our fancy tools that we like for whatever reason, but, you know, there are other people that need to express their ideas as well.
Paul Boag: And don't be afraid of nondesigners producing wireframes and getting sucked into that process, because it helps them … helps you to understand what their expectations are. Going back to what we were saying earlier about them just telling you but not really enough. You need to be able to see it. It'll also help you head off any problems early, because if they start doing ridiculous things, you want to know upfront. And also, let's face it, nondesigners can have good ideas, too. It's not an exclusive thing which is something that you taught me, Marcus, years back, and I think I was a bit kind of of the arrogant designer mindset. But, you know, even you have occasionally had a good idea, so that kind of proves the point, doesn't it?
Marcus Lillington: Ha. As I was just saying in the intro, some of the ideas that the client came up with in the initial meeting ended up in the design, yeah.
Paul Boag: Yeah, absolutely. So it's a really good collaboration tool basically. You should give it a go. You can give it a go by going to Balsamiq.cloud where you get a 30-day free trial so you can have a good play around with it. If you do decide to sign up for it, then I recommend adding in the code, Boagworld alongside your billing information because that will give you an extra three months for free which obviously is very worth it on top of the 30-day free trial as well. So yeah, check that out. Okay.
Paul Boag: So let's then dive into our interview for today. So we've done some fascinating interviews already this season. It's been really interesting, and we've talked to people from Virgin Atlantic. We've talked to people from University of Dundee, Cisco, IBM, but what all of those companies share in common is that they are all pre-digital companies. They've all been around for a long, long length of time, right? So they've got a lot of legacy and that kind of stuff. But what about a company that doesn't have all of that legacy, that's come around more recently. And obviously, as you would have heard from the introduction, we have Molly Stevens from Uber on the show. And she's going to talk to us a little bit about how a post digital company works, you know? How their … How they approach user experience and whether they take a different approach to things.
Paul Boag: So Molly, why don't you start off by telling us a little bit about your role at Uber. What is it that you do there?
Molly Stevens: Yeah, I'm the director of user experience research. That's a global role so it looks across all of our products and services and around the world in the 65 countries we're located in.
Paul Boag: Uber's now in 65 countries, is it? That's amazing.
Molly Stevens: Yeah, yeah. It's quite the scope and complexity that we have to deal with. It's very interesting.
Paul Boag: So what does your job kind of consist of on a daily basis?
Molly Stevens: So my current role, a lot of what we're looking at, I would say roughly falls into working with the product and engineering teams to ensure we're covering the top priorities, working with my managers to make sure things are flowing well on their teams, looking around the globe to make sure that we have the right coverage in the right locations, quite a bit of hiring and recruitment because we are growing the team quite a bit, and looking at how to, you know, maintain and grow our team vision and mission and what we work on … How we work, how we work as a Ux research team.
Paul Boag: So what does the kind of role of the team consist of? I mean, it's kind of in the name, isn't it? It's doing user research, but I'm kind of quite interested in how that process works. Are you doing research that is almost commissioned by particular parts of your organization? Or are you've just got an ongoing program of research? How's it structured?
Molly Stevens: So we have a mix of research. So first, in terms of methods, so my team has a pretty wide range of skills and experience and we work across highly qualitative research, like field research and ethnographies and anthropology-focused work, all the way to quantitative, survey analysis, in some cases logs analysis, and kind of everything in between. And then, we work on a kind of a semi-embedded model or a hybrid studio model, couldn't come up with any kind of name for it. But basically, our researchers are focusing on a particular specialty or area where we need to have deep expertise and user experience research, and they work closely with those product and engineering and design stakeholders to figure out what needs to be researched. In some cases, that might be something that one of their stakeholders has asked for, something specific that they're trying to find out, either for a near-term product or for possibly figuring out what goes on the roadmap.
Molly Stevens: Or it might be something that we've identified, so research that is happening because we have seen certain types of things in the field or we're identifying certain kinds of findings in surveys that indicate to us there's an unexplored research question. And we'll work with the teams to prioritize that and make sure that work happens. So it's a definite mix of both.
Paul Boag: So does your team kind of sit together, in the sense are they … Or are the members of your team kind of embedded in other groups across the organization? Does that make sense?
Molly Stevens: So we … Yeah. So we've recently started sitting together. We were more embedded until recently, and that's why I say hybrid studio model. We're sitting together now to see what kinds of efficiencies that brings us and what kinds of benefits we have. I'm a big believer that seating patterns and reporting structures and all those kinds of things are tools that you can use to kind of make certain things more important within your organization and make changes. If it's not making the change you want, then you change it back, you know?
Paul Boag: Yeah.
Molly Stevens: Lots of Ux teams have gone from being centralized to decentralized to back to centralized. And so we recently all moved to sitting together or in clusters as best we can across our different buildings and locations. And we'll see what kinds of efficiencies that brings us.
Paul Boag: Yeah, because there are pros and cons to both, aren't there? There's obviously efficiency gains to being together, not repeating yourself, etc. But on the other hand, there are benefits to being deeply embedded within teams in order to refocus them on the research and what they know about the users. So yeah, it's a tricky one to get right, isn't it?
Molly Stevens: It is. And I think it varies per team, too, right? So you have a product team that's … you have great trust with and they really believe the work you're doing and very connected to it. It kind of doesn't matter where you're sitting but how you get there, maybe you do need to be more present and able to answer those questions. So definitely, it varies a lot per team.
Paul Boag: How do you deal with the international aspect of it? Do you have researchers around the globe? Or are you trotting every now and again. I mean, I know at the moment you're in Amsterdam.
Molly Stevens: Correct.
Paul Boag: So is it that you're traveling around the world a lot or do you have localized expertise?
Molly Stevens: We do both. So I think it's important for us to have localized expertise from the region or who know the region really well who can help us do a better job of addressing regional issues. And our folks also travel for projects depending on what's happening. We've also recently started a program called the global scalable research program where we have locally embedded teams that can run at a regular cadence that can run studies to help folks back at headquarters who might need to test something but also don't want to be traveling a ton. And maybe it's something more discrete and kind of smaller, it's a not a full field study. And so we've been doing that … It started running in January, and we're getting great rewards from that.
Paul Boag: Ah, that sounds interesting. Because I mean, it is … That's the danger, isn't it, with user research. That yes, you want to do the more in-depth detailed stuff, but the danger is that it becomes perceived in the organization as onerous. So, you know, having that lighter weight model as well I imagine helps very much.
Molly Stevens: Definitely, definitely. It's one of the reasons why we … We needed it because one, we're global, two, we need quick turnaround on smaller lightweight projects, and three, we do need to also then build up our knowledge of those regions so we can do things like create design principles or common user needs principles from these, and if we don't have regular feedback from those regions, it's quite hard.
Paul Boag: So I mean, the big … There is one other kind of aspect to this that … the scope of your role that I'm quite interested in, is so far we've talked about the user, and I imagine in most people's mind when they think of the user, they think of the person that's getting into the vehicle being driven wherever. Do you do research on the driver side of things as well? Or are you exclusively on the customer side?
Molly Stevens: No, we definitely do research on the driver side. In fact, when I joined Uber about two years ago, I was managing the driver research team.
Paul Boag: Okay. All right. I see. So in some ways you almost see the drivers as much as users as the customer. Is that a fair comment?
Molly Stevens: Absolutely. Our drivers really are Uber in the wild. When you call an Uber, it is that … those people who show up at your door, and so it's really important for us to understand what they need, what are they looking for, what are their pain points? We recently launched a redesign of the driver app that was really focused on us working closely with drivers to understand a lot of their needs, and then build something that met those needs from what we understand.
Paul Boag: So I mean, that's quite interesting, cause you don't get into kind of broader culture within the organization. Because obviously, you can easily think of … if you're superficial, you can easily think of Uber an app. The app is your point of … your touchpoint. But actually, it's not. It's a part of the experience, and the experience includes the drivers and the rest of it. So I'm quite interested in … obviously Uber is a newer company compared to some of the others that I've been talking to on this season, so it's been built with the user in mind from the very beginning. But I'm quite interested in what that means beyond just the user interface, in terms of your drivers, in terms of company culture, in terms of how decisions are made within the organization. What do you think it is that's different about Uber's user perspective than maybe a more traditional company?
Molly Stevens: What I find that's really unique about Uber in terms of thinking about user experience that's very different than other products I've worked on before is the kind of immediacy within the world that what we change and what we work on has, so the neat kind of immediate impact, right? So if we improve the navigation app, and we're doing x million trips a day in the US, that is incredible, right? It has a very immediate impact. It can help get people home faster, avoid traffic. Drivers might get an extra tip if they do an extra good job. There's this like very immediate impact on the day-to-day experience of the folks who are both driving and riding. And so I find that's one of the most unique pieces about working at Uber, and it's one of the ones that I find most compelling, too.
Paul Boag: And being able to draw a very direct comparison between the changes that you make and the impact of those changes is so helpful. I work with a lot of organizations. For example, I do a lot in the higher education sector where drawing … a decision like that, what university do I go to, is a prolonged decision and the experience is a prolonged decision that lasts over years and it has a lifetime impact. While … And so it's quite hard to, therefore, measure the impact of things that you're doing because they might have an impact two, three, four years down the line. While I'm imagining in your situation, it's very instant. You make a change. You're getting data back straight away basically.
Molly Stevens: Yes, yes. That's really, really compelling. It also means, though, that we need to be really careful about not just focusing on short-term impacts but what the long-term effects are of the work that we do can be kind of seductive, to see that a number is going up and be like, ah, things are magical. Everything's great. And then, six months down the road you see unintended effects, and you have to be careful about that.
Paul Boag: Yeah, yeah. I mean, you see that all the time. And I know you guys have struggled a little bit with reputation because of that, and you're far from alone. There are many, many companies that have been fixated on certain metrics and that has then influenced the experience in unintended ways. And that actually brings me nicely onto the next question which is how does Uber balance those kind of user experiences? You want to provide a great experience both to the drivers and to their own customers with the kind of demands that go alongside running a company like Uber which is in terms of kind of financial growth, and there is this expectation that Uber grows rapidly. Do you find that there's sometimes conflict between those two things and how do you address that? How do you keep people focused on user needs?
Molly Stevens: I find that Uber pretty consistently is very focused on user needs-
Paul Boag: Oh, good.
Molly Stevens: … across the board which is great. I think the complexity that we run into is the fact that Uber primarily is a marketplace. The things that drivers want aren't necessarily things that riders want and vice versa. So we work very hard to try and understand the core needs that are consistent and can help balance out the experience across the whole marketplace. And then design and work on experiences around those things, right? And it's … Marketplaces are a very unique beast in terms of how they interact and the different levers you can pull and push on the different sides. I saw this a lot at Google when I worked in the ads area where basically we had Google AdWords on one side and Adsense and publishing on the other. It's very similar in some ways to what I've been experiencing at Uber.
Paul Boag: That does raise an interesting question of mine because you just described Uber as a marketplace, which I totally get. But do you think that's how most people see it? Is there a disconnect between how maybe Uber sees itself as being a marketplace and your experience of talking to users who perceive Uber like a taxi company? Are you trying to fight a kind of existing preconception of what they're getting?
Molly Stevens: Well I think that most companies have a variety of ways that they are seen by the people who use them, right? So an example would be, you know, Google, some people see them primarily a search, some as a phone company, you know? All the different aspects. So there's some of that that we can't necessarily change or effect very much.
Molly Stevens: I think the most important thing that we have been working on is making sure that the people who do use our tools understand our good intentions and that we are trying to improve the marketplace the best that we can for everyone who's involved. A good example of that would be this fall, back in August last year, we launched the 180 days of change campaign for drivers and the focus was on six months of different features and changes and launches to really talk to drivers about how we were trying to improve things for the better for them. And a lot of that was based on user experience research that my team had done that showed we were falling short in some areas and that we needed to be making improvements.
Molly Stevens: And I mean, I think the other thing is, too, is that Uber is very different things to different drivers around the world. For example, whereas a driver in India has a very different level of need and urgency on a platform than say a casual driver in Florida. And so we have to take all of those things into account.
Paul Boag: It's really fascinating for me to see the kind of history of someone … a company like Uber that obviously had come in and they've radically disrupted a market that was open for change because there were obvious huge holes in the user experience that needed fixing. So it's just a beautiful case study in that regard. A really crappy experience for some people. Somebody came in, offered a good experience and completely disrupted the market. But now, we're kind of moving into a phase where as Uber matures, you've got two things happening. You've got Uber becoming … beginning to experience some of the problems around scale, especially from the globalization point of view and then at the end type, you've got your old competitors effectively cloning and copying what Uber are doing in an attempt to run up, to catch up. It's a very interesting time, I'm imagining, to be involved with Uber.
Molly Stevens: It is. I'm consistently just amazed and excited and blown away by the kinds of research questions we get to dive into and that we get to have answers to and help our teams build great product. So it's a very fun and interesting time and I could definitely say I've never been bored in the two years at Uber.
Paul Boag: So tell me a little bit about how you structured the management of user experience within Uber. I mean, so you've got your team which is a user research team. But I'm imagining that there are other teams that do different aspects of user experience. Maybe you've got, those people obviously that are working on the user interface. There are people working maybe on the driver app. So how is it kind of organized?
Molly Stevens: So the design and research functions roll up to a central contact, that's our new VP of design, Michael Gough. So I report to him and the vast majority of the researchers at Uber report up to me. So we have a centralized Ux research and design team.
Paul Boag: Sure. Okay. So what about … and then what about other aspects? You've got head of design. Do you have anybody exclusively responsible for customer experience or is that responsibility kind of distributed across the entire company?
Molly Stevens: Yeah, we have some other groups at Uber that focus on say, the support experience, or the … we have these Greenlight hubs where in different … in all the cities where drivers can go to get problems solved and get coffee and meet each other. They have folks working on that almost service design experience. So there are definitely other folks at Uber worrying about other parts of the experience. And we typically would collaborate closely with them to make sure that we're creating as a whole [inaudible 00:38:06] experience as possible.
Paul Boag: Yeah, okay. Yeah, that makes … That kind of makes sense. What about in terms of things like metrics, you know? What metrics are you measuring around experience? Are they kind of general? Kind of company level metrics that you are tracking? Or is it all individual project level?
Molly Stevens: So we definitely for different projects will create success metrics that are relevant to the particular project we're trying to build or launch or get out into the world. And then, there's company level metrics also. Uber is known for the amount of data we have and how that data influences how we build and design our products, and we really look closely at that to make sure that we're measuring not just all the things we can but the things that really matter.
Paul Boag: So can you give me some examples of what metrics you keep an eye on from a user experience point of view? Are you using … I don't know, something as old school as a net promoter score? Or are you looking, travel time? Or what are you looking at?
Molly Stevens: Well, it's funny you bring up net promoter score. I can get on one of my favorite hobby horses which is why I feel that the net promoter score is often a poor metric for different kinds of experiences. It has its uses. But when I joined Uber, we were using that promoter score to understand the driver's point of view. And it really was not the right metric to use for a variety of reasons. One is that many drivers … so if you phrase the question, how likely would you be to recommend a friend or family to drive for Uber, in many of the places where we have drivers, cars are actually very expensive and you might answer that question, not because you don't like driving for Uber, but because you literally don't know anyone who has a car.
Paul Boag: Yeah.
Molly Stevens: Right? And so, you can go on and on. I have like three or four examples why this particular net promoter score question was bad for drivers.
Paul Boag: So what did you replace it with, then?
Molly Stevens: We've replaced it with a basic satisfaction question. So like, how satisfied are you with driving for Uber? We found that it's much more indicative of driver behavior. It gives us a much better measure for whether or not they're getting the experience they want on the platform, and we can set net promoter score aside in that case.
Paul Boag: So I'm guessing from the customer side, a lot of it is about the reviews and the ratings that they give to drivers. Is that the main metric you're looking at there? Or are there other ones that you're interested in?
Molly Stevens: It's definitely important and I can't get into any more specifics beyond that. I would say that it varies a lot per item. So, you know, one of the projects that we were heavily involved in, the driver's profile project, which said, you have a driver coming and you can tap on their picture and see a bunch of metrics and details about that driver, so how long they've been driving for, how many trips they have, whether or not they have compliments and things like that. That would have a very different set of metrics than something like a new payment method, right?
Paul Boag: Of course.
Molly Stevens: It varies pretty widely project to project.
Paul Boag: So what … I'm quite interested with the kind of projects that you work on. Are there any that you feel you could tell us about, on what you did and what you felt you achieved from doing that?
Molly Stevens: So we recently did a redesign of the driver app and what was very important to us was to try and make the experience of earning on the platform as seamless as possible, right? If a driver is coming on, and they would like to find rides and work for several hours, how do we make that a very positive experience and to do it globally? Not going to be launching something in just on market typically. Usually it's multiple markets.
Molly Stevens: So we worked closely with a number of folks outside of the US and did basically … well we started by collecting all the research we knew about the driver app and that experience and what the edges of that … so like what they were doing at home, when are they choosing to go online, what's that whole experience, and we use that to come up with some initial designs. And then, we did a huge amount of global testing regularly over the course of about nine months to ensure what we were designing wasn't going to break the experience in any of our major markets.
Molly Stevens: And then, at the very end … and there's a great video … if you want to see a lot of the folks on my team, there's a great video of the beta program that we participated in and had a large influence on where we launched to a certain number of drivers in all our major markets and really partnered with them. We called it the building together program where we had them, the beta testers for the app to, you know … we had them do diaries. We were partners with them on … we basically found some of them pen pals [inaudible 00:44:05]. We did a huge amount of data collection to really get a sense for what was happening on the ground with these beta users so we could make sure that when we started rolling it out, it would be as positive of an experience as possible.
Paul Boag: That's so interesting to hear. That kind of shows you what happens when you start working at that kind of scale, that not only do you have all those different global markets to consider, also you've got that … the impact of getting it rolled is enormous, isn't it? You really have to be sure. And so the idea of rolling it out … The number of times … I'm just amazed where people go, okay, we're done, and they just roll it out to everybody all in one go, and these are big organizations that really should know better.
Molly Stevens: Yeah.
Paul Boag: I just think that is such a gamble, you know? Yeah.
Molly Stevens: It was a gamble that we couldn't afford to take. And also, you know, it's really, really important for our drivers to understand that we're partnering with them on this. That the intention of this is really to make their experience better and just the super-positive engagement that we had from them throughout the course of the beta demonstrated that that was just really a great experience for them.
Paul Boag: And that's another big … what you just said ,there is another huge part of it, isn't it? It's not just what you get back, what you learn from the audience that you're working with, like drivers. It's also what it says to them about your attitude. By engaging with them and including them, you're saying, I care … We care about you. We care about what you're trying to achieve. We don't want to make things difficult for you. And that is as valuable as actually what you learn from them in many ways, isn't it?
Molly Stevens: Absolutely. It's a partnership.
Paul Boag: Yeah. Okay, one last question, and then I'll let you get on with your busy day. I just wanted to ask you, just to finish up on, a little bit about cross collaboration, right? Now, you've … we kind of touched on this a little bit throughout the chat that we've had, but I'm interested in any kind of ways that you've managed to work with other people across the organization. Because a big part of user research, in my experiences, isn't even so much the research itself, it's making sure that that research then becomes embedded within the thinking and behavior of the organization. So how do you go about making sure what you learn gets taken into account?
Molly Stevens: Yeah, there's a number of ways that we do that. So one of the main things is working hard up front to try and collaborate and coordinate with the other insights functions at Uber, like marketing and data science, some of our operations teams in the field, to make sure that we're on the same page about the research that's getting done, that we know what the intention of gathering that data is going to be, and whether or not they already have some information or background on it that would help us do a better job or maybe we don't need to do it at all, maybe they already know the question that the product team is asking. So we do a lot of upfront cooperation and collaboration. And then, during the research, involving as many stakeholders or interested parties as possible which is a good consistent practice across research experience in general.
Molly Stevens: And then, at the end, once you have your insights and your data together, your observations and knowledge gathered, and I find in general that it varies a lot in terms of how teams might want to digest or share knowledge. I've used everything from reports and kind of standard read-outs to collaborative sessions where you have everyone on the team watch a video and do a group debrief, so they really start to internalize the findings from some of the individual users, to taking a report, a document report, and putting it in a spreadsheet. Like what is the communication method that will really connect with your particular audience or individual. And trying out different ways of telling that story, what's the medium that you're trying to get it across, and figure that out.
Paul Boag: I mean, essentially, you're using good user experience and user research techniques on your own stakeholders to communicate with them in the best possible way, aren't you really?
Molly Stevens: Absolutely, yeah. We use that skill set as much as we can.
Paul Boag: Okay. Absolutely wonderful. So thank you so much for being so open and sharing insights to how you approach user research and user experiences within your organization. It's really interesting to kind of get a little sneak peek behind the doors. So thank you very much for your time.
Molly Stevens: Yeah, thank you for having me.
Paul Boag: So there you go. What did you think of that one, Marcus?
Marcus Lillington: I love the fact that they've got a kind of whole team dedicated to just user research. It's just this big company mentality that I'm not used to. As I've already said in the past on this show, I have no desire to go back to that big company sphere, but it's great that they have so much kind of granularity in the roles that they have.
Marcus Lillington: The thing that struck me most about it, and it's just … This is just as a user of Uber, is that I consider their user base to be me and other people like me. I just don't even consider that they have drivers who are a complete different audience. And basically, what the impression I got was they are the important audience to the company. Isn't that weird? It's kind of like … Cause they're in between Uber the company, drivers, and then us, the end customer. So it's kind of like the relationship that they have with the drivers, seems to be like if they really had to choose that would be their most important audience. I may be wrong in that, but that's the impression I got.
Paul Boag: I think it's true to a large extent because they're the public face of the company, and so they're the ones that have the biggest impact on … well to be honest on their revenue, I'm guessing. And so, yeah, they've been … certainly more recently. They've had a bumpy road on occasions with drivers, and so I think they're working particularly hard in that area now which is really good to see. Going back to what you were saying about having the user research teams and all that kind of stuff. I mean, that's … that I find very interesting because if you compare them as a post digital company to a pre-digital company of approximately the same size, doing the same kind of stuff, those companies wouldn't have a user research team. It wouldn't even occur … they'd have a marketing department. But, do you know what I mean?
Paul Boag: So things like user research, user experience, these are much bigger operations within post digital companies, which I find a really interesting thing. And you could argue, "Well, that's because Uber's primary contact with its clients is through a user interface," but these pre-digital companies don't even have big customer experience teams, right? Do you see … There's no one that owns the customer experience in that same way. And I think that's an interesting shift that-
Marcus Lillington: I think it's the … just the shear kind of volume of people dedicated to [crosstalk 00:52:30] that, probably large, and I'm guessing, but let's say with Unilever, for example.
Paul Boag: Yeah.
Marcus Lillington: They will have people that do user research by what I suspect a lot of them will be third-party agencies, but it won't be … it won't have the same focus that a company like Uber has, I'm guessing.
Paul Boag: Well, actually, that's very interesting you mentioned Unilever because I was doing some mentorship with someone from Unilever and that … you're exactly right. They will occasionally hire an agency to do that kind of work. Yes, they do have people with some experience with doing that kind of stuff, but they're very fragmented across the organization, very decentralized, and very little control over the direction that they're going. So it's a very different mindset. The power sits in a different place. So it's … Yeah, it's interesting to observe. Anyway, so that's Molly from Uber.
Paul Boag: Let's talk quickly about Fullstory, who is our other sponsor for the season. Fullstory is a session recorder as you probably would have gathered by now, and it's a really useful tool in your arsenal when it comes to doing user research and doing user experience stuff we've just been hearing. So if you haven't got a session recorder running on your site, get one, right? Whether it's Fullstory or not, make sure you have one running because analytics don't tell you the full story. Analytics are-
Marcus Lillington: No pun intended.
Paul Boag: I know, right? And analytics will point you … help you identify a problem, but won't really reveal the solution. And of course, you can use usability testing and you should use usability testing to help you understand what's causing the problem and investigate some possible solutions. But as soon as you put someone in a usability test session, they're not going to behave naturally. And they're going to want to please you, and they're going to want to … they're going to behave differently simply because they're being observed. And you might think, "Uh, I don't know whether that's true." But, I mean, just to show you how profound an impact being observed can have on behavior, they did a study where they had a honesty jar in an office, right? So whenever people had tea or coffee, they had to put money in the honesty jar. And what they started doing is one week they would put up a photograph of a pair of eyes, right, just above the jar, and then the other week, they'd just put a bunch of flowers, something very neutral, a picture of a bunch of flowers. And it … when the eyes were there, it caused something like a 44% jump in the amount of money being put in the jar, just the thought of being observed, right, changed people's behavior.
Paul Boag: So can you imagine … and that's not to say usability testing is not worthwhile. It absolutely is. But it should be a part of the mix alongside analytics and alongside session recorders. They give you some of the deeper insights you get from usability testing but also the numbers and scale of analytics. So now, just be clear, session recorders don't replace analytics and don't replace usability testing, but they are something else that should be a part of your testing approach and arsenal.
Paul Boag: And to be honest, if you're going to look at session recorders, Fullstory is simply the best out there. There are other services available, as they would say on the BBC, but I'm not BBC. Get Fullstory. I actually love it. You can sign up and get a free month for free of their pro account. And if you want to carry on using it, at that point then, great, you enter your credit card details at the end of your free month, so they're not tricking you into it or anything like that. But if you don't … if you feel it's not for you, or you can't just find the money, then you can obviously walk away, or you can continue to use it for free for up to a thousand sessions per month which is with smaller sites would be actually quite adequate. So to sign up, you can go to Fullstory.com/Boag.
Paul Boag: Okay. So that's pretty much it for this week's show. Marcus, what do you have for us joke wise?
Marcus Lillington: I have an appropriate joke this week.
Paul Boag: Ooh.
Marcus Lillington: Yeah. Two guys are in a bar. The first one says, "Do you know a good GDPR consultant?"
Paul Boag: Oh, for crying out loud.
Marcus Lillington: Second one says, "Yeah, yeah." And the first guy, says, "Oh, great. Can you give me his e-mail?" Second guy says, "No."
Paul Boag: I shouldn't laugh at that. I'm so fed up with GDPR.
Marcus Lillington: I think they're an art form. I mean, some of them are just like [inaudible 00:57:38]. But some of them are brilliantly worthy. I wish I'd saved them. Just to try and get your attention, "Oh hello. We're over here." Just … they're wonderful. But there are quite a lot of them. But like, we've changed our policy, end off.
Paul Boag: Yeah.
Marcus Lillington: You know? It's … how people are interpreting it is completely different. It's like some, "Oh, well you must sign up here and then you sign in again there and then change your preferences" and blah, blah, blah, cause there are a few things I do want to stay signed up for.
Paul Boag: Yeah.
Marcus Lillington: But other's have just gone, "We've changed our policies. Thought we'd ought to let you know."
Paul Boag: Yeah.
Marcus Lillington: Thanks.
Paul Boag: And you know, if you start having a discussion about what you should or shouldn't do as part of GDPR online, it turns into a firestorm of contradicting ideas. I have … I don't know whether you've been sucked into it, but I've just avoided talking about it. I've just basically not taken a position on it, because I just do not know. Every time I read about it, I get a slightly different interpretation of how it should be done. But what … the best one I saw was on Twitter where somebody says, described it as, "It's like an ex-girlfriend begging you to have them back." These are [inaudible 00:58:57] people you've got no memory of ever interacting with suddenly contact you and beg you, "Please, please fill in this form, please, please."
Marcus Lillington: Yeah. Funny, isn't it? Anyway, yes.
Paul Boag: It is.
Marcus Lillington: So there you go, appropriate joke of the week.
Paul Boag: Indeed. Okay. So that's pretty much it for this week's show. Hope you found it useful. We got a fun one for you next week. We've got Matt Curry coming back on the show. It's been a very long time since we've had him, and he's going to talk about probably the most unusual user experience challenge you will ever come across. If you don't know who Matt Curry is, I'm going to just leave it hanging at that, but it will … I promise you, it will be both entertaining and very, very insightful, and I think you'll learn a lot about the-
Marcus Lillington: The human psych … the psychological behavior of these people.
Paul Boag: Yeah. Yeah. Absolutely. I describe what Matt does as extreme user experience. It's like … it really is a little bit out there. So join us for that next week. But until then, thanks for listening.