Should You Be Paying More Attention to User Research?

Paul Boag

This week on the Boagworld Show, we explore the world of user research and ask how much attention all digital professionals should pay to it.

This week’s show is sponsored by Miro and TestingTime.


Paul Boag: This week on the Boagworld Show, we explore the world of user research and ask, "How much attention all digital professionals should pay to it?" This week's show is sponsored by Miro and Testing Time.

Hello, and welcome to the Boagworld Show, the podcast about all aspects of user experience design, digital strategy, and working in the world of digital. My name's Paul Boag, and joining me as always is Marcus Lillington and the Chatroom Crew.

Marcus Lillington: Ooh.

Paul Boag: That's what I'm now going to call them.

Marcus Lillington: Because they'll really like that. It'll make them feel-

Paul Boag: But it's better than Boagworlders, isn't it?

Marcus Lillington: Yes.

Paul Boag: Yes.

Marcus Lillington: Quite simply.

Paul Boag: Take two, so there you go.

Marcus Lillington: Here we are again, a day later.

Paul Boag: Yes, yeah. What those who aren't in the chatroom will not realize is, we tried to record this yesterday, but Marcus' wifi at his place of work, not his home, but at his place of work, was so shocking that we couldn't do it.

Marcus Lillington: Yeah. New office, you think, "Yeah, it's all great. It's all set up." It's a shared meeting room, but I booked it all in. But we said, "No, we don't need any hard wiring into connectivity. Wifi's fine."

Paul Boag: Ooh, yeah.

Marcus Lillington: That noise, I've got builders next door, so if you hear strange noises, it's not me, it's them.

Paul Boag: That's all right. Couldn't hear that.

Marcus Lillington: Stuff like that.

Paul Boag: Well, crosstalk 00:01:39.

Marcus Lillington: Here I am, back at home where everything works, and there are guitars behind me instead of a plain white wall.

Paul Boag: But you haven't exactly got the best internet connection at home, have you?

Marcus Lillington: Well, it's pretty good, and I get about 50 Mbps, but sometimes, it just dies to five.

Paul Boag: Wow.

Marcus Lillington: If I pull out the roots, I plug it back in again, it's all okay again. And apparently, most of where I live is like that. And also, just over the road, the most they get is eight, and I get 50.

Paul Boag: But didn't you say coming down your road is fiber, the main road. Or did I-

Marcus Lillington: Yeah, I'm on fiber.

Paul Boag: Yeah, but you're not on fiber back-

Marcus Lillington: Well, not back to the house, no.

Paul Boag: No.

Marcus Lillington: Fiber to the-

Paul Boag: Fiber to the cabinet.

Marcus Lillington: To the cabinet, yeah.

Paul Boag: See now, this is very interesting.

Marcus Lillington: Ooh, is it? Is it really?

Paul Boag: Because I've discubbered … Discubbered? What's discubbered? I've discubbered something.

Marcus Lillington: Something you keep in a … You discubber something in a cupboard. There you go.

Paul Boag: That would-

Marcus Lillington: Discupboard.

Paul Boag: That would be it.

Marcus Lillington: To discupboard.

Paul Boag: Yes.

Marcus Lillington: A new word.

Paul Boag: And once again, I'm looking forward to seeing how the transcriber manages to deal with that particular fluff. Hello, transcriber. Yes, you can actually request fiber to the premises.

Marcus Lillington: Can you?

Paul Boag: I didn't know you could do this. Obviously, all new builds-

Marcus Lillington: Surely that's going to cost a fortune.

Paul Boag: Not necessarily. See, here's the thing in the UK. Obviously, all new builds are fiber to the premises. That's just what they do now. If, however, you live, like the majority of people in the UK, in some old decrepit house-

Marcus Lillington: I certainly do.

Paul Boag: Yeah. It's more of a thing you do if you're going to do it from a business perspective, which obviously, you would be doing, is you have to pay Openreach £200 to do a survey, so I've just had my survey done. And you lose that money, whatever happens, but the government then will fund up to £2500 worth of work to get the fiber to your premises. In my case, as it so happened and as I suspected, there's fiber that goes just down our main road, which is a couple of hundred yards away.

And there is trunking between there and my house, with the exception of about a eight feet to 10 feet bit. All they've got to do is dig up that eight to 10 feet bit, which they could do for under £2500. Well, I don't know. I haven't got the survey back yet, but I suspect that you can. Then I don't pay anything additional beyond the £200 to get them to survey it, and I'll get fiber to the premises.

Marcus Lillington: Which is cool, and I was thinking about you and being lucky with money. What some news you got there, wow. We're all like, "Hell yeah, Boagworld." Or Boag, rather. "He always lands on his feet."

Paul Boag: I've got to say, this is … Although it's thrown up a very interesting dilemma, and I will be interested in people's opinions on my dilemma. Here is my bit of good news, which, to be fair, came out of some pretty bad news.

Marcus Lillington: Yeah, yeah, yeah. Absolutely.

Paul Boag: But when we were renovating-

Marcus Lillington: I'm very, very pleased for you.

Paul Boag: When we were renovating the house, we got an email from our builder that basically said, "Oh, we've changed our bank account details. Can you pay the latest invoice? This is the invoice number and here's all the details," from the invoice that we'd already received. "Can you pay into the new bank account?" We went ahead and paid it, and it turned out somebody had hacked his email. He knew all of the builder's invoice numbers and all the amounts and everything else, and then emailed people out saying, "Can you put it into this new account?"

And so we transferred a sizeable amount of money, because renovating is not cheap, across into this. And ah, that was the end of it. We contacted the bank in about two hours, but by the time the bank responded, it was too late. I was slightly upset that they didn't respond quicker, because it took them over a day to come back to us, and if they'd been quick, they could've put a freeze on the other bank account. Sure enough, a year and a half later, the financial ombudsman agrees with me. I'd long ago given up on it, presuming it wouldn't go anywhere. And we've got the whole lot back from the bank, plus interest, plus £200 compensation or something ridiculous.

Marcus Lillington: That's cool.

Paul Boag: Which is fair enough, but here's the interesting moral dilemma, Marcus. This is the Moral Maze with Marcus Lillingtonillington. What would you do in this situation? When the problem originally happened, the builder blamed us for paying the money into the wrong account, and we blamed the building for allowing his email to be hacked. And then we both realized that we were both victims, and screw whoever did it to us. We weren't going to fall out over it. And so we split it down the middle.

That means half of the compensation is due to the builder. Absolutely fine, no problem with that, except for the fact that, in the intervening one and a half years, he has closed his business, retired, and moved to Spain. None of his telephone numbers work. None of his email works. And we've contacted some of his other workers, the people that we worked with. None of them know how to contact him, so how hard do you try to return his half of the money?

Marcus Lillington: That's a tricky one. I guess … Yeah, look, it's like, "You've done your best." I was going to be a bit nicer than that to the builder man because, if he's retired, he could probably really do with a little bit extra, so I would go back to the people he works with and ask them, "Who are his friends that are still around?"

Paul Boag: Yeah, we've tried that. The last thing we've managed to do, our very last idea, was we've sent a letter to the address that the company was at, a physical letter, hoping his post will be forwarded on. What we've basically concluded is, look, we've tried all these different routes. Maybe it gets through to him somehow in the next couple of months. If it goes on longer than that, we're going to take the money and we're going to donate it to charity.

Marcus Lillington: Yeah, perfect.

Paul Boag: There's nothing else you can do, is there, really? It's a real tricky one. It's a very strange situation.

Marcus Lillington: We were trying. It could be a case of, "I've got the money now. He doesn't know. What do I do?"

Paul Boag: Yeah, "We won't tell him."

Marcus Lillington: At all.

Paul Boag: Yeah, yeah.

Marcus Lillington: "Let's not make any effort at all." Which is where I thought you were going to go with it, Paul.

Paul Boag: No, no, no. I'm not that bad. But it's interesting, because me and Cath-

Marcus Lillington: Paul in the chatroom says he'll find him for a cut. Excellent.

Paul Boag: There are people that do that, aren't there? People who find relatives that are due inheritance-

Marcus Lillington: I have had that happen to me.

Paul Boag: crosstalk 00:09:29 for a cut. That's an interesting job to do. Anyway, yeah. That's my little moral dilemma, because me and Catherine disagreed slightly about the degree of effort that you need to put in. I won't say which side I came down on, because I think it's fairly predictable. But it is an interesting one. And he totally deserves it, because he was very nice. Don't get me wrong. If we could've contacted him, I would've definitely contacted him, but at some point, you've got to go, "I don't know what else to try really, short of hiring someone like Paul from the chatroom to go and find him for us." Anyway, there you go. Fiber to the premises, people. Worth knowing if you live in the UK.

Marcus Lillington: Is a thing, yeah. Cool.

Paul Boag: Although Lewis, is Lewis in the room today? Lewis is always boasting about how ridiculously fast his internet is at home, so … He's not in. Oh yes, he is in. Yeah, Lewis, we hate you and your ridiculously fast internet. Anyway, Marcus, have you got a thought for the day today?

Marcus Lillington: I have got a thought for the day? Where are notes? Here they are.

Paul Boag: Why is it? It's something about English podcasters. We've got Gardeners' Question Time for the web, and Thought for the Day for the web. Men of a certain age, that's what it is.

Marcus Lillington: I don't know why I listened to Radio 4 in the morning for years. It just got annoying. But anyway, competitor reviews.

Paul Boag: Oh, yeah.

Marcus Lillington: I'm doing one at the moment, so it's front of mind. I thought I'd talk about it.

Paul Boag: Good, because I'm never quite sure. I always have mixed feelings about A, their value, and B, the best approach for them.

Marcus Lillington: Well, interesting you say that, Paul. Because-

Paul Boag: It's not like we've got this planned, because we really don't.

Marcus Lillington: Not at all. We've found that there are … Over the years, we've taken two approaches to competitor reviews. One, and we used to do these, was a very highly detailed, heuristics-based review. Or two, a more narrative-based, "have a look around, note what's good and bad," review. There's two different approaches. We used to do the heuristic reviews a lot, where we'd agree on a fairly large number of elements, probably between 30 and 50, so 40. Different elements that we'd measure each competitor against. You'd have, I don't know, donation process would be one of these heuristics, and you'd look at each one of the five or six different competitors, and you'd measure each one. You would be giving a score for each site for each of these 40 heuristics, as well as providing a little bit of commentary for each one.

Paul Boag: And that's … Sorry to-

Marcus Lillington: That's all right.

Paul Boag: … just cut across you, but that was the ones that I always particularly struggled with as to the real value of, "What are you going to do with that once you know it?"

Marcus Lillington: I'd say what's good about that kind of review is the results that you get are quantative. It's the reviewer's opinion, but there's a score. There are numbers, so what you can do is, once the document has been delivered, you can look at it as one of our clients and say, "We're doing better than Company X at this, but they're better than us at Y." And you can create really cool looking graphs and radar plots and all of that kind of stuff to show the result.

Paul Boag: That doesn't make it valuable, Marcus. Just because you can create a cool looking graph.

Marcus Lillington: Yes, okay, but I think that type of heuristic-based review takes a very long time, so therefore, they're expensive to carry out. They take forever and they're really boring to do. But the key point, I think, is I think you would find the most important takeaways using the more narrative approach. Where you might spend a week on a heuristic review, you could potentially discover the most important stuff, or the wins, within a day with the more random approach. I'm exaggerating to make the point. You might spend … It's just a little bit less time, usually.

The reason why I'm saying that is because we've ended up doing a more hybrid approach now to competitor reviews. Rather than having 40 or so very specific measures that you have to go through each site … It just takes forever, and it's mind-numbing. What we do is we'll measure the sites against much more broader categories, like branding, messaging, design, content, navigation, calls to action, and so on. This still provides useful comparisons, so you can still say, "From a branding or messaging point of view, we're doing better than Company X, but we're doing worse than Company Y." We, as reviewers, aren't straight-jacketed into answering absolutely everything when we don't need to. That is my thought for the day.

Paul Boag: Yeah, and I think I probably agree with that. It's all about return on investment, I think, really. I think a lot of people commission these kinds of reviews without having a clear idea going in what they're going to do when they discover that. Because you're right. A good heuristic review can let you know, "Okay, we need to be investing more in design or usability or donation process," or whatever it be. And that kind of-

Marcus Lillington: Yeah, the point of them is to tell you what's wrong or learn stuff. "They're doing it really well. We should copy them."

Paul Boag: Yeah, but often-

Marcus Lillington: With the … Sorry, carry on.

Paul Boag: But I'm just not convinced people often do that, do you know what I mean? I think they go, "Oh, that's interesting," and then they carry on with whatever it was they were going to be doing in the first place, do you know what I mean?

Marcus Lillington: Maybe. That's even more cynical-

Paul Boag: Am I being too cynical?

Marcus Lillington: That's even more cynical than me, Paul, for once. But anyway, it's-

Paul Boag: I don't like it-

Marcus Lillington: They're quite enjoyable if you're not straight-jacketed by, "I've got to do this one, and then that one." If you can just look around and find stuff that's interesting, they're actually quite interesting to do.

Paul Boag: Yeah. Don't get me wrong. I do them fairly often, but I almost always go towards the narrative, because it's much lighter weight. Unless somebody has got a very strong opinion, "We want to do this, so that we can make this judgment, and then do this." I would go with a much lighter weight approach, I think. Because oftentimes, you feel like the competitive review is in there, as Andrew points out in the chatroom, because the powers that be want us to review the landscape. And that's about as much depth as there is into it.

Marcus Lillington: Yeah, and I think there's nothing wrong with doing that, as long as you don't spend too much effort/money doing it.

Paul Boag: Absolutely.

Marcus Lillington: Because crosstalk 00:16:39.

Paul Boag: Yeah, yeah. Because that is a good thing to do, but not if it takes three weeks of work. There are better things to spend money on, aren't there?

Marcus Lillington: I couldn't spend three weeks on a competitive review. I would die. I'd go mad.

Paul Boag: Yeah, I know, right? You know what you should be spending that money on instead? Doing some user research, which happens to be our … Cat is going, "Three weeks!" No, I just pulled that out of my ass. I've never spent three weeks doing a competitive review in my life, so don't use that as a benchmark. Yeah, "Paul Boag says the standard competitive review is three weeks long." Oh my word.

Marcus Lillington: I've spent a week on more than one occasion.

Paul Boag: Yeah, no. I've spent a week, yeah. Anyway, what I prefer to spend-

Marcus Lillington: Yes. User research though …

Paul Boag: Yeah, I prefer to spend the money on user research because, to me, that's more competitive. More … not competitive. More applicable. Because that's the other thing I have about competitive reviews. If you're just spending all your time looking at your competition, then you're always going to be following them. You're always going to be one step behind them. Although there is a value in it, it can't be enough by itself, and that's what brings us on to today's subject, which is user research. Aye.

Marcus Lillington: Aye.

Paul Boag: But, of course, we're talking about user research within the context of the season, and the context of the season is that everybody should care about the subjects we're talking about, whether you be a designer, a developer, whatever. I have to now explain to you all why I think that, even if you're a developer, or even if you are a manager, you need to care and know a bit about user research.

I had to think about this quite heavily, because I wasn't entirely convinced that I was convinced that everybody needed to understand this. But the more I thought about it, the more I've decided, "No, I'm going to stick with my throwaway decision to put this in the season." Because I think there are two types of research we're talking about here. There is research about the person themselves, that you're researching. And then there is the research about how that person uses and perceives your products and services. Depending on your role, you might favor one of those areas more than another.

Marcus Lillington: I have to say that, without thinking about it at all, that I agree. I think this is something that everybody should care about. I don't know why. You're going to teach me.

Paul Boag: Well, it's more of a gut reaction. What I think made me think about it was, I was challenging my own assumptions, because obviously I'm going to think it's important, because I'm a user experience designer. But as I was … I want this season to be more than just big user experience designers pushing their agenda of what it is we think's important, do you know what I mean?

Marcus Lillington: Mm-hmm (affirmative).

Paul Boag: So I was debating about whether to include it or not, but I actually think, if you look at it within those two areas, so either about the person or about how the person is using or perceiving a product or service, then I think whatever you role, I think that applies. If you're working in digital fields, you need to know people better. The big challenge obviously in this is developers. Do developers really need to know this? They maybe don't need to know as much about the person like, "Do they read the Guardian and drive an Audi?

But they do need to know about how the person is using their products and services that they're involved in building. While a marketer is probably more interested in the fact that they read the Guardian and drive the Audi than that they want to complete this particular task on this particular thing within a website. It does depend on what your role is, but yes, I think everybody should care about user research.

Marcus Lillington: You know … Paul, I'm going to cut across you, and I apologize.

Paul Boag: No, go.

Marcus Lillington: Last week, I was going, "Hmm, I'm not sure if developers need to know about sales and marketing." You said, "Well, yes they do. They do, because they, often, the work that they'll be doing, it's been directed or it's been kicked off by the marketing department. You need to have an understanding of why you're doing something, and that's the business objective side of it. This side of it, the user research side of it, is the user requirement side of why you're doing something. Both are important, so you need to have an understanding of both.

Paul Boag: Yeah, no. You're absolutely right. And of course, the outcomes of user research is going to affect everybody, because, if you're doing it right, your projects should be user-centric. The result of that is that everyone is going to be affected by the user research. But there is another flip side, another reason why I think it's really good to have an understanding of user research techniques, and that's because you can apply it to anybody.

If you know good user research techniques, then you can apply it to your colleagues as well, to get to know them better, to get to understand what makes them tick, what motivates them, which is great for working in organizations. And I'm amazed that user experience professionals don't do this more. I was talking to … I was doing a mentoring session with a user researcher actually, at a big organization, and I was talking her through. She was really struggling with internal stakeholders.

I was talking her through all of these different techniques I use to understand them and their motivations. It was like this light bulb went on, and she said, "Well, that's what I do with users all the time." Yes, yeah. You understand what motivates them, what they care about, and you frame everything in that context. Absolutely, that's really useful for that. And of course, user research is going to lead to better products, because you'll be able to provide a better experience.

And also, it means that you're going to be more successful from a business perspective, because you're going to get more people recommending the product, because they like using it, etc, etc. You know all of this stuff. I don't need to go over that endlessly. But it even helps you define what your projects are. For example, with customer journey mapping. When I've been working on customer journey mapping before, as we go through the process of mapping the customer journey, you actually find on, "Well, hang on a minute. We've been spending all our money and resources at this part of the journey, because we thought that the most important."

And there's this glaringly big hole where we're crap further down the line. This actually happened with one large charity that I worked with. We were like, "Whoa, okay. We need to stop spending so much money on raising awareness, where actually, the awareness isn't too bad., and actually pour money into this part of the experience, where it's falling apart at the moment." User research can help uncover those kinds of problems and define your projects. And not just what projects you do, but also which projects you should prioritize based on their impact on the user.

And even the features and specification for the project, based on what it is that users need to know, what they want to do, what objections they've got, and that kind of stuff. But one of the primary reasons I love user research so much is it's a brilliant tool for resolving arguments within projects. If you get two parties that disagree, just test it. It'll be quicker to test it than it will be to argue about it, so absolutely consider that.

User research will speed up projects, because it avoids that endless debate. Take, for example, something like design sign-off. Design sign-off is a frigging nightmare because, "Oh, I don't like the green." And another person goes, "I do like the green, but I don't like the typography." And you have to have a meeting to discuss it. Everybody disagrees, and then they try and reach a consensus, and it comes out gray.

If you just take your keywords that you want the design to represent, and put the design in front of users with a mixture of keywords, and the users select the right keywords, then you know the design's working. Move on. Let's do the next thing. Anyway, that's turning into a rant now.

Marcus Lillington: It's funny though, isn't it? I'm going to get off on a bit of a tangent here, because I-

Paul Boag: Oh yeah, you rant too.

Marcus Lillington: Well, is this a rant? It's more of an observation. It seems that some senior people, CEOs, heads of businesses, to them, inputting their character into a design seems to be … Character, that's too strong. They want to input, and they want … It's almost like they think that it's important that they stamp their … Not authority, but their … I've come back to the word "character." I can't find a better one.

They want to be part of it, and they want to like it. I suppose that's what it is. And I'm going through that at the moment. That's something I'm going through, and as long as you recognize that, it's fine. But I guess where it gets to be a little bit difficult is if that might go against what user research is saying. It's like, "Hang on a minute. How do we deal with that?" I suppose you've got to find a balance.

Paul Boag: Well, I haven't even got a problem with that, as long as that leader is being honest with themselves. Ultimately, they're paying you. That person is paying you to produce the design, and if they want the design to look a certain way because of their ego or because of their desires for the business or whatever else, I'm happy just to deliver what they want. What annoys me is where they disagree with either the user research or my personal opinion, and instead of just saying, "I don't care. I want to do it this way anyway," they'll go, "Well, the user research is flawed," or they'll want me to agree with them. That, I've got a problem with, because that's not being honest with yourself about your motivations. But if they want it to be a certain way, and they're paying the bills, that's no problem.

Marcus Lillington: I think it's better than somebody that doesn't care. Or is it? It's a tricky one.

Paul Boag: I don't know whether it is, because it depends. It depends to what extent they don't care. I haven't got a problem with someone not caring about a design. I've got a problem with someone who doesn't care about the project as a whole, which happens occasionally in big organizations. "I've been dumped with this, and now I've got to deliver it."

Marcus Lillington: I suppose the worst thing of all is the leader that … We've had this. We had a really bad case of this a couple of years ago, the leader that says, "Well, I'm hiring you guys because you're the experts. I'm going to go completely hands-off and you're going to deliver something I love." And then the day it went live, he came back with a 20-point list of things that were wrong with it. It's like, "Okay."

Paul Boag: Yeah, but that goes back to what I was saying about not being honest with himself.

Marcus Lillington: Yeah, absolutely. Anyway, I've gone on a massive tangent there, but it is-

Paul Boag: crosstalk 00:29:10 tangent. The final reason that I think user research is something that we should all be paying attention to is, ultimately, it's going to save us money. Because we're going to avoid building things that people don't actually want, which often happens in projects. Which I don't care what … Okay, it might not benefit you personally. You won't save money personally, Mr. Developer, but it means you don't have to buy or build shit that you know nobody cares about, which I think we've all done from time to time.

And also, you're not going to have to fix costly mistakes further down the line, because you're going to pick up on any potential problems early rather than later. To my mind, it's a no brainer. And to a lesser or greater extent, it should be involved in every project that we do and everything that we do. I mean, to the point, you look at some that are like the UK Government Digital Service, that say, "In order to be a stakeholder in a project, you have to have spent at least two hours with a user in the last six weeks." That's how important they take it. I'd love to work on projects like … Could you imagine? Nobody in the room that hasn't spent time with users in the last six weeks. I wonder if they really do that, or whether it's just something they shoved on a poster?

Marcus Lillington: Yes, I'm feeling slightly cynical. Obviously, they have their user researchers who are part of every team, will have to have done that, but really? Everyone? Well, I thought-

Paul Boag: That's what they say.

Marcus Lillington: Okay.

Paul Boag: In their service manual. Lewis says he knows no one that actually likes the output of GDS. I like them. What's wrong with their output? I think it's pretty good.

Marcus Lillington: Generally speaking, it's very easy to use, and that's the point.

Paul Boag: Yeah, perhaps I'm misunderstanding what he's getting at.

Anyway, let's talk about our first sponsor, which is Testing Time. Perfect one, the perfect sponsor to have on this show, and I shall be using them forthwith and withforth. Well, I actually am, actually. Hopefully next week. I said it when … Last time they were on, I was saying, "Hey, I'm going to be using them next week," and I didn't, because the whole project got delayed a little bit. But this week, this is the week that I'm going to be using them.

Testing Time make doing user research easy, because it takes the hassle out of the biggest problem with user research, which is recruiting test users. Whether you're looking for people for usability testing, focus groups, interviews, surveys, whatever, these guys can help you find them. They can find people for online and remote testing, but they could do also offline and on-site testing. They'll help you do that as well.

And they've got a pool of over 350,000 people that they draw upon, so wow, what an amazing service that is really useful. The order process is really simple and straightforward. They calculate the price in real time, based on the number and the profile of the people that you want. All you've got to do is visit their website, so you enter the number of people that you need, the type of study you're doing, the criteria, standard criteria though, like demographics and that kind of stuff, any special requirements you have. And normally, they deliver you participants within 48 hours, so it really doesn't need to slow up your projects much. Put it in, and you get responses back pretty quick. And you've got this whole customer support team there to help you, from one end to the other. If you want to find out more about them, go to

There you go. That is why you should do user research, and a great tool for overcoming the greatest barrier of doing so. The question then becomes is, "Where do we need to start?" Right. I want to break this down into three bits. I want to share with you a little bit about what you want to know about users, in my opinion. A little bit about how you go about learning that, and then I want to wrap up with a few ways that you can then take that and make use of that knowledge, and visualize that knowledge.

Let's go through from the first one. What you need to know about users: Obviously, this is utterly dependent on your job and your project and all of that kind of stuff, but it makes a really boring show if you just say, "Well, it depends," and then stop there. I'm just going to give you the kind of things that I find myself wanting to know a lot about users. And that will be, some of these things will be more applicable to you than others. Insert disclaimer here, etc, etc.

Marcus Lillington: Caveat, caveat, caveat.

Paul Boag: I've got boring in my old age. I never used to do disclaimers. I used to make huge sweeping generalizations on this show.

Marcus Lillington: You did.

Paul Boag: It's a shame.

Marcus Lillington: It's sad.

Paul Boag: I know.

Marcus Lillington: You sound like me: "Oh, I'm terribly sorry. Just in case somebody gets offended, I didn't mean it."

Paul Boag: I'll say something completely offensive, and then it's all right.

Marcus Lillington: No, no, because then I've got to edit it.

Paul Boag: Okay, so first thing I find myself wanting to know is, I've already mentioned, this idea of the user journey. The idea of the progression that the user goes on through time. That is useful in all kinds of ways. It's going to shape your messaging. It's going to shape your calls to action. It's going to shape which channels you're focusing on, what products that you produce, where you place your investment. The list could just go on and on and on. Customer journey mapping is absolutely essential, I think, in pretty much all contexts. I could do a whole … Well, I do do a whole talk on customer journey mapping, because it's a really interesting topic.

Second thing that I'm looking at a lot is I'm really interested in the questions that users have. I often use, when it comes to content creation, I almost always encourage people to start with user questions. What do users want to know? Let's write answers to those questions, and then we can put them together into pages. Oftentimes, even things like information architecture, I start off by organizing the questions into piles, and that is the basis of my information architecture.

Marcus Lillington: We tend to basically add user questions into each section of the user journey, which is a unusual way to do it.

Paul Boag: Absolutely, yes. That should totally be part of the user journey mapping. Another thing that I'm very keen on, which is not something that the UX community focuses on hugely, but sales and marketing do, and actually I think the UX community could learn from sales and marketing on this one, which is user objections. What's stopping the user from acting? We don't often ask ourselves that question, and I don't really know why, but anyway. That's a thing that I care about a lot.

Because, if you can … Say, for example, you want someone to sign up for a newsletter. Why might they not sign up for a newsletter? Well, they're worried about spam or they're worried about privacy or they're worried about how often they're going to receive emails, whether it's going to be relevant to them. They've got all these concerns. If you address those in your call to action, then your conversion rate's going to go through the roof. Yay, you know?

Marcus Lillington: Woo.

Paul Boag: Exactly. Then, of course, is the user tasks. What is it the user wants to do at each stage on their journey? Really important. What's influencing them? And I don't just mean they read the Guardian, although that is a factor, but I mean, are there a lot of review sites out there? Are they being influenced by social proof on your own website? Are they being put off by some of your communications to them earlier on in their journey? What is influencing them?

What interactions have they had, or will they have, with the company? Again, this is the kind of thing you cover in your customer journey mapping. Have they spoken to you on Facebook? Have they come and watched a podcast live? Whatever. That kind of thing is pretty useful to know. How they feel at various stages on the journey is another really important thing to know. It's more important in some cases than others. When I did customer journey mapping for the Samaritans, how people feel was pretty important, as the Samaritans is for people that are feeling suicidal.

But if you're talking about how they feel about buying household insurance, it's probably fairly indifferent, if not slightly irritated, and that's about it. But then you have something like booking a holiday, when actually, how you feel will change massively over the period of time, from being quite stressed about getting the right booking and everything sorted out, to actually huge enjoyment. Feelings can be pretty important.

What's the user's ultimate goal? Because nobody comes to your website, because they want to come to your website. That's not their goal. Obviously, they want to achieve something, and often, we don't dig deep enough into that. Often, we think, "Uh yeah, we provide …" Oh, let's take Testing Time, we've just talked about. "We provide … There are people whose ultimate goal is to recruit test participants." No, it's not. Their ultimate goal is to understand the user, so they can improve the experience and make more money as a business, right?

Looking at that ultimate goal is a really good one, and there's a very similar one to that. The flip side of that is, what is the pain point they're hoping you're going to overcome? And again, if we take Testing Time as an example, because we've just been talking about them. The pain point is the hassle of recruiting people. Are you clear on what the pain point that people are hoping you're going to solve for them is? Those are some of the things that I like to know about users. Have I missed anything, Marcus? Is there anything in that list that you think, "Oh, that should've been in there?"

Marcus Lillington: I suppose things like demographic, age and cultural type stuff, which you would use.

Paul Boag: Yes, that's very good. That's a good point, actually. That's blindingly obvious. I completely missed that.

Marcus Lillington: Yeah, because that would feed into personas, bias.

Paul Boag: Yeah, and also, for a long time, I was a bit like, "Do we really need to know that?" But yes, you do, don't you? I think, me feeling like that was a very, very … I'm talking about 20 years ago, I used to feel like that. I think that's the sign of being a young white man, that you just presume the rest of the world is like you. Well, in truth, the older you get, your attitudes and the way you approach things changes quite dramatically. Let alone cultural considerations, let alone gender and all of these other influencing factors, so yeah, you're entirely right.

Okay, so how do we go about finding all of that stuff? There's a proverbial shit-ton of different techniques that you can use, and it's really trial and error as to what's going to work in your particular circumstance, because it depends on your audience and how you … I'm doing another depends now. I'm not doing it this time. This is the definitive list of things that you should do. You shouldn't do anything that's not on this list, and anything that isn't on this list is a waste of time. I don't think people believe me, but there you go.

Surveys, I find surveys really useful if they are done well. The big problem with surveys is legislation. When you table a piece of legislation, people add amendments on to it, and it gets longer and longer, and more and more bloated, until nobody ever votes for it. And that is exactly what happens with the average survey. "Oh, you're having a survey. Can you just ask this too? And then, and …" You want the most focused, concise, to-the-point survey possible. You need to go in wanting to know a single thing, and to only ask the questions that answers that single thing.

For example, when I ran a survey recently on one of the courses that I was running, the Video Masterclass, Conversion Rate Masterclass, all I wanted to know is what the primary reason was people weren't buying. Very simple, so I asked a one question survey that only appeared on exit, because we wanted to get people that really weren't buying. And I asked the one question, "Which of these following reasons did you not purchase today? Was it the dadadadada or other?" And that was it. Sorry, I just got distracted there, because somebody in the room said they'd bought it yesterday, and that made me happy. Yay. Actually, I knew you bought it, because I'm watching you. Right.

Marcus Lillington: Surveys. I-

Paul Boag: Creepy.

Marcus Lillington: I disagree, Paul. They're-

Paul Boag: Oh, do you?

Marcus Lillington: No, I do agree, but I think you can do surveys that are affective that are more than just one question, and that-

Paul Boag: Oh yeah, yeah, yeah. Yes, you can, but it's the idea of them being focused, I think is what I'm getting at.

Marcus Lillington: What we've done a lot of lately, and when I say lately, I mean over the last couple of years, is making them just focused on top tasks. So rather than having lots of opinion-based stuff that's an utter nightmare to wade through once you get the results back, doing top tasks where you've worked out that there are 60 tasks that the average user could do on your site, and get them to pick five, which you can all do very easily on automated … sorry, online surveys. I think surveys are great for that kind of thing.

I think people doing them get annoyed, because they're like, "I want to pick the other one," or "It's such a long list, blah." But actually, the results we get-

Paul Boag: That's part of the reason.

Marcus Lillington: Yeah. The results we get from it are great, rather than-

Paul Boag: Can I-

Marcus Lillington: … just, "What do you think about this?"

Paul Boag: Can I point something out, Marcus?

Marcus Lillington: Mm-hmm (affirmative).

Paul Boag: A top task analysis is just one question.

Marcus Lillington: It is.

Paul Boag: With 60 potential answers.

Marcus Lillington: Mm-hmm (affirmative), yeah. Well, then you pick the best five of, yeah.

Paul Boag: Yeah.

Marcus Lillington: Okay, then you still … I'd still think you'd be asking people to identify the type of user they are as well. crosstalk 00:44:39.

Paul Boag: Yeah, yeah. That is true. There's normally a … Yeah, I don't know. I'll give you that. If you haven't tried top task analysis, there's a brilliant List Apart article, so if you Google "top task analysis" and "List Apart …" I can't be bothered to add it to the show notes. I don't care that much. Gerry McGovern, who's the guy that came up with this technique, breaks it down very specifically of how it works, etc. Check that out.

But yeah, there are lots other. Another thing you can do is user interviews. We do a lot of those, very useful. They're really good, because you can ask follow-up questions. You can deep dive onto stuff. You can see the faces that people pull, because the faces that people pull are often very enlightening, in my experience. So yeah, interviews is really good.

Marcus Lillington: And you can get people to show you. That's a really good thing in an interview.

Paul Boag: Yeah, that's a really good one as well. Yeah, good point. Then we've got customer journey mapping workshops. You could do those with users. Unfortunately, that seems to not happen anywhere near as much as it should, but it's by far the best way of doing them. You could do participatory design, where you actually do things like, "Let's … The company wants to communicate these words to you. They want to be seen as progressive. They want to be seen as exciting, etc. Let's create some mood boards that you feel represent those kinds of words." That would be one example.

Or even doing wire framing with people, users, is pretty interesting as well. You get some interesting results. Not always applicable, but it's good. Then, of course, there's card sorting with end users. Do that quite a lot. That's a really useful following tool from doing a top task analysis, because your top task analysis will narrow down what people actually care about. Because, typically, you have far too many things in order to do a card sorting exercise. You use top task analysis to narrow it down, and then you can use card sorting to organize that around people's mental models.

There's usability testing, obviously. In-field studies, I really like. This is where you go into someone's home, and you get their permission first. Don't just turn up. They don't like it. I've tried it before, and especially when you climb through the window. That really, apparently, is wrong. But yeah, you go to people's houses, you sit down with them. You have a chat with them. You do maybe interview them, maybe do a bit of usability testing while you're there. I'll tell you what, it is absolutely transformative in terms of the way you think about it.

I always thought usability testing is really good. What I've learned mind is it's a very artificial environment. You do usability testing and you bring people in, you sit them down in front, in a quiet room in front of a big screen. They're giving it their full attention, because they're being paid by you to do this job, and so it goes on. And, of course, when you go into their house, a cat jumps on their lap, and the doorbell rings halfway through. They're distracted, and life is around. It's a much more real representation. Also, you just learn more about people by being able to have a nose around their house. Probably shouldn't say it like that, but that's basically what you're doing.

Marcus Lillington: You're not nosing around their house, Paul. That would be up in the bedroom, bathroom.

Paul Boag: Yeah.

Marcus Lillington: Is that what you do?

Paul Boag: Is that not … yeah.

Marcus Lillington: Okay.

Paul Boag: Don't you do that? Two of you go together, and then one of them keeps them distracted, while the other one noses around the house and helps yourself to a few bits. That's the technique, isn't it?

Marcus Lillington: I believe you've been watching too much television.

Paul Boag: I thought that was how it's … Then there's social media monitoring, obviously. People used to spend a fortune on market research stuff, didn't they?

Marcus Lillington: Mm-hmm (affirmative).

Paul Boag: And now, everybody just lays it out on the internet for them. I don't understand why more people aren't going, "Ooh, who's this person that follows me?" I do it all the time when people sign up for my newsletter. Every now and again, I'll just go and have a look at the list, and I go, "Ooh, what's that company?" And I go and have a little nose at them and look at what they're posting on social media, just to get a taste of who these people are.

It's not quantative. I'm not doing it in any kind of organized way, but it's really interesting to get little glimpses of the kind of people that are interested in your content and are showing interest in it. The other thing I used to do, I don't do this anymore, because I'm a lazy bugger, but the other thing is, I used to identify key questions that people would ask on Twitter. If they ever asked … I'd have a search running with searched on question mark, and then a whole series of different keywords, like "UX," "user experience," "web design," blah blah blah.

Then, I could see all of the questions that people were asking about my field, which was very enlightening as well. And then, obviously, there's things like site monitoring, site session recorders, looking at your analytics. But again, that's a classic thing. Analytics is a classic thing where, it's a bit like surveys. If you don't go in knowing what you want out, it's a bit of a waste of time. You've got to go into your analytics … If you just log in to Google, "Oh, my bounce rate is 87%." Is that good? Is that bad? What you want to do is you want to go in having a specific thing in mind you want to know. Chris Scott taught me that really. He always goes in with a set of questions that he wants answering. And then, there's finally user diaries is the other one I have done, but I've never done them.

Marcus Lillington: I don't even know what that is, Paul. What's a user diary?

Paul Boag: It's where you ask people to keep a track of what their experience over normally a prolonged period of time. For example, if you were creating a health app for people who had diabetes, you might ask them to keep a diary of their experiences relating to-

Marcus Lillington: Using the app, okay.

Paul Boag: … diabetes. But to be honest, again, it suffers from that same problem as competitive reviews, that it's a hell of a lot of work to go through all the data that you get back from something like that. It's not something I've done a lot with, but I know a lot of people swear by it. Oh, Cat. "We're doing user diaries for the first time right now, with first year students." See, I can imagine that's a really good scenario, because Cat works in a higher education institution, so understanding what those first year students are … It would even be better to get people to do a user … Well, not better, but I would be interested in getting a user diary of people that have been offered a place but haven't yet accepted the place.

I think that's a period of time when universities let down people quite a lot. People do a lot of switching, where they say they're going to go to one university and end up going to another. Yeah, they've absolutely got a use. It's just not something I've gotten to do a lot of, which is kind of a shame, because I like playing with new toys. The very last thing, I'll do this bit quickly, what do you do with all this data once you've got it?

You do all of this effort. I think it's very valuable to visualize it in some way, because otherwise, it all just, "Oh, that's interesting," and then everybody forgets about it and gets on with their projects as normal. You want something you can tangibly look at regularly. At the most granular level, you can start to do things like story cards, use the story cards, so when you're defining a project, you don't define it in terms of features and specifications of what the project is going to do, but rather you define it in a series of user story cards, things that you're going to help users do through this project.

I am a first year student at a university. I want to get access to my timetable, so I can succeed in my studies, for example. You'll have a whole batch of these that'll be based on the user research that you've done, and that defines your projects. Use the story cards. You can also do empathy maps, which are basically personas by another name. But they tend to have more of a focus on the thing that I've talked about being interested in: tasks, questions, feelings. Those kinds of things, rather than the demographic stuff.

But personas, it all depends on how you use things. It's all semantics, isn't it? Personas, user empathy maps. Call it what you want, as long as it's got relevant information in, based on what you need to know for your projects. And then, of course, there's the customer journey mapping we talked about already. And the reason that I like customer journey mapping is because the questions people have, the tasks they want to do, how they're feeling change over time, and it's very easy to skip over that kind of stuff.

There you go. There's some suggestions for user research. Hopefully, that has at least shown you the value of doing it even if it's not something you're going to be doing regularly. Hopefully, it's given you some tips on things you should pay attention to, etc. There you go. Let's talk about our second sponsor, which is Miro. Miro is a visualization collaboration platform. Wow, that sounds really fancy.

Marcus Lillington: What?

Paul Boag: I know, yeah. What? This is what happens when you read the copy that people write about their own sites. Let me explain what it actually is. Basically, it's a virtual environment that lets you do things like virtual white-boarding. You could use it for conducting user research or collecting information on users and put it all in one place. You could do card sorting on it. You could share feedback, and so, essentially, it's a place where you can create this single source of truth for complex projects, so you could keep everything together.

Which, of course, enables you to create designs out of it, do research out of it. It's a kind of communication repository tool. I totally understand why they struggle to explain it, because it does quite a lot. It's one of those things where, you can use it in lots of different ways. Does that make sense? Depending what your role is. It's going to be great really where you're dealing with cross-functional teams. You've got your product manager. You've got your project managers, your designers, your developers, your marketers, your user researchers.

They've all got to work together somehow, so you use a tool like Miro, because it's flexible, and because it can accommodate those different roles. It enables you to virtually engage with distributed, remote, colocated teammates across lots of formats and lots of timezones and all that kind of stuff. But it's got some really powerful features for designers as well. It's got an endless canvas, where you can just collect bits and bobs. It's like having a white board without edges.

It's got hundreds of built-in templates you could do, for collaborating on design. It's got chat in it, video conferencing, presentation modes, and it integrates with 20+ different other platforms as well. It's used by huge names, like 3M and Twitter and Netflix and, you know, list impressive sounding companies here. If you want to learn more about it and give it a go, because it is worth a try at least. I have to say, it's a tool that is incredibly useful. I work with a lot of clients that are spread all over the place, and Miro, it's a go-to tool for that kind of stuff. You can just go and find out more about them by going to

So there you go. Right, that about wraps it up for this week's show. I'm going to put a whole load of different resources in the show notes that apparently, according to people in the chat room, nobody ever goes to look at, so I don't know why I frigging bother. But there are going to be a load of resources that you can check out in the show notes. I've got my 10 techniques to get started and improve your user research, how to get started with the usability testing, how to test design comp, and then I'll make some recommendations for three books as well.

Though obviously, obviously, you have to put Don't Make Me Think in that list, as the kind of definitive starters guide to all things usability and user research orientated. Then there's Steve Krug's, because Steve Krug wrote Don't Make Me Think. He wrote a really good, practical guide to making that happen called Rocket Surgery Made Easy. And I've also through in a great book called … It's quite an old book now, but when I read it, it was really good.

I imagine it's still good. It didn't feel like something that would date, which is called The User Experience Team of One. I recommend you check that out. If you're the only one who gives a shit about this kind of stuff in your company, that will be a really good book for you as well. Marcus, do you have a joke to wrap us up with?

Marcus Lillington: I do. I was on Twitter the other day, which is a rare day for me, and I saw Drew McClellan share this joke, so here we go. "For the longest time, I resisted going potholing, but eventually, I caved."

Paul Boag: Oh dear. Oh dear, oh dear.

Marcus Lillington: That's pretty good.

Paul Boag: Thank you, Drew. I think? I think. I don't know why Cat's going, "Oh, Marcus." The very first thing that Marcus did was distance himself from the joke …

Marcus Lillington: crosstalk 00:59:19.

Paul Boag: By blaming Drew.

Marcus Lillington: Cat crosstalk 00:59:23.

Paul Boag: That was crosstalk 00:59:25.

All right, thank you very much for that, I think. On next week's show, we're going to look at organizational skills, but until then, thank you for listening, and goodbye.

Stock Photos from pathdoc/Shutterstock