This week on the Boagworld Web Show we are joined by Leisa Reichelt to discuss user research, its importance and how to begin.
Paul: Hello and welcome to Boagworld.com, the podcast for all those involved in designing, developing and running websites on a daily basis. My name is Paul and joining me as always is Marcus. Hello Marcus!
Marcus: Hello Paul! How are you today? Where are you today?
Paul: Where am I? Yes, very good point. I’m off in the Motorhome. I’m in sunny Minehead at the moment, just outside of Exmoor. We’re heading slowly westward into Cornwall and it’s gorgeous and it’s lovely and I’ve been seeing Castles and wandering round National Trust Gardens and eating Cream Teas.
Marcus: How divine. Umm yes, it’s gorgeous here today, although it’s supposed to be clouding over a bit tomorrow which is a shame. But I have had a cup of tea and some cake in the garden earlier.
Paul: Oh that’s very civilized.
Marcus: It was cold, but it was not so cold that I actually managed to drink the entire cup of tea.
Paul: Well we are British aren’t we, so we have to go outside at the first glimmer of sunlight.
Marcus: Yes, well Caroline made me because as you know I’ve been sick. And she said ‘You’ve got to come outside to get some fresh air’.
Paul: Oh you’re not still ill?
Marcus: I’m just on the end of it. I will cough at some point and you’ll hear it. And it still sounds nasty but I am past the worst of it by a long way.
Marcus: Just got an irritating cough now, one of those tickly ones that won’t go away.
Paul: Oh yes. I’m fed up with talking about it because my life is perfect at the moment so I don’t want to hear about your ills.
Marcus: See, here we go.
Paul: I want to live in blissful isolation in my perfect little Campervan world. Did you see that green screen I did? We’ve been recording video while away and it looks like I’m in an amazing office and its great isn’t it?
Marcus: For about a second I thought ‘Bloody hell, Paul’s done his office up!’
Marcus: And then I thought ‘Ahh, no.’
Paul: But that’s pretty good that it fooled you.
Paul: And that was recorded in the back of a Motorhome with no proper lighting, just natural light and it was really good! I came out really well, so I am never ever going home ever again.
Marcus: You’d like that wouldn’t you?
Paul: I would.
Marcus: You’d get grubbier and grubbier though.
Paul: What do you mean grubbier? You ought to see this motorhome, it’s not grubby, let me reassure you of that!
Marcus: You haven’t got a full walk in shower or anything like that have you?
Paul: Ahh yes.
Paul: A walk-in shower cubicle with a door that closes, yes.
Marcus: And the water comes out…
Paul: …comes out of a proper shower head, yes.
Marcus: At about one mile an hour if you’re lucky?
Paul: No! Seriously the only problem you have is that the water runs out reasonably quickly. But it comes out at exactly the same power as it would do at home. So there you go.
Marcus: I don’t believe you.
Paul: See. You just stick to your single house world while I travel and see the glory of the world around me.
Marcus: You do that Paul. I kind of like my house.
Paul: Well you’re not a huge traveller are you? You’ve got your group of friends, you’ve got your local pub, that’s what you like isn’t it?
Paul: Which is fair enough.
Marcus: Lovely nice places to go and walk and all of that kind of stuff and I am happy. Very easily pleased, me.
Paul: Yes, while I am not easily pleased at all. But I am very much enjoying blurring the lines between work and life. Which is nice. Do a bit of work, go for a walk for a bit, come back and do a bit more work. Work a bit in the evenings. It’s great. I’m very happy.
Marcus: So if you’re going down the North side of Cornwall and Devon..
Marcus: Yes because you are in Somerset at the moment, you will go past where I was born.
Paul: Oh will we?
Paul: How exciting!
Marcus: And then a bit further on from there you get to Clovelly. That’s great, you must drop into Clovelly.
Paul: I’m looking forward to going to see the monument in your home town to you, because I am in no doubt that there is one.
Marcus: I could tell you the house that I was born in.
Paul: Is there a plaque outside? Marcus Lillington…
Marcus: One of those blue ones, you know? A blue round plaque, yes. Marcus Lillington was born here forty-eight years ago.
Paul: My word.
Marcus: So yes. Bideford’s a nice little town. Clovelly is lovely. But if you keep going down to Cornwall you get to…. Oh I can’t remember what it’s called, the really famous Castle. Tr- something.
Paul: Oh Tr-… neither can I now. But yes, that was on our list of places to go.
Marcus: That’s fantastic and if you can get down onto the beach while the sea is out you can walk through the cave to the next beach. I’m quite envious.
Paul: Yes, it’s all very exciting.
Marcus: I love all that part of the world.
Paul: So shall we talk about this week’s show?
Marcus: Devon and Cornwall are much more interesting.
Paul: I know, but people expect us… actually if I was today’s guest I would be deeply insulted by that. You’re saying that Leisa is not a good guest?
Marcus: I didn’t say that. I said it’s more interesting talking about sponsors, so there you go.
Paul: Oh I like the way you transferred into talking about Sponsors. Because we do need to talk about our first sponsor, which is MailChimp who are back again. Oh I like this, when people keep giving us money. It’s very nice of them.
So MailChimp, oh now… look at what I’ve just done. I was about to go into talking about MailChimp but now I’ve lost the script that tells me what I’ve got to say about MailChimp.
Marcus: Because you need a script…
Paul: I have to be… we no, not with MailChimp or MediaTemple, I know them both really well, but I kind of make a few notes. And actually although I know MailChimp really well, they do a lot of things that I didn’t realise they did. You know when you use a product all the time and you get it into your head that you use just certain things. It’s like Photoshop you only ever know like a tenth of the things that they do.
Marcus: That’s generous with my use of Photoshop.
Paul: Absolutely. But MailChimp do loads of other… for example I didn’t know how much research they do into User Behaviour. And how Users use email and respond to email and which sectors have got the best open rates and all this kind of stuff. So they’ve got loads of information freely available – you don’t need to be a MailChimp customer to make use of those. They’ve also got a great blog which now I’ve discovered it, I’ve actually subscribed to. Because again, loads of kind of research and User Behaviour information and advice about Email Marketing generally. So there is that to check out as well.
And of course all of this advice and research that they are providing just helps you improve the campaigns which is why they do it, obviously because they want to see you get a big mailing list and they want to see you have a lot of success with your email campaigns. So they provide all this additional information for free. But they don’t just do it for their customers—although obviously it’s going to help their customers—they do it for everybody, which I think is a really great attitude. And I love that kind of open attitude. But it will improve your campaigns and why if you’re going to learn all these wonderful things, not make use of the company which has provided you with all this incredible information.
And as I’ve said many times before on this show, you can get two thousand subscribers on your mailing list and not pay a penny for MailChimp.
Marcus: Nothing at all.
Paul: Nothing. Exactly. I like good deals like that and for a long time… they suck you in mind, it’s evil really. You build up your mailing list and of course they’ve got all these export options, you could in theory take it away. But you hit your two thousand as I did, and it’s like ‘I can’t be bothered to move, I’ll just stay here’. And I’ve been happy with them ever since. So there you go. So they suck you in.
Marcus: You just called our sponsor evil.
Paul: Yes. Evil suckers.
That’s probably not a good thing, right?
Marcus: Well I don’t know. Depends….
Paul: They’ve got a sense of humour.
Marcus: I’m sure they have.
Paul: Yes, they have a Chimp as their logo…normally that’s doing some kind of rude activity so they’ve definitely got a sense of humour. So find out more about them at Boagworld.com/MailChimp.
So that’s that. Yes, so interview…. Leisa—I’m terrified of Leisa’s second name–…
Marcus: Do you want me to say it?
Paul: Yes, go on.
Marcus: It’s Reichelt.
Paul: Oh I did say it right then. Because I say it in the pre-roll and I was wondering if I had gotten it wrong and was going to have to re-record the pre-roll.
Leisa Reichelt is on the show today. It’s a very German name isn’t it…
Marcus: It is, yes.
Paul: …except she’s Australian. Living in England. Very confusing. But she’s absolutely awesome. Leisa is brilliant. She’s got an amazing mind and very, very insightful. You can tell she’s an incredible person because she was the editor of my book, Digital Adaptation.
By editor, I mean kind of technical advisor editor – not technical as in coding but I passed everything I did by her because she’s done stuff that… you know, she’s lived what I was writing about, on a daily basis, so I wanted her opinion of it all. So she must be amazing if she’s associated with me.
Marcus: [Sighs] Yes, yes, yes.
Paul: People expect a certain level of arrogance from me.
Marcus: It would be wrong without it.
Marcus: They got a bit of trouncing recently…
Paul: Yes, on Reddit. Yes. It made me a bit annoyed that post. I think… basically there is a load of people that moan about it. And if you go in the comments there are a load of people who are moaning ‘it can’t do this, it can’t do that and whatever’. Now, the problem is, is that in every single case I was reading about on there, it was either people that had had a problem in the early days when they first set up the website – and they always said from the very beginning that this is a site that evolves and improves over time. So for example, I had an argument with somebody, ‘I found it really difficult to find out what age you can start to learn to drive’ and I was thinking ‘That’s a really main use case, I can’t believe that’. And I went to Gov.uk and I typed in age of driving and it instantly turned up the right result. And I said ‘Oh when did you have this problem?’ ‘Oh like 1996 or something’. No, it wasn’t that far back, but it was in the very early days of gov.uk.
So there’s that kind of thing. And then most of the complaints are really about obscure edge cases and the argument is that the gov.uk site is dumbed down. It’s not serving these kind of more obscure use cases that exist. But the truth is, I don’t see why 90% of people should suffer for the needs of 10%, and if 10% have to work harder to find what they are after then I think that’s fine. That’s just good web design in my opinion.
Marcus: Good web design surely has reduced that to 0.1%?
Paul: And they will do, but it’s going to take time for you to get to that. And it’s always going to be the case that the only way that you can make the use case simpler for the majority is to make things slightly harder for the minority to find. It’s just the reality of it. Or, they’re promoting sub domains or referring people directly to specific parts of the site and that kind of stuff. You cannot have one site that does all things for all people but on the other hand, having a plethora of different Government sites makes it hard for the vast majority of people to find anything because they are going from one site to the other.
So yes, it’s an interesting debate isn’t it.
Marcus: It is.
Paul: And actually I think I’m even more pro-Gov.uk than Leisa is. Because of course Leisa is seeing the nitty-gritty behind the scenes all the time and I don’t think she fully appreciates how good a thing they have managed to create together, at least in my opinion.
So she is basically the Head of User Research at the Government Digital Service and she’s leading and building a team of User Researchers. I just think this is great. They have teams of User Researchers. How cool is that? Most organisations won’t bloody pay for one Usability test session let alone have teams of User Researchers. And they work as part of their agile multi-disciplinary teams as you’ll hear in a minute in the interview.
She’s a regular speaker, work-shopper, all that kind of stuff so I’ve got so much time for Leisa, so here’s the interview I did with her.
Marcus were you in on this one? You weren’t were you?
Marcus: I was yes.
Paul: You were?! Oh ok.
Marcus: I think.
Paul: So you’ve no idea, have you.
Marcus: No I am pretty sure I was.
Paul: Anyway, here’s the interview.
Paul: Hi Leisa, it’s good to have you on the show. Thank you so much for coming on. It’s very much appreciated.
Leisa: That’s my pleasure, thanks for having me.
Paul: And it’s really nice to actually talk to you… like, we still haven’t met face to face, but we’ve mainly communicated via you picking apart my last book.
Leisa: That was very fun though, wasn’t it?
Paul: It was actually. I really enjoyed the entire experience.
Leisa: It was good.
Marcus: Sorry Leisa to interrupt. I’ve edited your work many times in the past, and I wouldn’t describe it as fun, Paul?
Paul: Yes well, Leisa didn’t have to edit it in terms of grammar and things like that.
Leisa: I tried to, but he kept stopping me.
Leisa: Yes, that was good. And also we were using that nice software that doesn’t exist anymore.
Paul: Yes, Editorially.
Leisa: Yes, that was kind of fun as well.
Paul: Yes it was. It’s a nice bit of software.
So obviously we’ve got you on the show to talk about User Experience.
Paul: And in particular, User Research. Because…. I get very confused… what is your job? You work for Gov.uk.
Leisa: I work for Government Digital Services (GDS) and one of the things that GDS does is Gov.uk. We do lots of other stuff as well.
Paul: I knew that.
Marcus: You didn’t.
Paul: So what’s your job title and what does that actually involve?
Leisa: I am the Head of User Research here at GDS and that involves lots of different things. One of the main things that it involves is trying to kind of grow the practice and the capability within GDS and more broadly within Government around doing User Research when we are designing Digital Services. So that means trying to make sure that the right people are involved in projects at the right time and doing the right things and that people who have done projects have got kind of a better idea of what good looks like when it comes to doing User Research in their projects as well.
And then we have got a community of User researchers within GDS and also throughout Government and we learn a lot from working with each other about how to do our jobs better and in particular, how to do them in a Government context in particular as well. So there is all that, and giving people lots of different advice and sticking my nose into projects where probably people don’t want me to stick my nose into, that kind of thing.
Paul: So I mean you talk about User Research. That’s a relatively—as a discipline to be a User Researcher—is a relatively new thing. As we’ve gone along in the web I think we’ve become more and more specialised haven’t we. You used to be a User Experience designer and so to see it so…. I’m just waffling now… it’s not a term I’ve come across as much in the past as I am coming across it now. And I am interested in… do you feel it’s evolving as a separate discipline outside of larger organisations or is it still the preview of big organisations such as yours?
Leisa: No, I think it’s probably been called different things in different places at different times. But I think this is what we do, we rename ourselves all the time. But there’s a long history of people observing how people are currently solving problems and how effective the new solutions are meeting people’s needs and solving people’s problems. We’ve been doing that since forever.
Paul: Yes, that’s true.
Leisa: And so people who would do this kind of work would often come from a background of anthropology maybe or psychology, that kind of thing. Doing research in those kind of contexts goes back since – you know, I’d hate to think how long a time.
Leisa: So for me it’s like applying really quite old techniques but just in a relatively new environment of Digital Services.
Paul: Yes. I think more for me, it’s been that seeing that as a separate and unique full-time role within a web team, rather than something that’s tagged onto the edge of a designer’s job or whatever else. That it’s becoming seen as a more and more important component, that you need somebody constantly researching and understanding the User’s behaviour.
I think that was what I was driving at basically.
Leisa: I think you’re right about that. I think that organisations have to reach a certain level of maturity before they recognise the value of doing this kind of work in an ongoing way brings to the need to design good services. So I think that we’re maybe much more familiar with maybe doing a bit of kind of upfront research and then doing some usability testing at the end. But that’s…
Leisa: Well I was going to say it’s like an immature sort of stage that companies kind of get to where they are dipping their toes in and hopefully they will continue to mature if they see the value of it and do it in a richer and more useful way. But there are plenty of organisations who do both of those bits not particular well and so it’s going to be difficult for them to kind of move forward.
Paul: So talk us through then, what your more matured role looks like. In terms of what are you actually doing? What types of research are you doing? How does that fit alongside the kind of project lifecycle and that kind of thing?
Leisa: Ok, so for the way that we work here, we start with a discovery phase at the beginning which is a relatively short phase. Probably eight to twelve weeks for a discovery phase, and in that stage what we are doing there is contextual research. So going out and trying to understand how people are solving the problem that we want to help solve, at the moment.
Leisa: For Government and for loads of other organisations as well I think, you’re not solving a brand new problem. Usually you are trying to solve a problem that already exists. And so people are doing the job that you are trying to help them do more effectively or more delightfully or however it is you are trying to help them do it. And going out and understanding what they are actually doing right now. What their real problems are, whey they really experience right now, is a really good way to start your project. Because if you start in a place that’s not based on reality then it’s kind of difficult to get it right after that.
So that’s the first thing that we do is go out and understand what are people actually doing when they’re trying to do the thing that’s going to involve the service that we’re designing. Trying to understand what the needs are that they have so that we can understand how to design services that will meet those needs. So that’s a lot of contextual research, it’s going out and into the place where people are doing the thing that they are trying to do, so that could be in their home, or in their office or it could be in a passport office or it could be in lots of different places. So that’s the kind of the first stage and from that you’ve got this foundation of understanding what the user journey is and understanding what the user needs are and then you can start thinking about what service are we going to have. Are we going to design a solution that’s going to make this experience better for the people?
So that’s stage one, and then what we go into is the stages where we start deal with solutions that we think might meet the need. For us that starts with Alpha where we really quickly put together like a proof of concept, or a bunch of proof of concepts to see which one has got the best legs on it. And then when we are doing research we start to work really closely with the designers and the front end developers and the content people and the product designers in particular to make prototypes and put them in front of people as quickly as we can and get feedback on what’s working and what’s not. What’s understood, what’s not understood and so we can do a iteration of the prototype and take it back out again and start to get this iterative cycle happening really quickly because that’s how you get to the best design outcome. Is by getting it wrong really fast and really early and learning why it’s wrong and starting to experiment with how you can make it better along the way.
And that’s pretty much what we do from thereon. So what we try and do here is make sure that within every iteration which is roughly about every two weeks, you are doing a round of research. So you are not doing too much designing before you are putting it back in front of people again and getting it that reality check that you get from Users. And so we would very often get five or six people and bring them into a lab and do a mixture of a kind of interview with them to understand who they are to understand the attributes that they have that can impact how they interact better or worse with the service and then we would give them some basic tasks, usability stuff and see how they go with what we’ve designed so far.
And we might start to do that just with paper to begin with. We move pretty quickly into code here because we’ve got proper multi-disciplinary teams so it’s easy to get things into code quickly so we tend not to go down the [*unintelligible] path too much.
And then as the project matures you get opportunities to do different kinds of research to solve different kinds of problems. So you might have something, you might have to make a really important design decision that you’re just not going to be completely convinced that with seeing five or six people whether or not it’s the right thing to do, so we might do a larger scale piece where will put a prototype online and put lots of analytics into it and put some survey piece at either end of it and run hundreds of people through that to see what behaviours they exhibit and how that impacts the way that they comprehend and respond to the prototype and various things like that. So that’s sort of a larger scale test that we can do that will give us greater volumes but is still very tied into the actual design decisions that are being made.
So that’s the running cycle of things that we do, there’s a huge toolkit of different kinds of research that you can do and it’s just a matter of trying to match understanding what your research question is and thinking about what’s the best way that we can do some research to answer that question.
Paul: You talk about different tools there and having a toolkit on stuff to draw upon. You’ve already mentioned lots of analytics, usability testing…
Marcus: Can I ask just a quick question about usability testing? If you get five or six people are you trying to fit people very closely to the actual end user or are you just getting anyone in to test what you are designing and anyone’s better than no one kind of thing?
Leisa: No, we tend to take our recruitment pretty seriously so we want to make sure that the people coming in are representative of those who would actually use the service. So if we’re designing a service for Farmers to use then pretty much what we spend our time doing then is going out and finding farms and trying to hook up to the internet from the farm and then doing the test there. Or we use recruitment companies, research recruitment companies and then give them a screen and say ‘Here’s the kind of people that we want’. And they will find those particular people for us and so we try… I think it is really important to make sure that the people you are seeing are representative of the audience that are going to be using it. And even when you’ve got the whole of the UK audience, which we often do, there’s going to be little sub-sets of people in that that are going to have a particular kind of need.
For example when we were doing the design work for the individual electoral registration there were loads of little sub-groups within there that had specialist needs like kids that were off at University for example. Where they kind of have two addresses, so where do they register to vote and how do we design that so it makes sense to them, and people who were in the military and people who were abroad and all these edge cases. And so we needed to go out and find those specific people to make sure that the way that we were designing it worked for them as well as for the general public.
Paul: I mean, that sounds like a massive undertaking. To deal with all those different edge cases and to find those people. Because that’s always the big one for a lot of organisations that stumps them, is getting hold of people to test that are representative. Especially if you are talking about a commercial organisation. They can’t really wander up to their clients and say—they probably can to some of their friendlier clients—but then there is how do you approach prospective clients to do it? It’s quite a tricky business, recruitment, isn’t it?
Leisa: It is, and it’s a real skill too. So I guess the first thing is you need to be really clear about who it is you want to talk to. So writing a screen up that’s going to get you the people that you want is a bit of a skill. It’s not rocket science, it’s not that hard. As an organisation you should kind of know who would be people who would most likely potentially become your customers in the future, your clients in the future. It shouldn’t be too hard to make a profile of that and go and find them. If you’re commercial, the tricky thing can be about how open you want to be about you as a brand, exploring the different kind of directions that you are going in? So that’s the theory, some people get a bit sensitive about that but that’s not too difficult to manage. You can make up a fake brand, pretend you are a new company. You lose a lot of the benefit of, or value that your brand would bring, but it’s really doable.
The thing is that you want to start at the beginning with the easiest possible people. So don’t worry at the beginning about identifying all of the sub groups and all that kind of thing because you’ll just get to them as you need to. If you start to think about all of that complexity you won’t do anything. So we always start with the low hanging fruit of all of the people who should be able to do this and should want to do this, go and find a handful of them and start with them and you will still learn so much from them. And then once everything is fine for them, once you’ve got the experience working for them really well, then well go ‘Ok, who else? Where else do we think there should be some complexity?’ and go and do that. So that’s the way that we approach it.
The more you learn about what’s working and what’s not working and where the tricky bits are, the process will teach you who you should go and see next.
Paul: Yes. I can imagine that. I mean, there must be a degree as well where it depends on what you’re trying to find out as to what kind of people you need in the sense that if you are doing straight usability testing it may be that the demographics on some of that usability testing is less important than say if you were trying to do user research and you were trying to understand the user better. Because some problems you encounter in usability testing, it doesn’t matter who you are you will still stumble at the same kind of hurdles. Is that a fair comment?
Leisa: I think so, I think so. But you’ve mentioned demographics I think that a lot of the time we get hung up on demographics when really they don’t actually really matter that much.
Paul: Yes, that’s what I was getting at really.
Leisa: You want to pick out the things about ‘What is it about the people who would use this service that makes them different to other people’. Then are there any kind of variability’s within that that’s really important. So for example if we’re going and seeing students for electoral registration, it doesn’t matter to us whether they are mathematics students or humanities students, that doesn’t matter. It doesn’t even matter if they are male or female. It probably doesn’t even matter how old they are, probably the main thing that matters is that they’ve got two addresses. Maybe it matters if they are in London or in a regional area.
Leisa: So you want to pick out only the things that really matter. You want to reduce the complexity as much as you can. Because it’s more important that you do some of this, than you do it perfectly.
Paul: Yes. Because I do think a lot of people give up on this kind of stuff because they are overwhelmed by the potential complexity of it all and so they just throw their hands up in the air and go ‘There’s no way I can do this’.
Leisa: Exactly. And that’s one of the things we’ve done a lot here is really try to simplify the guidelines of what to do. So we say make sure you’ve got a researcher in the team for at least three days a week, make sure that you are doing research every two weeks, make sure that everyone in the team is coming and watching the research at least two hours every six weeks. So we make these little sort of guidelines and in those two weeks, get six people in and do a little bit of interviewing at the beginning just to understand who they are and what they understand about the service and do some task-based stuff and do your analysis like this. So we try to make it as simple as possible and then once you start doing that you learn enough about it that you can go, ‘Ok, we need to try something a little bit more complicated now’.
So you start with something really, really simple and just get the ball rolling. And that’s the most important thing. Once you get started doing this, you get a bit addicted to it and you want more and you learn more and you get better at it. Even the best teams, if they stop doing this, it’s really hard to get back into it again, so you just need to keep your hand in and keep doing it on a regular basis.
Paul: It’s interesting isn’t it. One of the things you touched on there was getting the whole team to watch the usability testing, so there’s another aspect to this which is not just doing the research, but then communicating the research and allowing that to impact the final product rather than it just being…
Because a number of times I have known people that have created personas and then they’ve been shoved in a draw somewhere and then forgotten. So how do you ensure that all this research you do actually makes a difference to the final product?
Leisa: So that’s incredibly important. That’s the thing that I think we are most concerned with, is to make sure that we’re not just doing this for the sake of it. And we’re not doing stuff for people to go ‘Oh that’s a really interesting story’ and then just getting on with what they were going to do anyway.
We were talking about this at the end of last year and thinking about what’s the portion of time that we spend doing the different kind of things that we do as researchers? And we worked out that probably we spend about 30% of our time doing the research and about 30% of the time trying to communicate the research. That’s kind of what the breakdown is. So when people say ‘I don’t need a researcher full time in my team because I only want to do a bit of research once or twice a month’ that’s what they spend the rest of the time doing, is making sure that they actually get value from the research. And it takes a lot of effort.
So for us, things that we do is that we really, really, really encourage as many people in the team as possible to come and watch the research on a regular basis. If you’ve got smart people in the team and they are watching people use the service regularly you almost don’t need to do any analysis because they will see so much stuff that they really understand is problematic. In particular getting Senior people to come along regularly as well, gives them a realistic sense of how well the project is going.
Leisa: Which is really important and can help them make good decisions about budgets and timelines and how they talk about the project to their bosses and all those kind of things that can get projects in trouble frequently. Getting people to come and watch research is really useful. The other thing is that a lot of the time, the things that cause you big problems with user experience are not things you can solve through the interface. They’re business process things, they are marketing things – they are stuff that you can’t fix on a wire frame or a prototype. So getting more Senior people in the organisation who’ve got a [*unintelligible] across the organisation not just the digital side of things is really useful for them unblocking those other aspects that are causing you problems that you can’t solve. So that’s really important to us and we always, you know, we’ve got posters that say ‘User Research is a team sport’. That’s something that’s really important to us. We always say that we don’t research for us, we do research for the team and our job is not to understand End Users, our job is to make sure the team understands End Users and how well our services are meeting their needs. So we try to think of ourselves as being facilitators rather than owners of the research if that makes sense?
Paul: Yes, yes. Absolutely.
Paul: So what do you do beyond the, getting somebody to come in and sitting in a usability session—yes that makes perfect sense, I can understand that—what about in terms of getting a sense of who these people are that you’re serving? Because that’s where you really start to empathise with people, that’s where you start to think of the nuances of their context etc. How do you communicate that to your team?
Leisa: So again wherever possible we get them to come along to contextual visits as well. So we’ll try and have… we’ll set up a series of interviews and we’ll get one person at a time to come with the researcher and be like a glorified note taker. So they might take some notes, which would be useful, but most of the time what they are there for is really just to actually see for themselves what a real person is like and what a real lived experience is like for these people. The other thing that we do is try to tell a lot of stories, so we tend not to use personas a lot. Some projects do but that’s not something that we do a huge amount of. But what we do instead is tell a lot of individual stories. So we’ll come back and we’ll say ‘We went out and we met this person and this is their story and these are the problems that they came up against and these are the pictures from inside their home…’ and all that kind of stuff. So really bring that to life. And that works really well. People love getting the… at the moment, the way that we do it is that at the end of the day of research we’ll sit down and just write the stories. ‘We went out and saw Hannah and this is her story…’ or ‘We went out and saw Bob and this is his story…’ And just email that out to the team that’s interested in the research for the project and that’s really powerful.
And then the other thing we like doing is try to build models of what we know as well. So try to condense all this down into a way of visualising the knowledge so that it’s something that people can draw. And that’s quite helpful as well. So it might be the stages of the User Journey that people go through or it could be a model of different information sources and which ones are trusted and which ones aren’t. Various things like that. So it’s breaking things down into like nuggets of knowledge that people can access easily when they are thinking about how to do the design. It’s not dug into a persona too much.
Paul: I mean it’s interesting. You’ve talked a lot about going and meeting with users, and interviewing users and going to their homes etc. What about the analytic side of things? Because data can become very overwhelming and very abstract for people so do you use that much when you are communicating with your teams, or is that when you are getting more into specific ‘People are dropping out at this point, we’ve got a problem here’ kind of stuff?
Leisa: Yes. We use a lot to support what we are finding in research. So it’s really important that we can triangulate what we think we know from what we are observing and what people are saying with their actual real behaviour as well. So yes, we use the analytics on the site to tell us what people are doing to confirm whether or not different things that we are seeing. Particularly now that you go and you do usability testing and you see five or six people do something, you think probably that’s a thing and you can go back and have a look at the analytics and see if anything’s there to support whether or not that problem exists or that behaviour exists. So that’s a good way of helping to validate what you are seeing in the research.
We also have little feedback forms on the site as well, where anybody who is using it on a regular basis can kind of say when something went wrong, theoretically they can say when something went right. And going back and kind of looking through all of that as well is a really good way to triangulate the data and the research with the feedback and make sure that everything makes sense together.
Because the thing with data is that it really hinges on what question you ask.
Marcus: Totally. Yes.
Leisa: So then you’re looking at data, thinking that you are trying to prove something, and it’s almost like you can make it tell you what you want it to tell you at the time. We’ve had a few examples where we’ve looked at the data and the data has told us that lots of people are going there, so we think ‘Well that must be working really well, because lots of people are going there’. But then you look at the feedback and you realise lots of people are going there, but they don’t want to be there. They actually want to be somewhere else but can’t find that other place.
So if we don’t use the difference sources of insight then we can look at the data and go ‘We’re awesome’ where actually we’ve completely stuffed something up. So it’s about using it as part of the mix.
Paul: Yes, I can understand that. Yes. It’s asking the right questions, and it’s being very careful about how you interpret the results. I like the idea of using it to validate stuff as much as anything else. I think that’s a nice way of viewing it. I mean ‘Are we correct in this hypothesis? Is there data to back that up?’ kind of thing.
Leisa: So it’s hypothesis during design that I think is really important and that’s where doing research as well and having the designer and the researcher and the analyst all working together is really powerful. Because you can identify a problem in usability testing and so we’ll think this is a problem. You’ll look at the analytics and see that ‘Yes, it looks like it’s definitely a problem’ and then you sit down with the designer and go ‘Ok, how are we going to fix this, here are some ideas’. And you can sit down together and go ‘If this is fixed, this is what we would expect to see. We’d expect to see this in usability testing and this in the analytics’. That just gives you a much more rigorous way of making sure we are actually solving problems and we’re not just shifting deckchairs.
Paul: I mean, one of the challenges that you must face is, and that you touched on this earlier, is that you must reach a point with some of the problems that you are trying to solve where it’s not just within your remit to solve them. For example if you are doing something like Tax Discs for example, you can create the ability to apply for Road Tax and all the rest of it, but that’s going to go into a back-end system that’s going to affect legacy stuff that’s been around for years and not just from a technical database point of view, but the way that the Road Tax people work and it affects work flows and all the rest of it. And so how far can you push it? How deep do you go into all this kind of stuff?
Leisa: As far as we possibly can!
Paul: Good answer.
Leisa: We deliberately talk about a Digital Service Design because we understand that the work that we are doing fits into a broader context in the way that services are being delivered. As you know, you go in some usability testing and that kind of stuff, you discover loads more issues than what you get just from the problems that the interface is causing. So yes, this is why we try to make sure that we are bringing along colleagues from other parts of the organisation and not just confining it to Digital. We want to try to make sure that we’ve got policy people coming along, we’ve got people at the service management level coming along. Things like security causes us all manner of problems all the time, but you just get the security guy to come along and see the problem so that’s amazing how they can just go ‘Well actually we don’t really need to do that, we can do this instead’ and problem solved. But unless you can get that person to come and feel as though they are part of the team and to observe what we actually experience here… and it’s so much better for them to actually come along and experience it for themselves than it is for you to go back and present it to them. You tend to get a much better response because you’re not railroading them into having to make a decision or putting them in a situation where they’ve done something wrong and you’ve discovered it. You don’t want to be the usability police as such.
Paul: Yes, you want them to discover the problem for themselves almost.
Leisa: Exactly, and for them to be able to say ‘Oh I know how we can solve that’ rather than you going ‘There’s a problem that you’ve caused’. So getting those people to come along and observe is really, really, really useful. It’s part of the reason we try not to have teams of job titles that include the term UX We try not to have a UX team or we try not to have UX designers, or UX researchers or UX this or that or the other, is because of the massive impact that people in policy and procurement and security, just to name three, but there are loads more… you can imagine how this would play out in a commercial environment as well. There’s often people who would have a huge—a huge—impact on user experience and it’s much easier for you to say we’re all responsible for the User Experience if somebody is somebody’s job title isn’t User Experience Designer or User Experience Architect, or User Experience Manager or the User Experience Team. It’s much easier to dial out of that and go well I’m just [*unintelligible].
Paul: Yes… ‘It’s not my problem’.
Leisa: Exactly, exactly. So if you really believe the User Experience is the responsibility of the entire team—as we do—then it doesn’t really make sense to have a team who’s called User Experience.
Paul: Which I totally agree with, but I can think of two potential problems that I am interested in how you guys solve.
One is, if everybody is responsible, then nobody is, if that makes sense? The problem of things falling between the gaps because there’s nobody that’s got ownership over that. And the other potential problem is if you are including all of these people, and I totally understand and accept that that’s a good idea, how do you remain agile? How do you keep the speed and momentum going with what sounds like a lot of meetings?
Leisa: So, the person who is ultimately responsible in our case is going to be the Service Manager – the person who is ultimately responsible for the service, is ultimately responsible for the experience and the performance of that service.
Leisa: It’s their job to make sure that resources are allocated properly and that things are prioritised properly. So ultimately for the whole coherent experience, they are the ones responsible.
Obviously you still need a good interaction designer, a good researcher, a good front-end developer. You need all the same people that comprises what you would call elsewhere a UX team. Those people still have got the same responsibilities as they would be acting on normally anyway, so I don’t think things fall through the cracks. What you are doing is elevating the responsibility for User Experience, higher up the hierarchy. And I think the higher up you can get somebody to really take ownership and responsibility, the better the outcome is going to be.
Paul: Yes. I can see where you are coming from and that. The only thing with pushing it up the hierarchy potentially is that it’s only ever going to get part of that person’s attention because they are going to have other responsibilities and stuff, so there’s a potential risk there I guess. But the idea of having a Service Manager, that makes entire sense to me.
What about the other issue of involving that number of individuals in the decision-making process and in the team? How do you prevent that from impacting speed of movement so to speak?
Leisa: I think… what do I think? I think that…
Paul: The answer might be it does, and you just need to live with that.
Leisa: The thing that I’m thinking about is that it’s really easy to focus on delivery and saying we’ve got to deliver lots of stuff and here we are moving the cards across the board and therefore we are delivering a lot of stuff and our velocity is high. But if what you’re delivering is something that is really fundamentally flawed and you’re not addressing the key issues that are going to define whether or not your project is successful or not successful then you are going to go back and either fail or have to redo a bunch of stuff anyway. So you’re going to have to have those meetings sometime, so the earlier you have them the better and the earlier you have them, the shorter and easier they are. The other thing is too, if you can get all the team into an experimental mind-set and you can explore different ways of doing things without having to ultimately commit to them. Then people are really much more open to making the decisions and seeing how they go and coming back and having the option to change their mind, having learnt a little bit more. So you don’t have to sit down in a really hard core heaving meeting to make the decision. You just need to make sure everybody is aware of what the problem is, come up with some ideas as to how you might solve it, start seeing how feasible and successful they are going to be and then revisit it again. And as long as those people are actively engaged in the project on a regular basis anyway, it’s not been my experience that you end up having loads and loads more meetings than you otherwise have.
Paul: I think the key there—that I really liked what you said there—was this thing of you’re not including these people where they have to sign off finally and commit themselves in blood to this and I think that really does make things much more fluid and much more lightweight when people don’t feel they are signing their life away and people are committing themselves to something major that might come back and bite them. I totally agree that people are a lot more flexible when they don’t have that pressure placed upon them.
Leisa: I think also the continual involvement throughout the project is really important. If they have to come into a room and brought up to speed and convinced and all that kind of thing, then that’s really time consuming.
Leisa: If they acknowledge that there is an issue because they’ve been aware by coming along to sessions and being actively involved in the team, you can skip that whole part of the meeting and just start digging into ‘Ok, what are some things that we can do?’.
Paul: Yes, that makes a lot of sense. Leisa thank you so much for coming on the show. We need to kind of draw it to a close, but I could carry on talking about this kind of stuff forever. It’s the kind of stuff certainly Headscape encounters a lot, and I do as well. So it’s really interesting to hear about it, and thank you for spending the time talking with us.
Marcus: Yes, thank you.
Leisa: Thank you. Bye. Thanks a lot.
Paul: So that was our interview with Leisa. I am now sure that everyone listening to the show is in complete agreement that she’s amazing. So that’s Leisa.
Marcus: Yes, and everyone wants to have a whole team of User Researchers.
Paul: Yes. Oh what I could do with a team of User Researchers…. I don’t know, but I am sure it would be good.
Marcus: You’d research things.
Paul: I would research stuff – I would research the hell out of stuff. I would know so much about Users.
Marcus: You’d research things to within an inch of their lives.
Paul: I would do. It would be research-ageddon or something. I don’t know.
Anyway, let’s talk about our second sponsor! Our second sponsor is MediaTemple. The thing I want you to check out this week, so stop whatever you are doing—oh no, you are listening to this podcast—after you have finished listening to this podcast, go and check out MediaTemple’s new WordPress hosting service. I think I need this actually. I think what I’ve got at the moment might be a bit overkill, and this is better suited to me, but I am lazy and I probably won’t get around to moving. But anyway.
It’s a new WordPress hosting service and they’ve got a lovely little package that they give you. Ideally suited if you run a WordPress site, so they also just throw in as an aside, Google apps which obviously I am using in order to do my email and that kind of stuff and you guys at Headscape use it as well. But as part of their kind of WordPress package you’ve got automated WordPress updates which just kind of happen, which is wonderful. They are constantly scanning your plug-ins to see if there are any malicious plug-ins that you ought to remove. They even through in an SSL certificate which these days are really important because Google have started to take into account whether you are a secure website or not in terms of their ranking. So that’s a really cool added thing they throw in. It has daily back-ups which, as someone as I said in last week’s show, completely ballsed up my website, daily backups just being done in the background would be very nice indeed. I mean, we do backups, obviously, but it’s nice that Media Temple do it automatically for you. And then there’s this whole kind of set of developer tools that allow you to design, build and deploy your WordPress website like a Pro. They’ve got One-click staging, site cloning, SSH access, GIT integration all built into this single nice, custom control panel that looks really good. So if you fancy trying that out there’s a special discount code that you can use as a Boagworld listener which is the Promo code: BOAG which you can use to get 25% off your web hosting. Go to Boagworld.com/Media Temple and enter that Promo code at signup and then you’ll get 25% off at that amazing deal.
I wonder if I’m allowed to use my own promo code? That’s an interesting one.
Marcus: I am sure you could use it, it’s whether you should…
Paul: Yes, I might have to ask politely if I am allowed to. Because this package is perfect for me. But anyway.
Marcus do you have a joke for us?
Marcus: I do!
I found a Tommy Cooper Twitter account. I don’t think it is actually Tommy Cooper because he is dead.
Paul: He is dead. So that would be quite remarkable.
Marcus: But it’s just an endless source of Tommy Cooper jokes. Rather good.
Paul: Did you find that yourself, or did someone send that to you?
Marcus: I found it all by myself.
Paul: Well done. Have you been getting any jokes through since our appeal?
Marcus: One person sent me some jokes, but a couple of them I’ve said before and another one wasn’t… I didn’t think was funny. I’m quite hard to please on the jokes front, despite me saying earlier how easy I am to please.
Paul: So we need more jokes to firstname.lastname@example.org
Marcus: Yes. Yes please!
Paul: So go on then drum roll
Marcus: I went to the doctors with jelly and custard stuck in my ears.
He asked ‘What seems to be the problem?’
So I said ‘I’m a trifle deaf’.
Paul: Yes. Tommy Cooper is kind of better… Tommy Cooper jokes are better when they are told by Tommy Cooper.
Paul: There is something about his delivery. Hey I know. Because some people might not know Tommy Cooper, so we need to put… there must be videos of Tommy Cooper on youtube?
Meg! Who does the transcriptions for the show – hello Meg. Can you find us a Tommy Cooper video on youtube, cause I am too lazy to do it myself and so I am going to get Meg to do it and put it in show notes.
Marcus: Shall I do another one?
Paul: Yes do another one.
Marcus: I visited the offices of the RSPCA today. It’s tiny, you couldn’t swing a cat in there.
Paul: Haha, now that’s funny! Now you do need to know that the RSPCA is the Royal Society for the Protection of Animals. I like that one. That’s good.
Marcus: There you go.
Paul: Well done, you’ve made me very happy now. Alright Marcus, well go away because I’ve got a distinct feeling you are going to infect me over the phone.
Marcus: No, I am way past being infectious.
Paul: Listening to you cough is disgusting enough.
Marcus: Snivelly me.
Paul: Yes, absolutely.
Marcus: No I’m alright. I’m fine. So have a lovely time in the West Country and say hello to my ancestors. My ancestors? Yes.
Paul: Yes you’re right. And I’ll talk to you again next week, and our dear listeners as well.