Some thoughts on user testing with kiosks and mobile devices
User testing sessions mark the first point of contact that our software and designs have with the people who will ultimately be using them. Showing the product to fresh pairs or eyes invariably exposes issues that were invisible to the people working on it, allowing them to be identified and fixed before the final build.
In our studio, a user testing specialist will join carefully chosen members of the general public along with a prototype version of the product. An adjacent room is filled with observers analysing how the product is being received, including the designers and techies nervously awaiting a response to their lovingly crafted creations, monitoring its triumphs and troubles through a combination of one-way glass, microphones, projectors and cameras. We’ve had a great deal of experience at doing this over the years for websites, but it’s always slightly trickier setting the scene just right for technologies that live on other platforms, such as kiosks or mobile devices.
It’s important that the technology used in the testing set-up doesn’t create a barrier between the user and their interaction with the product, but it’s something of a fine art getting this right while at the same time making sure all of the user’s performance is visible to the observers. The result is an usual mix of theatre and gadgetry.
We’ve had a couple of interesting physical set-ups recently which have given us an excuse back-stage to get our hands dirty with masking tape and glue:
The first came when we needed to test a kiosk system at the same time as a web site. We’re used to testing interactives, often for institutions like The Great North Museum or The British Library (and often involving a good number of hastily soldered LED circuits), but it’s unusual for us to have to do this at the same time as another screen. Using Morae, our house favourite recording device, was a no-brainer for the web site, which we used to transmit a view of the computer screen along with a view of the user’s face looking at it.

For the kiosk, we used a 19 inch ELO touch screen, which you can mount on a sturdy arm bracket which itself attaches to tables, book shelves; pretty much anything. We stuck a camera on the screen to get a front-view of the user’s face. We then used a separate (third) computer to record the user’s interaction with the touch screen, because of course there is no visible cursor being recorded. It was angled in such a way as to record the screen’s display, but also to see the way users tend to hover their fingers above the screen before committing to a press. These details are often very revealing about how comfortable the user is with the interface.
Here’s a schematic of the final set-up:

We’ve also recently held some fascinating user testing sessions for an iPad application we’re developing. The main issue with this is that it is essential that the user should be able to pick up the iPad, and hold it in a natural way. However, the majority of positions that people hold the device in means that it is very difficult to clearly record what is happening on the screen. We looked at just using some screen capture software, but this would not record people’s finger movements in the way we wanted.
Then we stumbled upon a very useful online tutorial which described how to make a sled for testing on iPhones. After some creative discussions and a few hours making a lot of noise in the office, and we came up with this:

To the best of my knowledge this rig is a world first, and will pave the way for DIY iPad sleds of the future. Here’s a quick run down on what we did:
- Find a piece of hard plastic, preferably not too psychedelic in colour.
- Heat a small area of it over the heating device of your choice (you can buy some expensive equipment for this bit, but with a little imagination you need not leave the office…)
- Slowly bend the plastic to your desired shape.
- Drill a few holes in, and use cable-ties to attach a lightweight camera to it.
The results are exactly what we were hoping for: there’s relatively little wobbling, and with the right lighting you can get a perfect view of both the screen and the user’s fingers, no matter how they choose to hold the device.
I’ve no doubt that with the all the interesting mobile devices of the future being prophesised at the moment, this won’t be the last device-specific rig we’ll need to make.

Comments
Great blog.
One of the aspects of testing that often seems to get in the way is the recording device itself, whether it's a person with a clipboard in hi-viz or other.
I wonder how the i-pad sled affects how the user holds and interacts with the physical, and hence the interface itself?
With kiosk testing, how do you get around the users feeling like they are being tested, and therefore introducing hesitation?
I agree; the recording device does create a barrier, and ultimately it will always need to be taken into account that the testing room isn't a perfect reflection of how a device would be used at home. No matter how cunning the technology, I think it's the facilitator of the session who has the most influence over how relaxed and natural the recruit performs.
In this case, the iPad sled was pretty light, so the user could pick it up and rotate it fairly naturally. A small hindrance was the wire coming out of the camera, and in future sessions it would be nice to use a camera with Wi-Fi to avoid this. Aside from that, the user could do pretty much anything apart from spin the device 360 degrees (a dangerous and increasingly popular act: http://il.youtube.com/watch?v=c3JHGVSSW9w).
For kiosks, we've used various methods for different set-ups. What we nearly always do is introduce the environment before the testing room is even entered - tell the recruit to imagine that they are going to enter a particular museum (for example), and then see how they react when they enter the studio and see the kiosk. We often add to the illusion by putting some relevant print-outs on the wall that might exist in the room that the kiosk will eventually live in. This role-playing also adds a level of humor to the session, which helps the recruit relax a bit.
Kiosks are particularly difficult to test on, and again I think it's the skill of the facilitator that makes the real difference.
Thanks for the response Joshua – I can see that humour is an important tool in relaxing the recruit – and I can see that being honest about the test, recreating elements of the environment and effectively saying to the recruit, "we're not perfect either", would give them confidence and relax them, rather than over doing it and attempting to present a perfect illusion, and creating something more like an exam.
[...] Some thoughts on user testing with kiosks and mobile devices | Cogapp cogapp.com/blog/some-thoughts-user-testing-kiosks-and-mobile-devices – view page – cached Cogapp build websites, interactive installations and next-generation mobile applications. [...]
Post new comment