Testing prototypes: dreams can be really powerful

In the last article I was writing about the prototyping. Actually, we spent a lot of time doing that for mobile applications we create, as we definitely need some way to estimate our ideas before they are developed. The next step – when you have the prototype ready – to show it to your users. On one hand you want to give the feedback from people as soon as possible, but on the other hand to get tha feedback you need to show them something that works.

So how we would test our rapid prototypes with users?

The everyone’s goal is to create the prototype cheap and fast – without writing much code and then show it to people and get feedback. It is not an easy thing – minimize costs of the first prototype creation and then getting the valuable feedback. Each project usually requires individual solution, some fresh ideas.

2012-08-02 17.14.26Sketching

But there are some common ideas we can use from the people who did big job on investigation of the user testings and prototyping, when there was no even real technical solutions for their ideas.

Wizard-Of-Oz prototyping

The one thing which can help you much is that:

Dreams can be really powerful.

You don’t need much to create a “show” and make people understand your idea. Yes, it is a kind of “faking”, but it works!

In the lecture we had the example from the Wizard-Of-Oz book – the wizard seemed to be a big and scaring, but in the end appeared, that a small man was hidden behind the curtain. But for the visitor of the city his image of the scaring wizard was real. It really illustrates what you need to achieve, when you create your “fake” prototype – people should think they are working with a real system.


The idea is that you have to simulate the machine behavior with human operators, who do all the things behind the curtains.


  • Make an interactive application without writing much code: front-end interface, which is controlled by the wizard.
  • Get feedback from users: users things it is more real, but still it gives them license to suggest changes.
  • Makes sense when it is faster/easier/chipper to make the than the real thing

It appeared in around 1980 – when the speech recognition UI prototyping was performed. Wizard-of-Oz prototyping helped to understand what will be in speech interfaces. If we assume that the speech recognition works perfect – what the interfaces will be, how people will interact with them? Wizard-of-Oz helped to put themselves in the future, when the speech recognition will work and the interfaces will be needed.

Real life example:

When you have the complicated search with “suggestions” in the mobile application – you may use this technique to test what be the most helpful and interesting for users. The same can be done, for example, when you are working on the “Recommendations” of the content for the users (my team had that task recently) – and  you have to define what will be really helpful for your users and how to give them this relevant content. Building this system can take a huge amount of time and a lot of code should be implemented. But we can write that code and later will have to through it away, as the first idea can be not working with our users. So to save up our client’s budgets we have to invent some user testings to identify, if our suggestion was right. All these activities seem to be rather expensive, but it is a tiny amount of money in comparison to the sum you can lost, if your decision was wrong. So test early.

2012-09-27 17.11.57Testing 2 types or “basket”.

One more thing:

If the people like it in the super crappy form then it is worth building, because they ill like it even more when we do the real thing.

Damon Horowitz, co-founder of Aardvark

Tips on creating your own Wizard-Of-Oz prototype:

  • Map out scenarios and application flow. The main question here to answer is: what should happen in response to which behavior.
  • Create UI skeletons
  • Decide how the wizard will provide input and what kind of input the wizard is allowed to offer.
  • Rehearse the wizard role with a colleague

Tips on running it:

  • Practice before involving users
  • You should have the facilitator and wizard. it is difficult to share these roles. Facilitator provides tasks and makes notes. It is very important to make the notes of your observations. Wizard – operates interface.
  • User feedback: think aloud, retrospective, some heuristic evaluation(you create the groups of problems).

The last tip – give your “users” some “thank you” present. We provided chocolates to our volunteers in the office 🙂

Also you can use this kind of testing continuously – as your system is developed. Int he beginning the wizard takes a big part of the interaction – almost all the functionality is “simulated” by as you are developing – you can show the ready parts in combination to the “fake”:


So you fill in by the wizard only the parts that are not done yet.


  • Fast to make
  • You can create multiple variations easily
  • More “real” than paper-prototyping
  • Identifies bugs and problems with current design
  • Places the user in the center of development
  • Designers learn by playing wizard


  • Wizards require training
  • Playing wizard can be exhausting
  • Some features are impossible to simulate effectively
  • May simulate technologies that do not exist

Materials used:

Human-computer interaction course on Coursera by Scott Klemmer

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s