Usability Test Results

In the last weeks mockups for the new GUI for OB ERP were tested on 12 users. Let me share the findings with you.

What was tested: three clickable mockups

Test methodology: participants were asked to execute a number of tasks, as described in the scenarios. They were asked to think out loud while doing so. All mouse movements were recorded, as well as the audio (participant talking). A web cam was used to record the user´s facial expression. The facilitator observed the participant, took notes and asked questions in case of hesitation, mistakes and other deviations from the pre-determined scenario path. Other than with the contextual inquiries (user observations) that were conducted in December 2008, I have decided not to post-process(annotations, etc.) the videos as the findings were extremely consistent across all sessions. The sessions were split in parts: 4 tests were done at first and findings were immediately processed into a modified design (biggest change was in the ribbon toolbar). The remaining 8 tests were done using both the initial design and the modified designs.

Findings: The most important findings are:

* Users have difficulty navigating the different master detail views from parents to childs, from grids to forms and back. There is definitely a learning curve but cognitive load seems to be high. I think part of it has to do with visual design but I´m concerned that this can only fix parts of the problem.

* Users have troubles finding menu items (buttons) in the ribbon toolbar. I noticed that the labels I initially used (such as "record") were not ideal, and changing them to more common labels (such as "edit") significantly improved the user´s performance. Still, it took most users a while to get used to the concept of "buttons hidden on tabs". Some users even thought the the form or grid below the toolbar was part of the tab. Here visual design and labeling play an important role but I have the feeling we should simplify, dumb it down. Plonking all buttons (icons) in one bar is perhaps the way to go. Users don´t seem to notice a difference between generic buttons and object specific process buttons but I still want to keep this clean separation, as in the earlier concepts.

* Forms work great. Users love the layout, the nifty sections and the color coding on field and section level for required fields.

* It was not always clear when records are saved and when not. Users did not seem to worry about it at first but seem to wonder what saving mechanisms are applied and demand clear feedback on the status. Auto saving is not always good: an implicit save action is requested for forms to avoid unfinished forms to be saved or even processed.

* Users double click rows in grids

* Users get confused about which tab contains what because of missing (id) labels on the tabs

* Three users had concrete examples of use cases where flicking through the headers while observing a related lines grid update is needed.

The detailed findings can be found here.

Now the next step is going back to the drawing board and solve the issues pointed out by our users. After that, another round of usability test might be necessary. As always, you are welcome to participate in all stages of the design process by posting on the UX Lab Forum.