Usability Test Results

In the last weeks mockups for the new GUI for OB ERP were tested on 12 users. Let me share the findings with you.

What was tested: three clickable mockups

Test methodology: participants were asked to execute a number of tasks, as described in the scenarios. They were asked to think out loud while doing so. All mouse movements were recorded, as well as the audio (participant talking). A web cam was used to record the user´s facial expression. The facilitator observed the participant, took notes and asked questions in case of hesitation, mistakes and other deviations from the pre-determined scenario path. Other than with the contextual inquiries (user observations) that were conducted in December 2008, I have decided not to post-process(annotations, etc.) the videos as the findings were extremely consistent across all sessions. The sessions were split in parts: 4 tests were done at first and findings were immediately processed into a modified design (biggest change was in the ribbon toolbar). The remaining 8 tests were done using both the initial design and the modified designs.

Findings: The most important findings are:

* Users have difficulty navigating the different master detail views from parents to childs, from grids to forms and back. There is definitely a learning curve but cognitive load seems to be high. I think part of it has to do with visual design but I´m concerned that this can only fix parts of the problem.

* Users have troubles finding menu items (buttons) in the ribbon toolbar. I noticed that the labels I initially used (such as "record") were not ideal, and changing them to more common labels (such as "edit") significantly improved the user´s performance. Still, it took most users a while to get used to the concept of "buttons hidden on tabs". Some users even thought the the form or grid below the toolbar was part of the tab. Here visual design and labeling play an important role but I have the feeling we should simplify, dumb it down. Plonking all buttons (icons) in one bar is perhaps the way to go. Users don´t seem to notice a difference between generic buttons and object specific process buttons but I still want to keep this clean separation, as in the earlier concepts.

* Forms work great. Users love the layout, the nifty sections and the color coding on field and section level for required fields.

* It was not always clear when records are saved and when not. Users did not seem to worry about it at first but seem to wonder what saving mechanisms are applied and demand clear feedback on the status. Auto saving is not always good: an implicit save action is requested for forms to avoid unfinished forms to be saved or even processed.

* Users double click rows in grids

* Users get confused about which tab contains what because of missing (id) labels on the tabs

* Three users had concrete examples of use cases where flicking through the headers while observing a related lines grid update is needed.

The detailed findings can be found here.

Now the next step is going back to the drawing board and solve the issues pointed out by our users. After that, another round of usability test might be necessary. As always, you are welcome to participate in all stages of the design process by posting on the UX Lab Forum.

Try a Clickable Prototype

The design concepts for OB ERP´s new user interface are getting more mature by the day and we continue using feedback at any stage, whether it be through the forums, peer reviews or usability tests. This time I have built a clickable prototype that shows a number of tasks around expense reports. It intends to verify how well our master-detail concept (Kompressor) works in real-life scenarios.

Download the zip file, unpack and run Expense Reports Scenario.htm. I will use this for a couple of usability tests with end users but I am also really interested in your thoughts while running it.

Normally I ask users to speak out loud while performing a task but that won´t work remotely (and your colleagues/wife/husband/house mates will look funny at you) so better write down your thoughts when running the scenario and post them here. If you are a colleague of mine and want to do the usability test, just let me know and we will do the test together in the office.

Remember that we are testing our software, not you, so don´t feel stupid when you did not click the right spot at first. It just means that the design is not right yet. The user is always right!

Running the scenario more than once can be interesting too, to see if there is a learning curve.

Choose your Super Hero: Grid Splitter or Kompressor

Guys, I am torn between two concepts for master-detail: "Grid Splitter" and "Kompressor". They sound like the villains from The Running Man, I know, but other than in the film, this time we want one of them to become an actual winner. Sorry Arnold.

Grid Splitter is the concept that shows a header (parent) grid together with a lines (child) grid in one view. Kompressor only shows one grid at a time and compresses the top grid into a single summarized row when the bottom grid is activated. This way the user is not distracted by the header grid while working on the lines, which is in fact a variant of the progressive disclosure interaction design technique we already used to hide the lines grid while working in the header grid.

You can view an abstraction of their behavior in these diagrams (1 & 3 show Grid Splitter, 2 & 4 Kompressor) but perhaps it's easier to go through a scenario for the new concept (Kompressor). Use right-arrow key on your keyboard to flick through the scenario.

Choose your favorite super hero now!

If my doubt persists, I will have to take them to the streets, for some serious usability testing. There can only be one winner.

Live Search or Query Suggestions?

In the proposed concepts for a new GUI we showed a keyword suggestions feature, similar to what many other applications and sites use nowadays. You type the first couple of characters and a little flyout menu starts suggesting keywords.

Until now I assumed that these keywords suggestions should be based on a mix of popularity, recency and bookmarks. I even asked you if you liked those. This is what you said:

Query suggestions based on bookmarks
Very unimportant 0.00%
Unimportant 12.50%
Neutral 37.50%
Important 37.50%
Very important 12.50%

Query suggestions based on recent searches
Very unimportant 0.00%
Unimportant 4.35%
Neutral 34.78%
Important 52.17%
Very important 8.70%

Query suggestions based on frequency / popularity
Very unimportant 0.00%
Unimportant 8.70%
Neutral 30.43%
Important 52.17%
Very important 8.70%

[Full survey result details here...]

Compared to your appreciation for other features, you weren´t that excited about it apparently. Now maybe I know why. During the World Conference Last weekend a business partner showed us a customized screen using a sort of search suggestion based on live search. So while typing it actually searches in real time. Pretty impressive. I thought this would be too slow. Perhaps with 1 million records it would be. But look at this demo of a dhtmlx-grid (Select Loading from big Datasets > 50,000 records in grid from the tree menu on the left and tick the Enable Autosearch checkbox)...and start typing away. This rocks!

Having seen this, I can´t suppress my enthusiasm for live search suggestions but we need to make sure this is what you want. Compared to search QUERY suggestions. The latter is used in consumer applications, such as iTunes or Google Suggest. Music stores use them all the time, as what the masses want, is probably what you want (typing "b" in iTunes will inevitably show Britney Spears and Beyoncé, whether you like them or not).

So what will it be, dear user? Query suggestions or Live Search? Share your thoughts on the UX Lab. Tightly related is the discussion about Endless Scrolling versus Pagination. Give us your 2 cents here as well please.

Cheers, Rob

It's Time for Form Design

We've been blatantly ignoring them in the first redesign excercise but now it's time to give them the attention they deserve: Forms

The current form (we are talking about the view you get when you edit a record) is far from ideal. The main points that need to be addressed are:

• Awkward field length variations
• Color coding only for mandatory fields: this is not accessible
• Label wrapping for long labels
• Grouping / clustering possibilities only limited
• No hide/show possibilties on user level (directly via the GUI)

A lot of research has been done already on form design, although most of it focuses on web form design, e.g. for a registration form but most of the heuristics are applicable for application forms as well.

Two column design: Although in most cases not recommended, I think we should keep this layout, as OB ERP has a lot of fields and a lot of horizontal space. Using a one column layout would be a waste of real estate.

Jakob Nielsen did a bit of research on forms versus apps and although outdated, it still offers some great takeaways:

• In-context help: Having this information on the main screen is preferable because users dislike going off to separate help screens
• Eliminating irrelevant steps: Users never need to see questions and options that don't apply to them.
• Customizability: (this example is really spot-on!) For example, an expense reporting application might require users to enter driving distances to get reimbursement for personal automobile use. If an employee often travels the same routes (say, to the airport or an important customer site), the application could provide one-click access to frequently used distances and list them by name (say, "office to JFK").

Luke Wroblewski wrote a great article on form layout. The most important conclusions here are:

• Vertical alignment of labels is best when the time to complete a form needs to be minimized and the data being collected is mostly familiar to users.
• Bold labels are preferred to give them visual weight and not have them compete with field values over the user's attention.

Enough for the first stab. I've modeled the first concepts which can be found on the UX Lab Web Album (images 35 and beyond). The concept with top aligned labels is quite unusual for application forms as it takes up a bit more vertical space due to the top labels. However, the two-column layout and collapsible sections offset that. Gray labels in collapsed sections are clickable: this opens the section and moves focus straight into the field. The first section contains help/guidance. I'd say that the user can choose to save the view and reuse the settings for all other forms of the same kind. This way you can hide sections that you don't use all the time and reach a super fast data entry, also using the keyboard. Next to the section label, we could place a "check" or "x" sign, indicating whether all mandatory fields in the sections are satisfied or not.

Special attention to:

How to emphasize mandatory fields: using *, using icons, using field coloring. In that same set you can also find a concept with and without seperating horizontal lines.

Process guidance using in-form bubbles that are triggered on completion of certain fields. The same thing but now using the top status bar. I really like the last idea, imagine how easy this can be for novice users! Of course you should be able to switch it off. Same hint could also be placed on the form itself, using the "introduction" section.

A naughty look at possible things to come for mobile: the resolution used in the concepts do actually fit on a 800x480 px smart phone.

Concepts with labels to the left of the fields, similar to the current OB form design, should not be ruled out. In fact, right aligned labels are commonly used for these kind of forms. Left-aligned labels are a good alternative as they support quick vertical scanning but risk higher completion time when the distance between field and label gets too big.

This should keep us busy for a while. Let me know your reactions via the UX Lab Forum so we can take it to the next level. Examples of Great Form Design out there and personal experiences with e.g. rapid order entry will also be highly appreciated.

You Have Spoken: Design Concepts Survey Results

Less than two months ago we presented design concepts for a new user interface for Openbravo ERP and asked you to give feedback by filling out the surveys or by discussing the design work on the User Experience Lab forum. And so you did! No less than 127 surveys were completed and many users participated in lively discussions on the forum.

The forum discussions continue and I'm getting more and more excited to see what beautiful things are happening there. With your help I was able to lift the design concepts to the next level. What about the new Virtual Child idea: a more elegant solution for Linked Items, while offering a new home for meta data, notes and attachments? You guys loved the Column Search for the grids, but couldn't care less about First-Letter-Search. Good to know. About the proposed Advanced Search: You liked its power, but many users pointed out that it just takes up too much space. So, after some discussion on the forum, here's a more simplified version. We also worked a bit more on the Task Paths idea and introduced the Label Context Bubble. Now let's hear your feedback so we can sculpt it even further.

Back to the survey results. If you are into stats and charts: check the results for Overall, Master Detail, Search and MyWorkspace . Cutting a long story short: You liked our concepts very much. Over 96% of you think the new stuff is "better" or "much better" than the current Openbravo ERP GUI. The other 4% voted “neutral” and no negative votes were given.

Really encouraging were the comments you gave, spread out across all four surveys. Lots of "Bravos" and "We want it!" and most of them are about the concept as a "holistic solution". This is especially helpful as we intended to design a complete new GUI framework that needs to serve as the basis for future functionality. All the different functional areas need to work together as a whole. You gave us the impression that we are heading in the right direction here.

The concepts shown to you were all about the GUI of our product but obviously there are many more features that deserve their place on the road map. For the GUI related features we now have a good idea of what is important and what not. For the bigger list of features, please visit the Uservoice Openbravo page where you can vote for features.

Now the next step is to use all the feedback that you have given – or will give soon – in the next round of design iterations. We will keep on doing this till the moment we are going to build it. The UX Lab is the place to go. I will post new design work as we go and you can throw stones or flowers at it. At the moment I'm publishing all the latest design mockups in a Picasa web album. These images are really snapshots in time of ongoing design work so the latest greatest are the last ones in the sequence.

What we're doing here is called Collaborative Design. I will blog about this phenomenon soon into more detail.

Rob

Concepts for Openbravo ERP Redesign

We are continuously improving our products and raise the bar with every release. Recently we have started to look at some very fundamental parts of the User Interface of Openbravo ERP and came up with ideas for a future redesign. User feedback told us that there is a need for more powerful and flexible search functionality. We also learned that many user like to edit the grids (tables) directly in the cells and that switching between different records at the same time would be very useful. Furthermore, we are looking at offering more functionality in the context of tasks and we also believe that there is a need for more high level and summarized views on data.



Here we would like to give you a peek preview of the stuff we are working on and give you the opportunity to give your feedback. Our approach is holistic: we believe that all functionality should be working in the context of user journeys rather than just plugging in new features left and right. That's why we also want to propose a complete new Master-Detail concept in which all other functionality is embedded.

Images say more than words so check out the Demo Videos and discuss them on the User Experience Lab forum:

Please also fill out the surveys for Master Detail, Search & Filter, My Workspace and Overall Impression.

I am looking forward to a lively discussion.

Thanks, Rob