B is for Baseline Study

What is a Baseline Study?

User experience design is design based on research. The customer experience (CX) compared with the business goals is what should drive the specifics of any design. Though this is the ideal situation, most projects do not have this benefit.

When coming into a project that has not been validated through user research knowing where to begin can be very difficult. Usually easy-win changes are quickly spotted, but the starting point is not always as apparent. 

The most important step to any redesign is the baseline study. 

While one might be tempted to begin with the goals and outstanding feature list provided by a project owner, manager, or other lead, the user is the one who understands the project best. Whatever the companies’ priorities, the fact that users for this project exist speak volumes. This data will be lost, or at least corrupted beyond usefulness if it is not properly captured before changes are made.

Where a user group exists a set of expectations and processes exist. Even if they are not ideal, they can provide critical insight to those who take the time to set proper markers. 

A baseline study should cover any aspect that is expected to change. Change involves a starting point, an end point, and the time it took to transition. If your design team waits to begin tracking performance after the new changes are implemented the risk is that all progress up until the latest changes could be lost. The only way to show progress is to document the start point. If you want to take credit for the improvements your team and processes create you absolutely must perform a baseline study to show where the progress began.

Using a study as a baseline is not limited to a specific user study method. Any method can be used. The only difference is in how future studies are compared to this standard that is set. Should performance drop in a new version a rollback is not always immediately mandatory. Future revisions may raise the usability of the product. However, until a new baseline study is set, the total level of improvement is measured against the last baseline study. When a products feature set has changed significantly from that in the baseline a new baseline may be necessary, even if the statistics show a lower performance since the last baseline.

How to perform a Baseline Study

A baseline study should be no different then any other user study. For the study to be truly a baseline it must be identical to future studies. This does not tie a baseline to any specific testing method. All methods should have baseline studies performed and metrics tracked against. 

A baseline studies key features is deciding what you want to track. Business goals almost never align with the user’s goals. This can make tracking the real value of a product difficult. When insight is needed into a product’s value to users, an observed natural use session can be all that is needed.

When to perform a Baseline Study

A baseline study should be performed whenever a base performance needs to be set. This is at the beginning of a project, and whenever a significant feature set changes. It is not necessary for a study to be labeled a ‘baseline’ when it is performed. As long as future tests are performed in the same way, any test can be used as a baseline. The important part is that a standard is set to show real improvement connected to specific change.

Leave a Comment