Usability study for medical devices

Insight in Brief

The European MDR and the US FDA regulatory frameworks require the application of usability engineering to medical devices.

To develop a device with the best user interface, two usability studies shall be performed during the development process, so-called Formative and Summative Usability evaluation. This article describes the key contents of each Usability evaluation stage.

Introduction

Usability engineering is a field that focuses on devising human–computer interfaces that have high user-friendliness. It provides structured methods for achieving efficiency and elegance in interface design. The goal of medical device manufacturers is to develop a medical device with an effective and efficient user-device interface that doesn’t result in any safety-related use errors or harm to the user. Devices that lack an intuitive design and fail to take into account the user-device interaction can jeopardize patient and user safety.

The user interface defines everything designed into a device with which the user may interact.

Usability is a characteristic of the user interface, which ensures an effective and efficient use of the device and finally satisfied end users in their intended use environment. Helpful methods therefore are on the one hand the formative evaluation which will be performed early during the device development process. The results of the formative evaluation will be considered to complete the device development. After that, a summative evaluation will be finally performed on the other hand.

Figure 1: Placement of formative and summative evaluation during the development process

Formative Evaluation

The formative evaluation will be performed by users early during the device development process. The aim of this developmental evaluation is to determine the strengths and weaknesses of the user-device interface and unanticipated use errors.

To perform a formative evaluation, it shall be planned and documented …

… as well as justified, which methods (e.g. cognitive walkthrough, usability test with prototypes, verification against heuristics, interviews, questionnaires) are used;
… when the evaluation takes place;
… which part of the user-device interface shall be evaluated;
… which criteria shall be fulfilled;
… who will be the evaluators/ which evaluation environment will be chosen.

A common method is a heuristic analysis where the following heuristics could be useful:

Will the system status always be visible and the user knows what’s going on?
Will the system match the real world and speak the user’s language?
Will the system allow a clear way to exit current interactions?
Will the system support a consistency that meets the customer’s expectations (e.g. wording, actions)?
Will the system prevent the occurrence of errors?
Will the system support recognition to reduce the amount of cognitive effort required from users?
Will the system allow flexibility and efficiency of use?
Will the system have an aesthetic and minimalist design without irrelevant information?
Will the system help users to recognize, diagnose, and recover from errors (e.g. comprehensible error messages)?
Will the system command help and documentation content that is easy to search?

Summative Evaluation

The summative evaluation – a common method is a usability test – will be performed by end users after the device development phase is completed. The aim of this final evaluation is to evaluate the user-device interface and to get evidence of a safe usable user-device interface.

To perform a summative evaluation it’s recommended to do the following steps:

Choose safety-related use scenarios that the end user shall perform.
Choose and document user profiles if the final summative evaluation shall be performed as usability tests.
Create a summative evaluation plan to document …
- … and justify which method (e.g. usability tests, cognitive walkthrough, inspection through usability experts) is used;
- … which part of the user-device interface shall be evaluated;
- … which criteria shall be fulfilled. These criteria are the basis for deciding whether the device can be used safely;
- … the use environment, and methods for collecting and evaluating the data of the use errors in case of usability tests.
Finally, perform the summative evaluation by the end users. If some safety-related use errors occur, the user-device interface has to be updated respectively it has to be decided how to improve the user-device interface. Decisions have to be justified by taking the risks into account.

Summary

As a medical device manufacturer, you need to ensure that your device has an effective and efficient user-device interface that doesn’t result in any safety-related use errors or harm to the user. For a formative and finally a summative evaluation, the manufacturer decides under consideration of the risk analysis, which parts of the user-device interface shall be evaluated, and which criteria shall be fulfilled.

Furthermore, he also defines and decides the used evaluators and evaluation method and documents it in a formative evaluation plan/ summative evaluation plan. In addition to these plans, the implementation of the formative/ summative evaluation, its results, and their evaluations must also be documented.

Do you have further questions about usability studies or in the area of regulatory affairs & quality management? Our experts would gladly answer any questions you have – contact us here.

Go back