Usability study for medical devices

Katharina Maul, Regulatory- and Quality Manager
Reading time: 3 min

Insight in Brief

The European MDR and the US FDA regulatory frameworks require the application of usability engineering to medical devices.

To develop a device with the best user interface, two usability studies shall be performed during the development process, so called Formative and Summative Usability evaluation. This article describes key contents of each Usability evaluation stage.

Introduction

Usability engineering is a field that focuses on devising human–computer interfaces that have high user friendliness. It provides structured methods for achieving efficiency and elegance in interface design. The goal of medical device manufacturers is to develop a medical device with an effective and efficient user-device interface which doesn’t result in any safety related use errors or harm to the user. Devices that lack an intuitive design and fail to take into account the user-device interaction can jeopardize patient and user safety.

User interface defines everything designed into a device with which the user may interact.

The usability is a characteristic of the user interface, which ensures an effective and efficient use of the device and finally satisfied end users in their intended use environment. Helpful methods therefore are on the one hand the formative evaluation which will be performed early during the device development process. The results of the formative evaluation will be considered to complete the device development. After that, a summative evaluation will be finally performed on the other hand.

Usability EN

Figure 1: Placement of formative and summative evaluation during the development process

 

Formative Evaluation

The formative evaluation will be performed by users early during the device development process. The aim of this developmental evaluation is to get strengths and weaknesses of the user-device interface and unanticipated use errors.

To perform a formative evaluation, it shall be planned and documented …

  • … as well as justified, which methods (e.g. cognitive walkthrough, usability test with prototypes, verification against heuristics, interviews, questionnaires) are used;
  • … when the evaluation takes place;
  • … which part of user-device interface shall be evaluated;
  • … which criteria shall be fulfilled;
  • … who will be the evaluators/ which evaluation environment will be chosen.

 

A common method is a heuristic analysis where the following heuristics could be useful:

  • Will the system status always be visible and the user knows what’s going on?
  • Will the system match between the real world and speaks the user’s language?
  • Will the system allow a clear way to exit current interactions?
  • Will the system support a consistency which meets the customer’s expectations (e.g. wording, actions)?
  • Will the system prevent the occurrence of errors?
  • Will the system support recognition to reduce the amount of cognitive effort required from users?
  • Will the system allow flexibility and efficiency of use?
  • Will the system have an aesthetic and minimalist design without irrelevant information?
  • Will the system help users to recognize, diagnose and recover from errors (e.g. comprehensible error messages)?
  • Will the system command a help and documentation content which is easy to search?

 

Summative Evaluation

The summative evaluation – a common method is a usability test – will be performed by end users after the device development phase is completed. The aim of this final evaluation is to evaluate the user-device interface and to get an evidence of a safe usable user-device interface.

 

To perform a summative evaluation it’s recommended to do the following steps:

  • Choose safety related use scenarios that the end user shall perform.
  • Choose and document user profiles if the final summative evaluation shall be performed as usability tests.
  • Create a summative evaluation plan to document …
    • … and justify which method (e.g. usability tests, cognitive walkthrough, inspection through usability experts) is used;
    • … which part of user-device interface shall be evaluated;
    • … which criteria shall be fulfilled. These criteria are the basis for deciding whether the device can be used safely;
    • … the use environment, and methods for collecting and evaluating the data of the use errors in case of usability tests.
  • Finally, perform the summative evaluation by the end users. If some safety related use errors occurred, the user-device interface has to be updated respectively it has to be decided how to improve the user-device interface. Decisions have to be justified by taking the risks into account.

 

Summary

As a medical device manufacturer, you need to ensure that your device has an effective and efficient user-device interface which doesn’t result to any safety related use-errors or harm to the user. For a formative and finally a summative evaluation, the manufacturer decides under consideration of the risk analysis, which parts of user-device interface shall be evaluated, and which criteria shall be fulfilled.

Furthermore, he also defines and decides the used evaluators and evaluation method and documents it in a formative evaluation plan/ summative evaluation plan. In addition to these plans, the implementation of the formative/ summative evaluation, its results and their evaluations must also be documented.

Do you have further questions about usability studies or in the area of regulatory affairs & quality management? Our experts would gladly answer any questions you have – contact us here.

Are you interested in reading further articles?

Leave your e-mail address here and we will inform you as soon as we publish new contributions.

This might also interest you

Outsourcing
Reading time: 3 min.

Ensuring successful outsourced engineering projects

How do you ensure success of your outsourced engineering project from the start?
Usability study for medical devices
Reading time: 3 min.

Usability study for medical devices

Preventing safety related use-errors or harm to the user using usability studies.
Event-based
Reading time: 4 min.

Why event-based software architecture?

This article explains the event-based software architecture, its advantages, & possible disadvantages for the development...
Expert Blog Architekturprozess
Reading time: 4 min.

The systematic creation of a system and software architecture

This article deals with the development of a good system architecture...
IMT_AdditiveFertigung_Blog
Reading time: 5 min.

Additive manufacturing - reliable enough for medical technology?

For a customer's device, a dynamically rotating system was developed within a short period of time...
knowwhatyoudonwknow_cover
Reading time: 4 min.

Know what you don't know

Typical classification algorithms assign a class to every sample, even though that sample may be far off...