Summative evaluation is one of the methodology specified by the IEC 62366 for the evaluation of the usability of medical devices. Usability is one of the key factors to take in consideration for design of medical devices and it is highly connected with safety, labelling and risk management.
We have already been discussing about usability, and in particular about the main requirements associated to IEC 62366. Moreover, within our 4EasyReg Shop, a checklist to support compliance to IEC 62366 has also been made available.
-
Usability File Template€64,00
In this article, we will discuss in details about summative evaluation, that is one of the key factor for the Human Factor Engineering Process and the evaluation of the usability for a medical device.
Few Considerations about Summative Usability Evaluation
Summative evaluation is the last step of the usability process and it basically coincides with the so-called usability validation.
The requirements for Summative Evaluation of usability for medical devices have been established within IEC 62366. The process of usability evaluation can be summarised based on the following steps:
- Preparing use specification
- Identifying UI characteristics related to safety and possible use errors
- Identifying known or foreseeable hazards and hazardous situations
- Identifying and describing hazard-related use scenarios
- Selecting hazard-related use-scenarios for a summative evaluation
- Establishing a UI specification
- Establishing a UI evaluation plan (formative and summative evaluations)
- Implementing and verifying the UI
- Validating the UI
If we focus specifically on the summative evaluations we can say that is a formal evaluation with specific acceptance criteria. The summative evaluation shall be performed on the final product, on the version that will be brought to the market after regulatory approval. The primary goal of the summative evaluation of usability is to get an objective evidence that the medical device is safe to use. This means that the chance of committing dangerous use errors is minimized.
Usually summative evaluation is performed for specific selected hazard-related use scenario. It is possible to perform a single test event or multiple tests , depending on the nature of the product and variation of use cases. If during summative evaluation it turns out that specific risk control measures are not adequate to mitigate specific risks, then the usability process shall restart and the formative evaluation shall be repeated.
Requirements for Summative Evaluation
Based on the IEC 62366, there are specific requirements associated to the summative evaluation for medical device usability assessment.
First fo all, there are specific requirements associated to the preparation of the summative usability plan, that shall contain the following elements:
- the evaluation method being used and a rationale that the method produces objective evidence;
- which part of the user interface is being evaluated;
- where applicable, the criteria for determining whether the information for safety is perceivable, understandable and supports correct use of the medical devices;
- the availability of the accompanying documentation and provision of training during the summative evaluation;
- For a usability test:
– the test environment and conditions of use and a rationale for how they are adequately representative of the actual conditions of use; and
– the method of collecting data during the usability test for the subsequent analysis of observed use errors.
Steps to Properly perform a Usability Evaluation
In this last paragraph, we want to provide the overall steps that need to be considered for the design of a summative usability study. The general steps to do are the following:
- Creating and Finalizing the Test Plan : this involves Choosing the Test Location, Test Personnel, Choosing the Tasks, Pilot Testing
- Recruiting Participants;
- Conducting the Test;
- Analyzing the Data;
- Writing the Report;
The activity related to the recruitment of the participants is of key importance and the following steps shall be undertaken:
- Take a decision on the number of participants;
- Definition of the user profile;
- Perform an initial evaluation of the participants through specific screening questionnaire;
- Inviting participants to actually perform the test.
Subscribe to 4EasyReg Newsletter
4EasyReg is an online platform dedicated to Quality & Regulatory matters within the medical device industry. Have a look to all the services that we provide: we are very transparent in the pricing associated to these consulting services.
Within our WebShop, a wide range of procedures, templates, checklists are available, all of them focused on regulatory topics for medical device compliance to applicable regulations. Within the webshop, a dedicated section related to cybersecurity and compliance to ISO 27001 for medical device organizations is also present.
As one of the leading online platforms in the medical device sector, 4EasyReg offers extensive support for regulatory compliance. Our services cover a wide range of topics, from EU MDR & IVDR to ISO 13485, encompassing risk management, biocompatibility, usability, software verification and validation, and assistance in preparing technical documentation for MDR compliance.
Do not hesitate to subscribe to our Newsletter!
