Comparison of Usability Evaluation Method between Heuristic Evaluation and Cognitive Walkthrough on Website SIMRS Del Egov Center
View Publication
PROJECT OVERVIEW
The hospital information system is a website which is an information and communication technology system that processes and integrates the entire flow of service processes and hospital activities that are used to obtain precise and accurate information. The hospital information system as one of the public service systems must have a good usability side so that users have no difficulty and can use it correctly according to its function. To achieve this, usability evaluation is needed that can help identify and analyze usability problems of a website. In this study, evaluation will be carried out using heuristic evaluation and cognitive walkthrough methods, and usability testing to find the most effective method for solving usability problems.
GOAL
Redesigning a Web-based SIMRS Del Egov Center
Evaluating the usability of the SIMRS Del Egov Center using the heuristic evaluation method and cognitive walkthrough
Comparing heuristic evaluations and cognitive walkthroughs to find which methods are more effective
Build recommendations for design improvements in the form of interactive user interface prototypes in the hospital information system based on the results of the usability evaluation
DESIGN PROCESS
DESIGN USER
INTERFACE
COGNITIVE
WALKTROUGH
HEURISTIC
EVALUATION
USABILITY TESTING
USER VALIDATION
DESIGN USER INTERFACE
STORYBOARD
Prepare stationery such as ballpoint pens or markers and paper as well as compose a story line, namely a series of scenes which are illustrations of using the system to complete tasks.
SKETCH
Sketching to explore early design ideas and realize ideas in the early stages of design
WIREFRAME
Wireframing to provide a realistic perspective on how the user interface will be formed
HIGH- FIDELITY PROTOTYPE
Sketching to explore early design ideas and realize ideas in the early stages of design. See the full prototype here.
COGNITIVE WALKTROUGH
Implementation of usability evaluation with the cognitive walktrough method evaluation involves three evaluators The three evaluators are experts who have the expertise or who work in the user interface field with at least 1 year of experience. The evaluation process consists of two stages, namely the preparation of the cognitive walkthrough and the evaluation of the cognitive walkthrough.
Based on the results of the evaluation carried out using the cognitive walkthrough method at the SIMRS Del Egov Center, the three evaluators found 5 problems each by Evaluator 1, 9 problems by Evaluator 2, and 11 problems by Evaluator 3. The results of the problems found by the evaluator were mapped with the usability aspect by the evaluator.
HEURISTIC EVALUATION
Implementation of usability evaluation with the heuristic evaluation method evaluation involves the same evaluator with the cognitive walkthrough method.
Based on the results of the heuristic evaluation that has been carried out, a total of 21 usability problems were found by the three evaluators. The three evaluators found 12 problems, 2 problems, and 7 problems each that violated Nielsen's principles. The problems found by the evaluators are mapped with usability aspects
REFINE THE DESIGN
Based on the insights from the cognitive walkthrough and heuristic evaluation method, I refine the design in prototype form. This prototype will be used in usability testing. See the prototype here.
USABILITY TESTING
Usability testing is carried out twice, using an interactive prototype improvement based on the results of heuristic evaluations and cognitive walkthroughs. The participants who took part in this test were 10 people who represented the actual users of the SIMRS Del Egov Center website, 5 doctors, and 5 nurses.
Usability testing based on interactive prototype improvements based on the results of the heuristic evaluation resulted in an average SUS questionnaire final score of 57% and was included in the Neutral category and obtained 9 problems that were found through observation and think aloud as well as the results of interviews conducted after participants had tested.
Usability testing based on the improvement of the interactive prototype based on the results of the cognitive walkthrough resulted in an average final score of the SUS questionnaire of 54.5% and included in the Neutral category and 10 problems were found through observation and think aloud as well as the results of interviews conducted after the participants had tested.
REFINE THE DESIGN
Based on the insights from the cognitive walkthrough and heuristic evaluation method, I refine the design in prototype form. This prototype will be used in usability testing. See the prototype here.
USER VALIDATION
After doing usability testing on the SIMRS Del Egov Center website, validation is done with the user. The validation carried out by the researcher is related to the implementation of the improvement suggestions submitted by the user when doing usability testing. After validation, all users receive the improvements that have been made
COMPARISON
After analyzing the problems found using the heuristic evaluation and cognitive walkthrough methods, it was found that there were 7 problems that were the same between the two methods. After eliminating the same problems between the Heuristic evaluation and cognitive walkthrough methods, there were 10 remaining problems in the evaluation using the heuristic evaluation method and the remaining 15 problems in the evaluation using the cognitive walkthrough method.
Comparison of usability evaluation at the web-based SIMRS Del Egov Center between the heuristic evaluation and cognitive walkthrough methods in terms of three aspects, namely the usability problems found, the level of usability problems, and the end user's response to the website which was evaluated using usability testing. The comparison results show that for the first comparison aspect, namely mapping with aspects of usability problems found, evaluation using the Heuristic Evaluation method is better on efficiency, memorability and satisfaction while Cognitive Walktrough finds more usability problems in aspects: learnability and error. In the second aspect of comparison, namely the level of usability problems using a severity rating, the Cognitive Walktrough method is more effective in finding usability problems with a higher severity level with an average severity rating of 3, while the Heuristic Evaluation method with a severity rating of 2. In the third comparison aspect, namely the end user's response to the website based on usability testing, the Heuristic Evaluation method has a higher SUS questionnaire score with a score of 57 which means that the improvement in user interface design is better built based on the results of evaluation recommendations using the Heuristic Evaluation method with an SUS score of 54.5. So it can be concluded that based on these three aspects, the Heuristic Evaluation method is better at finding usability problems at the SIMRS Del Egov center.