Introduction
Back in March of 2019, I decided to join the very first Neuro Nexus brain health hackathon at the University of Calgary. During this competition, I was intrigued by a particular problem presented by the PsychPGx Lab and decided to join the team.
Depending on your genetics, you can metabolize drugs differently from others. This means that not all drugs are effective for everyone. This is especially critical for things like mental health, where proper drug dosages are important. We created a tool called Sequence2Script that helps clinicians determine the best drug to prescribe for their patients based on their genotypes.
The Problem and Existing Solution
When doctors are given a patient’s genetic data, they would normally need to contact the University of Calgary’s pharmacogenetics consult service. The consultant (such as our challenge lead) would have to search multiple databases to find the relevant information. Compiling these data into a single recommendation was both time-consuming and tedious.
The goal of Sequence2Script was to automate this process and aggregate all of the information from these separate databases into a single report for clinicians. This frees up time for the consultants and reduces a lot of inefficient searching through different services.
Neuro Nexus Prototype
For the Neuro Nexus hackathon, I took the role of product design lead and built UI wireframes in Figma, based on the specifications provided by our challenge lead. Because our timeline was limited, we did weren’t able to conduct interviews with potential users directly. Instead, we decided to base our decisions on our challenge lead’s criteria, because they would be the one who would be using this tool first.
We started by brainstorming what the user’s workflow would be, as well as how the information would be pulled in from different data sources. From there, we established what kind of information should be available in the report.
The first build of the software incorporated four genes, drawing information from two different databases to provide recommendations for antidepressant prescriptions. This minimum viable prototype was presented at the end of our eight-week challenge.
Alpha Build
After the competition ended, we decided to expand the scope of the project to include more drugs, more genes, and more features. Many of the original team members moved on and we brought on a new developer to continue the project. This developer refactored the existing code into an alpha build.
I was not involved with the design of this alpha build’s UI, which converted my one-column design and into a two-column input form. We were planning to conduct usability testing on the tool soon after this transfer, so this layout was meant to be a stopgap.
Usability Testing
I was tasked to lead the usability testing sessions and bring our findings back to the team to drive further development of the tool.
We ran two rounds of qualitative usability testing. For the first round, we conducted six user testing sessions with 10 participants between December 2019 and February 2020. The second round was conducted with 10 more participants from May to June 2020, done remotely over Zoom due to COVID-19 safety measures.
The participants tested came from a variety of backgrounds, including physicians, pharmacologists, lab technicians, industry professionals, and pharmacists. Many of these participants were contacts and colleagues of our challenge lead.
We prepared a number of scenarios for participants, which we believed would be similar to their day-to-day work. For example, we would ask a doctor to prescribe a particular drug to a patient or we would ask a lab technician to do data entry and report generation.
During these sessions, we asked participants to complete these tasks with the alpha build. Participants would verbalize their thoughts by “thinking aloud.” I minimized interactions with the participants during the session and only asked open-ended probing questions if they forgot to think aloud.
After the tasks were completed, we interviewed the participants to hear their first impressions and address any questions they had.
Key Findings from Usability Testing
Since the alpha version had the form split across sections in two columns, I had hypothesized that this would create a friction point with users. This was validated in the usability testing sessions quite quickly. The alpha version had fields in two columns, but we discovered that this did not have enough visual hierarchy to show what the workflow was for the user. Many users expressed confusion and hesitation when looking for what to do next.
The study also revealed surprising insights into the users’ expectations about how this software worked. One thing we learned was that Sequence2Script’s workflow was inconsistent with the workflows of similar drug interaction tools that our participants were used to.
In Sequence2Script, users would enter drugs that a patient is currently taking and show resulting drug interactions in the report. However, many participants were used to tools where they entered every drug into the same bucket and expecting the results to show interactions between all of them.
Most interestingly, some users would actually add drugs into the “drugs being considered” box knowing that they didn’t belong there, but wanted them “just in case” they missed out on potential drug interactions. We also discovered that adding additional written instructions alone did not completely eradicate this problem.
These sessions revealed some major assumptions that we made about the users’ mental models of how this software worked. We were concerned that in its current state, a user might enter the wrong information, believe it was right, and prescribe patients the wrong drug. That was an unacceptable risk.
Major Changes for Beta Launch
We made some major changes in the beta version based on the insights we gained from the usability testing.
Form and report restored to single-column layout
For one, we restored the layout of the input form to be a single vertical column, separated into different steps with short, simple instructions on what kind of input is expected. We also restructured output report into a single column for easier scrolling and printing.
Additional step added in forms to reduce workflow ambiguity
We discovered in our testing that users were confused between “drugs they were considering” and “drugs the patient was currently taking.” To address this, we created an additional step on the input form specifically for users to enter drugs they are considering.
By creating two buckets for users to sort this information into, we hope that it will reduce input errors by showing users that these are two different concepts. While it does result in a longer workflow, we hope that it makes the steps less ambiguous and therefore reduce the errors produced by wrong inputs.
As a bonus, the drugs listed in this new step would be put on the report in their own dedicated section so clinicians can see that information more easily without needing to scroll through an entire list of drug recommendations.
Outcomes and Future Directions
With the Sequence2Script beta launch, I am hoping to continue conducting usability testing with early adopters and continue gaining insights on how we can improve the experience. So far, the feedback has been positive and the interface has been described as “clean” and “intuitive. That said, I hope that future usability tests will provide more data as to whether or not the additional step has reduced the input error rate.
For future development, I also intend to work with our developer to make the UI responsive for different screen sizes and explore the best design patterns to view the data tables on narrow mobile devices.