Product Features and Evaluation
We are now at the develop and deliver section of the double diamond design process model.
Product Features
The following section will expound on product features that we have designed to address user pain points and help them accomplish their tasks.
To accomplish this user task, we have developed a contractor marketplace for users to find contractors. The marketplace offers multiple contracting categories to cater most types of contracting services. Features such as Search Bars, "Sort by" or "Previously contracted" were all purposefully designed and incorporated in ContractMe! to aid users in searching for contractors more conveniently.
To ensure reliability of our application, we have made sure to include listing details, "Verified"/"Preferred" filters, as well as reviews for users to look for reliable contractors.
The second icon (from the left) on the navigation bar holds a comparison feature, allowing users to compare contractors easily. Users can pin their favorite listing and add as many contractors as they want to compare against. ContractMe! then generates an easily-digestible yet comprehensive table of comparison for users to compare all contractors at once.
There is another feature on the app that allows for comparisons. For users who are in the early phases of comparing and just want a small quick comparison against the current top favourite listing, they can click on the "Compare" icon at the top right hand corner of the detailed listing. An overlay would popup, comparing only the user's favorite and the current listing being viewed.
Finding it difficult to identify the home repair issues necessitates an identification feature that lets users know the type of contractor they need to find. Our camera feature is equipped with advanced technology to help with that. Users would only need to take a photo of facilities that require home repair and ContractMe! will lead them to a page with the contractors they need to find, helping them simplify the identification process. Furthermore, if a user does not require this additional feature and process, they can simply search based on their experience for services they require.
To help users communicate with homeowners more easily, ContractMe! has a chat feature that allows users to communicate directly with homeowners. Quotations are also offered and accepted on the chat to help better communicate expectations, allowing users to feel secure about prices they were quoted.
Evaluation
Our team conducted usability evaluation to discover usability deficiencies and collect data while observing our participants using our application to perform tasks. We used the "Think Aloud" technique with our participants to better understand the user journey and whether user expectations are met.
Our team conducted usability evaluation to discover usability deficiencies and collect data while observing our participants using our application to perform tasks. We used the "Think Aloud" technique with our participants to better understand the user journey and whether user expectations are met.
The purpose of the usability testing is to assess the extent to which our interactive ContractMe prototype accomplishes our intended user tasks. This assessment consequently helps to improve our application both functionality and aesthetically.
As our application aims to be inclusive and allow most to obtain home services, we aim to test with a diverse group of participants. Some form of segmentation was based on whether or not they have obtained home services through other means previously.
We have built usability testing tasks around our key user tasks to ensure a high level of focus on our initial objectives. In so doing, we would be able to ensure a more valid/accurate measure of how well our application accomplishes the tasks.
| Tasks | Details |
|---|---|
| 1.1 | Register for an account with your name and address. |
| 1.2 | Use the application to identify the problem shown on the whiteboard (picture given). |
| 1.3 | Look for a contractor that you would be interested to engage. |
| 1.4 | View the listing for more details of what the contractor offers. |
| 1.5 | Look for another contractor that you might be interested in and compare against the previous. |
| 1.6 | Communicate/Ask for a quotation from the preferred contractor. |
| 1.7 | Accept the quotation. |
| 1.8 | Make payment. |
The participants are invited to a common room. We prepared a board with a photo of a broken switch so that they can use it to complete task 1.2. The participants were given a laptop with the Figma interaction page. Instructions were to the participants by facilitators seated beside them.
We took on different roles when facilitating. One of us took on the role of engaging with the participants. He began by introducing Contract Me! and its goals, followed by the context/scenario and subsequently, the tasks. A new task is only given after the participant successfully completes it or wants to give up. The session then ends with the facilitator debriefing and collecting final thoughts of the participant regarding the use of our application.
The other facilitator takes on the task of collecting quantitative metrics such as time taken to complete each task, as well as error rates.
The last 2 facilitators focused on collecting more qualitative information. This includes noting down expressions and emotions expressed, writing down what each participant shares with the "Think Aloud" technique and test/task level satisfaction.
Briefing note:
Thank you very much for participating in this study. The goal of this project is to evaluate the interface of ContractMe. ContractMe is an application designed to simplify the process of looking for a contractor. The results of our evaluation will be summarized and used to improve our application further.
You will be asked to use ContractMe (the prototype) to conduct a series of tasks. You will be asked to "think aloud" as you interact with the prototype. What you say as you carry out the activities may be recorded. Your identity will remain confidential. As you use ContractMe for each task, please keep in mind that it is ContractMe that is the subject of this evaluation - not you.
We used the following table of metrics to record and note how successful the participants were in completing the tasks.
| Metric | Description |
|---|---|
| Successful task completion | Percentage of tasks that test participants complete correctly |
| Time on task | The amount of time it takes participants to complete a task successfully |
| Task level satisfaction | To flag a difficult task |
| Test level satisfaction | To evaluate how the participant felt about the overall experience |
| Error-free rate | Percentage of participants who complete a task successfully without any errors. |
| Critical Errors | Errors that block the user from completing a task successfully, e.g. due to participant workflow deviation. |
| Non-critical errors | Errors that are recovered by the participant and do not affect their ability to complete the task. |
Findings
Successful Aspects
We highlight some of the areas that we have done well based on Nielsen’s Usability Heuristics. We determined which of these aspects have been achieved based on the feedback given as well as the observations we made while conducting our usability testing.
The search and identify workflow was when users had to use the identify feature to diagnose an issue then afterwards search for the related contractor.
Camera feature that helps identify issuesUsers were satisfied with the way identification works and stated that this feature was incredibly useful. Based on our metrics, all users successfully completed the task under 60 seconds and no users experienced errors. We also notice that this matches with one of Nielsen Heuristics H2: Match between system and real world. This was due to the fact that taking pictures to diagnose issues is very similar to asking a contractor to identify the issue for you.
Chat functionalities which include messaging and invoicingOur chat feature is integrated with the entire contractor-homeowner interaction. It includes communication through text messages, as well as offering proposals by a contractor. After a user is satisfied with the details (price, date time etc), the user can accept the offer with all the required payment details. This reduces the number of other screens involved when completing the service agreement, which participants enjoyed. This is also related to H2: Match between system and real world in that users typically communicate back and forth with a contractor before a deal is set.
Information and confirmation dialogs in ContractMe! AppWe consider informative dialogs and information text to be the popup’s that help with error recovery and error prevention. This allowed users to cancel or make changes to their actions. In general, we noticed that these dialogs helped to prevent the users from making critical and non-critical errors and also improved the task level satisfaction. We note that this is also related to H5: Error Prevention and H9: Help users recognize, diagnose, recover from errors.
Detail of users are saved and reusedWith reference to H5: Error prevention, the participants were happy to see that the details of the contact person and service address needed when accepting the offer can be saved and reused. This reduces the chance of incorrect information provided to the contractor.
Areas for Improvement and changes made
We also noted some areas where we could improve upon that we noticed where pain points for users. Based on these findings, our team incorporated the feedback into our final prototype to better address our user needs. In particular, we focused on addressing any violations of the Usability heuristics and enhancements that can improve the application further.
The current application only supports messaging via chat, which can be quite troublesome when there is a need to have a lengthy discussion with a contractor. In view of that, suggestions were raised to include a video/audio feature so that users can easily hop on a call with the contractor.
Based on the feedback and results of the usability evaluation, we have included a video chat functionality as seen in the figures above within the messaging feature to allow for users to call and have a video consultation with a contractor directly. This provides the flexibility for homeowners and contractors to communicate and diagnose problems in a way that suits their communication style either through text messages or through call/video chat. This feature could also replace the on-site consultation session for a contractor to provide a quotation.
The current application only supports messaging via chat, which can be quite troublesome when there is a need to have a lengthy discussion with a contractor. In view of that, suggestions were raised to include a video/audio feature so that users can easily hop on a call with the contractor.
Based on the usability results, we started iterating on the compare services design and came up with a feature that allows for comparison of multiple listings, as well as a “quick compare” feature that would allow for a user to compare on the fly without going into the compare page.
This improvement is justified as it allows for flexibility for the users to compare as many or as little service listings as they require. In addition, users are also able to select which criteria they would like to compare the services on to improve their decision making.
While users found the identification feature very useful, they felt that the process was too long. There were too many confirmation pages to narrow down the issue. As such, comments were mainly to simplify and reduce the number of sections and clicks required to identify the problem.
Based on the feedback, we sought to simplify and reduce the number of sections and clicks required a user needs to go through before a user identifies the problem.
We do this through smart prediction that allows us to recommend services and asking the user if the answer suits their needs rather than making users go through a fixed number of steps. Through the use of additional hardware aids like the camera/gallery in addition to prompting service category types and potential service names we can prevent errors (Nielsen Heuristic H5) and reduce short-term memory load (Shneiderman’s Golden Rules, Nielsen Heuristic H6). As seen in the figures above, we provide both the option for using their cameras to identify the issue, or allow them to use an existing photo in their gallery to identify what issue they are facing.
Final Prototype
Now, with all the user evaluation steps done and changes made, we can showcase the final prototype after many iterations!
© 2022 NUS CS3240 Team 15 - ContractMe!