User Experience and Testing
User experience testing has been a continuous, iterative process involving usability evaluations and A/B testing to optimize the interface's effectiveness. The primary objective of this research has been to rigorously test and refine UX flows and data accessibility features, with a strong emphasis on enhancing data clarity. This clarity is crucial as it directly contributes to ease of use and a more intuitive user interface, ultimately leading to a more engaging and efficient user experience.
Each phase of testing was rigorously planned and executed while being guided by extensive multidisciplinary literature reviews. These reviews drew from fields such as cognitive psychology, visual design, human-computer interaction, and user behavior studies. The insights gained from this comprehensive research provided a solid foundation for the methods and implementations applied in each testing phase, ensuring that the process was both scientifically grounded and practically relevant.
-
Phase 1: Representational Data Comprehension Testing
-
Phase 2: Radial Data Visual Display Complexity Testing
-
Phase 3: Information Displays Formats (Defines company information displayed)
-
Phase 4: Data Flower Development (Develop Unique Data Representation)
-
Phase 5: Refine Design (Remove grading systems/change locations of objects)
-
Phase 6: Click Through and UX Flow Testing (Improved Usability)
While further testing is planned, the insights and results gathered thus far have already led to actionable solutions that have progressively refined the design throughout each phase. These findings will serve as a critical benchmark as the process and methodology continue to scale and evolve, ensuring that the interface remains aligned with user needs and industry standards. Below are concise summaries of each phase, highlighting the key takeaways and implications for future development.
Phase 1:
Representational Data Comprehension Testing
This study was designed to inform the design of legal data graphical representations that would be widely accepted and easily understood by the majority of users. Literature review strongly focused on educational and phycological fields to compare data approaches and their impacts on participant's retention and literacy rates.
Number of participants: 50
Age of Participants: 18-30
Interview length: 5 Minutes
Location of research venues: Virtual - Discord App
Task:
Assess multiple types of data representations which were identified in literature by their efficiency in delivering complex legal data into easily accessible formats.
Research Synopsis
User's were shown a random screen of of a possible 15 screens each displaying a different companies legal information. Each design displayed identical information with a change in the format of the visual data represented. The study examined use of standard data-forms like pie-charts, histograms, bar graphs, and unique data representations to determine which was most effective in delivering the quality of data desired.
Participants given an Adobe XD model the screen for 30 seconds before the display was turned off. Participants we're asked a series of questions following their experience that tested for data comprehension and retention of for key points of knowledge identified in the research.
Key Competencies Tested:
-
Total Number of Violations:
-
What was the total number of legal violations of the company on the page you looked at?
-
-
Categorical Method Understanding:
-
How many categories of violations were present?
-
Name the violation categories that you remember seeing in the interface.
-
-
Relational Understanding
-
Which category did the company you looked at have the most violations in?
-
Which category was the smallest?
-
How wide of a gap did you perceive there was numerically between their highest and lowest violation categories?
-
-
Data Source Transparency
-
Using this interface, how does one navigate to a record of a particular violation?
-
What was the companies biggest violation?
-
Which agency/entity cited the company with that violation?
-
Findings Summary
Out of the 15 data representations tested, only 5 satisfactorily encapsulated all desired data displays (bar graphs, sunburst diagram, radar chart, radial column chart and histogram) while demonstrating a strong ability to quickly translate to retainable knowledge. Both the histogram and bar graph proved to be key representations with the largest amount of participants able to quickly identify, companies and analyze data presented in these formats. Surprisingly, the radar chart and radial column charts scored higher in certain categories like total number violation recall and categorical method understanding that warranted further investigation.
This study resulted in a narrowing in on effective data visualization techniques adding both the histogram and bar graph as core data representations and demonstrated a clear need for further testing of radial visualization types.
Phase 2:
Radial Data Visual Display Complexity Testing
The study was designed to test variations of radial display formats and their ability to translate various levels of complexity. Five data visualizations where tested for user understanding and knowledge retention. Literature review focused on cognitive load testing, numerical inference, and grouping theories.
Number of participants: 60
Age of Participants: 18-30
Interview length: 5 Minutes
Location of research venues: Virtual - Discord App
Task:
Assess radial data visual displays which were identified in the pervious phase and demonstrate by their ability to quickly translate our dataset.
Research Synopsis
A total of 15 radial display variants were made. For each of the five identified data visualization three variants were created that subdivided company legal data into 4, 6, and 8 axis categories respectively. This was done to test not just the effects of visual complexity on retention.
Participants were given access to an Adobe XD model of the randomly chosen display variant and allowed 30 seconds to interact with the content. Immediately after the interaction, participants were asked a series of questions that tested data retention and understanding.
Radial Data Visualizations Types Tested
-
Radial Bar Chart
-
Radial Column Graph
-
Sunburst Diagram
-
Nightingale Rose
-
Radar chart
In addition the the question in step one, two additional questions were added to more accurately judge user familiarity with radial data visualization types.
Additional Competencies Tested:
-
Familiarity of Representation
-
Can you talk about a time that you have seen data represented in this format?
-
How frequently do you experience data in the format that you viewed here today?
-
Findings Summary
Overall, both Radial Column Chart and Radar Chart excelled in this test. Across the board, each chart type performed better with fewer data points than it's more complex counterpart. The Radar chart performed remarkably, with a 90% correct response rate in the four axis option, but was also most notable in declining rate of correct answers at higher complexity levels with an 8 point radial diagram performing among the worst in the study. Interestingly, Radial Bar Chart and Radial Column Graph were the most consistent regardless of data point complexity, exhibiting the least variance between 4 data axis and 8 data axis alternatives.
This study again narrowed the types of data visualizations considered for the application display; resulting in the inclusion of radial column graph as the primary data visualization type. This study helped determine how complexity of data visualization effects knowledge retention and tested users familiarity with the given representations.
Phase 3:
Company Information Display Relevance Testing
This phase was designed to identify the most relevant demographic information about parent companies that users deem crucial when making informed judgments about a company's operational efficacy. The study aimed to test various data points for their recall and impact, ensuring that the most critical information is prioritized in the interface. The literature review focused on information processing, recall, ethical consumption, and consumer behavior to guide the study's methodology.
Number of participants: 26
Age of Participants: 18-30
Interview length: 5 Minutes
Location of research venues: Virtual - Discord App
Task:
Participants were tasked with assessing the impact of various demographic data points on their interest and recall levels. The goal was to better understand which pieces of company information are most relevant to users when forming judgments about corporate ethics and operational practices.
Research Synopsis
Participants were presented with an Adobe XD model showcasing a randomly selected company information display. Each display variant included different combinations of data points to evaluate which details had the most significant impact on user recall and interest. Participants were given 30 seconds to review the display before it was removed, followed by a series of questions designed to assess their retention and understanding of the information presented.
-
HQ Location
-
Profit Margin
-
Total Profit
-
Total Revenue
-
Total Global Employees
-
Total US Employees
-
Y/Y Trends
-
Executive Pay Ratios
-
Key People (C-Suite)
-
Stock Price
-
Date Founded
-
Key Sectors
-
Entity Type
-
Total Charitable Contributions
In addition to the primary questions, participants were also asked about their familiarity with the types of data visualizations used in this phase, helping to gauge how prior exposure to such formats might influence their comprehension.
Findings Summary
The results from this phase highlighted a clear preference among participants for financial statistics, employee counts, stock prices, and the founding date of the company. These data points were the most frequently recalled and considered the most relevant for assessing a company’s operational and ethical standing. Participants demonstrated a strong inclination toward using these metrics to quickly gauge a company’s stability and performance, indicating their central role in user decision-making processes.
Data points such as Key People (C-Suite), Executive Pay Ratios, and Key Sectors were less frequently recalled and generally regarded as less relevant. This suggests that while these details may hold significance in specific contexts, they do not resonate as strongly with users when forming initial judgments about a company. The lower recall and relevance of these data points highlight the need to prioritize information that directly impacts user evaluations of corporate efficacy.
These findings will inform the prioritization of information in future interface designs, ensuring that users are presented with the most impactful data first. By focusing on the metrics that users find most relevant, we can enhance both the usability and effectiveness of the interface, ultimately supporting more informed and efficient decision-making.
Next:
Phase 4:
Data Flower Development
The focus for this phase was on developing and testing unique data representations designed to enhance the association, familiarity, and readability of complex datasets. The aim was to explore innovative visualization techniques that could effectively communicate intricate information while being intuitive and accessible to users. This phase was informed by literature on visual cognition, data aesthetics, and information design, guiding the creation and evaluation of new representational forms.
Number of participants: 27
Age of Participants: 18-30
Interview length: 5 Minutes
Location of research venues: Virtual - Discord App
Task:
Participants were assigned one of several randomized variants of a unique data representation, referred to as the "Data Flower." Each variant was designed to test different aspects of data visualization, including the ability to foster intuitive associations, user familiarity with the format, and overall readability. Unlike previous phases, participants were allowed to reference the data while answering questions to assess how well the design facilitated ongoing comprehension.
Research Synopsis
The "Data Flower" visualization was conceptualized as a multi-faceted, radial design that combined elements of traditional charts with innovative visual motifs. Each variant differed in how data was segmented and displayed, with variations in color schemes, petal shapes, and central focal points to test their impact on user engagement and understanding. Participants interacted with their assigned variant while answering a series of questions designed to evaluate their ability to accurately interpret and recall the presented information.
Questions Assessed:
-
Association Accuracy:
-
What relationship can you identify between the central and outer data points?
-
-
Familiarity:
-
Have you encountered similar visual representations in other contexts?
-
-
Readability:
-
How easy was it to understand the information presented in this format?
-
-
Data Comprehension:
-
What was the key data trend or pattern you observed?
-
-
Information Navigation:
-
How did you locate specific data points within the design?
-
Findings Summary
The results from this phase provided valuable insights into the effectiveness of the "Data Flower" as a unique data representation. Participants generally found the visualization to be visually engaging, with most reporting a strong association between the central and outer data points. However, familiarity with the format varied significantly, with many participants indicating that they had not encountered a similar design before. Despite this, the majority found the design to be readable and were able to accurately navigate and interpret the data presented.
The variants that utilized simpler, more uniform petal shapes and color schemes were rated higher in terms of readability and comprehension. In contrast, more complex designs, while visually striking, sometimes led to confusion and required additional cognitive effort to decode. This suggests that while innovative data representations can enhance engagement, they must be carefully balanced with clarity and ease of use.
Phase 5:
Refined Design Testing
In this phase, the focus was on refining the design elements of the data visualizations, specifically by removing unnecessary grading systems and adjusting the spatial arrangement of key objects. The objective was to streamline the interface, reduce cognitive load, and enhance both usability and visual appeal. This phase drew on research in visual hierarchy, cognitive load theory, and user interface design to guide the iterative modifications made to the visualizations.
Number of participants: 30
Age of Participants: 18-30
Interview length: 10 Minutes
Location of research venues: Virtual - Discord App
Task:
Participants interacted with updated versions of the data visualizations where grading systems (such as color-coded scales or numeric ratings) were removed, and key data elements were repositioned for improved visual flow. They were asked to evaluate these refined designs and provide feedback on their clarity, usability, and effectiveness in conveying information. A comparative analysis was also conducted, asking participants to reflect on their experience with previous versions of the designs.
Research Synopsis
The design refinements involved the systematic removal of grading systems, which had been identified as potentially problematic in earlier testing phases. It was observed that these grading elements often pre-supposed a level of user understanding and recall that could not be consistently assumed, particularly when dealing with specific data points about a company. The assumption that users could make nuanced judgment calls based on these grading systems was not always valid, leading to confusion and misinterpretation.
Key data points were relocated within the visualizations to enhance the logical flow of information. These changes aimed to reduce visual clutter, ensure that the interface remained intuitive, and facilitate a clearer focus on the most relevant data.
Participants were tasked with navigating the refined designs and responding to a series of questions assessing their ability to comprehend and engage with the visualizations. The feedback provided was instrumental in evaluating the success of these refinements and identifying any further improvements necessary to optimize the design.
Questions Assessed:
-
Clarity of Information:
-
How clearly were the key data points presented in this version?
-
-
Ease of Navigation:
-
Was it easier to locate and interpret the information without the grading systems?
-
-
Visual Flow:
-
How effective was the new arrangement of objects in guiding your attention?
-
-
Comparative Usability:
-
How does this refined design compare to the previous version in terms of ease of use?
-
-
Overall Satisfaction:
-
How satisfied were you with the overall design after these changes?
-
Findings Summary
The refinements made in this phase resulted in significant improvements to both the usability and clarity of the data visualizations. Participants reported that the removal of grading systems reduced visual noise and allowed them to focus more effectively on critical data points. The assumption that users could make accurate judgment calls based on these grading elements was shown to be flawed, particularly when participants lacked specific knowledge about the company in question. The elimination of these systems thus improved user comprehension by presenting data in a more straightforward manner.
The redesigned spatial arrangement of objects within the visualizations was also positively received. Participants noted that the new layout facilitated a more intuitive flow of information, making it easier to navigate and understand the data presented. This led to a marked improvement in overall satisfaction and ease of use compared to earlier design iterations.
These findings underscore the importance of simplifying visual representations and avoiding elements that presuppose user expertise or detailed prior knowledge. The insights gained from this phase will be critical as we move forward to the final phase of testing, where comprehensive usability assessments will determine the design’s readiness for broader application.
Phase 6:
Click Through and UX Flow Testing
The final phase of the research focused on evaluating the overall usability of the interface through Click Through and UX Flow Testing. The goal was to assess how well users could navigate the interface, interact with various elements, and complete tasks efficiently. This phase built on the refinements made in earlier phases, testing the cumulative effects of design changes on user experience. The literature guiding this phase emphasized human-computer interaction, task efficiency, and interface usability.
Number of participants: 16
Age of Participants: 18-30
Interview length: 15 Minutes
Location of research venues: Virtual - Discord App
Task:
Participants were asked to perform a series of tasks that required them to navigate through the interface, locate specific information, and complete actions such as comparing company data or finding detailed reports. The tasks were designed to mimic real-world use cases, providing a robust test of the interface’s usability. Participants’ interactions were tracked to analyze click patterns, task completion times, and any difficulties encountered during the process.
Research Synopsis
This phase involved the comprehensive testing of the interface’s click-through paths and overall user experience (UX) flow. After implementing design refinements from previous phases, the focus was on ensuring that users could efficiently and intuitively navigate the interface without encountering unnecessary friction. The study sought to identify any remaining usability issues and to evaluate how well the design facilitated smooth and logical user interactions.
Participants were given a set of predefined tasks that required them to engage with multiple aspects of the interface. These tasks were carefully structured to assess key usability metrics such as task completion time, error rates, and user satisfaction. Additionally, participants provided qualitative feedback on their experience, highlighting any areas where the interface excelled or fell short.
Metrics Assessed:
-
Task Completion Time:
-
How quickly were participants able to complete each task?
-
-
Click Efficiency:
-
How many clicks were required to accomplish the tasks?
-
-
Error Rate:
-
How often did participants encounter errors or dead-ends in the navigation flow?
-
-
User Satisfaction:
-
How satisfied were participants with the overall usability of the interface?
-
-
Intuitiveness of Flow:
-
How natural and logical did participants find the sequence of actions required to complete tasks?
-
Findings Summary
The findings from this phase demonstrated significant improvements in overall usability and user satisfaction compared to earlier iterations of the interface. The streamlined design and refined UX flow allowed participants to complete tasks more quickly and with fewer clicks, indicating a higher level of efficiency. The reduced error rates further underscored the effectiveness of the design changes, with participants encountering fewer navigation issues and dead-ends.
Participants generally found the interface intuitive, with many noting that the flow of actions felt natural and logically sequenced. This was particularly evident in tasks that required accessing detailed reports or comparing data, where the refined click-through paths facilitated quick and straightforward navigation.
The feedback highlighted that the design changes, such as the removal of redundant grading systems and the reorganization of key data points, contributed significantly to the improved user experience.
Some areas for further refinement were identified. A few participants noted that while the overall flow was much improved, there were still occasional moments of hesitation when locating less prominent features or information. This suggests a need for minor adjustments to further enhance discoverability and streamline the UX flow.
Confirmed that the design is now well-aligned with users’ needs and expectations, offering an efficient and intuitive experience. The insights gained will guide the final adjustments before the interface is deployed in a broader context, ensuring it meets the highest standards of usability.