A peek into the research being done around the dark or light visual theme for PatternFly 4 navigation
A major goal with PatternFly 4 was to create an updated, modern look and feel with our visual design language. To meet that goal, we revamped every component to provide something that looks great and is flexible enough to be applied across the system.
One of the most obvious differences between the new visual language and previous versions of PatternFly is the direction to proceed with a light versus a dark visual theme for the navigation.
So far, the new light theme has been met with mixed reactions from the PatternFly community, so we leaned on research to help us make an informed decision on the best direction forward. To proceed with the light theme, or adjust to a dark theme?
- Light theme vs. dark theme
Here’s a recap of what’s been done and what’s still to come..
At Red Hat Summit 2019, we conducted a study to gather both objective and subjective feedback on two visual design variations for PatternFly’s navigation system. The light theme was designed for PatternFly 4, and the dark theme came from a previous version of PatternFly (see image above). In addition to the navigation color, the themes also differed in font, overall spacing, and layout.
During the study, we gathered 68 survey responses and worked with 6 participants in a moderated research setting. We also worked remotely with 61 Mechanical Turk users from IT domains to reach higher confidence in reporting the results. In total, we had 135 respondents who were primarily Engineers, SysAdmins, and Developers.
We looked at the designs from three different user perspectives: performance, perception, and preference. Participants were asked to complete two first-click tasks and two general tasks within the mock-ups to gather performance data. Then we asked participants to provide feedback on their perception and preferences.
- Performance: There were no statistically significant differences between the two variations
- Perception: There were no statistically significant differences between the two variations when users evaluated each theme individually
- Preference: Users generally preferred the dark variation when they evaluated the two themes side-by-side
A good number of respondents suggested that the light theme seemed higher quality and they liked the simplicity of it. However, most respondents also stated that the light version appeared “too white/plain and need[ed] more contrast”. They also mentioned that they liked the clear separation of space in the dark variation.
Results showed that the dark variation is generally preferred mostly due to the higher contrast between different sections of the page. However, respondents suggested that the light theme looked more modern and high quality compared to the dark theme.
Also, while performance data suggested that the light theme resulted in a slightly higher success rate (though not statistically significant), we can’t be confident this result is due to the color of the navigation. We suspect that the bigger font and better spacing in the light theme may have made the text more legible and easier to read, but we can not be sure at this point.
- Since the two variations were different in terms of font, spacing and layout, and navigation color, we can’t say which UI factor played a more prominent role in the results, especially when it comes to the legibility of the designs.
- We utilized a survey as the main method of data collection at the Summit because that seemed to be the most feasible method for that context. But that resulted in presenting design variations in a non-realistic format.
- The mockups of PF3 that were used in the survey missed a vertical visual pointer on the left side of a category in the navigation bar. We suspect that mistake could have impacted the success rate of a task in the dark variation.
- Additionally, according to our previous research we suspect the content areas may also affect user preference and perception and requires further investigation.
Given the constraints, in the next round of study we’ll:
- Have consistent font, spacing, and layout across variations to investigate the effect of color more precisely.
- Use an unmoderated platform that will allow participants to see the designs in their real states.
- Make sure the test materials are identical and that the only difference is the color theme.
- Include two different variations of the content area (list view vs. dashboard).
Our goal is to address the following questions in the upcoming follow-up research to help the team to make an informed decision about the color theme:
- Which color theme results in a better performance?
- Does the content within the content area affect performance?
- Do color and content interact to affect performance?
- Do users prefer one color over the other?
We expect to complete this study and have a summary of the results in the coming weeks. Once we have this, we will post another update sharing the findings and explaining PatternFly’s next steps. Feel free to reach out in the meantime with any questions or comments.