Hack for LA is a nonprofit that brings together a network of civic-minded technologists who contribute their skills for the local government and community service of Los Angeles.
Project: Civic Tech Jobs
The project I worked on within the Hack for LA organization was called Civic Tech Jobs. The purpose of this project was to create a separate platform from the Hack for LA site to specifically post and search for roles within the organization.
Role: UX Researcher
Project duration: On-going
My research role: Oct 2021 - July 2022
Case focus: This case focuses on the role posting process - Specifically how Project Leads post and manage roles for their teams within the Hack for LA organization.
Tools: Survey Monkey, Figma, Zoom
The major hurdle at Hack
Hack for LA has a ton of great projects that need technical support. Anyone looking to advance their IT skillset has a variety of efforts to choose from. However, major difficulties exist in both finding available projects and figuring out how to get started on them. The process is not straightforward.
This case focuses on developing a solution to clarify and simplify the role posting process.
The proposed change
The method for managing volunteer roles through Github is confusing and cumbersome. The team set out to create a platform specific solution to showcasing volunteer opportunities for technical professionals that would not require the use of Github.
The UX design team came up with some lower fidelity prototypes to test before moving forward with a high fidelity option. We wanted to gain feedback on an initial concept so that we could move towards a more efficient solution. I was a member of the UX research team for this project and I conducted interviews to test these prototypes.
What do the Project Leads think of these changes?
We interviewed and tested project leads that were actively involved in various projects across the Hack for LA organization. We had mix of project leads who were familiar with the Github process as well as project leads who had not yet managed roles using Github.
Number of Participants: 11 Project Leads
from Ops, Development, and UXR
Duration: 25 - 40 minutes
Platforms: Zoom with screen sharing and recording and
Figma for prototypes
Screen Size: Desktop
Research focus:
Assess current Github process of posting a role
- Areas that are working
- Pain points
Understand what users think of the draft dashboard
- Flow
- Points of confusion
- Suggestions for improvement
Determine specific fields needed for role posting
- Are there aspects/categories that are missing?
- How would they modify this to fit their needs?
Recommendation method
The UX research team came up with standardized recommendations so that we could provide consistent feedback across the study. Our goal was to present our findings as well as to express the significance of each result. In giving the significance of our findings, it helps the UX design team to better control their product roadmap on the way to creating a Minimum Viable Product.
Must Have
These findings are vital to the product. Without adhering to these recommendations, the product won’t work or would become useless. Priority item for MVP.
Should Have
These findings are important, but not absolutely mandatory. These items support core functionality (that will be painful to leave out), but the product will still work without them. If schedule is an issue, these can be left out for MVP, but should be added as soon as resources permit.
Nice to Have
These findings are not essential, but wanted and nice to have. They have a small impact if left out of the MVP.
Starting with the Github look and feel
All members of Hack for LA are required to have a Github account. To gain access, members must communicate their Github account information to the Hack for LA coordinator so that they may be connected to the Hack Github repository. Once access is allowed, all members must use this system to post and search for roles.
Thoughts on the current situation
We knew that the current process to sign up and post roles was not ideal; however, we wanted to get details about what exactly the users did not like so these issues could be addressed in the future iterations.
After interviewing the Project Leads, 3 major issues were identified.
Disconnect between joining a project via Github and actually starting the role
Github serves as a repository for users to post and view roles. The available position listings state the job function as well as a description about the position. What is missing are any of the actual logistics when it comes to starting the role.
Leads complained of lack of clarity in posting process
There are no set standards for how and when Project Leads should post roles to Github.
Role Posting techniques differed among leads
The Project Leads we interviewed had varying methods for managing roles.
- Some leads stated that they post one role within their project and keep it open indefinitely to have a continuous recruitment cycle
- Other leads reporting using vague role descriptions so that the posting could be a catch-all
- 1 lead reported using ranges of open roles for role posts (1-2 people, 2-3 people) depending on the commitment people were willing to give
The Prototypes
The Dashboard
The first frame in the flow shown to our participants displays the current available roles according to position type and allows for the management of roles.
Frame insights
This initial frame elicited plenty of strong reactions. 3 features stood out that participants had negative comments about. They are listed below.
Dual sides
The first observation the majority of participants commented on was the dual screens. They used words like "confusing" and "hard to focus". They also did not like that they felt like they were multiasking when viewing the screen.
“I don’t care about the right side. No need to complicate it more with this.”
Level of Accessibility
When this simple frame was designed the Civic Tech Jobs team was thinking about projects being managed on an individual team level. We did not take into consideration that there could be a use for managing projects at the enterprise level. It was interesting to learn through interviews that the majority of testers assumed the dashboard was at the enterprise level and not the team level. Many comments were made that supported an enterprise level dashboard. Additionally, because we had "assumed" the frame was at a team level, there was no label to let users know one way or the other.
"Not clear from the jump."
From these comments, we learned that co-led teams, the continual flow of changing leads, and that some teams recruited from other teams. All of these reasons supported a more accessible dashboard to allow for Project Leads to see beyond their own individual team activities.
Last Posted
The "last posted" title on the frame was off putting to users. Because this frame had the ability to contain several projects, the words "last posted" were found to be confusing since it wasn't clear which project the "last posted" was actually referring to.
Recommendations
Based on the insights gained from the interview portion regarding Github experiences as well as the initial frame we showed to users, the following encouragements were given the the team.
Must haves
Findings: Some leads are confused with current role posting process. The steps that take place after a listing is posted on Github is not clear.
Recommendations: The recruitment process needs to be made transparent. Direct language, diagram, or tutorial should be added explaining the recruitment process. If Project Leads are confused about the project, those just starting out are bound to be confused about the process as well.
Findings: Dual sides of the frame are distracting. Many participants felt their attention was split.
Recommendations: Only focus on one function per frame at a time. The frame that displays open roles should be distinct from the frame that allows Project Leads to manage roles.
Findings: Project Leads were confused by the dashboard access level as there was no information regarding which projects will be presented. Some Project Leads shared their responsibilities with others on the team and expressed the need for multiple users to have access at the same time. Other Project Leads managed multiple teams and needed insight into all their teams on a single dashboard.
Recommendations: Make dashboard accessible at the enterprise level and label it accordingly for clarity.
Should haves
Findings: Leads want roles to be identified beyond Community of Practice level. The Community of Practice simply identifies the subject area of the role. Examples of this include the subjects of UX Design, Developers, and Project Management. To identify roles beyond Communities of Practice would be to further break down the overall subject areas into more specific categories. Suggestions include breaking down UX Design into UX Design and UX Research. Developers could be separated by Front-End, Back-End, and Full-Stack Developers.
Recommendations: Allow for the functional role title such as UX Researcher, Back-end Developer, etc. to allow for further detail so projects may be distinguished among each other.
Findings: "Last Posted" identification does not make sense when multiple roles are represented.
Recommendations: Remove "Last Posted" from the current page. It does not make sense when multiple roles are listed under same Community of Practice.
Nice to haves
Findings: One participant requested the current status of positions and the number of openings. It was discovered during interviews that there is a benefit to Project Leads having insight to projects at an enterprise level. These two findings complemented one another.
Recommendations: List current status of all team positions, filled and those that are open, to allow for a more holistic view of the team compositions.
Dashboard Screen 2
The intention of this second frame was a two-fold. On the left side of the screen, the aim was to allow the open roles to be shown on a more granular level. The right side of the screen served to further the path of posting a role.
Research Insights
The second frame was better understood
The participants had a much more positive reaction to this screen. Some found this page to be much clearer that the previous frame. Others liked the indent and color change to show individual roles. The “Last Posted On” title was found to be useful and made sense to the participants.
Still some confusion
Within the project management roles section, to differentiate the roles, the Civic Tech Jobs team labeled them as "Project Manager I" and "Project Manager II". Some participants viewed this as a manner of hierarchy instead of simply to differentiate between two different roles with the same function.
Recommendations
Must haves
Findings: Listing roles titles with “1” and “2” to distinguish them from one another creates wrong impression and signals a hierarchy to many.
Recommendations: Use the actual titles such as "Full Stack Developer" instead of simply stating "Developer" when possible. Allow for greater description to be included within each title so that similar roles may be distinguished from one another.
Should haves
Findings: Page 1 is redundant to Page 2; Page 1 simply has less info.
Recommendations: Start the dashboard at Page 2. Design can allow for a dropdown feature to collapse roles within a Community of Practice if a more condensed look is preferred.
Nice to haves
Findings: It's not always clear that roles listed represented individual roles within the same Community of Practice
Recommendations: Use a dropdown feature to see individual roles within Community of Practice, or hover for more information
Searching for New Roles
This frame shows how Project Leads start to post their available roles.
Research Insights
Frame title
Labeling the frame "Volunteer Roles Portal" made the testers think that this screen was for the volunteers to select a role as opposed to where Project Leads would post a role.
Time Commitment dropdown
We were surprised to learn that there were so many different ways to interpret the time commitment feature. Because there was no unit of measure, participants understood this term in a number of ways. This dropdown could refer to the number of hours per week one would need to commit, this field could also mean the number of times per week volunteers were expected to meet, and finally the time commitment number could refer to the number of months one would need to commit to the project.
Spots Available
The term "spots" was not understood by all. Hack for LA recruits users from around the world so it is important to use plain language that may be understood by all.
"Spots available is a bit weird. I would say how many roles are open."
Recommendations
Must haves
Findings: A lot of confusion revolved around the title “Volunteer Roles Portal”. Participants were not clear about who had access to the page. They also did not find it clear that this was an area where Project Leads would post roles as opposed to it being a page for the volunteers.
Recommendations: Change the title so that it is clear this for Project Leads to manage roles.
Findings: The term "time commitment" was interpreted multiple ways.
Recommendations: Further define what type of time commitment is being requested and allow for both length of time and number of hours per week to be input on each position as many Project Leads voiced a need for these areas to be identified.
Nice to haves
Findings: Word "spot" is not easily understood by all.
Recommendations: Use an alternative such as "open positions" so more participants understand what this section identifies.
Dashboard Screen 3
This screen gets deeper into the role posting flow and allows descriptive information explaining the project activity and expectations in greater detail.
Research Insights
Each of the entry fields is very important to the posting process. Useful insights were gathered for each area.
Project dropdown
The participants had different expectations for what they felt should appear in the dropdown. Some assumed all Hack for LA projects would be listed here. Others guessed that only their own team projects would appear in the dropdown. Some testers did mention they would like the ability to have a blank field or add-in entry field to be able to add projects in case they were not listed. Additionally, some had complaints about the project name not being fully displayed within the dropdown.
Role title dropdown
Participants remarked again on the role title not allowing for enough characters to show the full project title. Another point of contention was that Project Leads wanted to be able to clearly read if the posted role was for leads or teammates.
Role description
For this field our main goal was to find out if our Project Leads would benefit from the field being auto-populated. The answer was a unanimous "yes". Any actions that could be taken to reduce the amount of time they had to spend on posting roles would be welcomed.
Skills
This field was of particular importance to the Civic Tech Jobs team as well as for the Project Leads. The skills section could help to identify the exact needs of the team. However, as we learned through testing, a skills section could also be used to exclude or dissuade potential volunteers from joining the team. Some Project Leads liked the term “skills” and thought it suited their needs. However, some preferred terms such as “technology” or “tools” as these words were more specific than general skills. 6 people really liked “willing to learn” as it was more inclusive to recruitment.
"I would list the tools we use
and some common methodologies. My hesitation here is that I don't want it to
sound too limiting, like we only use interviews and usability tests. Language
would be: including but not limited to..."
UXR Lead
"In my team, if you're willing
to learn the different languages and technologies, you can join. It might be
different in other teams. So this is fine as long as I can leave ‘required’
blank to just have willing to learn in my posting.”
Development Lead
Recommendations
Must haves
Findings: Project leads were concerned that all projects titles would not appear in the drop down. The Hack for LA organization has many moving pieces and one cannot assume that the tool used to post jobs will always be updated in real time.
Recommendations: Because this was a strong concern for the Project Leads, measures must be put in place to ensure all projects have the ability to be listed. Allow for blank or add-in entry for projects so no project gets left out. Future A/B tests and usability could help to determine the most ideal way for the project dropdown to appear and function.
Findings: Project dropdown field is a bit short and may cut off teams’ names.
Recommendations: Extend the dropdown field to allow for longer names.
Findings: Project Leads perceive a differentiation between recruiting for team members versus recruiting for more Project Leads even when the position title and function read the same.
Recommendations: Add and option to identify a role as a lead role. An option is to include a checkbox for “Team Lead” to go along with each role selected when posting.
Findings: Auto population for all fields was found to be preferred, but only if the edit feature was allowed.
Recommendations: Allow for text to be autopopulated or use placeholder text for the role title, project description, and meetings. Having the edit function is a must for each of these areas.
Findings: Teams have different needs regarding the use of a field titled, "skills". Some Project Leads preferred the term skills while others preferred having the option of “required” and “willing to learn”.
Recommendations: Allow for “required skills” and “willing to learn” to be left empty as needed. There is also potential to add the term “soft skills” or other verbiage. Allow flexibility in language as different roles and different teams have different needs.
Should haves
Findings: Role: Leads wanted to further define positions beyond Community of Practice
Recommendations: Use actual position titles instead of Communities of Practice
Findings: Leads are unable to navigate within the posting.
Recommendations: Allow for back buttons as well as an auto-save feature so that users may move about the flow without information being lost.
Findings: Some leads would like to have previous examples for their posts.
Recommendations: Autopop and placeholder text should help to support this. Also, the ability to go back in frames without losing information would be useful.
Nice to haves
Findings: 2 Project Leads reported that they would like to specify a date for when the volunteer must be available.
Recommendations: Add this category separately, or could this information could be made part of the “role description” category.
Findings: Some leads wanted to see a preview before the posting goes live.
Recommendations: Add preview screen before posting as this feature would be useful to those wishing to proofread their postings before going live.
Retrospective
Lessons Learned
I found it very useful to conduct this testing with flows that were not fully flushed out. I was able to gain very useful insight that helped to guide the design teams overall strategy moving forward. Getting feedback early on ensured that we did not work too far ahead of ourselves. None of us on the Civic Tech Jobs team were married to any of the designs and we were all adaptable to change. Using these insights to come up with more developed designs was extremely helpful.
I do feel that we could have gained even more insight if we as a team would have taken a bit more time to review this initial prototype and create a few more of the flows that would be used frequently in the future. For example, we explored the role posting process, but we did not touch upon the management of the roles as shown in the dual screen frame. The frame gives the option of removing and editing the role, but these two paths were not created at all at this point. Low fidelity frames for both of these flows could have helped the team to gain even more valuable information for future designs.
Next Steps
The UX Design team has used the insights from this study to make changes to the flow above. At the time of this case, this project is still in progress.