Turns out STEM vocabulary has nothing to do with fruits that have stems, and an apple a day won’t exactly keep the AI away. We used artificial intelligence to customize games based on student knowledge while learning STEM vocabulary. The game has the power to generate images for abstract words and customize the levels based on student’s skills. Now that’s something we could sink our teeth into.
This project had a team of four UX designers. My job during the project was of UX Research and Product Designer.
6 Months
XD, Illustrator, Lucidchart, Miro
TapNotion designs and builds systems for generating STEM vocabulary games and activities forthe education and digital media markets. Based on established and novel educationalpedagogies developed with linguistics and educational experts, the dynamic TapNotion systemproduces personalized games and activities tailored to the needs of individual users.
Luckily, I help the developer design a poster to present the game's algorithm to the Python community. During this time, I worked closely with the leading developer to understand the logic of the process and iterate ideas to simplify complexity. The creation of the poster helped me to understand how the game would work and gave me ideas on how to approach the research.
Because I knew this project was too complex and a lot of work needed to be done, I reach out to three UX designers who volunteer their time to this novel project. After putting the team back to speed, we began gathering ideas on how to approach this project by exploring most efficient methods.
What is the purpose of the tool? to help students learn STEM vocabulary.
Who am I designing for? K12 Students
What is the primary goal? Motivate students to pursue STEM fields.
Who is the user? Students and Teachers
What is the timeframe? 7 weeks
After a large cup of coffee, the next step was to create a list of essential topics to monitor success and startup requirements.
Before we began our interviews, the team had to negotiate participants age with startup founders. Due to USA regulations, conducting research with minors would have been time consuming and challenging to work with. Therefore, we have to work with University students for most of our research.
Surveys
Through our survey responses, we found that all participants except one have used a learning platform, whether it be currently or in the past. The majority of our participants (77.8%) have used an online learning platform such as Khan Academy or Lynda, which shows their willingness to learn on a web-based application.
Access to a dictionary
Simple, straight forward game design
The ability to play both alone or with friends
Gain points and exchange those points for help within the game
Interviews
We also recruited participants at the DePaul University library. All participants were between 20 to 27 years of age and had taken a STEM course in the last few years.
After obtaining informed consent, we followed an interview script to ask participants about 1) the participants' experience with technologies in the classroom, and 2) any experience with learning software in the past and present.
Ten interviews were conducted in cafes and at the DePaul University library, and two were conducted in the researchers' homes. The interviews took about fifteen minutes to complete. After the interviews were completed, we transferred all the data to a spreadsheet, which we then transferred to Miro (online whiteboard tool) to use for an affinity diagram. Here is an example of how we created an Excel sheet to collect our findings.
Students motivation when learning STEM vocabulary
Students understand the importance of STEM courses. There are many methods that currently help students learn, such as YouTube videos, flashcards or writing down words multiple times. Interesting enough, none of our participants were able to name a specific game that helps students learn STEM vocabulary.
Current games facilitating the learning process of abstract topics
Most students want to win, and if the students find the game too difficult, they could lose interest in the game. If this happens, students are not willing to play the game again. Consequently, the game needs to quiz the knowledge of the students before they start playing, as well as staying entertaining enough to keep the student engaged in the learning process.
Most students agree that games have the ability to engage students in learning difficult topics.
Stimulating intrinsic learning
Participants mentioned that they would like to play with friends or with others through social media. By adding social context to the game, students can challenge each other or collaborate on difficult topics.
Social aspects and competition are stimulating intrinsic learning. When students play against each other, they are motivated to learn faster to be able to keep up with other players. But we also found that students want to spend time playing alone as well, meaning that social aspects should be optional.
STEM domains are underserved and expensive to teach, therefore, it is important to bring down the cost of teaching STEM. A gamified learning tool is needed, to help students stimulate motivation and improve engagement in learning vocabularies from four disciplines such as Science, Technology, Engineering and Mathematics by automating learning experiences.
We used three methods to analyze data: 1) organized survey responses by the sections of the survey layout in a spreadsheet to find the most common issues and insights, 2) created a Flow Chart , and 3) created a Conceptual Model. We used both the survey data and the interview data to build personas for our prototype
The app will allow users to choose their preferences for travel purpose (for pleasure, business, or both), location, length of stay, and activities. Then the application can provide the users with a checklist of items to bring. In addition, the app will be able to estimate the baggage weights and give the user an idea of how much they must pay for overweight luggage. Once the list is completed, the application will tell the user to start the checking process. As soon as the checking process is completed, the app will advise the user base on the final destination on what documents and accessories the user needs for their trip. Finally, when the check list is complete, the application will ask the user if it needs a taxi or an uber.
A task flow does not explain all the details of the interactions, but it provides a comprehensive view of the user experience for a particular objective. The purpose of our user flow was to understand types of tasks, how students interact with each other, type of information needed and to shows the framework of the game. In order to get here, the team needed to work together to make sure everyone in the team agrees and Fully understand the flow, before we jump into our low Fidelity prototype.
The app will allow users to choose their preferences for travel purpose (for pleasure, business, or both), location, length of stay, and activities. Then the application can provide the users with a checklist of items to bring. In addition, the app will be able to estimate the baggage weights and give the user an idea of how much they must pay for overweight luggage. Once the list is completed, the application will tell the user to start the checking process. As soon as the checking process is completed, the app will advise the user base on the final destination on what documents and accessories the user needs for their trip. Finally, when the check list is complete, the application will ask the user if it needs a taxi or an uber.
First Round of Usability Testing
In the first round of user testing, we found various results, both between the tasks and between participants
Participants didn’t fully understand why they were being asked questions after they logged in, and just wanted to skip into the game.
Participant did not understand what the icons meant.
We wanted them to enter the game through clicking the pin on the map, yet most of them entered the game through the listed games at the top of the screen
Two participants noted that they thought that clicking on the “I don’t know” button would cause them to fail the game, which is why they were hesitant.
Second Round of Usability Testing
Based off the results from the first round of testing, we were able to iterate our prototype and gothrough a second round of testing to see if the usability issues had improved or been fixed.
Participants noted that they assumed they would be able to challenge friends through the ‘friend button’ on the dashboard, instead of having to go into the game.
Major issues we still ran into were participants commenting on the icons of the dashboard being too similar, noting that there seem to be too many paths to challenge a friend to play, and participants not being sure if they had finished as task due to there being no confirm page when they added a friend.
AB Testing
Our participants complimented the simple game design, as well as the usefulness of the application.
For our A/B testing, our findings were based on a small participant pool (n=8). Although we didnot find a significant difference between the two tasks, we feel that the results might have been different if we had a larger subject pool. In the future we recommend at least doubling the amount of participants. Based on our results and previous comments from user testing, wedecided to keep both pathways (inviting a friend). We want to give the player both options to beable to challenge friends from within the game, and also on the main screen to minimize the amount of clicks depending on where they were.
The infographic below shows the time in seconds it took for each participant to finish the task, for both the A test and the B test. After conducting our t-test, we found that there was no significant difference between the dashboard (M= 26.75, SD=21) and the game page (M= 31, SD=11.5) at alpha level .05 (T(6)= -.35, p=.74)
The app will allow users to choose their preferences for travel purpose (for pleasure, business, or both), location, length of stay, and activities. Then the application can provide the users with a checklist of items to bring. In addition, the app will be able to estimate the baggage weights and give the user an idea of how much they must pay for overweight luggage. Once the list is completed, the application will tell the user to start the checking process. As soon as the checking process is completed, the app will advise the user base on the final destination on what documents and accessories the user needs for their trip. Finally, when the check list is complete, the application will ask the user if it needs a taxi or an uber.