"Talking with Friends": An Interactive Social Story for Students with Autism (2024 Walden Award Winner)

Recipient of the Daniel Walden Prize for Outstanding Undergraduate Paper. Originally presented in the Play Studies Area of the Annual Conference of the Mid-Atlantic Popular & American Culture Association. November 2024, Atlantic City, NJ

Middle school and high-school students on the autism spectrum exhibit social communication deficits that impede their ability to build relationships. 1  They may need assistance initiating conversation, creating an appropriate response or staying on topic. 2  More importantly, students may already know these skills, but be hesitant to apply those skills within their classrooms because they fear the negative effects caused by unintentionally poor social choices. However, a safe and playful alternative to address these issues are interactive social stories (ISS): learning tools that may help students develop social skills in a classroom setting. 3  The overall project involves developing an ISS through the JavaScript library p5.js, the content creation platform Visme, and the text-to-image AI service Midjourney. Subsequent expansions may utilize the Python programming language, with each new iteration undergoing qualitative and quantitative human testing. This presentation will provide an overview of an ISS currently under development, detailing the processes for creating the narrative script, wireframe, and lastly, a working coding prototype. The coding prototype will encourage socialization through four social skills: identifying appropriate conversation topics, choosing appropriate responses, conversing, and staying on topic. 4

When drafting this particular social story, I wanted to make sure that I was following Carol Gray’s guidelines appropriately. If you didn’t know, Carol Gray basically leveraged its power as an educational tool for students with autism. She did a ton of research proving its effectiveness in various social situations and later published a handbook that ultimately helped other educators create their own social stories. 5  Part of her guidelines described that these learning tools should be customized to meet the audience’s needs, catering to their specific learning styles, abilities, and interests. 6

For my particular social story, I imagined a conversation between two individuals: The player (who is the student) and the classmate the player interacts with. Keeping Carol Gray’s guidelines in mind, I began writing the initial dialogue. As you can see from my Word document, the player asks the classmate what the classmate will have for lunch. The classmate says, “Pepperoni Pizza.” From there, I initially gave players two choices to pick from. A) “That’s cool! I like to eat pizza too” and B) “No comment.” If the player chooses the wrong response, they would receive feedback in a low-stakes setting. The feedback wouldn’t be the same for each wrong answer choice chosen. Rather, to enunciate on the interactive component of the interactive social story, the feedback would cater to whatever answer choice the player chose. If the player chooses Choice B, which essentially is choosing not to respond, the conversation will end. The game will then output helpful advice that encourages the reader to pick a response. In fact, the game will say: “If I want to talk to my friend, I have to say something.” From there, the game will restart Exercise One, displaying only the appropriate response/responses. When we get to the wireframe, you will see that there is more than one appropriate response. But for this specific section, only the appropriate response will be Choice A. Once the player clicks that, the player can move on to Exercise Two. 

Moving back, on the first try, if the player had initially clicked on the appropriate response, Choice A, the player would have already moved on to the next exercise. The same goes for Choice B. The game will give feedback on that choice, then allow the player to start over again, only showing the appropriate answer. This errorless learning process directly prompts the player to select that as their second try. This process enhances the possibility of players applying these learned concepts to classrooms and communities, as that is the ultimate goal. 7  Hence, translating my thoughts onto paper contributed positively to the project’s overall development, enabling me to get my thoughts in a physical medium, while simultaneously providing me with a visual needed to communicate my ideas to my first reader, second reader, and colleagues. Furthermore, it helped me develop a better vision of how I wanted to design the wireframe and the other exercises contained in it.

The next logical step was to develop the entire scope of the wireframe, and I did this utilizing Visme. The wireframe has its introductory components, two specific topics in fact, and a total of four exercises that branch out of those specified topics. However, due to time constraints, I will only discuss the contents in Exercise 1 under the food topic. 8  If I zoom in accordingly, you all will be able to see that I added a couple of vital components. First, I added a goal before Exercise 1 explaining what specific skillset the player would learn after completing this part. Second, I’ve added speech bubbles to cater to a comic book-style approach. Since I am not an artist, character illustrations will be created via Midjourney, an AI image generator. Thus, instead of listing everything line by line, the player will fully understand who is asking the questions and, more importantly, who is answering them. 

In this slide, you can see that I’ve also added some visual components that would enhance the game aesthetics and player experience. I made the frame mimic what would be displayed at a computer screen, which will help the player familiarize themselves within this new digital environment. Here, I limited color contrast to ensure that the player would be able to concentrate on the content. My first reader had informed me that children with autism typically get distracted when such a large emphasis is placed on a bright color palette, especially when playtesting a prototype, so I made sure to take her advice seriously as I created the final slides of my wireframe. On a more playful side, I put a little smiley face next to the “Time to Test!” section to excite players about playing the game. I’ve placed the question “What will you say to your friend?” to prepare them for the exercise at hand. I’ve also added more answer choices that would provide different perspectives on what the student believes the correct answer could be. This was done to represent various response types: an on-topic response, an off-topic response, and an adamant refusal to respond, each with their own sets of consequences.

In fact, the game aims to cover as many response types as possible to curate an expected set of feedback and consequences for each choice, whether the selection was appropriate or inappropriate for the conversational flow. 9  Choosing Choice A or Choice B would lead to positive feedback, reinforced by the smiley face on the top right-hand corner. It also leads to an enthusiastic response from the game, stating, “When I listen to my friend and respond, it makes them happy!” On the other hand, choosing Choice C, an off-topic response, would lead to the opposite, in which the game gives the player feedback on how to navigate the situation best. In this case, the feedback is “If I want to talk to my friend, I need good listening ears.” Well, what if the player doesn’t choose a response at all? Don't worry; we have a slide for that. If, by some chance, the player decides to select Choice D, which is not to respond at all, the conversation will be over. The game will output, “If I want to talk to my friend, I have to say something,” encouraging players to pick a response. To top it all off, a sad face is placed in the top right-hand corner to reinforce the classmate’s reaction to the response. Both choices are prompted to restart the conversation. The only difference is that in Round 2, only the appropriate response is placed as the choice. All others will be eliminated. However, in this specific exercise, there are two appropriate responses, because I wanted players to know that there are different ways to answer the same question appropriately. Once players click on either of those, they will get positive feedback, providing support that ensures they are on the right path in learning the skill. 10

In turn, by laying the groundwork for these personalized aspects we will have a game students will be motivated to play and do well in, as they would feel they would have something to contribute into a greater cause. 11  As mentioned before, this is part of a larger wireframe. There are other exercises in this wireframe, four in each conversational topic. However, they are all arranged in the same way. They each have a goal, a small conversational exchange, a multiple choice section, a feedback section, and a second round if the player chooses incorrectly. Hence, after getting the approval from my first and second readers, I began putting it all together through code.

My goal for the coding portion was to represent my ideas in a playable way visually. It’s a manner of translating what was projected through the wireframe in a way that resembles the barebones of a game. In a more comprehensive sense, this portion was all about bringing my vision to life. The narrative script and the wireframe already provided some sort of conceptual understanding, especially when communicating my ideas to my first and second reader. However, now is that time where all that thought and conceptualization gets put into action. In this stage, I kept the aesthetic portion precisely how it was. I screenshotted specific slides from the wireframe created from Visme and attached them as images in this creative coding platform, p5js. 12  This relieved the burden of drawing the slides through code, which would take up a significant amount of time, further delaying the development process and, most importantly, preventing me from finishing my thesis—after all, I am still an undergraduate student, and there are deadlines I must work with. As we’ve seen before, users are given a goal before they are exposed to the exercise. In this exercise, the goal is to engage in the conversation. Let’s hit “Next”. Once again, we are exposed to various options displayed on the gaming screen. Keep in mind that this is no longer a wireframe. So, in whatever choice I make, I will automatically be exposed to such consequences.

 Let’s go through this alphabetically. Let’s go through “A: That’s cool! I like to eat pizza too!” Hm. I wonder what would happen. Oh great! “When I listen to my friend and respond, it makes them happy!” The smiley face kind of signals that the user picked the correct answer, so once I hit that next button, it automatically goes to the next exercise, Exercise 2. Let’s pick a different answer. We shall restart this exercise and return to where we left off. Let’s go with Choice B. “I don’t like pizza, but I like pasta.” Where does that take us? It’s a good answer! Let’s try again. Let’s go with Choice C. Guess what? We get feedback. And the feedback is worded exactly the same as the wireframe. It’s just that since it is more easily accessible it brings the social story’s interactive component to life. And, you get the idea with Choice D. 

So to summarize, what happens if the student chooses the “wrong answer?” The student gets feedback. They’re not berated with criticism. They’re not berated by the lasting negative implications of a bad first impression. Most importantly, the feedback is made to encourage them to try and try again. How do the players know that they are making progress and are on the right track? They are prompted to start the next exercise. And the switch is very apparent. You can see “Exercise Two” in bold with a goal at the bottom of it. 

So, what can we hope players learn from this? We hope players and students alike will learn that making good decisions leads them to advance into the next exercise, where they will be exposed to and practice new skill sets. On the other hand, making wrong decisions in this game encourages them to try again in a consequence-free environment, giving them the space needed to grow and thrive as students and people in their community. 13

In completing my undergraduate thesis capstone, I aim to continue my interactive social story during graduate study, conferences, and beta-testing sessions with private special education schools around the tristate area. 

Footnotes

• 1

  Erik W. Carter et al., “Promoting Social Competence and Peer Relationships for Adolescents With Autism Spectrum Disorders,” Remedial and Special Education 35, no. 2 (2014): 91–101, https://doi.org/10.1177/0741932513514618.

• 2

  Nirit Bauminger, et al., “Peer Interaction and Loneliness in High-Functioning Children with Autism,” Journal of Autism and Developmental Disorders 33, no. 5 (2003): 489–507, https://doi.org/10.1023/A:1025827427901.

• 3

  Carol A. Gray and Joy D. Garand, “Social Stories: Improving Responses of Students with Autism with Accurate Social Information,” Focus on Autistic Behavior 8, no. 1 (1993): 1–10, https://doi.org/10.1177/108835769300800101.

• 4

   Leasha M. Barry and Suzanne B. Burlew, “Using Social Stories to Teach Choice and Play Skills to Children with Autism,” Focus on Autism and Other Developmental Disabilities 19, no. 1 (2004): 45–51, https://doi.org/10.1177/10883576040190010601.

• 5

  Carol A. Gray, The New Social Story Book: 10th Anniversary Edition (Future Horizons, 2010). 

• 6

   Carol A. Gray, The New Social Story Book, Revised and Expanded 15th Anniversary Edition: Over 150 Social Stories that Teach Everyday Social Skills to Children and Adults with Autism and their Peers (Future Horizons, 2015).

• 7

  Jichen Zhu et al., “Using Interactive Social Story Games to Teach Social Skills to Children with Autism,” in Proceedings of  1st International Joint Conference of DiGRA and FDG. (Digital Games Research Association, 2016), https://doi.org/10.26503/dl.v2016i1.783.

• 8

  https://my.visme.co/view/dm4dr3xv-exercise-1-wireframe

• 9

   Eliane Segers and Ludo Verhoeven, “Long-Term Effects of Computer Training of Phonological Awareness in Kindergarten,” Journal of Computer Assisted Learning 21, no. 1 (2005): 17–27, https://doi.org/10.1111/j.1365-2729.2005.00107.x.

• 10

   Jichen Zhu et al., “Toward Interactive Social Stories for Children with Autism,” in Proceedings of the First ACM SIGCHI Annual Symposium on Computer-Human Interaction in Play, 453–54, (New York: Association for Computing Machinery, 2014), https://doi.org/10.1145/2658537.2661321.

• 11

   Sunugal Sani-Bozkurt et al., “Design and Use of Interactive Social Stories for Children with Autism Spectrum Disorder (ASD),” Contemporary Educational Technology 8, no. 1 (2017): 1–25, https://doi.org/10.30935/cedtech/6184.

• 12

  https://editor.p5js.org/ashleyzingillioglu-aeolian/sketches/idXLjrFes.

• 13

   Jichen Zhu et al., “Using Interactive Social Story Games to Teach Social Skills to Children with Autism,” in Proceedings of  1st International Joint Conference of DiGRA and FDG. (Digital Games Research Association, 2016), https://doi.org/10.26503/dl.v2016i1.783.