The deaf (in the sample of NISH students) play many of the outdoor games like cricket and football but do not find many hearing friends to play with since they have some difficulties in hearing others during such games. So most of them do not play any such games in college. Some of them play video games and online games. Given the pandemic and the reality of hybrid interactions going forward we focus on games that can be played online or in hybrid mode using smartphones as the only technology requirement. Some of these games can certainly played in-person without the need for intermediating technology. However, one of the key side effects that we aim to achieve is that of ISL data collection and hence we restrict ourselves to games that need to be played with all players accessing the play with either a smartphone or laptop/desktop.
One such popular game is Kahoot! which is a trivia/quiz game that can be played online and with multiple players participating with their mobile phones and has hundreds of millions of users around the world. This is a popular game among the NISH students as well. Another game that NISH students enjoyed playing is the following game.
Two teams A and B take turns. All seated facing a projector screen. One member of the guessing team (say, A) stands in front facing the rest and not being able to see the projected object. Other members of team A try to convey to this person what is projected on the screen, without using the exact sign for the projected object. Team B watches to ensure that Team A does not use the sign for the object but only uses round about ways of helping the guesser identify the object. Once the object is identified or if the actual object sign was mistakenly used by anyone in team A, Team B takes the turn. This was an interesting game that is very similar to the popular card game Taboo!
An important method for teaching ISL to both the deaf and hearing is the use of stories. And just like hearing children are introduced to comic books to encourage them to read and build a vocabulary, comic books can also be used to increase the English vocabulary and reading skills of the deaf. The following are some of the games that we are experimenting with. The goal is to keep play and playfulness central to all of these efforts, with everyone involved enjoying the experience at all times. The intended side effects for any of the games are 1) increasing competence in English for the deaf 2) Increasing competence in ISL for the hearing and 3) collection of labelled continuous sign language data. Each of the games may not meet all of the objectives all the time. But our continuing effort is to iterate on the games to achieve the objectives while keeping the experience enjoyable.
A. Quiz Games
1) On Zoom: The quiz game on Zoom consists of 5 participants out of which 3 are deaf students from NISH and 2 hearing participants. All the participants’ videos were on. Each player takes turns as host, signing the quiz question and options. The host addresses each player by their sign name and asks them to answer the quiz. After each player has answered the quiz question, the host reveals the answer.
2) On WhatsApp: This is an inclusive quiz game with a competitive element that both hearing and deaf individuals can engage. The WhatsApp group consists of 8 deaf participants from NISH and 2 hearing participants in the quiz game. Each player takes turns to be the host each day where they post three questions. The deaf host records the video using their phone camera and signs the multiple- choice question, which has four alternative answers, one of which is correct. The host informs the players in the group that the quiz questions will be posted at a specific time, then posts the quiz question videos in the group and invites players to answer the quiz questions within the specified timeframe. Each player signs the multiple-choice question and the four options with the answers, records them using their phone camera, and posts it on the WhatsApp group, along with the text message interpreting the meaning of their signs in English. This is for other players to understand and validate the signs of each player’s video. The scores for each quiz question are determined by the correct answer, the articulated typed question and answer, clarity of signing , and the time taken by the players to answer the quiz. The scoring system provided players with immediate responses to post the answers to the quiz questions making the game fun and highly interactive. The game is time-based to ensure that sessions remains short and to promote replay.
Quiz screenshots from WhatsApp
B. Comic Story telling:
The idea of comics storytelling is to ignite interests in reading through short text and visuals for the deaf students from NISH. The participants are asked to choose and read the comic that interests them, then narrate it during the zoom call which consists of both deaf and hearing participants. The comic story is divided into comic strips, each with at least two visual panels, which are then sequenced in the PPT slides and shared with the participants during the Zoom call.
While the participants’ videos are on, Zoom allows presenters to be pinned and enabled in the spotlight, as well as rearrange the video windows of other participants who appear on their grid. This allows participants to see the deaf presenter’s video while watching the shared presentation. The deaf presenter reads each panel of the comic strip and signs the narrative, while the other participant tries to follow and understand what the deaf presenter is signing. In this method, participants learn to comprehend, translate, and interpret from one language to another. Both deaf and hearing individuals can benefit from ISL storytelling in terms of language development, language acquisition, entertainment, and so on.
Screenshot from Story telling
C. Deaf Taboo:
Taboo is a word guessing game which the participants played by signing and guessing. The participants included two sign language interpreters, a deaf NISH and two hearing individuals. Hearing participants will benefit from sign language interpreters who will help interpret what the signer is stating. The goal of the game is to guess the word that the signer is describing, however the signer is not allowed to use a list of taboo words. Signer should try to give different clues to guess the word without using the taboo words. Each word guessed right, will get points for the signer, and all the words that were taboo are a point for the players.
Screenshot from Taboo on desktop
D. Who wants to be a Millionaire:
This is a popular TV game show that has now had avatars in many languages around the world and has proved to be popular across language and cultural differences. in India, the largest is the show in Hindi, Kaun Banega Crorepati (KBC). In Tamil, it has recently morphed into a show called ’Koteshwari’ the term is equivalent to ’woman millionaire’ and the show is run by a popular movie actress turned producer and the participants are women. With the experience gained by trying out various versions of the quiz games as discussed above, we might explore how layer can be built on top to make it interactive and as engaging for players as it is for observers like the Millionaire game has been around the world.
The following are some questions from the KBC show:
• Which of the following musical instruments is NOT of foreign origin?
1) Flute
2) Sitar
3) Violin
4) Tabla
• The word ’Hindu’ as reference to the people of Hind (India) was first used by
1) Greeks
2) Romans
3) Chinese
4) Arabs
• METSAT the first exclusive meteorological satellite built by ISRO is a?
1) Polar orbit satellite
2) Geosynchronous satellite
3) Low earth orbit satellite
4) Deep space probe
• Which animal’s favourite food is bamboo shoots
1) squirrel
2) Panda
3) Koala
4) Rabbit
E. Signoot!:
We have created a new game1, that we presently call Signoot! (short for Signed Kahoot!) that borrows from the highly popular kahoot game mentioned earlier. The key elements of Kahoot! that have made it a billion-user app on the net are a) online competition with scoring tied to speed of response b) user created content on any topic that catches the creator’s fancy. The simple UX that allows for multiple players across multiple devices to play together is the technology that amplifies these game elements. We are creating an equivalent game that uses sign language for the questions and the four answer choices. The rest of the experience will remain the same, subject to the need for a better bandwidth to enable speedy downloads of the question and 4 answers in video, rather than simple text or images as in the case of Kahoot! The players choose answers by clicking on the chosen option directly as before. Thus, any player that can understand sign language can play without having the competence to sign and so this game will be good for the hearing people who want to improve their sign language comprehension. If popularised, this new platform may encourage the deaf to create a whole range of questions and answers on topics that are closer to their heart, like deaf culture, audism, etc. in the process, a large corpus of continuous sign language in diverse topics signed by expert users will become available.
F. Wall, gun, rabbit :
A variant of the rock-scissor-paper game was recalled by one of the authors as something they had played growing up. This has the potential for a whole set of games that depend on object signs and will be explored further.
G. Memory Game :
This is another game played by the deaf community. A game equivalent to the memory game played by hearing people: the players sit in a circle and the game starts by one person mentioning a random object and the next person recalling the previous object and adding one more to the list, verbally, and passing the turn to the next person. Every player has to recall all the objects in the order in which they were added to the list. If a player makes an error that player leaves and a new list is created. IN the deaf version of the game, each player ’drops’ a new object into a imagined bag/pot and passes it on to the next person. Each person has to recall the set of objects inside the pot in the order they were dropped in and then add a new object.
The first challenge in all of the above games is to make it inclusive, such that both hearing and the deaf can play and enjoy, along with a competitive element. The second challenge is to enable all of this to be played online, a major challenge since it will involve capturing and sharing of videos, understanding responses and responding back. The diverse bandwidth availability among participants is another challenge.
We are exploring the use of the Ludic Design methodology to address these two critical needs of the deaf and hard of hearing (DHH) community. We will also strive to create games and play that are language agnostic: any sign language and any spoken language combination should be possible by some changes in the underlying technology stack while preserving the essence of the play.