IntelliGames
Final Report
Avery Smith
Introduction
Imagine growing up as a child in the ‘post-Y2K’ era, with an abundance of technology at everyone’s fingertips, except yours. Imagine growing up with other siblings that play video and computer games with their friends. Imagine going to school as a child, and seeing all of your other classmates go to the computer lab while you stay behind and miss out on the rewarding experiences of using the computer. The scenario described above sounds quite depressing doesn’t it? Unfortunately, many children with visual impairments don’t have to imagine this scenario, because they live it everyday. Due to their minute shortcoming of having limited to no vision, they are not afforded the many joys of life as others are.
As mentioned in the scenario earlier, children with visual impairments often are not provided a comparable amount of exposure to the computer as their sighted counterparts. They receive very limited, if any, interaction with the computer. As they progress through school, their schoolmates have a significant advantage over them in computer-literacy, because they have interacted with the computer in various ways (e.g. email, internet, word processing, games) and more frequently. Also, when children with visual impairments are finally introduced to the computer, their computer spatial awareness or orientation is undeveloped because of their lack of exposure with the computer. The solution to the discrepancy of computer-literacy between typical children and children with visual impairments is early and frequent interaction with the computer for children with visual impairments. This solution can be realized through the implementation of IntelliGames. IntelliGames focuses on providing a means of early exposure and interaction with the computer for children with visual impairments.
IntelliKeys within IntelliGames
Regardless of sight capability, it is difficult for most children at a young age to operate the computer due to the complexity of interacting with the keyboard. A simple keystroke mistake can result in the success or failure in operating a computer, which makes using the computer an even more daunting task for a visually-impaired child. With the use of the IntelliKeys hardware, IntelliGames simplifies the intricacy of the keyboard into a form that is uncomplicated for a child with visual impairments.
IntelliKeys is a programmable alternative keyboard that allows a user to send input to the computer. Overlays for the alternative keyboard are used to guide the user in sending input to the computer. By operating the IntelliKeys hardware within IntelliGames, a child with visual impairments can send input to the computer by pressing tactile markers on the overlay. Children with visual impairments rely heavily on their tactile and auditory senses, which are both fully utilized by the IntelliKeys board and IntelliGames. As the child makes selections during the game using the tactile markers on the IntelliKeys board, IntelliGames will respond with audio feedback.
Game Implementation
In addition to the IntelliKeys hardware, there were other resources needed to implement IntelliGames. Python 2.4 was the programming language used to develop IntelliGames. Two Python extensions developed by Dr. Gary Bishop and Peter Parente, pyTTS and pySonic, were used for the text-to-speech, and audio playback functionality respectively. The IntelliTools Overlay Maker 3 software program was used in the design of the overlays and the configuration of the IntelliKeys board to operate according to an overlay’s specifications.
Game Description
At this time, three games have been developed for IntelliGames. Between each of the games, there is a progression in difficulty, such that the child must rely more on their senses and knowledge than in the previous game. The first game that a child can play is the touching game, or the ‘Hear that Sound’ game, where the user touches a tactile marker and hears a sound. The second game is the identification game, or the ‘Name that Sound’ game. In the game, the user is prompted to identify the sound they heard after touching a marker. The game attempts to reinforce what the child learned from playing the ‘Hear that Sound’ game. The ‘Name that Sound’ game is identical to the ‘Hark the Sound’ game, whereas the user must select the right answer from a randomized collection of possible answers. The last game that has been created is the matching game, or the ‘Match that Sound’ game. The user essentially selects two markers to see if their corresponding sounds match. Although the game appears to be less learning but more leisure-oriented, it tests the user’s memory of where sounds are located on the IntelliKeys board.
There are currently two types of sounds in IntelliGames: Animal sounds and Everyday/Environment sounds. These sounds are vital for young children to know and be able to identify because the children will likely encounter such sounds in their surroundings, and knowing the sounds ahead of time will allow them to be more comfortable in their surroundings. Also, in all of the games within IntelliGames, the user receives a reward sound (e.g. song, applause) and image after completing the game. Positive feedback is present throughout the game, even when the user does not answer a question or perform a task correctly.
Below is a sample sequence of events of the ‘Hear that Sound’ game.
(image 1 in center)
When IntelliGames starts, the user is welcomed to IntelliGame, and is prompted to start a game. The user selects one of the available games to play, and then the type of sounds they would like to hear (animal or everyday).
Shown above is a screenshot of the overlay created for the ‘Hear that Sound’ game. The red squares represent markers used to hear sounds. Although it is not necessary in the game, the green circle and yellow star are for moving through a list of choices and making a selection respectively. The blue diamond is similar to the escape button, which allows the user to go back in the game, eventually to the main menu.
(image 2 in center)
Within the ‘Hear that Sound’ game, the user touches a red square to hear a sound. Along with the playing of the sound, an image corresponding to the sound is displayed, and the name of the image is spoken. If the user touched a red square that played the sound of a baby’s toy, the image shown above in the screenshot would be displayed, and the phrase “baby’s toy” would be spoken.
(image 3 in center)
After hearing three sounds in the ‘Hear that Sound’ game, the user will hear a reward sound, see a reward image, and hear some congratulatory phrase praising the user of a job well done. The game will then restart at the beginning of ‘Hear that Sound’.
Evaluation and Future Work
IntelliGames was tested by children with visual impairments at the 2006 Maze Day held in Sitterson Hall at UNC. Below are some of the suggestions and responses that were given by the children and the adults that accompanied them.
1.) More sounds should be included in the game. The children responded positively to the sounds in the game with laughter and smiles. Some children asked if some specific animal and environmental sounds were present in the game.
2.) The option of hearing a sound again in the ‘Name that Sound’ game. When trying to correctly identify the sound heard, the child wanted to replay the sound before making their selection. Currently, the sound can not be replayed.
3.) If the child selected the wrong answer in the ‘Name that Sound’ game, the pointer in the list of the possible answers is not reset to the beginning. This makes it confusing for the child to remember what was the last selection and make the right one.
4.) Some children and adults asked why the overlay corresponding to a particular game had to be reloaded by the Overlay Maker software and the IntelliGames program restarted before another game could be played. Currently, the Overlay Maker software has to configure the IntelliKeys board to the desired overlay every time, which can not be changed. However, IntelliGames currently refuses a mouse or a character event (i.e. keystroke) after the user switches to another program and back. Therefore, the game would operate more smoothly if IntelliGames did not have to restart. The issue may lie within the panels of the game and their state (active or non-active).
5.) The majority of children and adults were frustrated at the quality of the text-to-speech voices. Although the clearest of the text-to-speech voices in pyTTS was chosen, the children stated that the voice was not always clear, and made listening to the directions or responses difficult. The adults suggested a more ‘human’ voice, whose intonation reflected a particular emotion. One adult suggested that I record various phrases with my own voice, because the child she accompanied responded well to it.
In addition to the above suggestions made by adults and children, the following are tasks that need to be completed for IntelliGames.
6.) Input Timing Functionality: During Maze Day, some of the children forced the program to crash due to the repeated input from the IntelliKeys board. Some touched a marker twice in the process of making a selection or touch multiple markers en route to making a selection. IntelliGames should include a function that skips other character events that quickly follow another. Dr. Bishop mentioned an OnCharDown / OnCharUp function that is used in Hark the Sound.
7.) Game Creation: A renovated framework for IntelliGames that permits the creation of a game. Currently, each game within IntelliGames is ‘hardcoded’, which hinders flexibility. Also, more games need to be developed. Potential games include: musical instruments, songs, and pre-braille.
8.) Cross-platform Portability: IntelliGames has only been tested and executed on Windows operating systems. Testing could be done for Linux and Mac operating systems.
Conclusion
The IntelliGames project has the potential to be a powerful tool to expose children with visual impairments to the computer. With the use of the IntelliKeys hardware, the children can fully utilize their tactile and auditory capabilities to interact with the computer. The games within IntelliGames reinforce what the children have learned, while creating a fun experience at the same time. The goal is for children with visual impairments to develop some computer spatial awareness so that they can make a smooth, gradual transition to the keyboard.
Acknowledgements
Indispensable assistance was given by Ms. Diane Brauner, Dr. Gary Bishop, and Ms. Victoria Taylor-Holloway. Ms. Brauner, a Communication, Orientation, and Mobility Specialist, assisted with a majority of the brainstorming for the project, specifically what functionality was needed for IntelliGames to be beneficial for children with visual impairments. Dr. Bishop provided valuable insight into Python, along with audio files and images necessary for the feedback component of the game. Ms. Taylor-Holloway, a research assistant at the Franklin Porter Graham Institute also helped in the design of the project to improve its efficiency.