Sunday, November 17, 2019

STIP Student Jesse McCandlish Explores American Sign Language Recognition Softwares

In high school, I worked with my robotics team to demonstrate our robots in Richmond’s Maker Fest. While teaching the kids to build engineering toys, a small boy came up to build. Initially, he was unperceptive to instruction and ignored our help. It wasn’t until the mother came over and signed in American Sign Language (ASL) did I realize the child could not hear us. At that point, I was unfamiliar with ASL and unable to communicate with the mother or her son. I regret to say that child wasn’t able to ask questions about robots and get answers because we were unable to sign to him.

My freshman year of college, I signed up for an ASL class. I wanted to be able to work with all people and I wasn’t going to let a language barrier stop me. Since then, I have had little chance to use ASL outside of class until my professor approached me about OSCAR’s Summer Impact Project researching ASL recognition software. I was immediately ready to use my new ASL knowledge as well as my coding skills to help research a bridge across the communication gap.

When my team meets, it is often a mix of work and fun. We all get along wonderfully and enjoy helping each other out. We tend to start with a run-down of what needs to get done each day and a check-in toward our long-term goals. I then spend my time either recording ASL hand movements, coding scripts to transform or read collected data, or explaining the technical code to the other team members who are unfamiliar with Python and its libraries. My team often will end up repeating tasks like moving or combining files and ask me to automate the process. While I am coding, my team might have a great idea or discovery and we take to the white board for them to explain what I missed.

It has been wonderful working with OSCAR. I have learned how to work with a team and work towards a larger goal in small steps. I also am faster now at finding relevant research material both in and outside the lab. Overall, I am just overjoyed that I have been able to make communication easier for future generations.