Two computer science researchers at New York Institute of Technology have received National Science Foundation (NSF) grants to help advance innovation in artificial intelligence (AI). One project seeks to enable machines to detect human emotions, as they occur in real life, by gathering data from multiple forms of human emotion, such as facial expression, body movement, gestures, and speech. The other project intends to design more efficient and secure AI deep learning accelerators that can reliably process and interpret extremely large-scale sets of data with little delay.

“These grants will enable our computer science faculty to perform cutting-edge research that is as important for the challenges they address as they are for the opportunities they afford New York Tech students and faculty collaborators at other leading universities,” said Babak D. Beheshti, Ph.D., NYIT College of Engineering and Computing Sciences dean.

$100,000 Exploratory Grant to Enhance Machine Detection of Human Emotion

Houwei Cao, Ph.D., assistant professor of computer science, investigates how machines can be taught to use enormous data sets to “recognize” a person’s emotions and respond accordingly. However, in real life, automatic emotion recognition is often complicated by spontaneous, subtle expressive behaviors that occur between humans and their environments. Such complications include background noise, music, overlapping voices, lighting, and partially covered faces, among others. For this reason, the approach is often most effective when multiple modalities, or forms of expression, are analyzed.

With a nearly $100,000 NSF grant, Cao will conduct a 12-month exploratory research project to address the challenges of understanding spontaneous emotions and imperfect audio and video signals in real-life versus controlled laboratory settings. The data collected will be used to develop a new multimodal emotion recognition system for real-world applications; the project is expected to lead to significant advances in the benchmarking of next-generation affective computing.

“Analysis and recognition of spontaneous emotion is a challenging task,” says Cao. “Systems that work well with acted emotion datasets in laboratory environments may not work well for real-world applications ‘in-the-wild.’ We aim to design an emotion recognition system for real-life human-computer interaction applications, such as spoken dialogue systems, which are computer systems that converse with a human with voice, and conversational AI devices that use speech recognition.”

The project also has the potential to make a broader impact on healthcare, particularly mental healthcare, where multimodal data used to sense and monitor a patient’s emotional state may be collected from wearable or mobile devices and incorporated into a healthcare tracking system.

The research project will also be instrumental in educating New York Tech undergraduate and graduate students, who can be involved through the Undergraduate Research and Entrepreneurship Program (UREP), the Research Experience for Undergraduate (REU) program, and via thesis projects.

$60,000 Collaborative Grant to Develop Powerful AI Accelerators

Supported by a $60,000 NSF grant, Jerry Cheng, Ph.D., assistant professor of computer science, is leading a research team, which includes experts from Rutgers University, Temple University, and Indiana University, that aims to design optimized, reliable AI accelerators that will interpret extremely large-scale deep-learning computations and models.

“Existing research has shown that large-scale data from various sources with high-resolution sensing or large-volume data-collection capabilities can significantly improve the performance of deep-learning approaches,” said Cheng. “However, today's state-of-the-art hardware and software do not provide sufficient computing capabilities and resources to ensure accurate deep-learning performance in a timely manner when using extremely large-scale data. The project develops a scalable and robust system that includes a new low-cost, secure, deep-learning hardware-accelerator architecture and a suite of large-data-compatible deep-learning algorithms.” Cheng said that New York Tech graduate research assistants will be actively involved in the project.

Deep learning is a type of machine learning that uses neural networks, or algorithms, loosely modeled after the human brain in order to recognize patterns. The neural networks detect sensory information, such as images, sound, and text, and store this layered data in numeric form. Like the human brain, each time the machine repeats a task, the layers become more fine-tuned and results improve. However, few existing deep learning processing machines, known as AI accelerators, have been designed to ensure that very large-scale sets of data are accurately processed in a timely manner.

The research team’s prototype will be specially designed to process the complicated datasets with low power consumption, a challenge faced by current deep learning processors. To help protect the neural network data, the team will also design innovative in-memory encryption schemes. In addition, they will develop data-modeling and statistical-learning algorithms to further reduce the costs associated with processing these huge data sets.

About New York Institute of Technology

New York Institute of Technology offers 90 undergraduate, graduate, and professional degree programs in more than 50 fields of study, including computer science, data, and cybersecurity; biology and biomedical studies; architecture and design; engineering; health professions and medicine; IT and digital technologies; management; communications and marketing; education and counseling; and energy and sustainability. A nonprofit, independent, private, and nonsectarian institute of higher education, New York Institute of Technology welcomes nearly 8,000 students worldwide. The university has campuses in New York City (Manhattan) and Long Island (Old Westbury), New York; Jonesboro, Arkansas; and Vancouver, British Columbia, as well as programs around the world.

New York Institute of Technology embraces its mission to provide career-oriented professional education, give all qualified students access to opportunity, and support research and scholarship that benefit the larger world. More than 107,000 alumni comprise an engaged network of doers, makers, and innovators prepared to change the world, solve 21st-century challenges, and reinvent the future.

Media Contact

New York Tech Media Relations
mediarelations@nyit.edu