The City College of the City University of New York
160 Convent Avenue
New York, NY 10031
Physics Department
Marshak Science Building, Room 419
Phone: 212-650-6832
fax: 212-650-6940
email:
physdept@ccny.cuny.edu
-----------------------------------------------------------------------------------------
Wednesday, September 16, 2020 from 04:00 PM to 05:00 PM
Where Zoom
Contact Name Sriram Ganeshan
Contact Email sganeshan@ccny.cuny.edu
Physics Colloquium
Dynamics of Gated Recurrent Neural Networks
Tankut Can
Postdoctoral Fellow
Institute for Theoretical Sciences
Graduate Center, CUNY
New York, NY
Abstract
Recurrent neural networks (RNNs) have long captured the attention and imagination of the scientific community. Initially studied as a model for the brain, they have now established a firm foothold in the modern practice of machine learning. RNNs can now achieve state-of-the-art performance on sequential learning tasks like speech recognition, translation, and text prediction, in large part due to a key feature: gating, a multiplicative interaction which controls the flow of information. Despite this progress, we still do not really know how gating shapes the dynamics of RNNs, which presents a hurdle to building even better learning systems.
In this talk, I will show how physics approaches can shed light on why gated RNNs work, as well as how they can be improved. Furthermore, I will argue that gated RNNs are interesting in their own right as models of complex interacting systems, and give rise to several novel phenomena, including robust line attractors and a first-order chaotic transition. Finally, I hope to convey a sense of the excitement in this growing field at the intersection of physics and machine learning.
--------------------------
The goal of ITS is to provide a home for research and teaching, emphasizing theoretical approaches to the natural sciences. Participants are interested in a wide range of phenomena, from the elementary building blocks of matter to the dynamics of social systems, from the collective behavior of electrons in solids to the collective behavior of neurons in the brain, and more. What ties these disparate problems together is the search for a compact and compelling mathematical description of the world around us.
Last Updated: 09/14/2020 14:12