Several of my projects have been work based primarily in Google Colab, and they are collected here. Many of these are still works in progress.
Code for the project resulting in our Geometry of Integration in Text Classification RNNs paper (accepted to ICLR 2021). This work uses tools from dynamical systems analysis to better understand recurrent networks trained on text-classiciation tasks. Note this work relies on the reverse-engineering neural networks (renn) library.
Follow-up work to the above that aims to analyze the dynamics of attention in Transformer-like and recurrent architectures. A paper based on this work has been submitted to ICML 2021. (See this for most recent coding samples.)
Also a follow-up to the multi-class RNN work above. I aim to leverage our understanding of how RNNs trained on such tasks behave in order to better understand the efficacy of echo-state networks.
Some early work attempting to apply our dynamical systems analysis to RNNs trained on reinforcement learning tasks.
Large learning rate work that is a follow-up to large learning rate and catapult behavior observed in networks with MSE loss. We aim to extend such results to network with cross-entropy loss.