Analyzing Transformer Attention and Domain Prediction for Homology Search

Abstract

We explore the utility of attention networks in analyzing protein data. While previous research has shown the ability of attention networks to learn contacts in protein sequences and tertiary structures, the underlying mechanisms of their learning process remain unclear. In this thesis, we propose the use of synthetic protein data to investigate this issue and demonstrate that attention networks are capable of effectively capturing hidden features within a dataset. Moreover, we show evidence of their capability to manage more intricate tasks, such as domain prediction. We then present a novel approach for improving the homology search through the application of trained attention networks.