10 Attention mechanisms
This chapter covers
- Understanding attention mechanisms and when to use them
- Adding context to an attention mechanism for context-sensitive results
- Handling variable-length items with attention
Imagine having a conversation with a couple of friends at a busy coffee shop. Around you are other conversations and people placing orders and talking on their cell phones. Despite all this noise, you, with your complex and sophisticated brain and ears, can pay attention to only what is important (your friends!), and selectively ignore the things occurring around you that are not relevant. The important thing here is that your attention is adaptive to the situation. You ignore the background sounds to listen to your friends only ...
Get Inside Deep Learning now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
