Date | Topic | Readings | Discussion Leaders | Todo's |
Aug 24 | Intro to the Course (slides) | SLP2 Chapter1 | Mohit | - |
Aug 31 | NLP Basics and Core Tasks 1: POS Tagging, Parsing, Coreference, Distributional Semantics (slides) | SLP2 Chapters 5-6, 13-14, 19-21; SLP3 Chapters 8-13, 15-16, 23 | Mohit | - |
Sep 7 | NLP Basics and Core Tasks 2: Compositional Semantics, Sentiment, Language Modeling, Machine Translation (slides) | SLP2 Chapters 4, 17-18, 25; SLP3 Chapters 4-5, 7, 16-18, 21, 26 | Mohit | - |
Sep 14 | Question Answering (KB-based and Passage-based) | (1) SLP2 Chapter 23; SLP3 Chapter 27; (2) "Semantic Parsing via Paraphrasing"; (3) "Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks"; (4) "Thorough Examination of the CNN/Daily Mail Reading Comprehension Task" | Anna Z (1, 2), Marquis H (3, 4) | 2 paper summaries (except readings) |
Sep 21 | Language Generation (Summarization, Dialogue) | SLP2 Chapters 23-24; SLP3 Chapters 25, 28; (1) "Abstractive Sentence Summarization with Attentive Recurrent Neural Networks"; (2) "What to talk about and how? Selective Generation using LSTMs with Coarse-to-Fine Alignment"; (3) "A Persona-Based Neural Conversation Model"; (4) "On-line Active Reward Learning for Policy Optimisation in Spoken Dialogue Systems" | Cheng-Yang F (1, 2), Yen-Chun C (3, 4) | 2 paper summaries |
Sep 28 | Human-like Language (Non-literal, Ambiguous) | (1) "Black Holes and White Rabbits: Metaphor Identification with Visual Features"; (2) "An LSTM Framework for Predicting Humor in Dialogues"; (3) "Interpreting Neural Networks to Improve Politeness Comprehension"; (4) "Harnessing Context Incongruity for Sarcasm Detection" | Patrick H (1, 2), Ying L (3, 4) | 2 paper summaries |
Oct 5 | Image-Text Alignment and Language Disambiguation via Visual Features | (1) "What are you talking about? Text-to-Image Coreference"; (2) "Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models"; (3) "Resolving Language and Vision Ambiguities Together: Joint Segmentation & Prepositional Attachment Resolution in Captioned Scenes" | Scotty C (2, 3) | 2 paper summaries |
Oct 12 | Image/Video Captioning and Referring Expressions | (1) "BabyTalk: Understanding and Generating Simple Image Descriptions"; (2) "Show, Attend and Tell: Neural Image Caption Generation with Visual Attention"; (3) "Modeling Context in Referring Expressions"; (4) "Sequence to Sequence -- Video to Text" | Chris Horning (1, 2), Ramakanth P (3, 4) | 2 paper summaries |
Oct 19 | Visual Question Answering and Madlibs | (1) "VQA: Visual Question Answering"; (2) "Visual7W: Grounded Question Answering in Images" (part of "Visual Genome"); (3) "Multimodal Compact Bilinear Pooling for Visual Q&A and Visual Grounding"; (4) "Visual Madlibs: Fill in the blank Description Generation and Question Answering" | Fan J (1, 2), Micah M (3, 4) | 2 paper summaries |
Oct 26 | Project Proposals (10 min presentations) | NLP Colloquium (Dipanjan Das, Google Research), then extended class after that | - | - |
Nov 2 | Traveling to EMNLP (make-up class on Oct 26); Project midterm write-ups due Nov06! | - | - | Project midterm write-ups due Nov06 |
Nov 9 | Text-to-Image Generation, Visual Storytelling | (1) "Text to 3D Scene Generation with Rich Lexical Grounding"; (2) "Generative Adversarial Text to Image Synthesis"; (3) "Visual Storytelling"; (4) "Sort Story: Sorting Jumbled Images and Captions into Stories" | Marc E (1, 2), Bhavya V (3, 4) | 2 paper summaries |
Nov 16 | Language for Robotics 1: Instructions for Navigation, Manipulation, Skill Learning | (1) "Listen, Attend, and Walk: Neural Mapping of Navigational Instructions to Action Sequences" (based on the task from MacMahonEtAl2006 and Chen&Mooney2011); (2) "Interpreting and Executing Recipes with a Cooking Robot"; (3) "Understanding Natural Language Commands for Robotic Navigation and Mobile Manipulation"; (4) "Tell Me Dave: Context-Sensitive Grounding of Natural Language to Manipulation Instructions" | Adam A (1, 2), Jaesung P (3, 4) | 2 paper summaries |
Nov 23 | Thanksgiving Recess (no class) | - | - | - |
Nov 30 | Language for Robotics 2: Grounding via Dialogue-based Games | (1) "Back to the Blocks World: Learning New Actions through Situated Human-Robot Dialogue"; (2) "Learning to Interpret Natural Language Commands through Human-Robot Dialog"; (3) "Natural Language Communication with Robots"; (4) "Learning Language Games through Interaction" | Shiwei F (1, 2), Hao T (3, 4) | 2 paper summaries |
Dec 7 | Last Class: Final Project Presentation (and some tips on research-style writing) | - | - | - |
Dec 16 | Project Write-ups Due | - | - | - |