Hossein Rajaby Faghihi author Roshanak Mirzaee author Sudarshan Paliwal author Parisa Kordjamshidi author 2020-jul text Association for Computational Linguistics Online conference publication We propose a novel alignment mechanism to deal with procedural reasoning on a newly released multimodal QA dataset, named RecipeQA. Our model is solving the textual cloze task which is a reading comprehension on a recipe containing images and instructions. We exploit the power of attention networks, cross-modal representations, and a latent alignment space between instructions and candidate answers to solve the problem. We introduce constrained max-pooling which refines the max pooling operation on the alignment matrix to impose disjoint constraints among the outputs of the model. Our evaluation result indicates a 19% improvement over the baselines. rajaby-faghihi-etal-2020-latent https://www.aclweb.org/anthology/2020.alvr-1.5 2020-jul 26 31