This giving season, consider making us your charity of choice when you shop on Amazon.

This site uses cookies

By using this site, you consent to our use of cookies. You can view our terms and conditions for more information.

Return to Session

Interactive Grounding and Inference in Instruction Following

Dario Salvucci
Drexel University

Learning by instruction is one of the most common forms of learning, and a number of research efforts have modeled the cognitive process of instruction following, with many successes. However, most computational models remain brittle with respect to the given instructions and lack the ability to adapt dynamically to variants of the instructions. This paper aims to illustrate modeling constructs designed to make instruction following more robust, including (1) more flexible grounding of language to execution, (2) processing of instructions that allows for inference of implicit instruction knowledge, and (3) dynamic, interactive clarification of instructions during both the learning and execution stages. Examples in the context of a paired-associates task and a visual-search task are discussed.

Larger chunks of procedural knowledge Last updated 5 months ago

HI Dario, Very nice presentation, and development of the instruction following theme. One of the problems I have struggled with in the past with instruction following, is that we typically try to make a one-to-one mapping between instructions and productions (or one to very few). Therefore the instructions often seem to be unnecessary detailed. Yo...

Dr. Niels Taatgen 2 comments