Future directions for interactive cognition using the VisiTor model eyes and hands and VisiGaze eye-tracking

Some recent advances in creating and testing models of interacting have theoretical and practical implications. We have had a simulated eye and hand (VisiTor) that works with an existing ACT-R model of a 14-min spreadsheet task to use two different, uninstrumented interfaces and compare task time on the two spreadsheet systems. The model uses oN-the-shelf vision algorithms to tie models to interfaces, and extends the knowledge used, such as visual icons, through its own simple interface. The model shows that the commercial system is faster and requires less visual, procedural, and declarative knowledge. A related system (VisiGaze) has been created to track eye movements in dynamic environments. These results suggest that engineering design based on user models is increasingly possible through modeling and testing (including high-level behavior representation languages) because VisiTor and VisiGaze may provide design patterns for this work. This work provides new types of insights (e.g., visual icons might need to be measured and taught), and that several new areas can be explored including error generation, error correction, and visual search can be more directly, and easily, and appropriately modeled. This approach also lets situations in which “milliseconds matter” to be more directly studied and predicted because now knowledge in the world (vs. in the head) will be more easily and more often available. All of which provides new topics for interactive cognition.