Plan Recognition Project

Natural language and spoken dialogue provide an intuitive and efficient way of communicating with a range of real world situated applications such as robotics, ambient intelligence, in-car assistance systems, and virtual characters for the gaming or entertainment industry. However, for spoken interfaces to move beyond the current command-and-control applications, systems need to better model and understand human behaviour. This project examines the role of language grammars and syntax in the modelling of human behaviour.

Plan Recognition is the problem of  identifying a person's goal, plans, and intentions by observing the actions that they are performing in their environment. We can use the  results of plan recognition to allow dialogue systems to go beyond a user's basic requests and commands. Unfortunately, the problem of plan recognition itself is very difficult and remains the subject of active research in a number of communites.

Figure 1 depicts the classic principles of plan recognition as used in human-human interaction.

Figure 2: Overview of a Plan Recognition Process.

The main goal of computation models of plan recognition is to determine what goal or goals are being pursued by a person given an observed sequence of actions.

Figure 3: Application of a plan recognition project.

In the context of a dialogue system, a plan recognition mechanism can be used to motivate the dialogue planning process and determine what dialogue acts might be most appropriate for a system in a given context. Similarly, the results of plan recognition can also be used to prime the language understanding process.

Corpus Resources

The SCARE corpus, developed at the Ohio State University, consists of a series of videos recordings of pairs of study participants performing a joint task in a simulated 3D environment. For this project the SCARE ccorpus has been extended with annotations of location, actions, and participant goals. This extended SCARE annotation can be downloaded and used subject to the SCARE licence agreement here.

Dialogue System Development

The DIT dialogue system framework is a dialogue management and systems integration toolkit that includes information state based plan and intention modelling. The framework extends work that been previously done in the University of Bremen in Germany, but has since been extended to include online dialogue system development and plan recognition.  Daisie is written in Java, open-source, and is freely available for download.

Download DIT core architecture.