Natural language processing: Dialogue, spoken language systems.

Human-computer interaction: Multimodal interfaces (speech+sketch/gesture/handwriting/vision, etc), digital pen/paper technology, augmented reality, virtual reality, human-centered design.

Artificial intelligence: Multi-agent systems, personal assistants (software and robotic), intelligent agents, theories of communicative action, intention and commitment, collaboration and teamwork.

Applications to: Virtual assistants, automotive human-machine interaction, human-robot interaction/teaming, mobile computing, gaming, healthcare informatics, field data collection, simulation, architecture/design, crisis management, planning, logistics.

I am currently engaged in building a new Laboratory for Dialogue Research, which will address a major problem for current technology. I have the goal of being the major dialogue research program worldwide, and will staff it with up to 20 researchers. The approach to be taken will be based on the plan-based approach to dialogue, which I began (with Allen and Perrault) at the University of Toronto in 1974-1978.Our research will be directed at building collaborative dialogue systems that are responsive to their interlocutors’ plans and goals. This research will be based on my formal theories of individual/joint intention, speech acts, planning/plan recognition, and knowledge of actions. Given the burgeoning interest in dialogue understanding among academic and corporate organizations, this deep approach to dialogue is both timely and impactful.

At Voicebox, I staffed and led the AI R&D efforts, with a team of 10 researchers (8 PhDs). The group was engaged in building conversational agents for mobile and automotive applications, incorporating speech recognition, natural language processing, planning and plan recognition. Specifically, we developed high performance cooperative conversational systems in multiple languages for inquiring about points of interest, and other topics. I both led the technical direction for the next generation of VBT products, as well as represented VBT’s AI technology to the external world, interfacing with industry executives, the press, the research community, and corporate leadership. I also was engaged in transfer of the next generation technology into existing products.