Brandeis University · Computer Science

James Pustejovsky

TJX Feldberg Chair in Computer Science

Director, Lab for Linguistics and Computation  ·  Chair, Computational Linguistics MS Program  ·  Chair, Linguistics Program

I develop computational and cognitive models of natural language meaning — how words encode knowledge, how events and time are represented, and how language grounds meaning in physical and social context. My work spans Generative Lexicon Theory, temporal and spatial semantics, multimodal dialogue, and large-scale language annotation.

Publications Research Areas Contact
277+ i10-index
75 h-index
33,677 Citations
James Pustejovsky
Google Scholar ACL Anthology ResearchGate LinkedIn jamesp@brandeis.edu Brandeis Brandeis CS
What I Work On

Research Areas

My research bridges formal linguistics and machine intelligence — building theories and computational tools for understanding how language encodes meaning in context.

📖

Generative Lexicon Theory

A formal framework for representing the rich, generative nature of word meaning — accounting for polysemy, coercion, and sense extension through qualia structure and type composition.

Lexical Semantics Polysemy Type Theory
⏱

Temporal Reasoning & TimeML

Chief architect of TimeML and ISO-TimeML, an international ISO standard for annotating events and temporal relations in text. Includes the TARSQI toolkit for temporal parsing and reasoning.

TimeML ISO Standard Event Semantics
🗺

Spatial Language & ISO-Space

Developing formal representations for spatial information in natural language — including locations, paths, motion events, and spatial relations — through the ISO-Space annotation framework.

ISO-Space Motion Events Spatial Reasoning
🤖

Multimodal Dialogue & HCI

Grounding language meaning in visual and physical context for human-robot and human-computer interaction, including the VoxWorld platform for embodied multimodal agents.

Embodied AI VoxWorld Common Ground
🏷

Language Annotation & NLP

Large-scale linguistic annotation for machine learning, including PropBank, TimeBank, UMR (Uniform Meaning Representation), and the CLAMS platform for multimedia NLP.

UMR PropBank CLAMS
🔗

Compositional Distributional Semantics

Integrating formal compositional meaning with distributional models — from noun-noun compound interpretation and metaphor detection to semantically enriched text generation.

Distributional Metaphor NLG
Brandeis University

Lab for Linguistics
& Computation

The BLLC focuses on computational and cognitive modelling of natural language meaning — how words combine to create meaningful texts, how context shapes interpretation, and how computational tools can capture this richness at scale.

Lab members work on annotation frameworks, formal semantics, embodied AI, and NLP systems with applications ranging from clinical text to archival media to human-robot interaction.

Visit Lab Website ↗

Lab Director & Affiliations

  • 🏛 Director, Lab for Linguistics and Computation
  • 📚 Chair, Computational Linguistics MS Program
  • 🔤 Chair, Linguistics Program
  • 🧠 Affiliated Faculty, Volen Center for Complex Systems
  • 📐 Head, ISO/TC37/SC4 Semantic Annotation Framework
  • 🤝 NSF AI Institute for Student-AI Teaming (iSAT)

Active Research Projects

Platform

VoxWorld

A multimodal platform for embodied agents enabling grounded language understanding through 3D simulation, gesture, and dialogue.

NLP System

CLAMS

Computational Linguistics Applications for Multimedia Services — NLP pipelines for the American Archive of Public Broadcasting.

Annotation

UMR (Uniform Meaning Representation)

Cross-lingual semantic annotation extending AMR with event structure, aspect, quantification, and temporal information.

Semantics

GLAMR

Augmenting Abstract Meaning Representation with GL-VerbNet event structure for richer semantic parsing and inference.

Dialogue

TRACE

Real-time multimodal common ground tracking in situated collaborative dialogues — tracking shared beliefs and intentions.

Standards

ISO-TimeML / ISO-Space

International ISO annotation standards for temporal and spatial information, developed through the ISO/TC37 working group.

DARPA

Communicating with Computers (CwC)

DARPA-funded program exploring how ideas are conveyed between humans and computers through language, gesture, visualization, and collaborative action in simulation environments.

Infrastructure

LAPPS Grid

Language Application Grid — an open web service platform for NLP research enabling interoperability of linguistic tools and resources across institutions.

Lexical Resources

VerbNet + Generative Lexicon

Augmenting VerbNet verb representations with GL event structures to better capture polysemy, argument alternations, and contextual meaning shift.

Selected Recent Work

Publications

A selection of recent papers (2023–2025). For the complete list of 286+ publications see ACL Anthology or Google Scholar.

2025
Dynamic Epistemic Friction in Dialogue
Timothy Obiso, Kenneth Lai, Abhijnan Nath, Nikhil Krishnaswamy, James Pustejovsky
Proceedings of CoNLL 2025
Beyond Benchmarks: Building a Richer Cross-Document Event Coreference Dataset with Decontextualization
Jin Zhao, Jingxuan Tu, Bingyang Ye, Xinrui Hu, Nianwen Xue, James Pustejovsky
Proceedings of NAACL 2025
TRACE: Real-Time Multimodal Common Ground Tracking in Situated Collaborative Dialogues
Hannah VanderHoeven, Brady Bhalla, Ibrahim Khebour, et al., James Pustejovsky
NAACL 2025 (System Demonstrations)
Enhanced Noun-Noun Compound Interpretation through Textual Enrichment
Bingyang Ye, Jingxuan Tu, James Pustejovsky
Proceedings of EMNLP 2025
ProPara-CRTS: Canonical Referent Tracking for Reliable Evaluation of Entity State Tracking
Bingyang Ye, Timothy Obiso, Jingxuan Tu, James Pustejovsky
Proceedings of IWCS 2025
A Graph Autoencoder Approach for Gesture Classification with Gesture AMR
Huma Jamil, Ibrahim Khebour, Kenneth Lai, James Pustejovsky, Nikhil Krishnaswamy
Proceedings of IWCS 2025
Representing ISO-Annotated Dynamic Information in UMR
Kiyong Lee, Harry Bunt, James Pustejovsky, Alex C. Fang, Chongwon Park
6th International Workshop on Designing Meaning Representations
Revisiting the ISO-TimeML Abstract Syntax
Harry Bunt, Alex Fang, Kiyong Lee, Volha Petukhova, Purificação Silvano, James Pustejovsky
21st Joint ACL–ISO Workshop on Interoperable Semantic Annotation
2024
Linguistically Conditioned Semantic Textual Similarity
Jingxuan Tu, Keer Xu, Liulu Yue, Bingyang Ye, Kyeongmin Rim, James Pustejovsky
Proceedings of ACL 2024
ACL Anthology
Building a Broad Infrastructure for Uniform Meaning Representations
Julia Bonn, Matthew J. Buchholz, Jayeol Chun, et al., James Pustejovsky
Proceedings of LREC-COLING 2024
Common Ground Tracking in Multimodal Dialogue
Ibrahim Khalil Khebour, Kenneth Lai, Mariah Bradford, et al., James Pustejovsky
Proceedings of LREC-COLING 2024
Encoding Gesture in Multimodal Dialogue: Creating a Corpus of Multimodal AMR
Kenneth Lai, Richard Brutti, Lucia Donatelli, James Pustejovsky
Proceedings of LREC-COLING 2024
GLAMR: Augmenting AMR with GL-VerbNet Event Structure
Jingxuan Tu, Timothy Obiso, Bingyang Ye, Kyeongmin Rim, Keer Xu, et al., James Pustejovsky
Proceedings of LREC-COLING 2024
ChainNet: Structured Metaphor and Metonymy in WordNet
Rowan Hall Maudslay, Simone Teufel, Francis Bond, James Pustejovsky
Proceedings of LREC-COLING 2024
Spatial and Temporal Language Understanding: Representation, Reasoning, and Grounding
Parisa Kordjamshidi, Qiang Ning, James Pustejovsky, Marie-Francine Moens
NAACL 2024 Tutorial
PropBank Goes Public: Incorporation into Wikidata
Elizabeth Spaulding, Kathryn Conger, Anatole Gershman, et al., James Pustejovsky
18th Linguistic Annotation Workshop (ACL 2024)
2023
The Coreference under Transformation Labeling Dataset: Entity Tracking in Procedural Texts
Kyeongmin Rim, Jingxuan Tu, Bingyang Ye, Marc Verhagen, Eben Holderness, James Pustejovsky
Findings of ACL 2023
Dense Paraphrasing for Textual Enrichment
Jingxuan Tu, Kyeongmin Rim, Eben Holderness, Bingyang Ye, James Pustejovsky
Proceedings of IWCS 2023
Integrated Annotation of Event Structure, Object States, and Entity Coreference
Kyeongmin Rim, James Pustejovsky
6th Workshop on Computational Models of Reference, Anaphora and Coreference (ACL 2023)
Scalar Anaphora: Annotating Degrees of Coreference in Text
Bingyang Ye, Jingxuan Tu, James Pustejovsky
6th Workshop on Computational Models of Reference, Anaphora and Coreference (ACL 2023)
Annotating Situated Actions in Dialogue
Christopher Tam, Richard Brutti, Kenneth Lai, James Pustejovsky
4th International Workshop on Designing Meaning Representations
Mapping AMR to UMR: Resources for Adapting Existing Corpora for Cross-Lingual Compatibility
Julia Bonn, Skatje Myers, Jens E. L. Van Gysel, et al., James Pustejovsky
21st International Workshop on Treebanks and Linguistic Theories
View All Publications on ACL Anthology ↗   Google Scholar Profile ↗
Selected Journal Work

Journal Articles

Key journal publications spanning lexical semantics, temporal reasoning, and computational linguistics.

The Generative Lexicon
James Pustejovsky
Computational Linguistics, 17(4):409–441 · 1991
The Syntax of Event Structure
James Pustejovsky
Cognition, 41(1–3):47–81 · 1991
Are You Sure That This Happened? Assessing the Factuality Degree of Events in Text
Roser Saurí & James Pustejovsky
Computational Linguistics, 38(2) · 2012
Situated Meaning in Multimodal Dialogue: Human-Robot and Human-Computer Interactions
James Pustejovsky & Nikhil Krishnaswamy
Traitement Automatique des Langues, 61(3) · 2020
The Qualitative Spatial Dynamics of Motion in Language
James Pustejovsky & J. L. Moszkowicz
Spatial Cognition & Computation, 11(1) · 2011
Coercion in a General Theory of Argument Selection
James Pustejovsky
Linguistics, 49(6) · 2011
FactBank: A Corpus Annotated with Event Factuality
Roser Saurí & James Pustejovsky
Language Resources and Evaluation · 2009
The TempEval Challenge: Identifying Temporal Relations in Text
M. Verhagen, R. Gaizauskas, F. Schilder, M. Hepple, J. Moszkowicz, James Pustejovsky
Language Resources and Evaluation, 43(2) · 2009
Type Theory and Lexical Decomposition
James Pustejovsky
Journal of Cognitive Science · 2006
Aspectual Coercion and Logical Polysemy
James Pustejovsky & Pierrette Bouillon
Journal of Semantics · 1995
Authored & Edited

Books

Selected monographs, edited volumes, and textbooks.

📘
The Generative Lexicon
James Pustejovsky
MIT Press, 1995

The foundational monograph introducing Generative Lexicon Theory — qualia structure, type coercion, and the computational treatment of polysemy.

📔
The Handbook of Linguistic Annotation
Nancy Ide & James Pustejovsky (eds.)
Springer, 2017

Comprehensive reference covering annotation theory, methodology, and frameworks across all levels of linguistic structure.

📙
Interpreting Motion: Grounded Representations for Spatial Language
I. Mani & James Pustejovsky
Oxford University Press, 2012

Formal and computational treatment of motion verbs, spatial language, and the ISO-Space framework.

📒
Natural Language Annotation for Machine Learning
James Pustejovsky & Amber Stubbs
O'Reilly Media, 2012

A practical guide to designing annotation schemes and building corpora for NLP and machine learning.

📕
Advances in Generative Lexicon Theory
J. Pustejovsky, P. Bouillon, H. Isahara & K. Kanzaki (eds.)
Springer, 2012

International research extending and applying Generative Lexicon Theory through compositional semantics techniques.

📗
The Language of Time: A Reader
I. Mani, J. Pustejovsky & R. Gaizauskas (eds.)
Oxford University Press, 2005

Foundational readings on temporal language, temporal reasoning, and TimeML annotation.

📓
Events as Grammatical Objects
Carol L. Tenny & James Pustejovsky (eds.)
CSLI Publications, 2000

Explores how natural language grammars structure and refer to events, bringing together linguistic and philosophical perspectives.

📖
Semantics and the Lexicon
James Pustejovsky (ed.)
Kluwer Academic, 1993

Integrates lexical semantics research with knowledge representation and computational linguistics perspectives.

📚
Lexical Semantics: The Problem of Polysemy
James Pustejovsky & Branimir Boguraev (eds.)
Cambridge University Press, 1996

Addresses lexical ambiguity from a computational linguistics perspective.

📑
Annotating, Extracting and Reasoning about Time and Events
F. Schilder, G. Katz & J. Pustejovsky (eds.)
Springer LNAI, 2007

State-of-the-art survey on TimeML and temporal information extraction, centered on the emerging ISO-TimeML standard.

📋
Corpus Processing for Lexical Acquisition
Branimir Boguraev & James Pustejovsky (eds.)
MIT Press, 1996

Addresses lexicon acquisition from corpora as essential for natural language processing systems.

📝
Readings in the Lexicon: Interdisciplinary Perspectives
James Pustejovsky & Yorick Wilks (eds.)
MIT Press, 2012

Cross-disciplinary perspectives on the lexicon drawing from linguistics, cognitive science, and AI.

Brandeis University

Teaching

Courses in natural language processing, computational semantics, and linguistic theory at the undergraduate and graduate levels.

CS 115B · Spring 2025

Fundamentals of Natural Language Processing

Core concepts and algorithms in computational linguistics and NLP — covering syntax, semantics, parsing, language models, and machine learning for text.

NLP Machine Learning Undergraduate
CS / LING 114A

Introduction to Computational Linguistics

An introduction to the computational modeling of language — finite automata, regular expressions, context-free grammars, parsing, and corpus-based methods.

Formal Grammars Parsing Undergraduate
CS / LING 216

Computational Semantics

Graduate seminar on formal and computational approaches to meaning — lambda calculus, type theory, Generative Lexicon, temporal semantics, and distributional models.

Semantics Type Theory Graduate
LING 130

Introduction to Generative Lexicon

A foundational course in lexical semantics covering qualia structure, type coercion, co-composition, and the computational treatment of word meaning and polysemy.

Generative Lexicon Lexical Semantics Graduate
CS 216 / LING 216

Language Annotation and Machine Learning

Hands-on course on building annotated corpora — annotation scheme design, inter-annotator agreement, active learning, and applications to NLP tasks.

Annotation Corpora Graduate
ESSLLI / LSA Summer Institutes

Invited Summer School Courses

Regular lecturer at ESSLLI (2017, 2018) and LSA Linguistic Institute (2019) on topics in temporal semantics, spatial language, and multimodal meaning.

ESSLLI LSA Institute Invited
Get In Touch

Contact

I welcome inquiries from prospective graduate students, collaborators, and colleagues. The Computational Linguistics MS program at Brandeis admits students with backgrounds in linguistics, computer science, and related fields.

Office
Volen 258, Brandeis University
415 South Street, Waltham, MA 02454

Affiliations & Programs

© 2025 James Pustejovsky · Brandeis University
Hosted on GitHub Pages