HTML conversions sometimes display errors due to content that did not convert correctly from the source. This paper uses the following packages that are not yet supported by the HTML conversion tool. Feedback on these issues are not necessary; they are known and are being worked on. Much text describes a changing world e. An earlier dataset, OpenPI , provided crowdsourced annotations of entity state changes in text.
However, a major limitation was that those annotations were free-form and did not identify salient changes, hampering model evaluation. On our fairer evaluation setting, we find that current state-of-the-art language models are far from competent. We also show that using state changes of salient entities as a chain-of-thought prompt, downstream performance is improved on tasks such as question answering and classical planning, outperforming the setting involving all related entities indiscriminately.
Tracking entity states in procedural texts Weston et al. To name a few, question answering about events e. While most recent work has relied on end-to-end language models LMs Huang et al. Procedural entity tracking is challenging in itself, requiring much understanding of an implicit environment as well as external knowledge of events and entities.
OpenPI contains annotations of entities, attributes, and state changes for each step e. Originally, different mentions of the same entity or attribute render evaluation difficult. Here, we prompt LMs to effectively cluster the entities and attributes. Entity Salience. Originally, a large amount of entities that undergo changes are listed in parallel. Here, we provide both human and model-predicted annotations of their salience. Regarding canonicalization, clustering different mentions e.
Moreover, as our task of predicting entities, attributes, and states is a generation task with imperfect and incomplete ground-truth references, we show that expanding each entity or attribute cluster with possible paraphrases thus providing more references is effective for reducing the false-negative rate.
