Evaluating Large-Scale Construction Grammars on the Tasks of Semantic Frame Extraction and Semantic Role Labeling

This study investigates the application of computational construction grammars to se-mantic frame extraction (SFE) and semantic role labeling (SRL) in natural languageprocessing. It provides an evaluation framework for large-scale construction gram-mars, combining the assessment of different simulations of sentence comprehensionin the context of an SRL task. The study evaluates these grammars, which em-ploy different configurations of heuristics - shortcuts or “rules of thumb” that guidehow the grammar processes and understands language. Just as humans use variousstrategies to comprehend sentences quickly, these computational heuristics help thegrammar efficiently apply constructions to extract meaning from an utterance. Thefocus lies on testing linguistically motivated heuristics, including (1) a preference forconstructions that occur more often in language use, (2) a bias towards relating nearbyelements in a sentence, and (3) a strategy that considers the strength of associationsbetween different constructions in the linguistic network. The analysis reveals thatdifferent heuristics perform better in processing different semantic frames, highlight-ing the importance of well-designed heuristics. Notably, the heuristic that leveragesthe interconnectedness of constructions demonstrated superior overall performance,particularly on frames with high support, such as those with communication, cogni-tion, and perception verbs. This suggests that considering how linguistic elementsrelate to each other in a broader network is crucial for accurate processing, especiallyin commonly occurring frames. The heuristic favoring local relationships showed ef-fectiveness in frames involving expressions of desire, necessity, intention, and actionsrequiring immediate relationships between agents and actions. In contrast, relyingsolely on how frequently a construction occurs showed subpar performance acrossmost frames compared to the other examined methods. These findings, derivedfrom the application of the evaluation framework, contribute to the further devel-opment and operationalization of large-scale computational construction grammars,particularly in optimizing heuristic selection based on semantic frame properties.