A Context-Free Framework for Evaluating Morphological Generalization in Lexical POS Classification

Abstract

Part-of-speech (POS) classification typically uses sentence context, making it difficult to isolate grammatical information available from lexical form alone. This study evaluates context-free lexical POS classification and morphological generalization using 370,100 English word types annotated with Penn Treebank-style POS labels. A majority-class baseline and four morphology-sensitive approaches—rule-based suffix, statistical suffix, character n-gram, and character-level BiLSTM models—were evaluated on coarse- and fine-grained classification tasks. Morphological transfer was examined separately through suffix hold-out experiments in which selected suffix families were excluded from training. Under standard stratified evaluation, the BiLSTM achieved the highest Macro-F1 in both coarse-grained (0.8323) and fine-grained (0.8412) classification. Pairwise randomization tests with Holm correction showed significant differences among the prespecified adjacent model comparisons. Hold-out performance, however, depended strongly on suffix-family composition. Near-perfect scores for -tion and -ness largely reflected the fact that both held-out families contained only NOUN instances, whereas the heterogeneous -ly and -ing families yielded substantially lower Macro-F1 scores. For these two families, the character n-gram model obtained the highest point estimates. Strong performance under standard lexical classification therefore did not necessarily correspond to transfer to unseen morphological families. Morphological generalization should be evaluated separately from standard classification and interpreted in relation to the grammatical composition of the held-out family.