Search papers, labs, and topics across Lattice.
This paper evaluates how well LLMs and their tokenizers handle Arabic root-pattern morphology, a complex non-concatenative system. They first assess tokenizer morphological alignment against gold-standard segmentation. Then, they probe LLM performance on generating novel root-pattern forms using a new test set. The key finding is that tokenizer morphological alignment doesn't guarantee better morphological generation in LLMs, challenging the assumed benefits of morphological tokenization.
Tokenizing Arabic text morphologically doesn't necessarily improve how well LLMs generate new word forms, suggesting that LLMs may learn morphological patterns independently of explicit tokenization.
This work investigates how effectively large language models (LLMs) and their tokenization schemes represent and generate Arabic root-pattern morphology, probing whether they capture genuine morphological structure or rely on surface memorization. Arabic morphological system provides a rich testbed for analyzing how LLMs handle complex, non-concatenative forms and how tokenization choices influence this process. Our study begins with an evaluation of morphological fidelity across Arabic and multilingual tokenizers against gold-standard segmentation, followed by an analysis of LLM performance in productive root-pattern generation using a newly developed test set. Our findings across seven Arabic-centric and multilingual LLMs and their respective tokenizers reveal that tokenizer morphological alignment is not necessary nor sufficient for morphological generation, which questions the role of morphological tokenization in downstream performance.
