Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces HeceTokenizer, a syllable-based tokenizer for Turkish that leverages the language's deterministic phonological structure to create a closed vocabulary of ~8,000 syllables. A BERT-tiny model trained with HeceTokenizer and a chunk-based retrieval strategy achieves 50.3% Recall@5 on TQuAD, outperforming a morphology-driven baseline using a model 200x larger. This demonstrates the effectiveness of syllable-based tokenization as a strong inductive bias for Turkish retrieval tasks, especially in resource-constrained settings.
A BERT-tiny model with syllable-based tokenization beats a much larger morphology-driven model on Turkish retrieval, highlighting the power of phonological inductive bias.
HeceTokenizer is a syllable-based tokenizer for Turkish that exploits the deterministic six-pattern phonological structure of the language to construct a closed, out-of-vocabulary (OOV)-free vocabulary of approximately 8,000 unique syllable types. A BERT-tiny encoder (1.5M parameters) is trained from scratch on a subset of Turkish Wikipedia using a masked language modeling objective and evaluated on the TQuAD retrieval benchmark using Recall@5. Combined with a fine-grained chunk-based retrieval strategy, HeceTokenizer achieves 50.3% Recall@5, surpassing the 46.92% reported by a morphology-driven baseline that uses a 200 times larger model. These results suggest that the phonological regularity of Turkish syllables provides a strong and resource-light inductive bias for retrieval tasks.