Shoshan — Hebrew lemmatizer

Paste Hebrew text and compare Shoshan with DictaBERT-lex, word by word. Shoshan retrieves a real lemma from a fixed bank, or makes a bounded edit of the word for unknown words; DictaBERT-lex predicts each lemma as a single token from its vocabulary. The first run downloads both models, so it takes a minute.

Shoshan: code · weights · data. DictaBERT-lex: dicta-il/dictabert-lex.