MITRA-lexicon: a corpus-based record of how Sanskrit Buddhist terms were rendered in Tibetan, Chinese and English

What was released

On 19 October 2026, Sebastian Nehrdich, Kengo Harimoto, Sven Sellmer and Oliver Hellwig posted a preprint on arXiv describing MITRA-lexicon, six directional lexica between Sanskrit and English, Tibetan and Chinese. The authors report 2.18 million headword entries and 10.4 million attestations; each attestation links to its sentence pair with the character positions of the Sanskrit word and its equivalent. Senses are ranked by how often they occur. ^[1]

The data were published the same day on GitHub as release v1.1, together with the code, evaluation data and a web interface in beta. The release adds a Chinese–Tibetan lexicon that is not described in the paper. The interface states that it is not a manually curated dictionary: the pairs were extracted automatically, so a match can be wrong, partial or misaligned, and the full sentence should be read. ^{[2] [3]}

How it was built

The Sanskrit–English lexica come from the Mitrasaṃgraha corpus (395,528 sentence pairs); the Sanskrit–Tibetan and Sanskrit–Chinese lexica come from MITRA-parallel v3 (566,068 and 340,691 pairs). Eighteen texts whose Sanskrit is wholly or partly a modern reconstruction from a Tibetan or Chinese translation were excluded. ^[1]

The Sanskrit side is segmented and lemmatised with the ByT5-Sanskrit analyser. A commercial language model (Gemini Flash) labelled 9,366 training examples; a smaller model based on MITRA Qwen3.5-9B was then fine-tuned to copy each translation and mark the span that renders each Sanskrit word. Mining the three corpora took about 36 GPU-hours. The alignment model is published separately, and its card notes that its scores measure agreement with the teacher model, not correctness. ^{[1] [4]}

How reliable it is

Compared with reference dictionaries under strict matching, the first-ranked sense agreed with the dictionary for 58.2% of Sanskrit–Tibetan headwords (Negi), 53.0% of Sanskrit–Chinese headwords (Hirakawa) and 36.6% of Sanskrit–English headwords (Monier-Williams with Apte). The authors call these figures lower bounds, because the dictionaries cover a narrower set of texts and contain their own errors. Agreement rises with frequency: for headwords with at least 100 attestations, the top Sanskrit–Tibetan sense was confirmed in 84–90% of cases. ^[1]

Gold alignments exist only for Sanskrit–English: 100 sentences aligned independently by three Sanskritists, with Krippendorff’s alpha of 0.77 among them; the model reached an F1 of 88.4 against their majority decision. The paper lists further limits: analyser and e-text errors propagate into wrong lemmas, inflected target forms split evidence across senses, the English lexicon is dominated by a few large epic and purāṇic texts, rare headwords contain more errors, and the labelling step depends on a commercial model. The repository describes the paper as under review for ISCLS 2027, so it is a preprint, not a peer-reviewed result. ^{[1] [2]}

Licences and reuse

Because every attestation reproduces its sentence pair, the lexica inherit the licences of their source texts. The Sanskrit–Tibetan, Sanskrit–Chinese and Chinese–Tibetan lexica are released under CC BY-SA 4.0. The Sanskrit–English lexicon comes in two versions: a CC BY-SA 4.0 version with the 92.3% of attestations from CC BY-SA texts, and the complete version under CC BY-NC-ND 4.0, which adds attestations from CC BY-NC-ND texts and from corpus splits with no stated licence. The gold and teacher data are CC BY-NC-ND 4.0, the code is MIT and the alignment model is Apache-2.0. ^{[2] [4]}

A licence on a lexicon does not settle the rights to any other corpus, scan or translation. Reading a term in the interface does not grant permission to republish the source texts or to train a model on them beyond their own licences. The authors ask public-facing applications to acknowledge the Dharmamitra project. ^[2]

A careful way to use it

Search for a term in Sanskrit, Tibetan (Unicode or Wylie) or Chinese, choose the target language and read the attestations in their sentence context rather than the headword alone. Treat frequency as evidence of translators’ usage, not as a ruling on the correct equivalent; a rare sense with one attestation may be an alignment error. Prefer headwords with several attestations, note the collection the attestation comes from, and record the release version and licence when you cite an entry. ^{[1] [3]}

For Matreya this is a practical check for terminology across our languages and a documented example of a permission-aware corpus resource: licences are stated per file and the construction can be reproduced from the published code and data. We have not reproduced the reported numbers ourselves, and the lexicon does not replace a specialist’s reading of a passage. ^{[1] [2]}

Editorial note

The authors acknowledge the Tsadra Foundation for funding the computing infrastructure and disclose the use of generative AI for programming and for the language of the paper. This article and its translations were prepared with AI assistance; specialist language review has not been completed. ^[1]