1. Define the task before choosing a model
Finding a quotation, comparing translations and introducing a beginner to a tradition are different tasks. They require different data and criteria. A broad promise to create “Buddhist AI” hides those distinctions and encourages infrastructure spending before anyone has defined success.
A possible first Matreya pilot would answer limited questions about an authorised corpus and identify supporting passages. Success would mean fewer incorrect references under documented conditions. Sometimes the correct outcome would be to decline a question because the necessary material is absent. This is a research proposal: corpus permissions, independent questions and qualified reviewers still need to be established before an experiment can support claims.
2. What retrieval adds to generation
Lewis and colleagues introduced a RAG formulation combining generation with access to an external document collection. [1] The practical distinction is between finding material and composing an answer. An index can be documented and updated separately from a model’s parameters, but retrieved material does not guarantee faithful use.
A reader-facing tool could show a short answer, its supporting passages and the surrounding source. It should also report when retrieval fails. Evaluation must examine the whole route: whether the correct document was found, whether the right passage was selected and whether the conclusion follows. Improving prose alone can conceal failures earlier in this chain. An architectural label is not a reliability certificate.
- 01Question→
- 02Source retrieval→
- 03Cited response→
- 04Human review
An editorial diagram by Matreya: a methodological illustration, not an experimental result.
3. A real reference can still mislead
A model can cite an existing publication while attributing an absent claim to it. It may overlook a negation or turn a cautious suggestion into certainty. With Buddhist texts, it may also confuse commentary with scripture or one school’s position with agreement across traditions.
ALCE treats factual correctness and citation quality as distinct evaluation concerns. [2] We propose checking three relations: the source exists; the cited passage contains relevant evidence; and the claim stays within that evidence. An answer could visibly distinguish source content, paraphrase and interpretation. A link should let the reader inspect the relevant version and translation. Where texts differ, the interface should preserve the difference rather than average it into apparent agreement.
4. Build an evaluation that can fail
A useful test set could include straightforward source questions, translation comparisons, questions with false premises and questions unanswered by the corpus. The last two categories reveal whether a system invents information when it should abstain.
Expected answers should be prepared independently of the model being tested. One reviewer might formulate a question and another verify its reference. Disagreements should be documented, and some questions retained for a final evaluation. A comparison with and without retrieval needs matching questions, recorded model versions and a clear resource budget. Reports should show failure categories alongside averages. Confusing traditions is a different problem from misspelling a translator’s name; treating them identically makes improvement harder.
5. Give the reader an addressable passage
A link to an entire book transfers most verification work to the reader. LongCite investigates finer-grained citation in long-context question answering. [3] Its relevance here is methodological: readers need enough precision to check a claim without repeating the whole search.
An address might be a stable segment identifier, a folio side or a volume and page. It should not depend on the current order of search results. Context must also remain available: an adjacent sentence may show that a position belongs to an opponent whom the author later criticises. Dividing documents into convenient search chunks can break such relationships. Evaluation therefore needs philological and traditional expertise alongside software engineering.
6. An index also uses someone’s materials
Reading, copying, displaying excerpts and transmitting them to an external service are distinct uses. BDRC’s access policy illustrates why collection items should not be assumed to share one unrestricted status. [4]
Before building a prototype, we propose a record of permitted actions for every collection: its source, version, storage and display conditions, deletion route and responsible contact. Sending passages to a model provider is another action requiring consideration. UNESCO’s AI ethics recommendation offers a reference for discussing accountability and cultural diversity. [5] It does not certify our own project. Practical responsibility means making actual data flows understandable and correctable rather than relying on statements of benevolent intent.
7. Reviewers need bounded responsibilities
A system built mostly from English translations may inherit their interpretive choices. Better-digitised traditions may dominate retrieval. Asking a model to be neutral cannot repair missing or uneven data; the collection’s composition and limits need documentation.
Volunteers could check a small set of references or identify genre confusion. They should not be assigned an undefined duty to guarantee universal Buddhist correctness. Corrections need a process, responsible people and a way to notify readers of significant changes. A search assistant should not claim authority to assess spiritual attainment or issue personal teachings on behalf of a tradition. Its scope should follow the task that has actually been evaluated, including in the interface’s normal behaviour.
8. A small result can be a useful result
Fine-tuning should address an identified failure. If the real obstacle is poor OCR, missing names or incomplete metadata, additional model training may not solve it. A modest retrieval tool with good documentation may offer more value and clearer research evidence.
Matreya’s proposed question is whether verified passages improve citation accuracy on a specified set of tasks. The answer could be positive, negative or conditional on question type. Each outcome is useful when methods and limitations are public. Improvement on one collection does not establish reliability across Buddhist literature. Trust grows when sources can be found, mistakes can be corrected and the system can acknowledge an unanswered question; fluency and scale alone do not establish wisdom.
Key points
- RAG separates retrieval and generation without guaranteeing a faithful answer.
- Check the existence, precise location and evidential relevance of every citation.
- A manageable pilot should include unanswerable questions and false premises, not only easy successes.
Sources & further reading
Links point to original publications, texts or official institutional pages. Access and reuse conditions may differ.
- Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksLewis et al. / arXiv, 2020
Original RAG architecture paper; its results do not automatically transfer to Buddhist texts.
- Enabling Large Language Models to Generate Text with CitationsGao et al. / EMNLP, 2023
The ALCE benchmark and paper on evaluating cited answers.
- LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QAZhang et al. / arXiv, 2024
Fine-grained citations in long-context QA, not evidence of our future tool’s reliability.
- Access PoliciesBuddhist Digital Resource Center
An example of differentiated access conditions.
- Recommendation on the Ethics of Artificial IntelligenceUNESCO, 2021
A normative reference for accountability, rights and cultural diversity.
An editorial article prepared by Matreya with AI assistance. It has not undergone external scientific peer review. Sources are provided for independent checking.
Editorial policy · Report an error or suggest an addition
