Machine Translation Lab
Open-source machine translation pipeline for classic literature
Buchenberg is a research project that translates public-domain books from Project Gutenberg into multiple languages using open-source machine translation models — then measures and ranks translation quality automatically, sentence by sentence.

The name Buchenberg is a deliberate echo of Gutenberg — honouring the spirit of the original project while extending it into the age of machine translation. Buch is German for book; the suffix -enberg ties us to our source.

How it works

Each sentence of a source book is translated into a target language by a pool of models — several general-purpose LLMs run at different temperatures, plus a dedicated machine-translation model. Each translation is then back-translated to English and compared to the original using cosine similarity on multilingual embeddings. A separate LLM, chosen only to judge and never to translate, blindly rates each candidate on grammar, naturalness, and fidelity. Combining both scores selects a winner for this first phase.

That winner then becomes the anchor for the next phase: self-refinement. The pipeline re-translates the current best translation, taking it as a hint and mutating it — an anchored mutation that keeps the sentence grammatical while searching for something better. This can repeat over several phases, each one anchored to the best result so far. The final winner is chosen across all phases played, so the finished document is a hybrid not of models but of phases: the best translation for each sentence, whichever model or phase produced it.

📐 Back-translation scoring
Translate to target language, then back to English. Measure cosine similarity between original and round-tripped text. A high score means the translation preserved meaning.
⚖️ LLM judge
A dedicated LLM, used only for judging, evaluates each candidate translation blindly on three axes: grammar, naturalness, and fidelity to the original. The judge score carries 60% of the final ranking weight.
🧬 Self-refinement
The first-phase winner is fed back as a hint and re-translated rather than translated from scratch. This anchored mutation keeps grammar intact while exploring for better wording — evolution over language itself.
🏆 Sentence-level winner
No single model — or phase — wins every sentence. The final document combines the best-scoring translation per sentence across every phase played — a hybrid that outperforms any individual model.
100% open source. All models used are freely available. No proprietary APIs, no cloud translation services. Source books are from Project Gutenberg — public domain, freely distributable. The pipeline runs on commodity hardware with a PostgreSQL backend and Ollama for local and cloud LLM inference.