Comment by maxloh
5 days ago
Background of the project:
Tectonic is forked from the XeTeX TeX engine. However, the build process for XeTeX (and all other mainstream TeX engines) is extremely baroque:
1. The original Knuth sources are written in a language called WEB (which is basically used by nobody else)
2. Those sources are then patched with a series of “change files” due to restrictions on distributing modified sources
3. These patched files are then converted to the Pascal language using some custom processing scripts
4. That Pascal code is then converted to C code using additional custom tools
5. Those C files are then compiled against a set of extension files and libraries written natively in C and C++
One of the big motivators for the launch of the Tectonic project was to break out of this ridiculously unwelcoming development process. Tectonic is based on the C/C++ files that emerge from the XeTeX build process, and is gradually translating that code to modern Rust.
While the core TeX architecture and XeTeX are largely stable, they do, however, evolve with time. This repository recreates the final XeTeX C/C++ files from their source, so that modifications can be ported into the main Tectonic codebase.
https://github.com/tectonic-typesetting/tectonic-staging/blo...
> However, the build process for XeTeX (and all other mainstream TeX engines) is extremely baroque
(Disclaimer: I'm on the TeX Live team)
Yes, the build process is rather unusual internally, but this is pretty well insulated from the user. The "standard" build commands used by lots of other open source projects
should mostly just work, and give you a functioning TeX engine at the end. The full steps are listed at [0] if you're interested in more details though. And TeX Live is regularly built on essentially every platform imaginable [1], so it's pretty unlikely that you'll ever need to work with the low-level build system.
And only some of the engines require this complex build process; LuaTeX is the recommended engine these days [2], and it's written entirely in C (with a few C++ libraries), and it uses a standard autotools build process [3].
> One of the big motivators for the launch of the Tectonic project was to break out of this ridiculously unwelcoming development process.
TeX development is pretty welcoming in my personal experience: the first time that I built TL (~5 years ago), I was able to follow the official instructions without any problems, and I got a working TeX engine by the end of it. And it was only 2 or 3 years after that that I became an "official" member of the TL team, so I like to think that the development process is fairly welcoming.
But if you have any suggestions on what we can do better, please let me know, either by replying to this comment or to the email address linked in my profile.
> Tectonic is based on the C/C++ files that emerge from the XeTeX build process
I haven't looked at the Tectonic source, but the problem is that the C code generated by Web2C is fairly unreadable, so it's not really very usable as source code.
> While the core TeX architecture and XeTeX are largely stable, they do, however, evolve with time.
XeTeX is essentially frozen for the time being [4], unless someone steps up to maintain it. We are cautiously optimistic regarding LLMs though: they seem to be fairly decent at writing WEB code, and since even the TL development team only has 2 developers who are proficient in WEB, we'll take any help that we can get. (We don't have any non-trivial AI-written code yet, but it's definitely something that we're looking into)
[0]: https://tug.org/texlive/build.html
[1]: https://ftp.math.utah.edu/pub/texlive-utah/
[2]: https://www.latex-project.org/news/2024/11/01/issue40-of-lat...
[3]: https://gitlab.lisn.upsaclay.fr/texlive/luatex
[4]: https://tex.stackexchange.com/a/755239/270600
So what does this fork do? No more WEB? No more Pascal? Does it produce exact results as XeTeX?