#9170 Read build files once and publish outputs from memory in OCaml rewatch

The compiler now reads and writes build files through Build_file_store. Rewatch enables it for the duration of each build: a file is read from disk at most once, a file written through the store is kept for later reads, and each domain keeps the interfaces it has unmarshalled until the file is written again. bsc leaves the store disabled and reads the disk as before. Writers render their output in memory first, so the bytes written are unchanged: every artifact of the fixtures below, .cmt files included, is byte-identical to master's.

The module graph now takes the dependencies of a module that wasn't parsed in this build from its published AST, which is a copy of the one in lib/bs, and those of a module parsed in this build from memory. The published ASTs are inventoried on all worker domains, and a header is read with one small read instead of through a channel that buffers 64 KiB of the AST.

Copies now take their contents from Build_file_store, where the compiler left what it wrote, and keep the published interfaces and .cmj files for dependents to read; ASTs, typed trees and JavaScript leave the store once published. Each parse publishes its AST and source on the worker that ran it. The source copy that compiling publishes again is written only if it differs from the one the parse published. Every rewatch-side write or removal of a file updates the store.

Codecov flagged the store's failure handling as untested. The new cases check that an interface cut short reads as corrupted from memory as it does from disk, that a failed write keeps nothing for its path, and that a failed request makes its domain read its interfaces again while a successful one keeps them.

The entry said clean builds became about a quarter faster, as measured with the disk nearly full. Measured again with free space, a clean build of the synthetic 2000-module project takes 2.21 s on master and 1.47 s with this branch, about a third less.

The guide still said the main domain publishes every artifact, which stopped being true when workers began publishing what they produce from memory.

Binary_ast.read_header finds where the dependency block and the source path end in the first bytes of an AST, or says they don't hold the whole header. read_ast_exn raises End_of_file for a file without one, and rewatch reads one block of a published AST, and all of it only when the header may be longer.

run_parses maps each parse task's result to publish it, and task and run_requests are as they were before the build file store. Compile_assets.create takes each directory's entries in turn. The store says why its enabled and generation fields are read without its lock.

Module_graph.initialize takes the paths that this build parsed successfully, which Build_preparation already knows, and reads their dependencies from their new AST in lib/bs. The store's copy of that AST only saves the disk read.

A domain caches interfaces whose contents add up to at most 16 MB, and starts over when the next one would exceed that. The runtime's interfaces take 2 MB. An interface rewritten during the build replaces its entry and its size.

Ext_io.unmarshal checks that the marshalled value ends within the contents, and raises End_of_file when it doesn't. The AST and .cmj readers use it, and read their file with Ext_io.load_file instead of their own copy of it.

Misc.remove_file forgets the path it removes, and publishing an implementation that has an interface evicts the .cmi that its compile read. Parallel.map shares the items among the domains that started. The store's interface no longer suggests keep:false for .cmt files, which rewatch copies out of the store.

New tests cover a removed file, a file that exists but can't be read, several domains using the store at once, more spellings of one path, and AST headers that end at the edge of the first read.

The new test publishes a module and checks that the published .cmi and .cmj stay in the store, while the typed tree, the AST, the sources, the JavaScript output and every copy leave it. It also publishes an implementation that has an interface, whose compile reads its .cmi in lib/bs again, and checks that this .cmi leaves the store too. Each check fails without the eviction it covers.

The worker now publishes only parses that printed nothing. Package_parse publishes the others itself, after it registers their cleanup, as it did for every parse before the worker took over publication.

A new test parses one clean source and one that warns, and checks that only the clean one is published.

The branch added one entry for the build file store and publication and another for the AST header scan. Each said it made builds about a third faster, without saying which builds, so they read as two similar claims. One entry now describes the three steps together, with the figures the pull request measured on the synthetic 2000-module project: a clean build takes 2.21 s on master and 1.47 s with the branch, a no-op build 0.154 s and 0.089 s, and a build after editing a leaf module 0.174 s and 0.100 s.