Why
A dependent only needs a module's .cmi to type check, but a build system that runs requests in process can only start it after that module has also produced its lambda, JavaScript, .cmj and .cmt.
A request now takes hooks from the build system. interface_computed receives the contents of the .cmi an implementation is about to write, once no warning reported as an error is pending; the contents are computed once (Cmi_format.cmi_contents) and also written. redirect_cmi hands Bs_cmi_load the contents of a .cmi that is not published yet, which it reads instead of the build file store; the store never sees them. before_effect is called before the request writes a .cmi, .cmt, .cmti, binary AST or a PPX's input, runs a command such as a PPX, or takes a command line action that writes a file (-e, -format); a build system that started the request early waits there, or raises to abandon it before any effect. Build_file_store.key is exported so that a build system matches paths as the store does. bsc attaches no hooks and behaves as before.
Review
- Can a redirected interface reach a later request through the store or the decoded interface cache? Only as the same bytes. The dependency writes the very string it announced through the store, and publishing copies it, so a domain's cache may reuse an interface it decoded from the redirect for the published file. A request that is abandoned or fails drops its domain's cache.
Why
On builds limited by a long dependency chain most job slots sit idle on the critical path: a 136 module library gets only 1.6x from 7 domains.
When a slot would otherwise idle, a module can now start as soon as each unfinished dependency's interface is final: an implementation without .resi has computed the .cmi it is about to write, or an interface file has compiled. The early request reads those .cmi files from memory and waits at a gate before writing a file, running a command or returning its result. The scheduler adopts it at the point it would have started it (dependencies complete, module still dirty), so its code generation sees the dependencies' .cmj files, and artifacts, diagnostics and compiler.log are unchanged. Otherwise it is abandoned without effects, together with anything that started on top of it.
Redirected paths are matched as the build file store keys them. The directories in which the build publishes interfaces are spelled from canonical package roots, both in the redirected paths and on the load path, so a lookup there that matches no redirected path names another file. A lookup of a redirected file name in any other directory, which may be an alias from user flags (with -nostdlib those come first), or one that differs only in case, waits for adoption instead. Whether an announced interface changes the published one is read from disk, so that an abandoned request never fills the store. Modules with js-post-build commands or lowercase .cmi names don't release their dependents early, and with one domain nothing changes.
Review
- Can a dependent see an interface that differs from master's? It reads the exact contents the dependency then writes, and it is only adopted after the dependency completed. If the dependency fails later, for example in code generation, the dependent is abandoned before it wrote, ran or reported anything.