@bnjbvr - compilersTechnical blog and random musings.Zola2021-02-17T19:00:42+00:00https://bouvier.cc/tags/compilers/atom.xmlA primer on code generation in Cranelift2021-02-17T19:00:42+00:002021-02-17T19:00:42+00:00
Benjamin Bouvier
https://bouvier.cc/tech/cranelift-codegen-primer/<script src="https://cdn.jsdelivr.net/npm/mermaid/dist/mermaid.min.js"></script>
<script>mermaid.initialize({startOnLoad:true});</script>
<p><a rel="noopener noreferrer external" target="_blank" href="https://github.com/bytecodealliance/wasmtime/tree/main/cranelift#cranelift-code-generator">Cranelift</a> is a code generator written in the Rust programming language that aims to be a fast code generator, which outputs machine code that runs at reasonable speeds.</p>
<span id="continue-reading"></span>
<p>The Cranelift compilation model consists in compiling functions one by one, holding extra information about external entities, like external functions, memory addresses, and so on. This model allows for concurrent and parallel compilation of individual functions, which supports the goal of fast compilation. It was designed this way to allow for just-in-time (JIT) compilation of WebAssembly binary code in Firefox, although its scope has broadened a bit. Nowadays it is used in a few different WebAssembly runtimes, including <a rel="noopener noreferrer external" target="_blank" href="https://github.com/bytecodealliance/wasmtime#wasmtime">Wasmtime</a> and <a rel="noopener noreferrer external" target="_blank" href="https://wasmer.io/">Wasmer</a>, but also as an alternative backend for Rust debug compilation, thanks to <a rel="noopener noreferrer external" target="_blank" href="https://github.com/bjorn3/rustc_codegen_cranelift">cg_clif</a>.</p>
<p>A classic compiler design usually includes running a parser to translate the source to some form of intermediate representations, then run optimization passes onto them, then feeds this to the machine code generator.</p>
<p>This blog post focuses on the final step, namely the concepts that are involved in code generation, and what they map to in Cranelift. To make things more concrete, we’ll take a specific instruction, and see how it’s translated, from its creation down to code generation. At each step of the process, I’ll provide a short (<em>ahem</em>) high-level explanation of the concepts involved, and I’ll show what they map to in Cranelift, using the example instruction. While this is not a tutorial detailing how to add new instructions in Cranelift, this should be an interesting read for anyone who’s interested in compilers, and this could be an entry point if you’re interested in hacking on the Cranelift <code>codegen</code> crate.</p>
<p>This is our plan for this blog post: each squared box represents data, each
rounded box is a process. We’re going to go through each of them below.</p>
<div class="mermaid">
graph TD;
clif[Optimized CLIF];
vcode[VCode];
final_vcode[Final VCode];
machine_code[Machine code artifacts];
lowering([Lowering]);
regalloc([Register allocation]);
codegen([Machine code generation]);
clif --> lowering --> vcode --> regalloc --> final_vcode --> codegen --> machine_code
</div>
<h2 id="intermediate-representations"><a class="zola-anchor" href="#intermediate-representations" aria-label="Anchor link for: intermediate-representations">🔗</a>Intermediate representations</h2>
<p>Compilers use <strong>intermediate representations</strong> (<em>IR</em>) to represent source code. Here we’re interested in representations of the <em>data flow</em>, that is instructions themselves and only that. The IRs contain information about the instructions themselves, their operands, type specialization information, and any additional metadata that might be useful. IRs usually map to a certain level of abstraction, and as such, they are useful for solving different problems that require different levels of abstraction. Their shape (which data structures) and numbers often have a huge impact on the performance of the compiler itself (that is, how fast it is at compiling).</p>
<p>In general, most programming languages use IRs internally, and yet, these are invisible to the programmers. The reason is that source code is usually first <em>parsed</em> (tokenized, verified) and then translated into an IR. The <em>abstract syntax tree</em>, aka AST, is one such IR representing the source code itself, in a format that’s very close to the source code itself. Since the raison d’être of Cranelift is to be a code generator, having a text format is secondary, and only useful for testing and debugging purposes. That’s why embedders directly create and manipulate Cranelift’s IR.</p>
<p>At the time of writing, Cranelift has two IRs to represent the function’s code:</p>
<ul>
<li>one external, high-level intermediate representation, called <strong>CLIF</strong> (for <em>Cranelift IR format</em>),</li>
<li>one internal, low-level intermediate representation called <strong>VCode</strong> (for <em>virtual-registerized code</em>).</li>
</ul>
<h2 id="clif-ir"><a class="zola-anchor" href="#clif-ir" aria-label="Anchor link for: clif-ir">🔗</a>CLIF IR</h2>
<p>CLIF is the IR that Cranelift embedders create and manipulate. It consists of high-level typed operations that are convenient to use and/or can be simply translated to machine code. It is in <a rel="noopener noreferrer external" target="_blank" href="https://en.wikipedia.org/wiki/Static_single_assignment_form">static single assignment (SSA) form</a>: each value referenced by an operation (SSA value) is defined only once, and may have as many uses as desired. CLIF is practical to use and manipulate for classic compilers optimization passes (e.g. <a rel="noopener noreferrer external" target="_blank" href="https://en.wikipedia.org/wiki/Loop-invariant_code_motion">LICM</a>), as it is generic over the target architecture which we’re compiling to.</p>
<pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="rust"><span class="giallo-l"><span style="color: #BB9AF7;">let</span><span style="color: #C0CAF5;"> x</span><span style="color: #89DDFF;"> =</span><span style="color: #C0CAF5;"> builder</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">ins</span><span style="color: #89DDFF;">().</span><span style="color: #7AA2F7;">iconst</span><span style="color: #89DDFF;">(</span><span style="color: #0DB9D7;">types</span><span style="color: #89DDFF;">::</span><span style="color: #C0CAF5;">I64</span><span style="color: #89DDFF;">,</span><span style="color: #FF9E64;"> 42</span><span style="color: #89DDFF;">);</span></span>
<span class="giallo-l"><span style="color: #BB9AF7;">let</span><span style="color: #C0CAF5;"> y</span><span style="color: #89DDFF;"> =</span><span style="color: #C0CAF5;"> builder</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">ins</span><span style="color: #89DDFF;">().</span><span style="color: #7AA2F7;">iconst</span><span style="color: #89DDFF;">(</span><span style="color: #0DB9D7;">types</span><span style="color: #89DDFF;">::</span><span style="color: #C0CAF5;">I64</span><span style="color: #89DDFF;">,</span><span style="color: #FF9E64;"> 1337</span><span style="color: #89DDFF;">);</span></span>
<span class="giallo-l"><span style="color: #BB9AF7;">let</span><span style="color: #C0CAF5;"> sum</span><span style="color: #89DDFF;"> =</span><span style="color: #C0CAF5;"> builder</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">ins</span><span style="color: #89DDFF;">().</span><span style="color: #7AA2F7;">iadd</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">x</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> y</span><span style="color: #89DDFF;">);</span></span></code></pre>
<p>An example of Rust code that would generate CLIF IR: using an IR builder, two constant 64-bits integer SSA values x and y are created, and then added together. The result is stored into the <code>sum</code> SSA value, which can then be consumed by other instructions.</p>
<p>The code for the IR builder we’re manipulating above is automatically generated by the <code>cranelift-codegen</code> build script. The build script uses a domain specific <em>meta</em> language (DSL)<sup class="footnote-reference" id="fr-2-1"><a href="#fn-2">1</a></sup> that defines the instructions, their input and output operands, which input types are allowed, how the output type is inferred, etc. We won’t take a look at this <em>today</em>: this is a bit too far from code generation, but this could be material for another blog post.</p>
<p>As an example of a full-blown CLIF generator, there is <a rel="noopener noreferrer external" target="_blank" href="https://github.com/bytecodealliance/wasmtime/tree/main/cranelift/wasm">a crate</a> in the Cranelift project that allows translating from the WebAssembly binary format to CLIF. The Cranelift backend for Rustc uses its own CLIF generator that translates from one of the Rust compiler’s IRs.</p>
<p>Finally, it’s time to reveal what’s going to be our running example! The Chosen One is the <code>iadd</code> CLIF operation, which allows to add two integers of any length together, with wrapping semantics. It is both simple to understand what it does, and exhibits interesting behaviors on the two architectures we’re interested in. So, let’s continue down the pipeline!</p>
<h2 id="vcode-ir"><a class="zola-anchor" href="#vcode-ir" aria-label="Anchor link for: vcode-ir">🔗</a>VCode IR</h2>
<p>Later on, the CLIF intermediate representation is <em>lowered</em>, i.e. transformed from a high-level one into a lower-level one. Here lower level means a form more specialized for a machine architecture. This lower IR is called <em>VCode</em> in Cranelift. The values it references are called <em>virtual registers</em> (more on the <em>virtual</em> bit below). They’re not in SSA form anymore: each virtual register may be redefined as many times as we want. This IR is used to encode register allocation constraints and it guides machine code generation. As a matter of fact, since this information is tied to the machine code’s representation itself, this IR is also target-specific: there’s one flavor of VCode per each CPU architecture we’re compiling to.</p>
<p>Let’s get back to our example, that we’re going to compile on two instruction set architectures:</p>
<ul>
<li>ARM 64-bits (aka aarch64), which is used in most mobile devices but start to become mainstream on laptops (Apple’s Mac M1, some Chromebooks)</li>
<li>Intel’s x86 64-bits (aka x86_64, also abbreviated x64), which is used in most desktop and laptop machines).</li>
</ul>
<p>An integer addition machine instruction on aarch64 will take three operands: two input operands (one of which must be a register), and another third output register operand. While on the x86_64 architecture, the equivalent instruction involves a total of two registers: one that is a read-only source register, and another that is an in-out modified register, containing both the second source and the destination register. We’ll get back to this.</p>
<p>So considering <code>iadd</code>, let’s look at (one of<sup class="footnote-reference" id="fr-4-1"><a href="#fn-4">2</a></sup>) the VCode instruction that’s used to represent integer additions on aarch64 (as defined in <code>cranelift/codegen/src/isa/aarch64/inst/mod.rs</code>):</p>
<pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="rust"><span class="giallo-l"><span style="color: #51597D;font-style: italic;">/// An ALU operation with two register sources and a register destination.</span></span>
<span class="giallo-l"><span style="color: #C0CAF5;">AluRRR</span><span style="color: #89DDFF;"> {</span></span>
<span class="giallo-l"><span style="color: #C0CAF5;"> alu_op</span><span style="color: #89DDFF;">:</span><span style="color: #C0CAF5;"> ALUOp</span><span style="color: #89DDFF;">,</span></span>
<span class="giallo-l"><span style="color: #C0CAF5;"> rd</span><span style="color: #89DDFF;">:</span><span style="color: #C0CAF5;"> Writable</span><span style="color: #89DDFF;"><</span><span style="color: #C0CAF5;">Reg</span><span style="color: #89DDFF;">>,</span></span>
<span class="giallo-l"><span style="color: #C0CAF5;"> rn</span><span style="color: #89DDFF;">:</span><span style="color: #C0CAF5;"> Reg</span><span style="color: #89DDFF;">,</span></span>
<span class="giallo-l"><span style="color: #C0CAF5;"> rm</span><span style="color: #89DDFF;">:</span><span style="color: #C0CAF5;"> Reg</span><span style="color: #89DDFF;">,</span></span>
<span class="giallo-l"><span style="color: #89DDFF;">},</span></span></code></pre>
<p>Some details here:</p>
<ul>
<li><code>alu_op</code> defines the sub-opcode used in the ALU (Arithmetic Logic Unit). It will be <code>AluOp::Add64</code> for a 64-bits integer addition.</li>
<li><code>rn</code> and <code>rm</code> are the conventional aarch64 names for the two input registers.</li>
<li><code>rd</code> is the destination register. See how it’s marked as <code>Writable</code>, while the two others are not? <code>Writable</code> is a plain Rust wrapper that makes sure that we <em>can</em> statically differentiate read-only registers from writable registers; a neat trick that allows us to catch more issues at compile-time.</li>
</ul>
<p>All this information is directly tied to the machine code representation of an addition instruction on aarch64: each field is later used to select some bytes that will be generated during code generation.</p>
<p>As said before, the VCode is specific to each architecture, so x86_64 has a different VCode representation for the same instruction (as defined in <code>cranelift/codegen/src/isa/x64/inst/mod.rs</code>):</p>
<pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="rust"><span class="giallo-l"><span style="color: #51597D;font-style: italic;">/// Integer arithmetic/bit-twiddling: (add sub and or xor mul adc? sbb?) (32 64) (reg addr imm) reg</span></span>
<span class="giallo-l"><span style="color: #C0CAF5;">AluRmiR</span><span style="color: #89DDFF;"> {</span></span>
<span class="giallo-l"><span style="color: #C0CAF5;"> is_64</span><span style="color: #89DDFF;">:</span><span style="color: #C0CAF5;"> bool</span><span style="color: #89DDFF;">,</span></span>
<span class="giallo-l"><span style="color: #C0CAF5;"> op</span><span style="color: #89DDFF;">:</span><span style="color: #C0CAF5;"> AluRmiROpcode</span><span style="color: #89DDFF;">,</span></span>
<span class="giallo-l"><span style="color: #C0CAF5;"> src</span><span style="color: #89DDFF;">:</span><span style="color: #C0CAF5;"> RegMemImm</span><span style="color: #89DDFF;">,</span></span>
<span class="giallo-l"><span style="color: #C0CAF5;"> dst</span><span style="color: #89DDFF;">:</span><span style="color: #C0CAF5;"> Writable</span><span style="color: #89DDFF;"><</span><span style="color: #C0CAF5;">Reg</span><span style="color: #89DDFF;">>,</span></span>
<span class="giallo-l"><span style="color: #89DDFF;">},</span></span></code></pre>
<p>Here, the sub-opcode is defined as part of the <code>AluRmiROpcode</code> enum (the comment hints at which other x86 machine instructions are generated by this same VCode). See how there’s only one <code>src</code> (source) register (or memory or immediate operand), while the instruction conceptually takes two inputs? That’s because it’s expected that the <code>dst</code> (destination) register is <em>modified</em>, that is, both read (so it’s the second input operand) and written to (so it’s the result register). In equivalent C code, the x86’s add instruction doesn’t actually do <code>a = b + c</code>. What it does is <code>a += b</code>, that is, one of the sources is <em>consumed</em> by the instruction. This is an artifact inherited from the design of older x86 machines in the 1970’s, when instructions were designed around an accumulator model (and representing efficiently three operands in a CISC architecture would make the encoding larger and harder than it is).</p>
<h2 id="instruction-selection-lowering"><a class="zola-anchor" href="#instruction-selection-lowering" aria-label="Anchor link for: instruction-selection-lowering">🔗</a>Instruction selection (lowering)</h2>
<p>As said before, converting from the high-level IR (CLIF) to the low-level IR (VCode) is called lowering. Since VCode is target-dependent, this process is also target-dependent. That’s where we consider which machine instructions get eventually used for a given CLIF opcode. There are many ways to achieve the same machine state results for given semantics, but some of these ways are faster than other, and/or require fewer code bytes to achieve. The problem can be summed up like this: given some CLIF, which VCode can we create to generate the fastest and/or smallest machine code that carries out the desired semantics? This is called <em>instruction selection</em>, because we’re selecting the VCode instructions among a set of different possible instructions.</p>
<p>How do these IR map to each other? A given CLIF node may be lowered into 1 to N VCode instructions. A given VCode instruction may lead to the code generation of 1 to M machine instructions. There are no rules governing the maximum of entities mapped. For instance, the integer addition CLIF opcode <code>iadd</code> on 64-bits inputs maps to a single VCode instruction on aarch64. The VCode instruction then causes a single code instruction to be generated.</p>
<p>Other CLIF opcodes may generate more than a single machine instruction eventually. Consider the CLIF opcode for signed integer division <code>idiv</code>. Its semantics define that it traps for zero inputs and in case of integer overflow<sup class="footnote-reference" id="fr-3-1"><a href="#fn-3">3</a></sup>. On aarch64, this is lowered into:</p>
<ul>
<li>one VCode instruction that checks if the input is zero and trap otherwise</li>
<li>two VCode instructions for comparing the input values against the minimal integer value and -1</li>
<li>one VCode instruction to trap if the two input values match what we checked against</li>
<li>and one VCode instruction that does the actual division operation.</li>
</ul>
<p>Each of these VCode instruction then generates one or more machine code instructions, resulting in a bit of a longer sequence.</p>
<p>Let’s look at the lowering of <code>iadd</code> on aarch64 (in <code>cranelift/codegen/src/isa/aarch64/lower_inst.rs</code>), edited and simplified for clarity. I’ve added comments in the code, explaining what each line does:</p>
<pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="rust"><span class="giallo-l"><span style="color: #C0CAF5;">Opcode</span><span style="color: #89DDFF;">::</span><span style="color: #C0CAF5;">Iadd</span><span style="color: #89DDFF;"> => {</span></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // Get the destination register.</span></span>
<span class="giallo-l"><span style="color: #BB9AF7;"> let</span><span style="color: #C0CAF5;"> rd</span><span style="color: #89DDFF;"> =</span><span style="color: #7AA2F7;"> get_output_reg</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">ctx</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> outputs</span><span style="color: #89DDFF;">[</span><span style="color: #FF9E64;">0</span><span style="color: #89DDFF;">]).</span><span style="color: #7AA2F7;">only_reg</span><span style="color: #89DDFF;">().</span><span style="color: #7AA2F7;">unwrap</span><span style="color: #89DDFF;">();</span></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // Get the controlling type of the addition (32-bits int or 64-bits int or</span></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // int vector, etc.).</span></span>
<span class="giallo-l"><span style="color: #BB9AF7;"> let</span><span style="color: #C0CAF5;"> ty</span><span style="color: #89DDFF;"> =</span><span style="color: #C0CAF5;"> ty</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">unwrap</span><span style="color: #89DDFF;">();</span></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // Force one of the inputs into a register, not applying any signed- or</span></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // zero-extension.</span></span>
<span class="giallo-l"><span style="color: #BB9AF7;"> let</span><span style="color: #C0CAF5;"> rn</span><span style="color: #89DDFF;"> =</span><span style="color: #7AA2F7;"> put_input_in_reg</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">ctx</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> inputs</span><span style="color: #89DDFF;">[</span><span style="color: #FF9E64;">0</span><span style="color: #89DDFF;">],</span><span style="color: #0DB9D7;"> NarrowValueMode</span><span style="color: #89DDFF;">::</span><span style="color: #C0CAF5;">None</span><span style="color: #89DDFF;">);</span></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // Try to see if we can encode the second operand as an immediate on</span></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // 12-bits, maybe by negating it;</span></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // Otherwise, put it into a register.</span></span>
<span class="giallo-l"><span style="color: #BB9AF7;"> let</span><span style="color: #89DDFF;"> (</span><span style="color: #C0CAF5;">rm</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> negated</span><span style="color: #89DDFF;">) =</span><span style="color: #7AA2F7;"> put_input_in_rse_imm12_maybe_negated</span><span style="color: #89DDFF;">(</span></span>
<span class="giallo-l"><span style="color: #C0CAF5;"> ctx</span><span style="color: #89DDFF;">,</span></span>
<span class="giallo-l"><span style="color: #C0CAF5;"> inputs</span><span style="color: #89DDFF;">[</span><span style="color: #FF9E64;">1</span><span style="color: #89DDFF;">],</span></span>
<span class="giallo-l"><span style="color: #7AA2F7;"> ty_bits</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">ty</span><span style="color: #89DDFF;">),</span></span>
<span class="giallo-l"><span style="color: #0DB9D7;"> NarrowValueMode</span><span style="color: #89DDFF;">::</span><span style="color: #C0CAF5;">None</span><span style="color: #89DDFF;">,</span></span>
<span class="giallo-l"><span style="color: #89DDFF;"> );</span></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // Select the ALU subopcode, based on possible negation and controlling</span></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // type.</span></span>
<span class="giallo-l"><span style="color: #BB9AF7;"> let</span><span style="color: #C0CAF5;"> alu_op</span><span style="color: #89DDFF;"> =</span><span style="color: #BB9AF7;"> if !</span><span style="color: #C0CAF5;">negated</span><span style="color: #89DDFF;"> {</span></span>
<span class="giallo-l"><span style="color: #7AA2F7;"> choose_32_64</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">ty</span><span style="color: #89DDFF;">,</span><span style="color: #0DB9D7;"> ALUOp</span><span style="color: #89DDFF;">::</span><span style="color: #C0CAF5;">Add32</span><span style="color: #89DDFF;">,</span><span style="color: #0DB9D7;"> ALUOp</span><span style="color: #89DDFF;">::</span><span style="color: #C0CAF5;">Add64</span><span style="color: #89DDFF;">)</span></span>
<span class="giallo-l"><span style="color: #89DDFF;"> }</span><span style="color: #BB9AF7;"> else</span><span style="color: #89DDFF;"> {</span></span>
<span class="giallo-l"><span style="color: #7AA2F7;"> choose_32_64</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">ty</span><span style="color: #89DDFF;">,</span><span style="color: #0DB9D7;"> ALUOp</span><span style="color: #89DDFF;">::</span><span style="color: #C0CAF5;">Sub32</span><span style="color: #89DDFF;">,</span><span style="color: #0DB9D7;"> ALUOp</span><span style="color: #89DDFF;">::</span><span style="color: #C0CAF5;">Sub64</span><span style="color: #89DDFF;">)</span></span>
<span class="giallo-l"><span style="color: #89DDFF;"> };</span></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // Emit the VCode instruction in the VCode stream.</span></span>
<span class="giallo-l"><span style="color: #C0CAF5;"> ctx</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">emit</span><span style="color: #89DDFF;">(</span><span style="color: #7AA2F7;">alu_inst_imm12</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">alu_op</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> rd</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> rn</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> rm</span><span style="color: #89DDFF;">));</span></span>
<span class="giallo-l"><span style="color: #89DDFF;">}</span></span></code></pre>
<p>In fact, the <code>alu_inst_imm12</code> wrapper can create one VCode instruction among a set of possible ones (since we’re trying to select <em>the best one</em>). For the sake of simplicity, we’ll assume that <code>AluRRR</code> is going to be generated, i.e. the selected instruction is the one using only register encodings for the input values.</p>
<h2 id="register-allocation"><a class="zola-anchor" href="#register-allocation" aria-label="Anchor link for: register-allocation">🔗</a>Register allocation</h2>
<div class="mermaid">
graph TD
vcode_vreg[VCode with virtual registers]
regalloc([Register allocation])
vcode_rreg[VCode with real registers]
codegen([Code generation])
machine_code(Machine code)
vcode_vreg --> regalloc --> vcode_rreg --> codegen --> machine_code
</div>
<h3 id="vcode-registers-and-stack-slots"><a class="zola-anchor" href="#vcode-registers-and-stack-slots" aria-label="Anchor link for: vcode-registers-and-stack-slots">🔗</a>VCode, registers and stack slots</h3>
<p>Hey, ever wondered what the V in VCode meant? Back to the drawing board. While a program may reference a theoretically unlimited number of instructions, each referencing a theoretically unlimited number of values as inputs and outputs, the physical machine only has a fixed set of containers for those values:</p>
<ul>
<li>either they must live in machine <strong>registers</strong>: very fast to access in the CPU, take some CPU real estate, thus are costly, so there are usually few of them.</li>
<li>or they must live in the process’ <strong>stack memory</strong>: it’s slower to access, but we can have virtually any amount of stack <em>slots</em>.</li>
</ul>
<pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="asm"><span class="giallo-l"><span style="color: #89DDFF;">mov</span><span> %</span><span style="color: #FF9E64;">edi</span><span>,-</span><span style="color: #FF9E64;">0x4</span><span>(%</span><span style="color: #FF9E64;">rbp</span><span>)</span></span>
<span class="giallo-l"><span style="color: #89DDFF;">mov</span><span> %</span><span style="color: #FF9E64;">rsi</span><span>,-</span><span style="color: #FF9E64;">0x10</span><span>(%</span><span style="color: #FF9E64;">rbp</span><span>)</span></span>
<span class="giallo-l"><span style="color: #89DDFF;">mov</span><span> -</span><span style="color: #FF9E64;">0x4</span><span>(%</span><span style="color: #FF9E64;">rbp</span><span>),%</span><span style="color: #FF9E64;">eax</span></span></code></pre>
<p><em>In this example of x86 machine code, %edi, %rsi, %rbp, %eax are all registers; stack slots are memory addresses computed as the frame pointer (%rbp) plus an offset value (which happens to be negative here). Note that stack slots may be referred to by the stack pointer (%rsp) in general.</em></p>
<h3 id="defining-the-register-allocation-problem"><a class="zola-anchor" href="#defining-the-register-allocation-problem" aria-label="Anchor link for: defining-the-register-allocation-problem">🔗</a>Defining the register allocation problem</h3>
<p>The problem of mapping the IR values (in VCode these are the <code>Reg</code>) to machine “containers” is called <strong><a rel="noopener noreferrer external" target="_blank" href="https://en.wikipedia.org/wiki/Register_allocation">register allocation</a></strong> (aka regalloc). Inputs to register allocation can be as numerous as we want them, and map to “virtual” values, hence we call them <em>virtual registers</em>. And… that’s where the V from VCode comes from: the instructions in VCode reference values that are <em>virtual</em> registers before register allocation, so we say the code is in <em>virtualized</em> register form. The output of register allocation is a set of new instructions, where the virtual registers have been replaced by <em>real registers</em> (the physical ones, limited in quantity) or stack slots references (and other additional metadata).</p>
<pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="plain"><span class="giallo-l"><span>// Before register allocation, with unlimited virtual registers:</span></span>
<span class="giallo-l"><span>v2 = v0 + v1</span></span>
<span class="giallo-l"><span>v3 = v2 * 2</span></span>
<span class="giallo-l"><span>v4 = v2 + 1</span></span>
<span class="giallo-l"><span>v5 = v4 + v3</span></span>
<span class="giallo-l"><span>return v5</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span>// One possible register allocation, on a machine that has 2 registers %r0, %r1:</span></span>
<span class="giallo-l"><span>%r0 = %r0 + %r1</span></span>
<span class="giallo-l"><span>%r1 = %r0 * 2</span></span>
<span class="giallo-l"><span>%r0 = %r0 + 1</span></span>
<span class="giallo-l"><span>%r1 = %r0 + %r1</span></span>
<span class="giallo-l"><span>return %r1</span></span></code></pre>
<p>When all is well, the virtual registers don’t conceptually <em>live</em> at the same time, and they can be put into physical registers. Issues arise when there’s not enough physical registers to contain all the virtual registers that live at the same time, which is the case for… a very large majority of programs. Then, register allocation must decide which registers continue to live in registers at a given program point, and which should be <strong>spilled</strong> into a stack slot, effectively <em>storing</em> them onto the stack for later use. This later reuse will imply to <strong>reload</strong> them from the stack slot, using a <em>load</em> machine instruction. The complexity resides in choosing which registers should be spilled, at which program point they should be spilled, and at which program points we should reload them, if we need to do so. Making good choices there will have a large impact on the speed of the generated code, since memory accesses to the stack imply an additional runtime cost. For instance, a variable that’s frequently used in a hot loop should live in a register for the whole loop’s lifetime, and not be spilled/reloaded in the middle of the loop.</p>
<pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="plain"><span class="giallo-l"><span>// Before register allocation, with unlimited virtual registers:</span></span>
<span class="giallo-l"><span>v2 = v0 + v1</span></span>
<span class="giallo-l"><span>v3 = v0 + v2</span></span>
<span class="giallo-l"><span>v4 = v3 + v1</span></span>
<span class="giallo-l"><span>return v4</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span>// One possible register allocation, on a machine that has 2 registers %r0, %r1.</span></span>
<span class="giallo-l"><span>// We need to spill one value, because there's a point where 3 values are live at the same time!</span></span>
<span class="giallo-l"><span>spill %r1 --> stack_slot(0)</span></span>
<span class="giallo-l"><span>%r1 = %r0 + %r1</span></span>
<span class="giallo-l"><span>%r1 = %r0 + %r1</span></span>
<span class="giallo-l"><span>reload stack_slot(0) --> %r0</span></span>
<span class="giallo-l"><span>%r1 = %r1 + %r0</span></span>
<span class="giallo-l"><span>return %r1</span></span></code></pre>
<p>And, since we like to have our cake and eat it too, the register allocator itself should be <em>fast</em>: it should not take an unbounded amount of time to make these allocation decisions. Register allocation has the good taste to be a <a rel="noopener noreferrer external" target="_blank" href="https://en.wikipedia.org/wiki/NP-completeness">NP-complete</a> problem. Concretely, this means that implementations cannot find the <em>best</em> solutions given arbitrary inputs, but they’ll estimate <em>good</em> solutions based on heuristics, in worst-case quadratic time over the size of the input. All of this makes it so that register allocation has its own whole research field, and has been extensively studied for some time now. It is a fascinating problem.</p>
<h3 id="register-allocation-in-cranelift"><a class="zola-anchor" href="#register-allocation-in-cranelift" aria-label="Anchor link for: register-allocation-in-cranelift">🔗</a>Register allocation in Cranelift</h3>
<p>Back to Cranelift. The register allocation contract is that if a value <em>must</em> live in a real register at a given program point, then it <em>does</em> live where it should (unless register allocation is impossible). At the start of code generation for a VCode instruction, we are guaranteed that the input values live in real registers, and that the output real register is available before the next VCode instruction.</p>
<p>You might have noticed that the VCode instructions only refer to registers, and not stack slots. But where are the stack slots, then? The trick is that the stack slots are <em>invisible</em> to VCode. Register allocation may create an arbitrary number of spills, reloads, and register moves<sup class="footnote-reference" id="fr-5-1"><a href="#fn-5">4</a></sup> around VCode instructions, to ensure that their register allocation constraints are met. This is why the output of register allocation is a new list of instructions, that includes not only the initial instructions filled with the actual registers, but also additional spill, reload and move (VCode) instructions added by regalloc.</p>
<p>As said before, this problem is so sufficiently complex, involved and independent from the rest of the code (assuming the right set of interfaces!) that its code lives in a separate crate, <a rel="noopener noreferrer external" target="_blank" href="https://github.com/bytecodealliance/regalloc.rs"><code>regalloc.rs</code></a>, with its own fuzzing and testing infrastructure. I hope to shed some light on it at some point too.</p>
<p>What’s interesting to us today is the register allocation <em>constraints</em>. Consider the aarch64 integer add instruction <code>add rd, rn, rm</code>: <code>rd</code> is the output virtual register that’s written to, while <code>rn</code> and <code>rm</code> are the inputs, thus read from. We need to inform the register allocation algorithm about these constraints. In regalloc jargon, “read to” is known as <em>used</em>, while “written to” is known as <em>defined</em>. Here, the aarch64 VCode instruction <code>AluRRR</code> does <em>use</em> <code>rn</code> and <code>rm</code>, and it <em>def</em>ines <code>rd</code>. This usage information is <em>collected</em> in the <code>aarch64_get_regs</code> function (<code>cranelift/codegen/src/isa/aarch64/inst/mod.rs</code>):</p>
<pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="rust"><span class="giallo-l"><span style="color: #89DDFF;">fn</span><span style="color: #7AA2F7;"> aarch64_get_regs</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">inst</span><span style="color: #89DDFF;">: &</span><span style="color: #C0CAF5;">Inst</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> collector</span><span style="color: #89DDFF;">: &</span><span style="color: #9D7CD8;font-style: italic;">mut</span><span style="color: #C0CAF5;"> RegUsageCollector</span><span style="color: #89DDFF;">) {</span></span>
<span class="giallo-l"><span style="color: #BB9AF7;"> match</span><span style="color: #C0CAF5;"> inst</span><span style="color: #89DDFF;"> {</span></span>
<span class="giallo-l"><span style="color: #89DDFF;"> &</span><span style="color: #C0CAF5;">Inst</span><span style="color: #89DDFF;">::</span><span style="color: #C0CAF5;">AluRRR</span><span style="color: #89DDFF;"> {</span><span style="color: #C0CAF5;"> rd</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> rn</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> rm</span><span style="color: #89DDFF;">, .. } => {</span></span>
<span class="giallo-l"><span style="color: #C0CAF5;"> collector</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">add_def</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">rd</span><span style="color: #89DDFF;">);</span></span>
<span class="giallo-l"><span style="color: #C0CAF5;"> collector</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">add_use</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">rn</span><span style="color: #89DDFF;">);</span></span>
<span class="giallo-l"><span style="color: #C0CAF5;"> collector</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">add_use</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">rm</span><span style="color: #89DDFF;">);</span></span>
<span class="giallo-l"><span style="color: #89DDFF;"> }</span></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // etc.</span></span></code></pre>
<p>Then, after register allocation has assigned the physical registers, we need to instruct it how to replace virtual register mentions by physical register mentions. This is done in the <code>aarch64_map_regs</code> function (same file as above):</p>
<pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="rust"><span class="giallo-l"><span style="color: #89DDFF;">fn</span><span style="color: #7AA2F7;"> aarch64_map_regs</span><span style="color: #89DDFF;"><</span><span style="color: #FF9E64;">RUM</span><span style="color: #89DDFF;">:</span><span style="color: #C0CAF5;"> RegUsageMapper</span><span style="color: #89DDFF;">>(</span><span style="color: #C0CAF5;">inst</span><span style="color: #89DDFF;">: &</span><span style="color: #9D7CD8;font-style: italic;">mut</span><span style="color: #C0CAF5;"> Inst</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> mapper</span><span style="color: #89DDFF;">: &</span><span style="color: #FF9E64;">RUM</span><span style="color: #89DDFF;">) {</span></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // ...</span></span>
<span class="giallo-l"><span style="color: #BB9AF7;"> match</span><span style="color: #C0CAF5;"> inst</span><span style="color: #89DDFF;"> {</span></span>
<span class="giallo-l"><span style="color: #89DDFF;"> &</span><span style="color: #9D7CD8;font-style: italic;">mut</span><span style="color: #C0CAF5;"> Inst</span><span style="color: #89DDFF;">::</span><span style="color: #C0CAF5;">AluRRR</span><span style="color: #89DDFF;"> {</span></span>
<span class="giallo-l"><span style="color: #89DDFF;"> ref</span><span style="color: #9D7CD8;font-style: italic;"> mut</span><span style="color: #C0CAF5;"> rd</span><span style="color: #89DDFF;">,</span></span>
<span class="giallo-l"><span style="color: #89DDFF;"> ref</span><span style="color: #9D7CD8;font-style: italic;"> mut</span><span style="color: #C0CAF5;"> rn</span><span style="color: #89DDFF;">,</span></span>
<span class="giallo-l"><span style="color: #89DDFF;"> ref</span><span style="color: #9D7CD8;font-style: italic;"> mut</span><span style="color: #C0CAF5;"> rm</span><span style="color: #89DDFF;">,</span></span>
<span class="giallo-l"><span style="color: #89DDFF;"> ..</span></span>
<span class="giallo-l"><span style="color: #89DDFF;"> } => {</span></span>
<span class="giallo-l"><span style="color: #7AA2F7;"> map_def</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">mapper</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> rd</span><span style="color: #89DDFF;">);</span></span>
<span class="giallo-l"><span style="color: #7AA2F7;"> map_use</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">mapper</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> rn</span><span style="color: #89DDFF;">);</span></span>
<span class="giallo-l"><span style="color: #7AA2F7;"> map_use</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">mapper</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> rm</span><span style="color: #89DDFF;">);</span></span>
<span class="giallo-l"><span style="color: #89DDFF;"> }</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // etc.</span></span></code></pre>
<p>Note this is reflecting quite precisely what the usage collector did: we’re replacing the virtual register mention for the defined register <code>rd</code> with the information (which real register) provided by the <code>RegUsageMapper</code>. These two functions must stay in sync, otherwise here be dragons! (and bugs very hard to debug!)</p>
<h3 id="register-allocation-on-x86"><a class="zola-anchor" href="#register-allocation-on-x86" aria-label="Anchor link for: register-allocation-on-x86">🔗</a>Register allocation on x86</h3>
<p>On Intel’s x86, register allocation may be a bit trickier: in some cases, the lowering needs to be carefully written so it satisfies some register allocation constraints that are very specific to this architecture. In particular, x86 has <em>fixed register constraints</em> as well as <em>tied operands</em>.</p>
<p>For this specific part, we’ll look at the integer shift-left instruction, which is equivalent to C’s <code>x << y</code>. Why this particular instruction? It exhibits both properties that we’re interested in studying here. The lowering of <code>iadd</code> is similar, albeit slightly simpler, as it <em>only</em> involves tied operands.</p>
<h4 id="fixed-register-constraints"><a class="zola-anchor" href="#fixed-register-constraints" aria-label="Anchor link for: fixed-register-constraints">🔗</a>Fixed register constraints</h4>
<p>On the one hand, some instructions expect their inputs to be in <em>fixed</em> registers, that is, specific registers arbitrarily predefined by the architecture manual. For the example of the shift instruction, if the count is not statically known at compile time (it’s not a shift by a constant value), then the amount by which we’re shifting must be in the <code>rcx</code> register<sup class="footnote-reference" id="fr-8-1"><a href="#fn-8">5</a></sup>.</p>
<p>Now, how do we make sure that the input value actually is in <code>rcx</code>? We can mark <code>rcx</code> as used in the <code>get_regs</code> function so regalloc knows about this, but nothing ensures that the input <em>resides</em> in it at the beginning of the instruction. To resolve this, we’ll introduce a <strong>move instruction</strong> during lowering, that is going to copy the input value into <code>rcx</code>. Then we’re sure it lives there, and register allocation knows it’s used: we’re good to go!</p>
<p>In a nutshell, this shows how lowering and register allocation play together:</p>
<ul>
<li>during lowering, we introduce a move from a dynamic shift input value to <code>rcx</code> before the actual shift</li>
<li>in the register usage function, we mark <code>rcx</code> as used</li>
<li>(nothing to do in the register mapping function: <code>rcx</code> is a real register already)</li>
</ul>
<h4 id="tied-operands"><a class="zola-anchor" href="#tied-operands" aria-label="Anchor link for: tied-operands">🔗</a>Tied operands</h4>
<p>On the other hand, some instructions have operands that are both read and written at the same time: we call them <em>modified</em> in Cranelift and regalloc.rs, but they’re also known as <em>tied operands</em> in the compiler literature. It’s not just that there’s a register that must be read, and a register that must be written to: they <em>must</em> be the same register. How do we model this, then?</p>
<p>Consider a naive solution. We take the input virtual register, and decide it’s allocated to the same register as the output (modified) register. Unfortunately, if the chosen virtual register was going to be reused by another later VCode instruction, then its value would be overwritten (clobbered) by the current instruction. This would result in incorrect code being generated, so this is not acceptable. In general we can’t clobber the value that was in an input value during lowering, because that’s the role of regalloc to make this kind of decisions.</p>
<pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="plain"><span class="giallo-l"><span>// Before register allocation, with virtual registers:</span></span>
<span class="giallo-l"><span>v2 = v0 + v1</span></span>
<span class="giallo-l"><span>v3 = v0 + 42</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span>// After register allocation, on a machine with two registers %r0 and %r1:</span></span>
<span class="giallo-l"><span>// assign v0 to %r0, v1 to %r1, v2 to %r0</span></span>
<span class="giallo-l"><span>%r0 += v1</span></span>
<span class="giallo-l"><span>... = %r0 + 42 // ohnoes! the value in %r0 is v2, not v0 anymore!</span></span></code></pre>
<p>The right solution is, again, to <em>copy</em> this input virtual register into the output virtual register, right before the instruction. This way, we can still reuse the untouched input register in other instructions without modifying it: only the copy is written to.</p>
<p>Pfew! We can now look at the entire lowering for the shift left instruction, edited and commented for clarity:</p>
<pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="rust"><span class="giallo-l"><span style="color: #51597D;font-style: italic;">// Read the instruction operand size from the output's type.</span></span>
<span class="giallo-l"><span style="color: #BB9AF7;">let</span><span style="color: #C0CAF5;"> size</span><span style="color: #89DDFF;"> =</span><span style="color: #C0CAF5;"> dst_ty</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">bytes</span><span style="color: #89DDFF;">() as</span><span style="color: #C0CAF5;"> u8</span><span style="color: #89DDFF;">;</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;">// Put the left hand side into a virtual register.</span></span>
<span class="giallo-l"><span style="color: #BB9AF7;">let</span><span style="color: #C0CAF5;"> lhs</span><span style="color: #89DDFF;"> =</span><span style="color: #7AA2F7;"> put_input_in_reg</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">ctx</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> inputs</span><span style="color: #89DDFF;">[</span><span style="color: #FF9E64;">0</span><span style="color: #89DDFF;">]);</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;">// Put the right hand side (shift amount) into either an immediate (if it's</span></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;">// statically known at compile time), or into a virtual register.</span></span>
<span class="giallo-l"><span style="color: #BB9AF7;">let</span><span style="color: #89DDFF;"> (</span><span style="color: #C0CAF5;">count</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> rhs</span><span style="color: #89DDFF;">) =</span></span>
<span class="giallo-l"><span style="color: #BB9AF7;"> if let</span><span style="color: #C0CAF5;"> Some</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">cst</span><span style="color: #89DDFF;">) =</span><span style="color: #C0CAF5;"> ctx</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">get_input_as_source_or_const</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">insn</span><span style="color: #89DDFF;">,</span><span style="color: #FF9E64;"> 1</span><span style="color: #89DDFF;">).</span><span>constant </span><span style="color: #89DDFF;">{</span></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // Mask count, according to Cranelift's semantics.</span></span>
<span class="giallo-l"><span style="color: #BB9AF7;"> let</span><span style="color: #C0CAF5;"> cst</span><span style="color: #89DDFF;"> = (</span><span style="color: #C0CAF5;">cst</span><span style="color: #89DDFF;"> as</span><span style="color: #C0CAF5;"> u8</span><span style="color: #89DDFF;">) & (</span><span style="color: #C0CAF5;">dst_ty</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">bits</span><span style="color: #89DDFF;">() as</span><span style="color: #C0CAF5;"> u8</span><span style="color: #89DDFF;"> -</span><span style="color: #FF9E64;"> 1</span><span style="color: #89DDFF;">);</span></span>
<span class="giallo-l"><span style="color: #89DDFF;"> (</span><span style="color: #C0CAF5;">Some</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">cst</span><span style="color: #89DDFF;">),</span><span style="color: #C0CAF5;"> None</span><span style="color: #89DDFF;">)</span></span>
<span class="giallo-l"><span style="color: #89DDFF;"> }</span><span style="color: #BB9AF7;"> else</span><span style="color: #89DDFF;"> {</span></span>
<span class="giallo-l"><span style="color: #89DDFF;"> (</span><span style="color: #C0CAF5;">None</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> Some</span><span style="color: #89DDFF;">(</span><span style="color: #7AA2F7;">put_input_in_reg</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">ctx</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> inputs</span><span style="color: #89DDFF;">[</span><span style="color: #FF9E64;">1</span><span style="color: #89DDFF;">])))</span></span>
<span class="giallo-l"><span style="color: #89DDFF;"> };</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;">// Get the destination virtual register.</span></span>
<span class="giallo-l"><span style="color: #BB9AF7;">let</span><span style="color: #C0CAF5;"> dst</span><span style="color: #89DDFF;"> =</span><span style="color: #7AA2F7;"> get_output_reg</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">ctx</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> outputs</span><span style="color: #89DDFF;">[</span><span style="color: #FF9E64;">0</span><span style="color: #89DDFF;">]).</span><span style="color: #7AA2F7;">only_reg</span><span style="color: #89DDFF;">().</span><span style="color: #7AA2F7;">unwrap</span><span style="color: #89DDFF;">();</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;">// Copy the left hand side into the (modified) output operand, to satisfy the</span></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;">// mod constraint.</span></span>
<span class="giallo-l"><span style="color: #C0CAF5;">ctx</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">emit</span><span style="color: #89DDFF;">(</span><span style="color: #0DB9D7;">Inst</span><span style="color: #89DDFF;">::</span><span style="color: #7AA2F7;">mov_r_r</span><span style="color: #89DDFF;">(</span><span style="color: #FF9E64;">true</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> lhs</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> dst</span><span style="color: #89DDFF;">));</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;">// If the shift count is statically known: nothing particular to do. Otherwise,</span></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;">// we need to put it in the RCX register.</span></span>
<span class="giallo-l"><span style="color: #BB9AF7;">if</span><span style="color: #C0CAF5;"> count</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">is_none</span><span style="color: #89DDFF;">() {</span></span>
<span class="giallo-l"><span style="color: #BB9AF7;"> let</span><span style="color: #C0CAF5;"> w_rcx</span><span style="color: #89DDFF;"> =</span><span style="color: #C0CAF5;"> Writable</span><span style="color: #89DDFF;">::</span><span style="color: #7AA2F7;">from_reg</span><span style="color: #89DDFF;">(</span><span style="color: #0DB9D7;">regs</span><span style="color: #89DDFF;">::</span><span style="color: #7AA2F7;">rcx</span><span style="color: #89DDFF;">());</span></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // Copy the shift count (which is in rhs) into RCX.</span></span>
<span class="giallo-l"><span style="color: #C0CAF5;"> ctx</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">emit</span><span style="color: #89DDFF;">(</span><span style="color: #0DB9D7;">Inst</span><span style="color: #89DDFF;">::</span><span style="color: #7AA2F7;">mov_r_r</span><span style="color: #89DDFF;">(</span><span style="color: #FF9E64;">true</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> rhs</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">unwrap</span><span style="color: #89DDFF;">(),</span><span style="color: #C0CAF5;"> w_rcx</span><span style="color: #89DDFF;">));</span></span>
<span class="giallo-l"><span style="color: #89DDFF;">}</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;">// Generate the actual shift instruction.</span></span>
<span class="giallo-l"><span style="color: #C0CAF5;">ctx</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">emit</span><span style="color: #89DDFF;">(</span><span style="color: #0DB9D7;">Inst</span><span style="color: #89DDFF;">::</span><span style="color: #7AA2F7;">shift_r</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">size</span><span style="color: #89DDFF;">,</span><span style="color: #0DB9D7;"> ShiftKind</span><span style="color: #89DDFF;">::</span><span style="color: #C0CAF5;">ShiftLeft</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> count</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> dst</span><span style="color: #89DDFF;">));</span></span></code></pre>
<p>And this is how we tell the register usage collector about our constraints:</p>
<pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="rust"><span class="giallo-l"><span style="color: #C0CAF5;">Inst</span><span style="color: #89DDFF;">::</span><span style="color: #C0CAF5;">ShiftR</span><span style="color: #89DDFF;"> {</span><span style="color: #C0CAF5;"> num_bits</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> dst</span><span style="color: #89DDFF;">, .. } => {</span></span>
<span class="giallo-l"><span style="color: #BB9AF7;"> if</span><span style="color: #C0CAF5;"> num_bits</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">is_none</span><span style="color: #89DDFF;">() {</span></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // if the shift count is dynamic, mark RCX as used.</span></span>
<span class="giallo-l"><span style="color: #C0CAF5;"> collector</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">add_use</span><span style="color: #89DDFF;">(</span><span style="color: #0DB9D7;">regs</span><span style="color: #89DDFF;">::</span><span style="color: #7AA2F7;">rcx</span><span style="color: #89DDFF;">());</span></span>
<span class="giallo-l"><span style="color: #89DDFF;"> }</span></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // In all the cases, the destination operand is modified.</span></span>
<span class="giallo-l"><span style="color: #C0CAF5;"> collector</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">add_mod</span><span style="color: #89DDFF;">(*</span><span style="color: #C0CAF5;">dst</span><span style="color: #89DDFF;">);</span></span>
<span class="giallo-l"><span style="color: #89DDFF;">}</span></span></code></pre>
<p>Only the modified register needs to be mapped to its allocated physical register:</p>
<pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="rust"><span class="giallo-l"><span style="color: #C0CAF5;">Inst</span><span style="color: #89DDFF;">::</span><span style="color: #C0CAF5;">ShiftR</span><span style="color: #89DDFF;"> { ref</span><span style="color: #9D7CD8;font-style: italic;"> mut</span><span style="color: #C0CAF5;"> dst</span><span style="color: #89DDFF;">, .. } => {</span></span>
<span class="giallo-l"><span style="color: #7AA2F7;"> map_mod</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">mapper</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> dst</span><span style="color: #89DDFF;">);</span></span>
<span class="giallo-l"><span style="color: #89DDFF;">}</span></span></code></pre><h3 id="virtual-registers-copies-and-performance"><a class="zola-anchor" href="#virtual-registers-copies-and-performance" aria-label="Anchor link for: virtual-registers-copies-and-performance">🔗</a>Virtual registers copies and performance</h3>
<p>Do these virtual register copies sound costly to you? In theory, they could lead to the code generation of a move instructions, increasing the size of the code generated and causing a small runtime cost. In practice,
register allocation, through its interface, knows how to identify move instructions, their source and their destination. By analyzing them, it can see when a source isn’t used after a given move instruction, and thus allocate the same register for the source and the destination of the move. Then, when Cranelift generates the code, it will avoid generating a move from a physical register to the same one<sup class="footnote-reference" id="fr-7-1"><a href="#fn-7">6</a></sup>. As a matter of fact, creating a VCode copy doesn’t necessarily mean that it will generate a machine code move instruction later: it is present just in case regalloc <em>needs</em> it, but it can be avoided when it’s spurious.</p>
<h2 id="code-generation"><a class="zola-anchor" href="#code-generation" aria-label="Anchor link for: code-generation">🔗</a>Code generation</h2>
<p>Oh my, we’re getting closer to actually being able to run the code! Once register allocation has run, we can generate the actual machine code for the VCode instructions. Cool kids call this step of the pipeline <em>codegen</em>, for code generation. This is the part where we decipher the architecture manuals provided by the CPU vendors, and generate the raw machine bytes for our machine instructions. In Cranelift, this means filling a code buffer (there’s a <code>MachBuffer</code> sink interface for this!), returned along some internal relocations<sup class="footnote-reference" id="fr-6-1"><a href="#fn-6">7</a></sup> and additional metadata. Let’s see what happens for our integer addition, when the times come to generate the code for its VCode equivalent <code>AluRRR</code> on <code>aarch64</code> (in <code>cranelift/codegen/src/isa/aarch64/inst/emit.rs</code>):</p>
<pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="rust"><span class="giallo-l"><span style="color: #51597D;font-style: italic;">// We match on the VCode's identity here:</span></span>
<span class="giallo-l"><span style="color: #89DDFF;">&</span><span style="color: #C0CAF5;">Inst</span><span style="color: #89DDFF;">::</span><span style="color: #C0CAF5;">AluRRR</span><span style="color: #89DDFF;"> {</span><span style="color: #C0CAF5;"> alu_op</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> rd</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> rn</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> rm</span><span style="color: #89DDFF;"> } => {</span></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // First select the top 11 bits based on the ALU subopcode.</span></span>
<span class="giallo-l"><span style="color: #BB9AF7;"> let</span><span style="color: #C0CAF5;"> top11</span><span style="color: #89DDFF;"> =</span><span style="color: #BB9AF7;"> match</span><span style="color: #C0CAF5;"> alu_op</span><span style="color: #89DDFF;"> {</span></span>
<span class="giallo-l"><span style="color: #C0CAF5;"> ALUOp</span><span style="color: #89DDFF;">::</span><span style="color: #C0CAF5;">Add32</span><span style="color: #89DDFF;"> =></span><span style="color: #FF9E64;"> 0b00001011_000</span><span style="color: #89DDFF;">,</span></span>
<span class="giallo-l"><span style="color: #C0CAF5;"> ALUOp</span><span style="color: #89DDFF;">::</span><span style="color: #C0CAF5;">Add64</span><span style="color: #89DDFF;"> =></span><span style="color: #FF9E64;"> 0b10001011_000</span><span style="color: #89DDFF;">,</span></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // etc</span></span>
<span class="giallo-l"><span style="color: #89DDFF;"> };</span></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // Then decide the bits 10 to 15, based on the ALU subopcode as well.</span></span>
<span class="giallo-l"><span style="color: #BB9AF7;"> let</span><span style="color: #C0CAF5;"> bit15_10</span><span style="color: #89DDFF;"> =</span><span style="color: #BB9AF7;"> match</span><span style="color: #C0CAF5;"> alu_op</span><span style="color: #89DDFF;"> {</span></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // other cases</span></span>
<span class="giallo-l"><span style="color: #C0CAF5;"> _</span><span style="color: #89DDFF;"> =></span><span style="color: #FF9E64;"> 0b000000</span><span style="color: #89DDFF;">,</span></span>
<span class="giallo-l"><span style="color: #89DDFF;"> };</span></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // Then use an helper and pass forward the allocated physical registers</span></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // values.</span></span>
<span class="giallo-l"><span style="color: #C0CAF5;"> sink</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">put4</span><span style="color: #89DDFF;">(</span><span style="color: #7AA2F7;">enc_arith_rrr</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">top11</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> bit15_10</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> rd</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> rn</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> rm</span><span style="color: #89DDFF;">));</span></span>
<span class="giallo-l"><span style="color: #89DDFF;">}</span></span></code></pre>
<p>And what’s this <code>enc_arith_rrr</code> doing, then?</p>
<pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="rust"><span class="giallo-l"><span style="color: #89DDFF;">fn</span><span style="color: #7AA2F7;"> enc_arith_rrr</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">bits_31_21</span><span style="color: #89DDFF;">:</span><span style="color: #C0CAF5;"> u32</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> bits_15_10</span><span style="color: #89DDFF;">:</span><span style="color: #C0CAF5;"> u32</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> rd</span><span style="color: #89DDFF;">:</span><span style="color: #C0CAF5;"> Writable</span><span style="color: #89DDFF;"><</span><span style="color: #C0CAF5;">Reg</span><span style="color: #89DDFF;">>,</span><span style="color: #C0CAF5;"> rn</span><span style="color: #89DDFF;">:</span><span style="color: #C0CAF5;"> Reg</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> rm</span><span style="color: #89DDFF;">:</span><span style="color: #C0CAF5;"> Reg</span><span style="color: #89DDFF;">) -></span><span style="color: #C0CAF5;"> u32</span><span style="color: #89DDFF;"> {</span></span>
<span class="giallo-l"><span style="color: #89DDFF;"> (</span><span style="color: #C0CAF5;">bits_31_21</span><span style="color: #BB9AF7;"> <<</span><span style="color: #FF9E64;"> 21</span><span style="color: #89DDFF;">)</span></span>
<span class="giallo-l"><span style="color: #BB9AF7;"> |</span><span style="color: #89DDFF;"> (</span><span style="color: #C0CAF5;">bits_15_10</span><span style="color: #BB9AF7;"> <<</span><span style="color: #FF9E64;"> 10</span><span style="color: #89DDFF;">)</span></span>
<span class="giallo-l"><span style="color: #BB9AF7;"> |</span><span style="color: #7AA2F7;"> machreg_to_gpr</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">rd</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">to_reg</span><span style="color: #89DDFF;">())</span></span>
<span class="giallo-l"><span style="color: #BB9AF7;"> |</span><span style="color: #89DDFF;"> (</span><span style="color: #7AA2F7;">machreg_to_gpr</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">rn</span><span style="color: #89DDFF;">)</span><span style="color: #BB9AF7;"> <<</span><span style="color: #FF9E64;"> 5</span><span style="color: #89DDFF;">)</span></span>
<span class="giallo-l"><span style="color: #BB9AF7;"> |</span><span style="color: #89DDFF;"> (</span><span style="color: #7AA2F7;">machreg_to_gpr</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">rm</span><span style="color: #89DDFF;">)</span><span style="color: #BB9AF7;"> <<</span><span style="color: #FF9E64;"> 16</span><span style="color: #89DDFF;">)</span></span>
<span class="giallo-l"><span style="color: #89DDFF;">}</span></span></code></pre>
<p>Encoding the instruction parts (operands, register mentions) is a lot of bit twiddling and fun. We do so for each VCode instruction, until we’ve generated the whole function’s body. If you remember correctly, at this point register allocation may have added some spills/reloads/move instructions. From the codegen’s point of view, these are just regular instructions with precomputed operands (either real registers, or memory operands involving the stack pointer), so they’re not treated particularly and they’re just generated the same way other VCode instructions are.</p>
<p>More work is done by the codegen backend then, to optimize blocks placement, compute final branch offsets, etc. If you’re interested by this, I strongly encourage you to go read <a rel="noopener noreferrer external" target="_blank" href="https://cfallin.org/blog/2021/01/22/cranelift-isel-2/">this blog post</a> by Chris Fallin. After this, we’re finally done: we’ve produced a code buffer, as well as external relocations (to other functions, memory addresses, etc.) for a single function. The code generator’s task is complete: the final steps consist in linking and, optionally, producing an executable binary.</p>
<h2 id="mission-accomplished"><a class="zola-anchor" href="#mission-accomplished" aria-label="Anchor link for: mission-accomplished">🔗</a>Mission accomplished!</h2>
<p>So, we’re done for today! Thanks for reading this far, hope it has been a useful and pleasant read to you! Feel free to reach out to me on the <a rel="noopener noreferrer external" target="_blank" href="https://twitter.com/bnjbvr">twitterz</a> if you have additional remarks/questions, and to go contribute on <a rel="noopener noreferrer external" target="_blank" href="https://github.com/bytecodealliance/wasmtime">Wasmtime/Cranelift</a> if this sort of things is interesting to you 😇. Until next time, take care of yourselves!</p>
<p>Thanks to <a rel="noopener noreferrer external" target="_blank" href="https://cfallin.org">Chris Fallin</a> for reading and suggesting improvements to this blog post.</p>
<section class="footnotes">
<ol class="footnotes-list">
<li id="fn-2">
<p>Really, Rust <em>is</em> the DSL. It was Python code before, that had the advantage to be faster to update. Yet it was doing a lot of magic behind the curtain, which wasn’t very friendly for new people trying to learn and use Cranelift. Despite a statically typed language helping for exploration through tooling, this meta-language is to partially disappear in the long run, see Chris’ <a rel="noopener noreferrer external" target="_blank" href="https://cfallin.org/blog/2020/09/18/cranelift-isel-1/">blog post</a> on this topic. <a href="#fr-2-1">↩</a></p>
</li>
<li id="fn-4">
<p>Aarch64 connoisseurs may notice that there are other ways to encode an addition. Say, if one of the input operands was the result of a bit shift instruction by an immediate value, then it’s possible to <em>embed</em> the shift within the add, so we end up with fewer machine instructions (and lower the register pressure). This other possible encoding is sufficiently different in terms of register allocation and code generation that it justifies having its own VCode instruction. <code>AluRRR</code> is simpler in the sense that it’s only concerned with register inputs and outputs, thus a perfect example for this post. <a href="#fr-4-1">↩</a></p>
</li>
<li id="fn-3">
<p>What’s an integer overflow for signed integer division? Consider an integer value represented on <code>N</code> bits. If you try to divide the smallest integer value <code>-2**N</code> by <code>-1</code>, it should return <code>2**N</code>, but this is out of range, since the biggest signed integer value we can represent on <code>N</code> bits is <code>(2**N) - 1</code>! So this will overflow and be set to <code>-2**N</code>, which is the initial value, but not the correct result. Good luck debugging this without a software trap! <a href="#fr-3-1">↩</a></p>
</li>
<li id="fn-5">
<p>Register moves may be introduced because a successor block (in the control flow graph) expects a given virtual register to live in a particular real register, or because a particular instruction requires a virtual register to be allocated to a <em>fixed</em> real register that’s busy: regalloc can then temporarily divert the busy register into another unused register. <a href="#fr-5-1">↩</a></p>
</li>
<li id="fn-8">
<p>The <code>c</code> in <code>rcx</code> actually stands for <code>count</code>; this is a property inherited from former CPU designs. <a href="#fr-8-1">↩</a></p>
</li>
<li id="fn-7">
<p>Unless this move carries sign- or zero-extending semantics, which is the case for e.g. x86’s 32-bits <code>mov</code> instructions on a 64-bits architecture. <a href="#fr-7-1">↩</a></p>
</li>
<li id="fn-6">
<p>Relocations are placeholders for information we don’t have <em>yet</em> access to. For instance, when we’re generating jump instructions, the jump targets offsets are not determined yet. So we record where the jump instruction is in the code stream, as well as which control flow block it should jump into, so we can <em>patch it</em> later when the final offsets are known: that’s the content of our relocation. <a href="#fr-6-1">↩</a></p>
</li>
</ol>
</section>
Making calls to WebAssembly fast and implementing anyref2018-07-04T18:00:42+00:002018-07-04T18:00:42+00:00
Benjamin Bouvier
https://bouvier.cc/tech/mozilla-2018-faster-calls-and-anyref/<p>Since this is the end of the first half-year, I think it is a good time to
reflect and show some work I’ve been doing over the last few months, apart from
the regular batch of random issues, security bugs, reviews and the fixing of 24
bugs found by our <a rel="noopener noreferrer external" target="_blank" href="https://en.wikipedia.org/wiki/Fuzzing">fuzzers</a>.</p>
<span id="continue-reading"></span><h2 id="bug-1319203-make-js-to-webassembly-calls-blazingly-fast"><a class="zola-anchor" href="#bug-1319203-make-js-to-webassembly-calls-blazingly-fast" aria-label="Anchor link for: bug-1319203-make-js-to-webassembly-calls-blazingly-fast">🔗</a>Bug 1319203: Make JS to WebAssembly calls <em>blazingly</em> fast</h2>
<p>If we want more WebAssembly (wasm) adoption, there shouldn’t be a big costly
barrier between the two universes. That is, calls from one world to the other
should be fast. For a very long time, calls from JS to asm.js/WebAssembly have
been quite slow in Firefox. In fact, we didn’t optimize them at all. For ease
and speed of implementation at the time, asm.js call activations (data
structures recording information about the function being currently called in
the VM) were very different from the JS ones. This difference indicated some
significant structural differences, like the capability to reconstruct call
stack information used by <code>Error()</code> stack frames, or just tracing the stack for
garbage collection purposes. After putting a lot of hard work into refactoring
and low-level changes over the last year, Spidermonkey was finally ripe for an
optimization.</p>
<p>When we call from JS to asm.js/wasm, the call passes through C++, does a bunch of
work and then calls into a piece of glue code directly written in assembly: the
<em>interpreter entry stub</em>. This stub is quite small: it just copies out the C++
arguments into the right places the wasm function being called expects, sets up
some small machine state, calls into the function, then does error checking and
eventually returns to the C++ caller. The critical part is JIT compilation. JIT
compilation means that the code is compiled to machine code by the just-in-time
compiler, IonMonkey. When a JS function has been JIT-compiled and it calls into
wasm, then the caller would have to go back to C++ first, before the control
flow is redirected to WebAssembly.</p>
<p><img src="https://bouvier.cc/tech/mozilla-2018-faster-calls-and-anyref/2018-07-interpreter-stub.png" alt="Diagram showing interpreter entry stub" /></p>
<p>Starting with Firefox 60, the JIT compiler makes no distinctions between
calling a JavaScript function or a WebAssembly function, meaning it uses the
same call optimizations for both kinds of function. A new piece of glue code,
the <em>JIT entry stub</em>, is generated for each exported function: it converts and
unboxes the arguments read from the JIT-compiled JS caller into the right
primitive types as expressed in the wasm function’s signature, sets up some
machine registers, calls into the wasm function being called and then converts
the result into a format the JS caller will understand.</p>
<p><img src="https://bouvier.cc/tech/mozilla-2018-faster-calls-and-anyref/2018-07-jit-stub.png" alt="Diagram showing JIT entry stub" /></p>
<p>As you can see, the C++ step that was originally required to call wasm from JS
has been completely eliminated!</p>
<p>This resulted in massive speedups over a variety of different situations: when
a wasm function is directly / indirectly / polymorphically called, or used as a
getter/setter, or called by <code>Function.prototype.call/apply</code>, when the call is
missing required arguments, etc. Here’s a brief summary of the results, but
there might be a full-blown blog post about these optimizations coming on
<a rel="noopener noreferrer external" target="_blank" href="https://hacks.mozilla.org/">Mozilla Hacks</a> at some point in the future.
(calling 1 billion times into very simple functions, lower is better)</p>
<p><img src="https://bouvier.cc/tech/mozilla-2018-faster-calls-and-anyref/2018-07-wasm-calls.png" alt="Charts showing evolution of performance" /></p>
<p>This work is not entirely done yet: we can still even better optimize in the
case of a function call from JS when the called wasm function is definitely
known to be a unique wasm target; see the <a rel="noopener noreferrer external" target="_blank" href="https://bugzilla.mozilla.org/show_bug.cgi?id=1437065">tracking
bug</a>.</p>
<h2 id="bug-1422043-lazy-entry-stub-generation"><a class="zola-anchor" href="#bug-1422043-lazy-entry-stub-generation" aria-label="Anchor link for: bug-1422043-lazy-entry-stub-generation">🔗</a>Bug 1422043: Lazy entry stub generation</h2>
<p>The previous bug resolution came with an important memory issue: every exported
function now generates a rather big chunk of code for the JIT entry, having an
impact on the memory occupied by the code itself. This would be fine in most
situations where the number of exported functions is generally low. But when
the wasm module exports a Table (think of the equivalent of a C++ function
table with signature checks), we have to assume that every single function,
including those not explicitly exported, needs entry stubs. Indeed, each
function can be eventually called through the Table, after calls to
<a rel="noopener noreferrer external" target="_blank" href="http://webassembly.github.io/spec/js-api/index.html#dom-table-set">WebAssembly.Table.set</a>.
In fact, the existing code already suffered from this because of the
interpreter entries, but it had been largely amplified by the much larger JIT
entry stubs.</p>
<p>To fix this, we’ve decided to lazily generate all the entry stubs for functions
exported through a table. That is, if a function is <em>explicitly</em> exported, its
stubs will be generated at wasm compile time, but other functions won’t have
stubs yet. If a non-exported function is called through a Table, we’ll generate
the entry stubs the first time it is called. This involves some fun
interactions with our <a rel="noopener noreferrer external" target="_blank" href="https://hacks.mozilla.org/2018/01/making-webassembly-even-faster-firefoxs-new-streaming-and-tiering-compiler/">tiered
compilation</a>
mechanism, which can compile functions and create new entry stubs in the
background while the running thread will generate lazy ones.</p>
<p>Not only this fixed the memory regression introduced by bug 1319203, but it
actually made the situation even better than the baseline, because we didn’t
need to generate those interpreter entries for table-exported functions by
default anymore:</p>
<p><img src="https://bouvier.cc/tech/mozilla-2018-faster-calls-and-anyref/2018-07-wasm-stubs-memory.png" alt="Charts showing evolution of memory usage" /></p>
<p>Since it’s not entirely readable from the chart: after the patches, the
AngryBots and ZenGarden entry stubs memory usages went down to respectively 262
and 362 KB. This was also a relatively huge win in compilation times, but on
such a low scale that it didn’t make a huge difference on total compile time.</p>
<h2 id="bug-1447591-remove-wasm-binarytotext"><a class="zola-anchor" href="#bug-1447591-remove-wasm-binarytotext" aria-label="Anchor link for: bug-1447591-remove-wasm-binarytotext">🔗</a>Bug 1447591: Remove wasm::BinaryToText</h2>
<p>WebAssembly is a binary format, and there is an equivalent human-readable and
debuggable text format: the WebAssembly Text format, or <em>WAT</em> format. While
SpiderMonkey once directly produced WAT for display in C++, it’s now easier for
<a rel="noopener noreferrer external" target="_blank" href="https://github.com/devtools-html/debugger.html">debugger.html</a> to do so in JS.
This also made the mapping between bytecode offsets and text offsets (source
maps) more consistent with the display, and it could be useful in other places
where this project is being used. Recently after confirming that the C++
implementation wasn’t used anymore, I was able to remove it. It’s not every day
that you get a net loss of around 5,500 lines of code, which is always nice:
less code means fewer bugs and less maintenance burden, especially when the code
is dead.</p>
<h2 id="bug-1445272-1450261-implement-basic-anyref-support"><a class="zola-anchor" href="#bug-1445272-1450261-implement-basic-anyref-support" aria-label="Anchor link for: bug-1445272-1450261-implement-basic-anyref-support">🔗</a>Bug 1445272 / 1450261: Implement basic <code>anyref</code> support</h2>
<p>A new proposal has been made to the WebAssembly specification committee a few
months ago: to add <a rel="noopener noreferrer external" target="_blank" href="https://github.com/WebAssembly/reference-types">reference
types</a> to the type system.
Reference types are a new way to represent a reference to any <em>host</em> values. In
a Web environment, this means being capable of playing with JavaScript values
within WebAssembly. This is a huge difference with the existing type system,
which only contains primitive types: integers represented on 32 or 64 bits,
IEEE754 floating-point numbers represented on 32 or 64 bits. This is also a
first step for implementing <a rel="noopener noreferrer external" target="_blank" href="http://github.com/webassembly/gc">garbage
collection</a> (GC) integration within
WebAssembly: since these reference values have been allocated on the GC heap in
JavaScript, they need to be traced during wasm execution.</p>
<p>The basic implementation of this feature in the first bug allows one to use a
new type, called <code>anyref</code>, as part of a function’s signature or in local
variables, be it in a function definition or an imported function. This allows
using JS variables within wasm and pass them around to other JS functions. The
second bug implemented the capability to read and write <code>anyref</code> values in wasm
Globals [1]. Since Globals can be manipulated outside of the wasm Module thanks
to their JS API, and garbage collections can happen at any time in JS, we
needed to implement GC barriers to make sure that the stored value would not be
marked as unused during tracing. There is good literature explaining why these
barriers are needed and what they do, so I will not expand too much on the
topic.</p>
<p>Here’s an example of usage according to latest spec drafts (and therefore
subject to change for now):</p>
<pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="common-lisp"><span class="giallo-l"><span>(module</span></span>
<span class="giallo-l"><span> (func $alert (</span><span style="color: #0DB9D7;">import</span><span style="color: #89DDFF;"> "</span><span style="color: #9ECE6A;">env</span><span style="color: #89DDFF;">" "</span><span style="color: #9ECE6A;">alert</span><span style="color: #89DDFF;">"</span><span>) (param anyref))</span></span>
<span class="giallo-l"><span> (global $global_ref (mut anyref) (ref.null anyref))</span></span>
<span class="giallo-l"><span> (func (</span><span style="color: #0DB9D7;">export</span><span style="color: #89DDFF;"> "</span><span style="color: #9ECE6A;">set_and_alert</span><span style="color: #89DDFF;">"</span><span>) (param $param anyref) (result anyref)</span></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;"> ;; Put the previous value of $global_ref on the virtual value stack.</span></span>
<span class="giallo-l"><span> get_global $global_ref</span></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;"> ;; Get the argument anyref value and store it in $global_ref.</span></span>
<span class="giallo-l"><span> get_local $param</span></span>
<span class="giallo-l"><span> set_global $global_ref</span></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;"> ;; Call the $alert method with the argument anyref value.</span></span>
<span class="giallo-l"><span> get_local $param</span></span>
<span class="giallo-l"><span> call $alert</span></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;"> ;; The previous value of $global_ref is still on the stack and will be</span></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;"> ;; returned.</span></span>
<span class="giallo-l"><span> )</span></span>
<span class="giallo-l"><span>)</span></span></code></pre>
<p><em>Example of wasm text format using <code>anyref</code>.</em></p>
<pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="javascript"><span class="giallo-l"><span style="color: #9ABDF5;">(</span><span style="color: #9D7CD8;font-style: italic;">async</span><span style="color: #BB9AF7;"> function</span><span style="color: #9ABDF5;">() {</span></span>
<span class="giallo-l"><span style="color: #9D7CD8;font-style: italic;"> let</span><span style="color: #89DDFF;"> {</span><span style="color: #BB9AF7;"> instance</span><span style="color: #89DDFF;"> } =</span><span style="color: #BB9AF7;font-style: italic;"> await</span><span style="color: #C0CAF5;"> WebAssembly</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">instantiate</span><span style="color: #9ABDF5;">(</span><span style="color: #C0CAF5;">wasmBinary</span><span style="color: #89DDFF;">,</span><span style="color: #9ABDF5;"> {</span></span>
<span class="giallo-l"><span style="color: #73DACA;"> env</span><span style="color: #89DDFF;">:</span><span style="color: #9ABDF5;"> {</span></span>
<span class="giallo-l"><span style="color: #7AA2F7;"> alert</span><span style="color: #9ABDF5;">(</span><span style="color: #E0AF68;">obj</span><span style="color: #9ABDF5;">) {</span></span>
<span class="giallo-l"><span style="color: #7AA2F7;"> alert</span><span style="color: #9ABDF5;">(</span><span style="color: #89DDFF;">`</span><span style="color: #9ECE6A;">Hello, </span><span style="color: #7DCFFF;">${</span><span style="color: #C0CAF5;">obj</span><span style="color: #89DDFF;">.</span><span style="color: #7DCFFF;">name}</span><span style="color: #9ECE6A;">!</span><span style="color: #89DDFF;">`</span><span style="color: #9ABDF5;">)</span><span style="color: #89DDFF;">;</span></span>
<span class="giallo-l"><span style="color: #9ABDF5;"> }</span></span>
<span class="giallo-l"><span style="color: #9ABDF5;"> }</span></span>
<span class="giallo-l"><span style="color: #9ABDF5;"> })</span><span style="color: #89DDFF;">;</span></span>
<span class="giallo-l"><span style="color: #C0CAF5;"> console</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">log</span><span style="color: #9ABDF5;">(</span><span style="color: #C0CAF5;">instance</span><span style="color: #89DDFF;">.</span><span style="color: #7DCFFF;">exports</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">set_and_alert</span><span style="color: #9ABDF5;">({</span></span>
<span class="giallo-l"><span style="color: #73DACA;"> name</span><span style="color: #89DDFF;">: '</span><span style="color: #9ECE6A;">world</span><span style="color: #89DDFF;">',</span></span>
<span class="giallo-l"><span style="color: #73DACA;"> secretVal</span><span style="color: #89DDFF;">:</span><span style="color: #FF9E64;"> 42</span></span>
<span class="giallo-l"><span style="color: #9ABDF5;"> }))</span><span style="color: #89DDFF;">;</span></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // alerts "Hello, world!", logs null</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span style="color: #C0CAF5;"> console</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">log</span><span style="color: #9ABDF5;">(</span><span style="color: #0DB9D7;">JSON</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">stringify</span><span style="color: #9ABDF5;">(</span><span style="color: #C0CAF5;">instance</span><span style="color: #89DDFF;">.</span><span style="color: #7DCFFF;">exports</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">set_and_alert</span><span style="color: #9ABDF5;">({</span></span>
<span class="giallo-l"><span style="color: #73DACA;"> name</span><span style="color: #89DDFF;">: '</span><span style="color: #9ECE6A;">there</span><span style="color: #89DDFF;">'</span></span>
<span class="giallo-l"><span style="color: #9ABDF5;"> })))</span><span style="color: #89DDFF;">;</span></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // alerts "Hello, there!", logs { name: 'world!', secretVal: 42 }</span></span>
<span class="giallo-l"><span style="color: #9ABDF5;">})()</span><span style="color: #89DDFF;">;</span></span></code></pre>
<p><em>Example of JavaScript using the module defined above, passing JS values and
reading them from WebAssembly.</em></p>
<p>This is a very preliminary prototype and it might change in the next few
months. If you feel adventurous, you can try it on Firefox Nightly by setting
the <code>about:config</code> pref <code>javascript.options.wasm_gc</code> to <code>true</code>; note that we
haven’t fully hooked this up to garbage collection yet, so your experimentation
might occasionally throw out-of-memory exceptions. In any case, if you see
something, <a rel="noopener noreferrer external" target="_blank" href="https://bugzilla.mozilla.org/enter_bug.cgi?product=Core&component=Javascript%3A%20Web%20Assembly">say
something</a>.</p>
<p>Say you are a compiler developer, and you would like to port your language to
WebAssembly, and your language uses a GC. At the moment, the only way you can
do this is by compiling your garbage collector to WebAssembly, and it would be
backed by the wasm Module’s memory. This works, but it won’t be very efficient.
Plus, there’s already a very efficient, solidly tested, constantly improving
garbage collector in your browser that uses all the possible dirty low-level
tricks known to mankind, which is the GC being used for JavaScript. What if we
could give you access to the garbage collector directly? Then you’d just need
to give a way to define structures, and then could use a set of opcodes to
allocate them, read and write fields on them, etc. At the moment, the reference
types proposal only allows you to move garbage-collected values around. There’s
also code in Firefox Nightly to experiment with defining your own data
structures and using them, but it is very very early. If you’re interested in
following us implementing more parts, this <a rel="noopener noreferrer external" target="_blank" href="https://bugzilla.mozilla.org/show_bug.cgi?id=1444925">tracking
issue</a> might be of
interest.</p>
<p>[1] Think of a C++ “global” value, not a JavaScript “global”.</p>
<h2 id="future-work"><a class="zola-anchor" href="#future-work" aria-label="Anchor link for: future-work">🔗</a>Future work</h2>
<p>There is still much more work to be done on the implementation of WebAssembly
in Spidermonkey, to implement other new proposals, to make it faster, or to
have even better generated code.</p>
<p>A big thank you for the proofreading to <a rel="noopener noreferrer external" target="_blank" href="http://whereswalden.com/">Waldo</a>,
<a rel="noopener noreferrer external" target="_blank" href="https://github.com/steveklabnik">steveklabnik</a> and
<a rel="noopener noreferrer external" target="_blank" href="https://twitter.com/ag_dubs">ashleygwilliams</a>. Extra thanks go to
Ashley who also drew the two diagrams showing how stubs evolved.</p>
Making asm.js/WebAssembly compilation more parallel in Firefox2016-04-22T15:00:42+00:002016-04-22T15:00:42+00:00
Benjamin Bouvier
https://bouvier.cc/tech/making-asmjs-webassembly-compilation-more-parallel/<p>In December 2015, I’ve worked on reducing startup time of asm.js programs in
Firefox by making compilation more parallel. As our
JavaScript engine, Spidermonkey, uses the same compilation pipeline for both
asm.js and WebAssembly, this also benefitted WebAssembly compilation. Now is a
good time to talk about what it meant, how it got achieved and what are the
next ideas to make it even faster.</p>
<span id="continue-reading"></span><h1 id="what-does-it-mean-to-make-a-program-more-parallel"><a class="zola-anchor" href="#what-does-it-mean-to-make-a-program-more-parallel" aria-label="Anchor link for: what-does-it-mean-to-make-a-program-more-parallel">🔗</a>What does it mean to make a program “more parallel”?</h1>
<p>Parallelization consists of splitting a sequential program into smaller
independent tasks, then having them run on different CPU. If your program
is using <code>N</code> cores, it can be up to <code>N</code> times faster.</p>
<p>Well, in theory. Let’s say you’re in a car, driving on a 100 Km long road.
You’ve already driven the first 50 Km in one hour. Let’s say your car can
have unlimited speed from now on. What is the maximal average speed you can
reach, once you get to the end of the road?</p>
<p>People intuitively answer “If it can go as fast as I want, so nearby lightspeed
sounds plausible”. But this is not true! In fact, if you could teleport from
your current position to the end of the road, you’d have traveled 100 Km in one
hour, so your maximal theoritical speed is 100 Km per hour. This result is a
consequence of <a rel="noopener noreferrer external" target="_blank" href="https://en.wikipedia.org/wiki/Amdahl%27s_law">Amdahl’s law</a>.
When we get back to our initial problem, this means you can expect a <code>N</code> times
speedup if you’re running your program with <code>N</code> cores if, and only if your
program can be <strong>entirely</strong> run in parallel. This is usually not the case, and
that is why most wording refers to <em>speedups <strong>up to</strong> N times faster</em>, when it
comes to parallelization.</p>
<p>Now, say your program is already running some portions in parallel. To make it
faster, one can identify some parts of the program that are sequential, and make
them independent so that you can run them in parallel. With respect to our car
metaphor, this means augmenting the portion of the road on which you can run at
unlimited speed.</p>
<p>This is exactly what we have done with parallel compilation of asm.js programs
under Firefox.</p>
<h1 id="a-quick-look-at-the-asm-js-compilation-pipeline"><a class="zola-anchor" href="#a-quick-look-at-the-asm-js-compilation-pipeline" aria-label="Anchor link for: a-quick-look-at-the-asm-js-compilation-pipeline">🔗</a>A quick look at the asm.js compilation pipeline</h1>
<p>I recommend to read this <a rel="noopener noreferrer external" target="_blank" href="https://blog.mozilla.org/luke/2014/01/14/asm-js-aot-compilation-and-startup-performance/">blog
post</a>.
It clearly explains the differences between JIT (Just In Time) and AOT (Ahead
Of Time) compilation, and elaborates on the different parts of the engines
involved in the compilation pipeline.</p>
<p>As a TL;DR, keep in mind that <a rel="noopener noreferrer external" target="_blank" href="http://asmjs.org/">asm.js</a> is a strictly
validated, highly optimizable, typed subset of JavaScript. Once
validated, it guarantees high performance and stability (no garbage collector
involved!). That is ensured by
mapping every single JavaScript instruction of this subset to a few CPU
instructions, if not only a single instruction. This means an asm.js program
needs to get <em>compiled</em> to machine code, that is, translated from JavaScript to
the language your CPU directly manipulates (like what GCC would do for a C++
program). If you haven’t heard, the results are impressive and you can run
<a href="https://bouvier.cc/tech/making-asmjs-webassembly-compilation-more-parallel/beta.unity3d.com/jonas/DT2/">video</a>
<a rel="noopener noreferrer external" target="_blank" href="https://www.unrealengine.com/html5">games</a> directly in your browser, without
needing to install anything. No plugins. Nothing more than your usual, everyday
browser.</p>
<p>Because asm.js programs can be gigantic in size (in number of functions as well
as in number of lines of code), the first compilation of the entire program is
going to take some time. Afterwards, Firefox uses a caching mechanism that
prevents the need for recompilation and almost instaneously loads the code, so
subsequent loadings matter less*****. The end user will mostly wait for the
first compilation, thus this one needs to be fast.</p>
<p>Before the work explained below, the pipeline for compiling a single function
(out of an asm.js module) would look like this:</p>
<ul>
<li>parse the function, and as we parse, emit intermediate representation (IR)
nodes for the compiler infrastructure. SpiderMonkey has several IRs,
including the MIR (middle-level IR, mostly loaded with semantic) and the LIR
(low-level IR closer to the CPU memory representation: registers, stack,
etc.). The one generated here is the MIR. All of this happens on the main
thread.</li>
<li>once the entire IR graph is generated for the function, optimize the MIR
graph (i.e. apply a few optimization passes). Then, generate the LIR graph
before carrying out register allocation (probably the most costly task of the
pipeline). This can be done on supplementary helper threads, as the MIR
optimization and LIR generation for a given function doesn’t depend on other
ones.</li>
<li>since functions can call between themselves within an asm.js module, they
need references to each other. In assembly, a reference is merely an offset
to somewhere else in memory. In this initial implementation, code generation
is carried out on the main thread, at the cost of speed but for the sake of
simplicity.</li>
</ul>
<p>So far, only the MIR optimization passes, register allocation and LIR
generation were done in parallel. Wouldn’t it be nice to be able to do more?</p>
<p><strong>*</strong> There are conditions for benefitting from the caching mechanism. In
particular, the script should be loaded
<a rel="noopener noreferrer external" target="_blank" href="https://developer.mozilla.org/en-US/docs/Games/Techniques/Async_scripts">asynchronously</a>
and it should be of a consequent size.</p>
<h1 id="doing-more-in-parallel"><a class="zola-anchor" href="#doing-more-in-parallel" aria-label="Anchor link for: doing-more-in-parallel">🔗</a>Doing more in parallel</h1>
<p>Our goal is to make more work in parallel: so can we take out MIR generation
from the main thread? And we can take out code generation as well?</p>
<p>The answer happens to be <em>yes</em> to both questions.</p>
<p>For the former, instead of emitting a MIR graph as we parse the function’s
body, we emit a small, compact, pre-order representation of the function’s
body. In short, a new IR. As work was starting on
<a rel="noopener noreferrer external" target="_blank" href="https://github.com/webassembly/design">WebAssembly</a> (wasm) at this time, and
since asm.js semantics and wasm semantics mostly match, the IR could just be
the wasm
<a rel="noopener noreferrer external" target="_blank" href="https://github.com/WebAssembly/design/blob/master/BinaryEncoding.md">encoding</a>,
consisting of the wasm opcodes plus a few specific asm.js ones*. Then, wasm
is translated to MIR in another thread.</p>
<p>Now, instead of parsing and generating MIR in a single pass, we would now parse
and generate wasm IR in one pass, and generate the MIR out of the wasm IR in
another pass. The wasm IR is very compact and much cheaper to generate than a
full MIR graph, because generating a MIR graph needs some algorithmic work,
including the creation of Phi nodes (join values after any form of branching).
As a result, it is expected that compilation time won’t suffer. This was a
large refactoring: taking every single asm.js instructions, and encoding them
in a compact way and later decode these into the equivalent MIR nodes.</p>
<p>For the second part, could we generate code on other threads? One structure in
the code base, the <em>MacroAssembler</em>, is used to generate all the code and it
contains all necessary metadata about offsets. By adding more metadata there to
abstract internal calls <strong>**</strong>, we can describe the new scheme in terms of a
classic functional <code>map</code>/<code>reduce</code>:</p>
<ul>
<li>the wasm IR is sent to a thread, which will return a MacroAssembler. That
is a <code>map</code> operation, transforming an array of wasm IR into an array of
MacroAssemblers.</li>
<li>When a thread is done compiling, we merge its MacroAssembler into one big
MacroAssembler. Most of the merge consists in taking all the offset metadata
in the thread MacroAssembler, fixing up all the offsets, and concatenate the
two generated code buffers. This is equivalent to a <code>reduce</code> operation,
merging each MacroAssembler within the module’s one.</li>
</ul>
<p>At the end of the compilation of the entire module, there is still some light
work to be done: offsets of internal calls need to be translated to their
actual locations. All this work has been done in <a rel="noopener noreferrer external" target="_blank" href="https://bugzilla.mozilla.org/show_bug.cgi?id=1181612">this bugzilla
bug</a>.</p>
<p><strong>*</strong> In fact, at the time when this was being done, we used a different
superset of wasm. Since then, work has been done so that our asm.js frontend is
really just another wasm emitter.</p>
<p>**** ** referencing functions by their appearance order index in the module,
rather than an offset to the actual start of the function. This order is indeed
stable, from a function to the other.</p>
<h1 id="results"><a class="zola-anchor" href="#results" aria-label="Anchor link for: results">🔗</a>Results</h1>
<p>Benchmarking has been done on a Linux x64 machine with 8 cores clocked at 4.2
Ghz.</p>
<p>First, compilation times of a few asm.js massive games:</p>
<ul>
<li><a rel="noopener noreferrer external" target="_blank" href="http://beta.unity3d.com/jonas/DT2/">DeadTrigger2</a></li>
<li><a rel="noopener noreferrer external" target="_blank" href="http://beta.unity3d.com/jonas/AngryBots/">AngryBots</a></li>
<li><a rel="noopener noreferrer external" target="_blank" href="https://github.com/lukewagner/PlatformerGamePacked">Platformer game</a></li>
<li><a rel="noopener noreferrer external" target="_blank" href="https://www.unrealengine.com/html5">Tappy Chicken</a></li>
</ul>
<p>The X scale is the compilation time in seconds, so lower is better. Each value
point is the best one of three runs. For the new scheme, the corresponding
relative speedup (in percentage) has been added:</p>
<p><img src="https://bouvier.cc/tech/making-asmjs-webassembly-compilation-more-parallel/2016-04-22_parallelization-times.png" alt="Compilation times of various benchmarks" /></p>
<p>For all games, compilation is much faster with the new parallelization scheme.</p>
<p>Now, let’s go a bit deeper. The Linux CLI tool <code>perf</code> has a <code>stat</code> command
that gives you an average of the number of utilized CPUs during the program
execution. This is a great measure of threading efficiency: the more a CPU is
utilized, the more it is not idle, waiting for other results to come, and thus
useful. For a constant task execution time, the more utilized CPUs, the more
likely the program will execute quickly.</p>
<p>The X scale is the number of utilized CPUs, according to the <code>perf stat</code>
command, so higher is better. Again, each value point is the best one of three
runs.</p>
<p><img src="https://bouvier.cc/tech/making-asmjs-webassembly-compilation-more-parallel/2016-04-22_parallelization-cpu-utilized.png" alt="CPU utilized on DeadTrigger2" /></p>
<p>With the older scheme, the number of utilized CPUs quickly rises up from 1 to 4
cores, then more slowly from 5 cores and beyond. Intuitively, this means that
with 8 cores, we almost reached the theoritical limit of the portion of the
program that can be made parallel (not considering the overhead introduced by
parallelization or altering the scheme).</p>
<p>But with the newer scheme, we get much more CPU usage even after 6 cores! Then
it slows down a bit, although it is still more significant than the slow rise
of the older scheme. So it is likely that with even more threads, we could have
even better speedups than the one mentioned beforehand. In fact, we have moved
the theoritical limit mentioned above a bit further: we have expanded the
portion of the program that can be made parallel. Or to keep on using the
initial car/road metaphor, we’ve shortened the constant speed portion of the
road to the benefit of the unlimited speed portion of the road, resulting in a
shorter trip overall.</p>
<h1 id="future-steps"><a class="zola-anchor" href="#future-steps" aria-label="Anchor link for: future-steps">🔗</a>Future steps</h1>
<p>Despite these improvements, compilation time can still be a pain, especially on
mobile. This is mostly due to the fact that we’re running a whole multi-million
line codebase through the backend of a compiler to generate optimized code.
Following this work, the next bottleneck during the compilation process is
parsing, which matters for asm.js in particular, which source is plain text.
Decoding WebAssembly is an order of magnitude faster though, and it can be made
even faster. Moreover, we have even more load-time optimizations coming down
the pipeline!</p>
<p>In the meanwhile, we keep on improving the WebAssembly backend. Keep track of
our progress on <a rel="noopener noreferrer external" target="_blank" href="https://bugzilla.mozilla.org/show_bug.cgi?id=1188259">bug
1188259</a>!</p>
Previous writings about Mozilla work2016-03-09T18:00:42+00:002016-03-09T18:00:42+00:00
Benjamin Bouvier
https://bouvier.cc/tech/previous-writing-about-mozilla-work/<p>I am currently a compiler engineer at Mozilla corporation, the company making
the Firefox browser among else. Our JavaScript virtual machine, Spidermonkey,
is split in several tiers, including an highly optimizing Just-In-Time (JIT)
compiler able to compile JavaScript to assembly at runtime. My previous work
has involved efficiently compiling Float32 arithmetic to hardware instructions
and implement a new SIMD API for the Web.</p>
<h2 id="about-float32-optimizations"><a class="zola-anchor" href="#about-float32-optimizations" aria-label="Anchor link for: about-float32-optimizations">🔗</a>About Float32 optimizations</h2>
<p>The full blog post is
<a rel="noopener noreferrer external" target="_blank" href="https://blog.mozilla.org/javascript/2013/11/07/efficient-float32-arithmetic-in-javascript/">there</a>.
It has been written in November 2013.</p>
<p>The main idea is that if you have float32 inputs to an operation; and you cast
them to doubles; and you apply an arithmetic operation to these inputs; and you
cast the result back to a float32, then you’d have the same result as if you
did the entire computation with float32 values and operations.</p>
<p>So we’ve introduced an operation in JavaScript that converts a Number to its
closest float32 IEEE754 representation: <code>Math.fround</code>. Said differently, the
above equivalence says that:</p>
<pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="javascript"><span class="giallo-l"><span style="color: #BB9AF7;">function</span><span style="color: #7AA2F7;"> f</span><span style="color: #9ABDF5;">(</span><span style="color: #E0AF68;">x</span><span style="color: #89DDFF;">,</span><span style="color: #E0AF68;"> y</span><span style="color: #9ABDF5;">) {</span></span>
<span class="giallo-l"><span style="color: #BB9AF7;font-style: italic;"> return</span><span style="color: #C0CAF5;"> x</span><span style="color: #89DDFF;"> +</span><span style="color: #C0CAF5;"> y</span><span style="color: #89DDFF;">;</span></span>
<span class="giallo-l"><span style="color: #9ABDF5;">}</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span style="color: #BB9AF7;">function</span><span style="color: #7AA2F7;"> g</span><span style="color: #9ABDF5;">(</span><span style="color: #E0AF68;">x</span><span style="color: #89DDFF;">,</span><span style="color: #E0AF68;"> y</span><span style="color: #9ABDF5;">) {</span></span>
<span class="giallo-l"><span style="color: #9D7CD8;font-style: italic;"> var</span><span style="color: #BB9AF7;"> xf</span><span style="color: #89DDFF;"> =</span><span style="color: #C0CAF5;"> Math</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">fround</span><span style="color: #9ABDF5;">(</span><span style="color: #C0CAF5;">x</span><span style="color: #9ABDF5;">)</span><span style="color: #89DDFF;">;</span></span>
<span class="giallo-l"><span style="color: #9D7CD8;font-style: italic;"> var</span><span style="color: #BB9AF7;"> yf</span><span style="color: #89DDFF;"> =</span><span style="color: #C0CAF5;"> Math</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">fround</span><span style="color: #9ABDF5;">(</span><span style="color: #C0CAF5;">y</span><span style="color: #9ABDF5;">)</span><span style="color: #89DDFF;">;</span></span>
<span class="giallo-l"><span style="color: #BB9AF7;font-style: italic;"> return</span><span style="color: #C0CAF5;"> Math</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">fround</span><span style="color: #9ABDF5;">(</span><span style="color: #C0CAF5;">xf</span><span style="color: #89DDFF;"> +</span><span style="color: #C0CAF5;"> yf</span><span style="color: #9ABDF5;">)</span><span style="color: #89DDFF;">;</span></span>
<span class="giallo-l"><span style="color: #9ABDF5;">}</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span style="color: #51597D;font-style: italic;">// For all x, y that can be represented exactly as float32:</span></span>
<span class="giallo-l"><span style="color: #7AA2F7;">assert</span><span style="color: #9ABDF5;">(</span><span style="color: #7AA2F7;">f</span><span style="color: #9ABDF5;">(</span><span style="color: #C0CAF5;">x</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> y</span><span style="color: #9ABDF5;">)</span><span style="color: #BB9AF7;"> ===</span><span style="color: #7AA2F7;"> g</span><span style="color: #9ABDF5;">(</span><span style="color: #C0CAF5;">x</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> y</span><span style="color: #9ABDF5;">))</span><span style="color: #89DDFF;">;</span></span></code></pre>
<p>Yes, <code>===</code>. The same <code>===</code> you’ve been told <strong>not</strong> to use for floating-point
Numbers. But here, we have <em>bitwise</em> equality, so we can use strict equality*.</p>
<p>Processors have special instructions for carrying out float32 arithmetic, which
have higher throughput than the equivalent double ones. With this result in
mind, we could add a pass that would spot opportunities where the computations
are equivalent (thanks to <code>Math.fround</code> hints) and emit float32 instructions
instead of double instructions. This sped up a some numerical applications and
games engines by a few points.</p>
<p>* a careful reader would object that this is wrong for <code>x = y = NaN</code>, which
I’ve put away for the sake of simplicity.</p>
<h2 id="about-simd-js"><a class="zola-anchor" href="#about-simd-js" aria-label="Anchor link for: about-simd-js">🔗</a>About SIMD.js</h2>
<p>The full blog post is
<a rel="noopener noreferrer external" target="_blank" href="https://blog.mozilla.org/javascript/2015/03/10/state-of-simd-js-performance-in-firefox/">there</a>.
It has been written in March 2015.</p>
<p>Nowadays, processors have instructions sets that allow them to execute several
simple arithmetic operations at once. For instance, let’s say you have two
arrays of integers and you want to add each element to the corresponding one in
the other array. If both arrays have size <code>N</code>, this means you’ll have to carry
out <code>N</code> scalar additions. But processors can actually group these into bundles
of several additions, with SIMD; for the case of 32-bits wide integers, on most
modern processors, you need at most <code>Math.ceil(N / 4)</code> instructions. The blog
post details what SIMD.js is and what bottlenecks we hit during implementation.</p>
<h2 id="conclusion"><a class="zola-anchor" href="#conclusion" aria-label="Anchor link for: conclusion">🔗</a>Conclusion</h2>
<p>This was a small reminder about previously written blog posts. If you’re into
JavaScript, compilers or low-level optimization, I can only recommend you to go
read the <a rel="noopener noreferrer external" target="_blank" href="https://blog.mozilla.org/javascript/">Mozilla’s JavaScript blog</a>.</p>