Technical blog and random musings. Zola 2021-02-17T19:00:42+00:00 https://bouvier.cc/tags/mozilla/atom.xml 2021-02-17T19:00:42+00:00 2021-02-17T19:00:42+00:00 Benjamin Bouvier https://bouvier.cc/tech/cranelift-codegen-primer/ <script src="https://cdn.jsdelivr.net/npm/mermaid/dist/mermaid.min.js"></script> <script>mermaid.initialize({startOnLoad:true});</script> <p><a rel="noopener noreferrer external" target="_blank" href="https://github.com/bytecodealliance/wasmtime/tree/main/cranelift#cranelift-code-generator">Cranelift</a> is a code generator written in the Rust programming language that aims to be a fast code generator, which outputs machine code that runs at reasonable speeds.</p> <span id="continue-reading"></span> <p>The Cranelift compilation model consists in compiling functions one by one, holding extra information about external entities, like external functions, memory addresses, and so on. This model allows for concurrent and parallel compilation of individual functions, which supports the goal of fast compilation. It was designed this way to allow for just-in-time (JIT) compilation of WebAssembly binary code in Firefox, although its scope has broadened a bit. Nowadays it is used in a few different WebAssembly runtimes, including <a rel="noopener noreferrer external" target="_blank" href="https://github.com/bytecodealliance/wasmtime#wasmtime">Wasmtime</a> and <a rel="noopener noreferrer external" target="_blank" href="https://wasmer.io/">Wasmer</a>, but also as an alternative backend for Rust debug compilation, thanks to <a rel="noopener noreferrer external" target="_blank" href="https://github.com/bjorn3/rustc_codegen_cranelift">cg_clif</a>.</p> <p>A classic compiler design usually includes running a parser to translate the source to some form of intermediate representations, then run optimization passes onto them, then feeds this to the machine code generator.</p> <p>This blog post focuses on the final step, namely the concepts that are involved in code generation, and what they map to in Cranelift. To make things more concrete, we’ll take a specific instruction, and see how it’s translated, from its creation down to code generation. At each step of the process, I’ll provide a short (<em>ahem</em>) high-level explanation of the concepts involved, and I’ll show what they map to in Cranelift, using the example instruction. While this is not a tutorial detailing how to add new instructions in Cranelift, this should be an interesting read for anyone who’s interested in compilers, and this could be an entry point if you’re interested in hacking on the Cranelift <code>codegen</code> crate.</p> <p>This is our plan for this blog post: each squared box represents data, each rounded box is a process. We’re going to go through each of them below.</p> <div class="mermaid"> graph TD; clif[Optimized CLIF]; vcode[VCode]; final_vcode[Final VCode]; machine_code[Machine code artifacts]; lowering([Lowering]); regalloc([Register allocation]); codegen([Machine code generation]); clif --> lowering --> vcode --> regalloc --> final_vcode --> codegen --> machine_code </div> <h2 id="intermediate-representations"><a class="zola-anchor" href="#intermediate-representations" aria-label="Anchor link for: intermediate-representations">🔗</a>Intermediate representations</h2> <p>Compilers use <strong>intermediate representations</strong> (<em>IR</em>) to represent source code. Here we’re interested in representations of the <em>data flow</em>, that is instructions themselves and only that. The IRs contain information about the instructions themselves, their operands, type specialization information, and any additional metadata that might be useful. IRs usually map to a certain level of abstraction, and as such, they are useful for solving different problems that require different levels of abstraction. Their shape (which data structures) and numbers often have a huge impact on the performance of the compiler itself (that is, how fast it is at compiling).</p> <p>In general, most programming languages use IRs internally, and yet, these are invisible to the programmers. The reason is that source code is usually first <em>parsed</em> (tokenized, verified) and then translated into an IR. The <em>abstract syntax tree</em>, aka AST, is one such IR representing the source code itself, in a format that’s very close to the source code itself. Since the raison d’être of Cranelift is to be a code generator, having a text format is secondary, and only useful for testing and debugging purposes. That’s why embedders directly create and manipulate Cranelift’s IR.</p> <p>At the time of writing, Cranelift has two IRs to represent the function’s code:</p> <ul> <li>one external, high-level intermediate representation, called <strong>CLIF</strong> (for <em>Cranelift IR format</em>),</li> <li>one internal, low-level intermediate representation called <strong>VCode</strong> (for <em>virtual-registerized code</em>).</li> </ul> <h2 id="clif-ir"><a class="zola-anchor" href="#clif-ir" aria-label="Anchor link for: clif-ir">🔗</a>CLIF IR</h2> <p>CLIF is the IR that Cranelift embedders create and manipulate. It consists of high-level typed operations that are convenient to use and/or can be simply translated to machine code. It is in <a rel="noopener noreferrer external" target="_blank" href="https://en.wikipedia.org/wiki/Static_single_assignment_form">static single assignment (SSA) form</a>: each value referenced by an operation (SSA value) is defined only once, and may have as many uses as desired. CLIF is practical to use and manipulate for classic compilers optimization passes (e.g. <a rel="noopener noreferrer external" target="_blank" href="https://en.wikipedia.org/wiki/Loop-invariant_code_motion">LICM</a>), as it is generic over the target architecture which we’re compiling to.</p> <pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="rust"><span class="giallo-l"><span style="color: #BB9AF7;">let</span><span style="color: #C0CAF5;"> x</span><span style="color: #89DDFF;"> =</span><span style="color: #C0CAF5;"> builder</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">ins</span><span style="color: #89DDFF;">().</span><span style="color: #7AA2F7;">iconst</span><span style="color: #89DDFF;">(</span><span style="color: #0DB9D7;">types</span><span style="color: #89DDFF;">::</span><span style="color: #C0CAF5;">I64</span><span style="color: #89DDFF;">,</span><span style="color: #FF9E64;"> 42</span><span style="color: #89DDFF;">);</span></span> <span class="giallo-l"><span style="color: #BB9AF7;">let</span><span style="color: #C0CAF5;"> y</span><span style="color: #89DDFF;"> =</span><span style="color: #C0CAF5;"> builder</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">ins</span><span style="color: #89DDFF;">().</span><span style="color: #7AA2F7;">iconst</span><span style="color: #89DDFF;">(</span><span style="color: #0DB9D7;">types</span><span style="color: #89DDFF;">::</span><span style="color: #C0CAF5;">I64</span><span style="color: #89DDFF;">,</span><span style="color: #FF9E64;"> 1337</span><span style="color: #89DDFF;">);</span></span> <span class="giallo-l"><span style="color: #BB9AF7;">let</span><span style="color: #C0CAF5;"> sum</span><span style="color: #89DDFF;"> =</span><span style="color: #C0CAF5;"> builder</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">ins</span><span style="color: #89DDFF;">().</span><span style="color: #7AA2F7;">iadd</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">x</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> y</span><span style="color: #89DDFF;">);</span></span></code></pre> <p>An example of Rust code that would generate CLIF IR: using an IR builder, two constant 64-bits integer SSA values x and y are created, and then added together. The result is stored into the <code>sum</code> SSA value, which can then be consumed by other instructions.</p> <p>The code for the IR builder we’re manipulating above is automatically generated by the <code>cranelift-codegen</code> build script. The build script uses a domain specific <em>meta</em> language (DSL)<sup class="footnote-reference" id="fr-2-1"><a href="#fn-2">1</a></sup> that defines the instructions, their input and output operands, which input types are allowed, how the output type is inferred, etc. We won’t take a look at this <em>today</em>: this is a bit too far from code generation, but this could be material for another blog post.</p> <p>As an example of a full-blown CLIF generator, there is <a rel="noopener noreferrer external" target="_blank" href="https://github.com/bytecodealliance/wasmtime/tree/main/cranelift/wasm">a crate</a> in the Cranelift project that allows translating from the WebAssembly binary format to CLIF. The Cranelift backend for Rustc uses its own CLIF generator that translates from one of the Rust compiler’s IRs.</p> <p>Finally, it’s time to reveal what’s going to be our running example! The Chosen One is the <code>iadd</code> CLIF operation, which allows to add two integers of any length together, with wrapping semantics. It is both simple to understand what it does, and exhibits interesting behaviors on the two architectures we’re interested in. So, let’s continue down the pipeline!</p> <h2 id="vcode-ir"><a class="zola-anchor" href="#vcode-ir" aria-label="Anchor link for: vcode-ir">🔗</a>VCode IR</h2> <p>Later on, the CLIF intermediate representation is <em>lowered</em>, i.e. transformed from a high-level one into a lower-level one. Here lower level means a form more specialized for a machine architecture. This lower IR is called <em>VCode</em> in Cranelift. The values it references are called <em>virtual registers</em> (more on the <em>virtual</em> bit below). They’re not in SSA form anymore: each virtual register may be redefined as many times as we want. This IR is used to encode register allocation constraints and it guides machine code generation. As a matter of fact, since this information is tied to the machine code’s representation itself, this IR is also target-specific: there’s one flavor of VCode per each CPU architecture we’re compiling to.</p> <p>Let’s get back to our example, that we’re going to compile on two instruction set architectures:</p> <ul> <li>ARM 64-bits (aka aarch64), which is used in most mobile devices but start to become mainstream on laptops (Apple’s Mac M1, some Chromebooks)</li> <li>Intel’s x86 64-bits (aka x86_64, also abbreviated x64), which is used in most desktop and laptop machines).</li> </ul> <p>An integer addition machine instruction on aarch64 will take three operands: two input operands (one of which must be a register), and another third output register operand. While on the x86_64 architecture, the equivalent instruction involves a total of two registers: one that is a read-only source register, and another that is an in-out modified register, containing both the second source and the destination register. We’ll get back to this.</p> <p>So considering <code>iadd</code>, let’s look at (one of<sup class="footnote-reference" id="fr-4-1"><a href="#fn-4">2</a></sup>) the VCode instruction that’s used to represent integer additions on aarch64 (as defined in <code>cranelift/codegen/src/isa/aarch64/inst/mod.rs</code>):</p> <pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="rust"><span class="giallo-l"><span style="color: #51597D;font-style: italic;">/// An ALU operation with two register sources and a register destination.</span></span> <span class="giallo-l"><span style="color: #C0CAF5;">AluRRR</span><span style="color: #89DDFF;"> {</span></span> <span class="giallo-l"><span style="color: #C0CAF5;"> alu_op</span><span style="color: #89DDFF;">:</span><span style="color: #C0CAF5;"> ALUOp</span><span style="color: #89DDFF;">,</span></span> <span class="giallo-l"><span style="color: #C0CAF5;"> rd</span><span style="color: #89DDFF;">:</span><span style="color: #C0CAF5;"> Writable</span><span style="color: #89DDFF;">&lt;</span><span style="color: #C0CAF5;">Reg</span><span style="color: #89DDFF;">&gt;,</span></span> <span class="giallo-l"><span style="color: #C0CAF5;"> rn</span><span style="color: #89DDFF;">:</span><span style="color: #C0CAF5;"> Reg</span><span style="color: #89DDFF;">,</span></span> <span class="giallo-l"><span style="color: #C0CAF5;"> rm</span><span style="color: #89DDFF;">:</span><span style="color: #C0CAF5;"> Reg</span><span style="color: #89DDFF;">,</span></span> <span class="giallo-l"><span style="color: #89DDFF;">},</span></span></code></pre> <p>Some details here:</p> <ul> <li><code>alu_op</code> defines the sub-opcode used in the ALU (Arithmetic Logic Unit). It will be <code>AluOp::Add64</code> for a 64-bits integer addition.</li> <li><code>rn</code> and <code>rm</code> are the conventional aarch64 names for the two input registers.</li> <li><code>rd</code> is the destination register. See how it’s marked as <code>Writable</code>, while the two others are not? <code>Writable</code> is a plain Rust wrapper that makes sure that we <em>can</em> statically differentiate read-only registers from writable registers; a neat trick that allows us to catch more issues at compile-time.</li> </ul> <p>All this information is directly tied to the machine code representation of an addition instruction on aarch64: each field is later used to select some bytes that will be generated during code generation.</p> <p>As said before, the VCode is specific to each architecture, so x86_64 has a different VCode representation for the same instruction (as defined in <code>cranelift/codegen/src/isa/x64/inst/mod.rs</code>):</p> <pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="rust"><span class="giallo-l"><span style="color: #51597D;font-style: italic;">/// Integer arithmetic/bit-twiddling: (add sub and or xor mul adc? sbb?) (32 64) (reg addr imm) reg</span></span> <span class="giallo-l"><span style="color: #C0CAF5;">AluRmiR</span><span style="color: #89DDFF;"> {</span></span> <span class="giallo-l"><span style="color: #C0CAF5;"> is_64</span><span style="color: #89DDFF;">:</span><span style="color: #C0CAF5;"> bool</span><span style="color: #89DDFF;">,</span></span> <span class="giallo-l"><span style="color: #C0CAF5;"> op</span><span style="color: #89DDFF;">:</span><span style="color: #C0CAF5;"> AluRmiROpcode</span><span style="color: #89DDFF;">,</span></span> <span class="giallo-l"><span style="color: #C0CAF5;"> src</span><span style="color: #89DDFF;">:</span><span style="color: #C0CAF5;"> RegMemImm</span><span style="color: #89DDFF;">,</span></span> <span class="giallo-l"><span style="color: #C0CAF5;"> dst</span><span style="color: #89DDFF;">:</span><span style="color: #C0CAF5;"> Writable</span><span style="color: #89DDFF;">&lt;</span><span style="color: #C0CAF5;">Reg</span><span style="color: #89DDFF;">&gt;,</span></span> <span class="giallo-l"><span style="color: #89DDFF;">},</span></span></code></pre> <p>Here, the sub-opcode is defined as part of the <code>AluRmiROpcode</code> enum (the comment hints at which other x86 machine instructions are generated by this same VCode). See how there’s only one <code>src</code> (source) register (or memory or immediate operand), while the instruction conceptually takes two inputs? That’s because it’s expected that the <code>dst</code> (destination) register is <em>modified</em>, that is, both read (so it’s the second input operand) and written to (so it’s the result register). In equivalent C code, the x86’s add instruction doesn’t actually do <code>a = b + c</code>. What it does is <code>a += b</code>, that is, one of the sources is <em>consumed</em> by the instruction. This is an artifact inherited from the design of older x86 machines in the 1970’s, when instructions were designed around an accumulator model (and representing efficiently three operands in a CISC architecture would make the encoding larger and harder than it is).</p> <h2 id="instruction-selection-lowering"><a class="zola-anchor" href="#instruction-selection-lowering" aria-label="Anchor link for: instruction-selection-lowering">🔗</a>Instruction selection (lowering)</h2> <p>As said before, converting from the high-level IR (CLIF) to the low-level IR (VCode) is called lowering. Since VCode is target-dependent, this process is also target-dependent. That’s where we consider which machine instructions get eventually used for a given CLIF opcode. There are many ways to achieve the same machine state results for given semantics, but some of these ways are faster than other, and/or require fewer code bytes to achieve. The problem can be summed up like this: given some CLIF, which VCode can we create to generate the fastest and/or smallest machine code that carries out the desired semantics? This is called <em>instruction selection</em>, because we’re selecting the VCode instructions among a set of different possible instructions.</p> <p>How do these IR map to each other? A given CLIF node may be lowered into 1 to N VCode instructions. A given VCode instruction may lead to the code generation of 1 to M machine instructions. There are no rules governing the maximum of entities mapped. For instance, the integer addition CLIF opcode <code>iadd</code> on 64-bits inputs maps to a single VCode instruction on aarch64. The VCode instruction then causes a single code instruction to be generated.</p> <p>Other CLIF opcodes may generate more than a single machine instruction eventually. Consider the CLIF opcode for signed integer division <code>idiv</code>. Its semantics define that it traps for zero inputs and in case of integer overflow<sup class="footnote-reference" id="fr-3-1"><a href="#fn-3">3</a></sup>. On aarch64, this is lowered into:</p> <ul> <li>one VCode instruction that checks if the input is zero and trap otherwise</li> <li>two VCode instructions for comparing the input values against the minimal integer value and -1</li> <li>one VCode instruction to trap if the two input values match what we checked against</li> <li>and one VCode instruction that does the actual division operation.</li> </ul> <p>Each of these VCode instruction then generates one or more machine code instructions, resulting in a bit of a longer sequence.</p> <p>Let’s look at the lowering of <code>iadd</code> on aarch64 (in <code>cranelift/codegen/src/isa/aarch64/lower_inst.rs</code>), edited and simplified for clarity. I’ve added comments in the code, explaining what each line does:</p> <pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="rust"><span class="giallo-l"><span style="color: #C0CAF5;">Opcode</span><span style="color: #89DDFF;">::</span><span style="color: #C0CAF5;">Iadd</span><span style="color: #89DDFF;"> =&gt; {</span></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // Get the destination register.</span></span> <span class="giallo-l"><span style="color: #BB9AF7;"> let</span><span style="color: #C0CAF5;"> rd</span><span style="color: #89DDFF;"> =</span><span style="color: #7AA2F7;"> get_output_reg</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">ctx</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> outputs</span><span style="color: #89DDFF;">[</span><span style="color: #FF9E64;">0</span><span style="color: #89DDFF;">]).</span><span style="color: #7AA2F7;">only_reg</span><span style="color: #89DDFF;">().</span><span style="color: #7AA2F7;">unwrap</span><span style="color: #89DDFF;">();</span></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // Get the controlling type of the addition (32-bits int or 64-bits int or</span></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // int vector, etc.).</span></span> <span class="giallo-l"><span style="color: #BB9AF7;"> let</span><span style="color: #C0CAF5;"> ty</span><span style="color: #89DDFF;"> =</span><span style="color: #C0CAF5;"> ty</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">unwrap</span><span style="color: #89DDFF;">();</span></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // Force one of the inputs into a register, not applying any signed- or</span></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // zero-extension.</span></span> <span class="giallo-l"><span style="color: #BB9AF7;"> let</span><span style="color: #C0CAF5;"> rn</span><span style="color: #89DDFF;"> =</span><span style="color: #7AA2F7;"> put_input_in_reg</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">ctx</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> inputs</span><span style="color: #89DDFF;">[</span><span style="color: #FF9E64;">0</span><span style="color: #89DDFF;">],</span><span style="color: #0DB9D7;"> NarrowValueMode</span><span style="color: #89DDFF;">::</span><span style="color: #C0CAF5;">None</span><span style="color: #89DDFF;">);</span></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // Try to see if we can encode the second operand as an immediate on</span></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // 12-bits, maybe by negating it;</span></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // Otherwise, put it into a register.</span></span> <span class="giallo-l"><span style="color: #BB9AF7;"> let</span><span style="color: #89DDFF;"> (</span><span style="color: #C0CAF5;">rm</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> negated</span><span style="color: #89DDFF;">) =</span><span style="color: #7AA2F7;"> put_input_in_rse_imm12_maybe_negated</span><span style="color: #89DDFF;">(</span></span> <span class="giallo-l"><span style="color: #C0CAF5;"> ctx</span><span style="color: #89DDFF;">,</span></span> <span class="giallo-l"><span style="color: #C0CAF5;"> inputs</span><span style="color: #89DDFF;">[</span><span style="color: #FF9E64;">1</span><span style="color: #89DDFF;">],</span></span> <span class="giallo-l"><span style="color: #7AA2F7;"> ty_bits</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">ty</span><span style="color: #89DDFF;">),</span></span> <span class="giallo-l"><span style="color: #0DB9D7;"> NarrowValueMode</span><span style="color: #89DDFF;">::</span><span style="color: #C0CAF5;">None</span><span style="color: #89DDFF;">,</span></span> <span class="giallo-l"><span style="color: #89DDFF;"> );</span></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // Select the ALU subopcode, based on possible negation and controlling</span></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // type.</span></span> <span class="giallo-l"><span style="color: #BB9AF7;"> let</span><span style="color: #C0CAF5;"> alu_op</span><span style="color: #89DDFF;"> =</span><span style="color: #BB9AF7;"> if !</span><span style="color: #C0CAF5;">negated</span><span style="color: #89DDFF;"> {</span></span> <span class="giallo-l"><span style="color: #7AA2F7;"> choose_32_64</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">ty</span><span style="color: #89DDFF;">,</span><span style="color: #0DB9D7;"> ALUOp</span><span style="color: #89DDFF;">::</span><span style="color: #C0CAF5;">Add32</span><span style="color: #89DDFF;">,</span><span style="color: #0DB9D7;"> ALUOp</span><span style="color: #89DDFF;">::</span><span style="color: #C0CAF5;">Add64</span><span style="color: #89DDFF;">)</span></span> <span class="giallo-l"><span style="color: #89DDFF;"> }</span><span style="color: #BB9AF7;"> else</span><span style="color: #89DDFF;"> {</span></span> <span class="giallo-l"><span style="color: #7AA2F7;"> choose_32_64</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">ty</span><span style="color: #89DDFF;">,</span><span style="color: #0DB9D7;"> ALUOp</span><span style="color: #89DDFF;">::</span><span style="color: #C0CAF5;">Sub32</span><span style="color: #89DDFF;">,</span><span style="color: #0DB9D7;"> ALUOp</span><span style="color: #89DDFF;">::</span><span style="color: #C0CAF5;">Sub64</span><span style="color: #89DDFF;">)</span></span> <span class="giallo-l"><span style="color: #89DDFF;"> };</span></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // Emit the VCode instruction in the VCode stream.</span></span> <span class="giallo-l"><span style="color: #C0CAF5;"> ctx</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">emit</span><span style="color: #89DDFF;">(</span><span style="color: #7AA2F7;">alu_inst_imm12</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">alu_op</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> rd</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> rn</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> rm</span><span style="color: #89DDFF;">));</span></span> <span class="giallo-l"><span style="color: #89DDFF;">}</span></span></code></pre> <p>In fact, the <code>alu_inst_imm12</code> wrapper can create one VCode instruction among a set of possible ones (since we’re trying to select <em>the best one</em>). For the sake of simplicity, we’ll assume that <code>AluRRR</code> is going to be generated, i.e. the selected instruction is the one using only register encodings for the input values.</p> <h2 id="register-allocation"><a class="zola-anchor" href="#register-allocation" aria-label="Anchor link for: register-allocation">🔗</a>Register allocation</h2> <div class="mermaid"> graph TD vcode_vreg[VCode with virtual registers] regalloc([Register allocation]) vcode_rreg[VCode with real registers] codegen([Code generation]) machine_code(Machine code) vcode_vreg --> regalloc --> vcode_rreg --> codegen --> machine_code </div> <h3 id="vcode-registers-and-stack-slots"><a class="zola-anchor" href="#vcode-registers-and-stack-slots" aria-label="Anchor link for: vcode-registers-and-stack-slots">🔗</a>VCode, registers and stack slots</h3> <p>Hey, ever wondered what the V in VCode meant? Back to the drawing board. While a program may reference a theoretically unlimited number of instructions, each referencing a theoretically unlimited number of values as inputs and outputs, the physical machine only has a fixed set of containers for those values:</p> <ul> <li>either they must live in machine <strong>registers</strong>: very fast to access in the CPU, take some CPU real estate, thus are costly, so there are usually few of them.</li> <li>or they must live in the process’ <strong>stack memory</strong>: it’s slower to access, but we can have virtually any amount of stack <em>slots</em>.</li> </ul> <pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="asm"><span class="giallo-l"><span style="color: #89DDFF;">mov</span><span> %</span><span style="color: #FF9E64;">edi</span><span>,-</span><span style="color: #FF9E64;">0x4</span><span>(%</span><span style="color: #FF9E64;">rbp</span><span>)</span></span> <span class="giallo-l"><span style="color: #89DDFF;">mov</span><span> %</span><span style="color: #FF9E64;">rsi</span><span>,-</span><span style="color: #FF9E64;">0x10</span><span>(%</span><span style="color: #FF9E64;">rbp</span><span>)</span></span> <span class="giallo-l"><span style="color: #89DDFF;">mov</span><span> -</span><span style="color: #FF9E64;">0x4</span><span>(%</span><span style="color: #FF9E64;">rbp</span><span>),%</span><span style="color: #FF9E64;">eax</span></span></code></pre> <p><em>In this example of x86 machine code, %edi, %rsi, %rbp, %eax are all registers; stack slots are memory addresses computed as the frame pointer (%rbp) plus an offset value (which happens to be negative here). Note that stack slots may be referred to by the stack pointer (%rsp) in general.</em></p> <h3 id="defining-the-register-allocation-problem"><a class="zola-anchor" href="#defining-the-register-allocation-problem" aria-label="Anchor link for: defining-the-register-allocation-problem">🔗</a>Defining the register allocation problem</h3> <p>The problem of mapping the IR values (in VCode these are the <code>Reg</code>) to machine “containers” is called <strong><a rel="noopener noreferrer external" target="_blank" href="https://en.wikipedia.org/wiki/Register_allocation">register allocation</a></strong> (aka regalloc). Inputs to register allocation can be as numerous as we want them, and map to “virtual” values, hence we call them <em>virtual registers</em>. And… that’s where the V from VCode comes from: the instructions in VCode reference values that are <em>virtual</em> registers before register allocation, so we say the code is in <em>virtualized</em> register form. The output of register allocation is a set of new instructions, where the virtual registers have been replaced by <em>real registers</em> (the physical ones, limited in quantity) or stack slots references (and other additional metadata).</p> <pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="plain"><span class="giallo-l"><span>// Before register allocation, with unlimited virtual registers:</span></span> <span class="giallo-l"><span>v2 = v0 + v1</span></span> <span class="giallo-l"><span>v3 = v2 * 2</span></span> <span class="giallo-l"><span>v4 = v2 + 1</span></span> <span class="giallo-l"><span>v5 = v4 + v3</span></span> <span class="giallo-l"><span>return v5</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span>// One possible register allocation, on a machine that has 2 registers %r0, %r1:</span></span> <span class="giallo-l"><span>%r0 = %r0 + %r1</span></span> <span class="giallo-l"><span>%r1 = %r0 * 2</span></span> <span class="giallo-l"><span>%r0 = %r0 + 1</span></span> <span class="giallo-l"><span>%r1 = %r0 + %r1</span></span> <span class="giallo-l"><span>return %r1</span></span></code></pre> <p>When all is well, the virtual registers don’t conceptually <em>live</em> at the same time, and they can be put into physical registers. Issues arise when there’s not enough physical registers to contain all the virtual registers that live at the same time, which is the case for… a very large majority of programs. Then, register allocation must decide which registers continue to live in registers at a given program point, and which should be <strong>spilled</strong> into a stack slot, effectively <em>storing</em> them onto the stack for later use. This later reuse will imply to <strong>reload</strong> them from the stack slot, using a <em>load</em> machine instruction. The complexity resides in choosing which registers should be spilled, at which program point they should be spilled, and at which program points we should reload them, if we need to do so. Making good choices there will have a large impact on the speed of the generated code, since memory accesses to the stack imply an additional runtime cost. For instance, a variable that’s frequently used in a hot loop should live in a register for the whole loop’s lifetime, and not be spilled/reloaded in the middle of the loop.</p> <pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="plain"><span class="giallo-l"><span>// Before register allocation, with unlimited virtual registers:</span></span> <span class="giallo-l"><span>v2 = v0 + v1</span></span> <span class="giallo-l"><span>v3 = v0 + v2</span></span> <span class="giallo-l"><span>v4 = v3 + v1</span></span> <span class="giallo-l"><span>return v4</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span>// One possible register allocation, on a machine that has 2 registers %r0, %r1.</span></span> <span class="giallo-l"><span>// We need to spill one value, because there&#39;s a point where 3 values are live at the same time!</span></span> <span class="giallo-l"><span>spill %r1 --&gt; stack_slot(0)</span></span> <span class="giallo-l"><span>%r1 = %r0 + %r1</span></span> <span class="giallo-l"><span>%r1 = %r0 + %r1</span></span> <span class="giallo-l"><span>reload stack_slot(0) --&gt; %r0</span></span> <span class="giallo-l"><span>%r1 = %r1 + %r0</span></span> <span class="giallo-l"><span>return %r1</span></span></code></pre> <p>And, since we like to have our cake and eat it too, the register allocator itself should be <em>fast</em>: it should not take an unbounded amount of time to make these allocation decisions. Register allocation has the good taste to be a <a rel="noopener noreferrer external" target="_blank" href="https://en.wikipedia.org/wiki/NP-completeness">NP-complete</a> problem. Concretely, this means that implementations cannot find the <em>best</em> solutions given arbitrary inputs, but they’ll estimate <em>good</em> solutions based on heuristics, in worst-case quadratic time over the size of the input. All of this makes it so that register allocation has its own whole research field, and has been extensively studied for some time now. It is a fascinating problem.</p> <h3 id="register-allocation-in-cranelift"><a class="zola-anchor" href="#register-allocation-in-cranelift" aria-label="Anchor link for: register-allocation-in-cranelift">🔗</a>Register allocation in Cranelift</h3> <p>Back to Cranelift. The register allocation contract is that if a value <em>must</em> live in a real register at a given program point, then it <em>does</em> live where it should (unless register allocation is impossible). At the start of code generation for a VCode instruction, we are guaranteed that the input values live in real registers, and that the output real register is available before the next VCode instruction.</p> <p>You might have noticed that the VCode instructions only refer to registers, and not stack slots. But where are the stack slots, then? The trick is that the stack slots are <em>invisible</em> to VCode. Register allocation may create an arbitrary number of spills, reloads, and register moves<sup class="footnote-reference" id="fr-5-1"><a href="#fn-5">4</a></sup> around VCode instructions, to ensure that their register allocation constraints are met. This is why the output of register allocation is a new list of instructions, that includes not only the initial instructions filled with the actual registers, but also additional spill, reload and move (VCode) instructions added by regalloc.</p> <p>As said before, this problem is so sufficiently complex, involved and independent from the rest of the code (assuming the right set of interfaces!) that its code lives in a separate crate, <a rel="noopener noreferrer external" target="_blank" href="https://github.com/bytecodealliance/regalloc.rs"><code>regalloc.rs</code></a>, with its own fuzzing and testing infrastructure. I hope to shed some light on it at some point too.</p> <p>What’s interesting to us today is the register allocation <em>constraints</em>. Consider the aarch64 integer add instruction <code>add rd, rn, rm</code>: <code>rd</code> is the output virtual register that’s written to, while <code>rn</code> and <code>rm</code> are the inputs, thus read from. We need to inform the register allocation algorithm about these constraints. In regalloc jargon, “read to” is known as <em>used</em>, while “written to” is known as <em>defined</em>. Here, the aarch64 VCode instruction <code>AluRRR</code> does <em>use</em> <code>rn</code> and <code>rm</code>, and it <em>def</em>ines <code>rd</code>. This usage information is <em>collected</em> in the <code>aarch64_get_regs</code> function (<code>cranelift/codegen/src/isa/aarch64/inst/mod.rs</code>):</p> <pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="rust"><span class="giallo-l"><span style="color: #89DDFF;">fn</span><span style="color: #7AA2F7;"> aarch64_get_regs</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">inst</span><span style="color: #89DDFF;">: &amp;</span><span style="color: #C0CAF5;">Inst</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> collector</span><span style="color: #89DDFF;">: &amp;</span><span style="color: #9D7CD8;font-style: italic;">mut</span><span style="color: #C0CAF5;"> RegUsageCollector</span><span style="color: #89DDFF;">) {</span></span> <span class="giallo-l"><span style="color: #BB9AF7;"> match</span><span style="color: #C0CAF5;"> inst</span><span style="color: #89DDFF;"> {</span></span> <span class="giallo-l"><span style="color: #89DDFF;"> &amp;</span><span style="color: #C0CAF5;">Inst</span><span style="color: #89DDFF;">::</span><span style="color: #C0CAF5;">AluRRR</span><span style="color: #89DDFF;"> {</span><span style="color: #C0CAF5;"> rd</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> rn</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> rm</span><span style="color: #89DDFF;">, .. } =&gt; {</span></span> <span class="giallo-l"><span style="color: #C0CAF5;"> collector</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">add_def</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">rd</span><span style="color: #89DDFF;">);</span></span> <span class="giallo-l"><span style="color: #C0CAF5;"> collector</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">add_use</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">rn</span><span style="color: #89DDFF;">);</span></span> <span class="giallo-l"><span style="color: #C0CAF5;"> collector</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">add_use</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">rm</span><span style="color: #89DDFF;">);</span></span> <span class="giallo-l"><span style="color: #89DDFF;"> }</span></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // etc.</span></span></code></pre> <p>Then, after register allocation has assigned the physical registers, we need to instruct it how to replace virtual register mentions by physical register mentions. This is done in the <code>aarch64_map_regs</code> function (same file as above):</p> <pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="rust"><span class="giallo-l"><span style="color: #89DDFF;">fn</span><span style="color: #7AA2F7;"> aarch64_map_regs</span><span style="color: #89DDFF;">&lt;</span><span style="color: #FF9E64;">RUM</span><span style="color: #89DDFF;">:</span><span style="color: #C0CAF5;"> RegUsageMapper</span><span style="color: #89DDFF;">&gt;(</span><span style="color: #C0CAF5;">inst</span><span style="color: #89DDFF;">: &amp;</span><span style="color: #9D7CD8;font-style: italic;">mut</span><span style="color: #C0CAF5;"> Inst</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> mapper</span><span style="color: #89DDFF;">: &amp;</span><span style="color: #FF9E64;">RUM</span><span style="color: #89DDFF;">) {</span></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // ...</span></span> <span class="giallo-l"><span style="color: #BB9AF7;"> match</span><span style="color: #C0CAF5;"> inst</span><span style="color: #89DDFF;"> {</span></span> <span class="giallo-l"><span style="color: #89DDFF;"> &amp;</span><span style="color: #9D7CD8;font-style: italic;">mut</span><span style="color: #C0CAF5;"> Inst</span><span style="color: #89DDFF;">::</span><span style="color: #C0CAF5;">AluRRR</span><span style="color: #89DDFF;"> {</span></span> <span class="giallo-l"><span style="color: #89DDFF;"> ref</span><span style="color: #9D7CD8;font-style: italic;"> mut</span><span style="color: #C0CAF5;"> rd</span><span style="color: #89DDFF;">,</span></span> <span class="giallo-l"><span style="color: #89DDFF;"> ref</span><span style="color: #9D7CD8;font-style: italic;"> mut</span><span style="color: #C0CAF5;"> rn</span><span style="color: #89DDFF;">,</span></span> <span class="giallo-l"><span style="color: #89DDFF;"> ref</span><span style="color: #9D7CD8;font-style: italic;"> mut</span><span style="color: #C0CAF5;"> rm</span><span style="color: #89DDFF;">,</span></span> <span class="giallo-l"><span style="color: #89DDFF;"> ..</span></span> <span class="giallo-l"><span style="color: #89DDFF;"> } =&gt; {</span></span> <span class="giallo-l"><span style="color: #7AA2F7;"> map_def</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">mapper</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> rd</span><span style="color: #89DDFF;">);</span></span> <span class="giallo-l"><span style="color: #7AA2F7;"> map_use</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">mapper</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> rn</span><span style="color: #89DDFF;">);</span></span> <span class="giallo-l"><span style="color: #7AA2F7;"> map_use</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">mapper</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> rm</span><span style="color: #89DDFF;">);</span></span> <span class="giallo-l"><span style="color: #89DDFF;"> }</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // etc.</span></span></code></pre> <p>Note this is reflecting quite precisely what the usage collector did: we’re replacing the virtual register mention for the defined register <code>rd</code> with the information (which real register) provided by the <code>RegUsageMapper</code>. These two functions must stay in sync, otherwise here be dragons! (and bugs very hard to debug!)</p> <h3 id="register-allocation-on-x86"><a class="zola-anchor" href="#register-allocation-on-x86" aria-label="Anchor link for: register-allocation-on-x86">🔗</a>Register allocation on x86</h3> <p>On Intel’s x86, register allocation may be a bit trickier: in some cases, the lowering needs to be carefully written so it satisfies some register allocation constraints that are very specific to this architecture. In particular, x86 has <em>fixed register constraints</em> as well as <em>tied operands</em>.</p> <p>For this specific part, we’ll look at the integer shift-left instruction, which is equivalent to C’s <code>x &lt;&lt; y</code>. Why this particular instruction? It exhibits both properties that we’re interested in studying here. The lowering of <code>iadd</code> is similar, albeit slightly simpler, as it <em>only</em> involves tied operands.</p> <h4 id="fixed-register-constraints"><a class="zola-anchor" href="#fixed-register-constraints" aria-label="Anchor link for: fixed-register-constraints">🔗</a>Fixed register constraints</h4> <p>On the one hand, some instructions expect their inputs to be in <em>fixed</em> registers, that is, specific registers arbitrarily predefined by the architecture manual. For the example of the shift instruction, if the count is not statically known at compile time (it’s not a shift by a constant value), then the amount by which we’re shifting must be in the <code>rcx</code> register<sup class="footnote-reference" id="fr-8-1"><a href="#fn-8">5</a></sup>.</p> <p>Now, how do we make sure that the input value actually is in <code>rcx</code>? We can mark <code>rcx</code> as used in the <code>get_regs</code> function so regalloc knows about this, but nothing ensures that the input <em>resides</em> in it at the beginning of the instruction. To resolve this, we’ll introduce a <strong>move instruction</strong> during lowering, that is going to copy the input value into <code>rcx</code>. Then we’re sure it lives there, and register allocation knows it’s used: we’re good to go!</p> <p>In a nutshell, this shows how lowering and register allocation play together:</p> <ul> <li>during lowering, we introduce a move from a dynamic shift input value to <code>rcx</code> before the actual shift</li> <li>in the register usage function, we mark <code>rcx</code> as used</li> <li>(nothing to do in the register mapping function: <code>rcx</code> is a real register already)</li> </ul> <h4 id="tied-operands"><a class="zola-anchor" href="#tied-operands" aria-label="Anchor link for: tied-operands">🔗</a>Tied operands</h4> <p>On the other hand, some instructions have operands that are both read and written at the same time: we call them <em>modified</em> in Cranelift and regalloc.rs, but they’re also known as <em>tied operands</em> in the compiler literature. It’s not just that there’s a register that must be read, and a register that must be written to: they <em>must</em> be the same register. How do we model this, then?</p> <p>Consider a naive solution. We take the input virtual register, and decide it’s allocated to the same register as the output (modified) register. Unfortunately, if the chosen virtual register was going to be reused by another later VCode instruction, then its value would be overwritten (clobbered) by the current instruction. This would result in incorrect code being generated, so this is not acceptable. In general we can’t clobber the value that was in an input value during lowering, because that’s the role of regalloc to make this kind of decisions.</p> <pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="plain"><span class="giallo-l"><span>// Before register allocation, with virtual registers:</span></span> <span class="giallo-l"><span>v2 = v0 + v1</span></span> <span class="giallo-l"><span>v3 = v0 + 42</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span>// After register allocation, on a machine with two registers %r0 and %r1:</span></span> <span class="giallo-l"><span>// assign v0 to %r0, v1 to %r1, v2 to %r0</span></span> <span class="giallo-l"><span>%r0 += v1</span></span> <span class="giallo-l"><span>... = %r0 + 42 // ohnoes! the value in %r0 is v2, not v0 anymore!</span></span></code></pre> <p>The right solution is, again, to <em>copy</em> this input virtual register into the output virtual register, right before the instruction. This way, we can still reuse the untouched input register in other instructions without modifying it: only the copy is written to.</p> <p>Pfew! We can now look at the entire lowering for the shift left instruction, edited and commented for clarity:</p> <pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="rust"><span class="giallo-l"><span style="color: #51597D;font-style: italic;">// Read the instruction operand size from the output&#39;s type.</span></span> <span class="giallo-l"><span style="color: #BB9AF7;">let</span><span style="color: #C0CAF5;"> size</span><span style="color: #89DDFF;"> =</span><span style="color: #C0CAF5;"> dst_ty</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">bytes</span><span style="color: #89DDFF;">() as</span><span style="color: #C0CAF5;"> u8</span><span style="color: #89DDFF;">;</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;">// Put the left hand side into a virtual register.</span></span> <span class="giallo-l"><span style="color: #BB9AF7;">let</span><span style="color: #C0CAF5;"> lhs</span><span style="color: #89DDFF;"> =</span><span style="color: #7AA2F7;"> put_input_in_reg</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">ctx</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> inputs</span><span style="color: #89DDFF;">[</span><span style="color: #FF9E64;">0</span><span style="color: #89DDFF;">]);</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;">// Put the right hand side (shift amount) into either an immediate (if it&#39;s</span></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;">// statically known at compile time), or into a virtual register.</span></span> <span class="giallo-l"><span style="color: #BB9AF7;">let</span><span style="color: #89DDFF;"> (</span><span style="color: #C0CAF5;">count</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> rhs</span><span style="color: #89DDFF;">) =</span></span> <span class="giallo-l"><span style="color: #BB9AF7;"> if let</span><span style="color: #C0CAF5;"> Some</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">cst</span><span style="color: #89DDFF;">) =</span><span style="color: #C0CAF5;"> ctx</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">get_input_as_source_or_const</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">insn</span><span style="color: #89DDFF;">,</span><span style="color: #FF9E64;"> 1</span><span style="color: #89DDFF;">).</span><span>constant </span><span style="color: #89DDFF;">{</span></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // Mask count, according to Cranelift&#39;s semantics.</span></span> <span class="giallo-l"><span style="color: #BB9AF7;"> let</span><span style="color: #C0CAF5;"> cst</span><span style="color: #89DDFF;"> = (</span><span style="color: #C0CAF5;">cst</span><span style="color: #89DDFF;"> as</span><span style="color: #C0CAF5;"> u8</span><span style="color: #89DDFF;">) &amp; (</span><span style="color: #C0CAF5;">dst_ty</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">bits</span><span style="color: #89DDFF;">() as</span><span style="color: #C0CAF5;"> u8</span><span style="color: #89DDFF;"> -</span><span style="color: #FF9E64;"> 1</span><span style="color: #89DDFF;">);</span></span> <span class="giallo-l"><span style="color: #89DDFF;"> (</span><span style="color: #C0CAF5;">Some</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">cst</span><span style="color: #89DDFF;">),</span><span style="color: #C0CAF5;"> None</span><span style="color: #89DDFF;">)</span></span> <span class="giallo-l"><span style="color: #89DDFF;"> }</span><span style="color: #BB9AF7;"> else</span><span style="color: #89DDFF;"> {</span></span> <span class="giallo-l"><span style="color: #89DDFF;"> (</span><span style="color: #C0CAF5;">None</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> Some</span><span style="color: #89DDFF;">(</span><span style="color: #7AA2F7;">put_input_in_reg</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">ctx</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> inputs</span><span style="color: #89DDFF;">[</span><span style="color: #FF9E64;">1</span><span style="color: #89DDFF;">])))</span></span> <span class="giallo-l"><span style="color: #89DDFF;"> };</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;">// Get the destination virtual register.</span></span> <span class="giallo-l"><span style="color: #BB9AF7;">let</span><span style="color: #C0CAF5;"> dst</span><span style="color: #89DDFF;"> =</span><span style="color: #7AA2F7;"> get_output_reg</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">ctx</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> outputs</span><span style="color: #89DDFF;">[</span><span style="color: #FF9E64;">0</span><span style="color: #89DDFF;">]).</span><span style="color: #7AA2F7;">only_reg</span><span style="color: #89DDFF;">().</span><span style="color: #7AA2F7;">unwrap</span><span style="color: #89DDFF;">();</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;">// Copy the left hand side into the (modified) output operand, to satisfy the</span></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;">// mod constraint.</span></span> <span class="giallo-l"><span style="color: #C0CAF5;">ctx</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">emit</span><span style="color: #89DDFF;">(</span><span style="color: #0DB9D7;">Inst</span><span style="color: #89DDFF;">::</span><span style="color: #7AA2F7;">mov_r_r</span><span style="color: #89DDFF;">(</span><span style="color: #FF9E64;">true</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> lhs</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> dst</span><span style="color: #89DDFF;">));</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;">// If the shift count is statically known: nothing particular to do. Otherwise,</span></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;">// we need to put it in the RCX register.</span></span> <span class="giallo-l"><span style="color: #BB9AF7;">if</span><span style="color: #C0CAF5;"> count</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">is_none</span><span style="color: #89DDFF;">() {</span></span> <span class="giallo-l"><span style="color: #BB9AF7;"> let</span><span style="color: #C0CAF5;"> w_rcx</span><span style="color: #89DDFF;"> =</span><span style="color: #C0CAF5;"> Writable</span><span style="color: #89DDFF;">::</span><span style="color: #7AA2F7;">from_reg</span><span style="color: #89DDFF;">(</span><span style="color: #0DB9D7;">regs</span><span style="color: #89DDFF;">::</span><span style="color: #7AA2F7;">rcx</span><span style="color: #89DDFF;">());</span></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // Copy the shift count (which is in rhs) into RCX.</span></span> <span class="giallo-l"><span style="color: #C0CAF5;"> ctx</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">emit</span><span style="color: #89DDFF;">(</span><span style="color: #0DB9D7;">Inst</span><span style="color: #89DDFF;">::</span><span style="color: #7AA2F7;">mov_r_r</span><span style="color: #89DDFF;">(</span><span style="color: #FF9E64;">true</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> rhs</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">unwrap</span><span style="color: #89DDFF;">(),</span><span style="color: #C0CAF5;"> w_rcx</span><span style="color: #89DDFF;">));</span></span> <span class="giallo-l"><span style="color: #89DDFF;">}</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;">// Generate the actual shift instruction.</span></span> <span class="giallo-l"><span style="color: #C0CAF5;">ctx</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">emit</span><span style="color: #89DDFF;">(</span><span style="color: #0DB9D7;">Inst</span><span style="color: #89DDFF;">::</span><span style="color: #7AA2F7;">shift_r</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">size</span><span style="color: #89DDFF;">,</span><span style="color: #0DB9D7;"> ShiftKind</span><span style="color: #89DDFF;">::</span><span style="color: #C0CAF5;">ShiftLeft</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> count</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> dst</span><span style="color: #89DDFF;">));</span></span></code></pre> <p>And this is how we tell the register usage collector about our constraints:</p> <pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="rust"><span class="giallo-l"><span style="color: #C0CAF5;">Inst</span><span style="color: #89DDFF;">::</span><span style="color: #C0CAF5;">ShiftR</span><span style="color: #89DDFF;"> {</span><span style="color: #C0CAF5;"> num_bits</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> dst</span><span style="color: #89DDFF;">, .. } =&gt; {</span></span> <span class="giallo-l"><span style="color: #BB9AF7;"> if</span><span style="color: #C0CAF5;"> num_bits</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">is_none</span><span style="color: #89DDFF;">() {</span></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // if the shift count is dynamic, mark RCX as used.</span></span> <span class="giallo-l"><span style="color: #C0CAF5;"> collector</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">add_use</span><span style="color: #89DDFF;">(</span><span style="color: #0DB9D7;">regs</span><span style="color: #89DDFF;">::</span><span style="color: #7AA2F7;">rcx</span><span style="color: #89DDFF;">());</span></span> <span class="giallo-l"><span style="color: #89DDFF;"> }</span></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // In all the cases, the destination operand is modified.</span></span> <span class="giallo-l"><span style="color: #C0CAF5;"> collector</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">add_mod</span><span style="color: #89DDFF;">(*</span><span style="color: #C0CAF5;">dst</span><span style="color: #89DDFF;">);</span></span> <span class="giallo-l"><span style="color: #89DDFF;">}</span></span></code></pre> <p>Only the modified register needs to be mapped to its allocated physical register:</p> <pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="rust"><span class="giallo-l"><span style="color: #C0CAF5;">Inst</span><span style="color: #89DDFF;">::</span><span style="color: #C0CAF5;">ShiftR</span><span style="color: #89DDFF;"> { ref</span><span style="color: #9D7CD8;font-style: italic;"> mut</span><span style="color: #C0CAF5;"> dst</span><span style="color: #89DDFF;">, .. } =&gt; {</span></span> <span class="giallo-l"><span style="color: #7AA2F7;"> map_mod</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">mapper</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> dst</span><span style="color: #89DDFF;">);</span></span> <span class="giallo-l"><span style="color: #89DDFF;">}</span></span></code></pre><h3 id="virtual-registers-copies-and-performance"><a class="zola-anchor" href="#virtual-registers-copies-and-performance" aria-label="Anchor link for: virtual-registers-copies-and-performance">🔗</a>Virtual registers copies and performance</h3> <p>Do these virtual register copies sound costly to you? In theory, they could lead to the code generation of a move instructions, increasing the size of the code generated and causing a small runtime cost. In practice, register allocation, through its interface, knows how to identify move instructions, their source and their destination. By analyzing them, it can see when a source isn’t used after a given move instruction, and thus allocate the same register for the source and the destination of the move. Then, when Cranelift generates the code, it will avoid generating a move from a physical register to the same one<sup class="footnote-reference" id="fr-7-1"><a href="#fn-7">6</a></sup>. As a matter of fact, creating a VCode copy doesn’t necessarily mean that it will generate a machine code move instruction later: it is present just in case regalloc <em>needs</em> it, but it can be avoided when it’s spurious.</p> <h2 id="code-generation"><a class="zola-anchor" href="#code-generation" aria-label="Anchor link for: code-generation">🔗</a>Code generation</h2> <p>Oh my, we’re getting closer to actually being able to run the code! Once register allocation has run, we can generate the actual machine code for the VCode instructions. Cool kids call this step of the pipeline <em>codegen</em>, for code generation. This is the part where we decipher the architecture manuals provided by the CPU vendors, and generate the raw machine bytes for our machine instructions. In Cranelift, this means filling a code buffer (there’s a <code>MachBuffer</code> sink interface for this!), returned along some internal relocations<sup class="footnote-reference" id="fr-6-1"><a href="#fn-6">7</a></sup> and additional metadata. Let’s see what happens for our integer addition, when the times come to generate the code for its VCode equivalent <code>AluRRR</code> on <code>aarch64</code> (in <code>cranelift/codegen/src/isa/aarch64/inst/emit.rs</code>):</p> <pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="rust"><span class="giallo-l"><span style="color: #51597D;font-style: italic;">// We match on the VCode&#39;s identity here:</span></span> <span class="giallo-l"><span style="color: #89DDFF;">&amp;</span><span style="color: #C0CAF5;">Inst</span><span style="color: #89DDFF;">::</span><span style="color: #C0CAF5;">AluRRR</span><span style="color: #89DDFF;"> {</span><span style="color: #C0CAF5;"> alu_op</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> rd</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> rn</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> rm</span><span style="color: #89DDFF;"> } =&gt; {</span></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // First select the top 11 bits based on the ALU subopcode.</span></span> <span class="giallo-l"><span style="color: #BB9AF7;"> let</span><span style="color: #C0CAF5;"> top11</span><span style="color: #89DDFF;"> =</span><span style="color: #BB9AF7;"> match</span><span style="color: #C0CAF5;"> alu_op</span><span style="color: #89DDFF;"> {</span></span> <span class="giallo-l"><span style="color: #C0CAF5;"> ALUOp</span><span style="color: #89DDFF;">::</span><span style="color: #C0CAF5;">Add32</span><span style="color: #89DDFF;"> =&gt;</span><span style="color: #FF9E64;"> 0b00001011_000</span><span style="color: #89DDFF;">,</span></span> <span class="giallo-l"><span style="color: #C0CAF5;"> ALUOp</span><span style="color: #89DDFF;">::</span><span style="color: #C0CAF5;">Add64</span><span style="color: #89DDFF;"> =&gt;</span><span style="color: #FF9E64;"> 0b10001011_000</span><span style="color: #89DDFF;">,</span></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // etc</span></span> <span class="giallo-l"><span style="color: #89DDFF;"> };</span></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // Then decide the bits 10 to 15, based on the ALU subopcode as well.</span></span> <span class="giallo-l"><span style="color: #BB9AF7;"> let</span><span style="color: #C0CAF5;"> bit15_10</span><span style="color: #89DDFF;"> =</span><span style="color: #BB9AF7;"> match</span><span style="color: #C0CAF5;"> alu_op</span><span style="color: #89DDFF;"> {</span></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // other cases</span></span> <span class="giallo-l"><span style="color: #C0CAF5;"> _</span><span style="color: #89DDFF;"> =&gt;</span><span style="color: #FF9E64;"> 0b000000</span><span style="color: #89DDFF;">,</span></span> <span class="giallo-l"><span style="color: #89DDFF;"> };</span></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // Then use an helper and pass forward the allocated physical registers</span></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // values.</span></span> <span class="giallo-l"><span style="color: #C0CAF5;"> sink</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">put4</span><span style="color: #89DDFF;">(</span><span style="color: #7AA2F7;">enc_arith_rrr</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">top11</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> bit15_10</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> rd</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> rn</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> rm</span><span style="color: #89DDFF;">));</span></span> <span class="giallo-l"><span style="color: #89DDFF;">}</span></span></code></pre> <p>And what’s this <code>enc_arith_rrr</code> doing, then?</p> <pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="rust"><span class="giallo-l"><span style="color: #89DDFF;">fn</span><span style="color: #7AA2F7;"> enc_arith_rrr</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">bits_31_21</span><span style="color: #89DDFF;">:</span><span style="color: #C0CAF5;"> u32</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> bits_15_10</span><span style="color: #89DDFF;">:</span><span style="color: #C0CAF5;"> u32</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> rd</span><span style="color: #89DDFF;">:</span><span style="color: #C0CAF5;"> Writable</span><span style="color: #89DDFF;">&lt;</span><span style="color: #C0CAF5;">Reg</span><span style="color: #89DDFF;">&gt;,</span><span style="color: #C0CAF5;"> rn</span><span style="color: #89DDFF;">:</span><span style="color: #C0CAF5;"> Reg</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> rm</span><span style="color: #89DDFF;">:</span><span style="color: #C0CAF5;"> Reg</span><span style="color: #89DDFF;">) -&gt;</span><span style="color: #C0CAF5;"> u32</span><span style="color: #89DDFF;"> {</span></span> <span class="giallo-l"><span style="color: #89DDFF;"> (</span><span style="color: #C0CAF5;">bits_31_21</span><span style="color: #BB9AF7;"> &lt;&lt;</span><span style="color: #FF9E64;"> 21</span><span style="color: #89DDFF;">)</span></span> <span class="giallo-l"><span style="color: #BB9AF7;"> |</span><span style="color: #89DDFF;"> (</span><span style="color: #C0CAF5;">bits_15_10</span><span style="color: #BB9AF7;"> &lt;&lt;</span><span style="color: #FF9E64;"> 10</span><span style="color: #89DDFF;">)</span></span> <span class="giallo-l"><span style="color: #BB9AF7;"> |</span><span style="color: #7AA2F7;"> machreg_to_gpr</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">rd</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">to_reg</span><span style="color: #89DDFF;">())</span></span> <span class="giallo-l"><span style="color: #BB9AF7;"> |</span><span style="color: #89DDFF;"> (</span><span style="color: #7AA2F7;">machreg_to_gpr</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">rn</span><span style="color: #89DDFF;">)</span><span style="color: #BB9AF7;"> &lt;&lt;</span><span style="color: #FF9E64;"> 5</span><span style="color: #89DDFF;">)</span></span> <span class="giallo-l"><span style="color: #BB9AF7;"> |</span><span style="color: #89DDFF;"> (</span><span style="color: #7AA2F7;">machreg_to_gpr</span><span style="color: #89DDFF;">(</span><span style="color: #C0CAF5;">rm</span><span style="color: #89DDFF;">)</span><span style="color: #BB9AF7;"> &lt;&lt;</span><span style="color: #FF9E64;"> 16</span><span style="color: #89DDFF;">)</span></span> <span class="giallo-l"><span style="color: #89DDFF;">}</span></span></code></pre> <p>Encoding the instruction parts (operands, register mentions) is a lot of bit twiddling and fun. We do so for each VCode instruction, until we’ve generated the whole function’s body. If you remember correctly, at this point register allocation may have added some spills/reloads/move instructions. From the codegen’s point of view, these are just regular instructions with precomputed operands (either real registers, or memory operands involving the stack pointer), so they’re not treated particularly and they’re just generated the same way other VCode instructions are.</p> <p>More work is done by the codegen backend then, to optimize blocks placement, compute final branch offsets, etc. If you’re interested by this, I strongly encourage you to go read <a rel="noopener noreferrer external" target="_blank" href="https://cfallin.org/blog/2021/01/22/cranelift-isel-2/">this blog post</a> by Chris Fallin. After this, we’re finally done: we’ve produced a code buffer, as well as external relocations (to other functions, memory addresses, etc.) for a single function. The code generator’s task is complete: the final steps consist in linking and, optionally, producing an executable binary.</p> <h2 id="mission-accomplished"><a class="zola-anchor" href="#mission-accomplished" aria-label="Anchor link for: mission-accomplished">🔗</a>Mission accomplished!</h2> <p>So, we’re done for today! Thanks for reading this far, hope it has been a useful and pleasant read to you! Feel free to reach out to me on the <a rel="noopener noreferrer external" target="_blank" href="https://twitter.com/bnjbvr">twitterz</a> if you have additional remarks/questions, and to go contribute on <a rel="noopener noreferrer external" target="_blank" href="https://github.com/bytecodealliance/wasmtime">Wasmtime/Cranelift</a> if this sort of things is interesting to you 😇. Until next time, take care of yourselves!</p> <p>Thanks to <a rel="noopener noreferrer external" target="_blank" href="https://cfallin.org">Chris Fallin</a> for reading and suggesting improvements to this blog post.</p> <section class="footnotes"> <ol class="footnotes-list"> <li id="fn-2"> <p>Really, Rust <em>is</em> the DSL. It was Python code before, that had the advantage to be faster to update. Yet it was doing a lot of magic behind the curtain, which wasn’t very friendly for new people trying to learn and use Cranelift. Despite a statically typed language helping for exploration through tooling, this meta-language is to partially disappear in the long run, see Chris’ <a rel="noopener noreferrer external" target="_blank" href="https://cfallin.org/blog/2020/09/18/cranelift-isel-1/">blog post</a> on this topic. <a href="#fr-2-1">↩</a></p> </li> <li id="fn-4"> <p>Aarch64 connoisseurs may notice that there are other ways to encode an addition. Say, if one of the input operands was the result of a bit shift instruction by an immediate value, then it’s possible to <em>embed</em> the shift within the add, so we end up with fewer machine instructions (and lower the register pressure). This other possible encoding is sufficiently different in terms of register allocation and code generation that it justifies having its own VCode instruction. <code>AluRRR</code> is simpler in the sense that it’s only concerned with register inputs and outputs, thus a perfect example for this post. <a href="#fr-4-1">↩</a></p> </li> <li id="fn-3"> <p>What’s an integer overflow for signed integer division? Consider an integer value represented on <code>N</code> bits. If you try to divide the smallest integer value <code>-2**N</code> by <code>-1</code>, it should return <code>2**N</code>, but this is out of range, since the biggest signed integer value we can represent on <code>N</code> bits is <code>(2**N) - 1</code>! So this will overflow and be set to <code>-2**N</code>, which is the initial value, but not the correct result. Good luck debugging this without a software trap! <a href="#fr-3-1">↩</a></p> </li> <li id="fn-5"> <p>Register moves may be introduced because a successor block (in the control flow graph) expects a given virtual register to live in a particular real register, or because a particular instruction requires a virtual register to be allocated to a <em>fixed</em> real register that’s busy: regalloc can then temporarily divert the busy register into another unused register. <a href="#fr-5-1">↩</a></p> </li> <li id="fn-8"> <p>The <code>c</code> in <code>rcx</code> actually stands for <code>count</code>; this is a property inherited from former CPU designs. <a href="#fr-8-1">↩</a></p> </li> <li id="fn-7"> <p>Unless this move carries sign- or zero-extending semantics, which is the case for e.g. x86’s 32-bits <code>mov</code> instructions on a 64-bits architecture. <a href="#fr-7-1">↩</a></p> </li> <li id="fn-6"> <p>Relocations are placeholders for information we don’t have <em>yet</em> access to. For instance, when we’re generating jump instructions, the jump targets offsets are not determined yet. So we record where the jump instruction is in the code stream, as well as which control flow block it should jump into, so we can <em>patch it</em> later when the final offsets are known: that’s the content of our relocation. <a href="#fr-6-1">↩</a></p> </li> </ol> </section> 2020-11-12T18:49:42+00:00 2020-11-12T18:49:42+00:00 Benjamin Bouvier https://bouvier.cc/tech/botzilla/ <p>In this post I reflect on my personal history of writing chat bots, and then present a panel of features that the bot has, some user-facing ones, some others that embody what I esteem to be a sane, well-behaved Matrix bot.</p> <span id="continue-reading"></span> <p>Over the last year, Mozilla has decided to shut down the IRC network and replace it with a more modern platform. To my greatest delight, <a rel="noopener noreferrer external" target="_blank" href="http://exple.tive.org/blarg/2019/12/19/over-the-line/">the Matrix ecosystem has been selected</a> among all the possible replacements. For those who might not know Matrix, it’s a modern, decentralized protocol, using plain HTTP JSON-formatted endpoints, well-documented, and it implements both features that are common in recent messaging systems (e.g. file attachments, message edits and deletions), as well as those needed to handle large groups (e.g. moderation tools, private rooms, invite-only rooms).</p> <h2 id="but-first-some-history"><a class="zola-anchor" href="#but-first-some-history" aria-label="Anchor link for: but-first-some-history">🔗</a>but first, some history</h2> <p>Back in 2014 when I was an intern at Mozilla, I made a silly IRC JavaScript bot that would quote the @horsejs twitter account, when asked to do so. Then a few other useless features were added: “karma” tracking <sup class="footnote-reference" id="fr-1-1"><a href="#fn-1">1</a></sup>, being a karma guardian angel (lowering the karma of people lowering the karma of some predefined people), keeping track of contextless quotes from misc people…</p> <p>Over time, it slowly transformed into an IRC bot <em>framework</em>, with <em>modules</em> you could attach and configure at startup, setting which rooms the bot would join, what should be the cooldowns for message sending (more on this later), and so much more! Hence it was renamed <em>meta-bot</em>.</p> <h3 id="an-aside-on-the-morality-of-bots"><a class="zola-anchor" href="#an-aside-on-the-morality-of-bots" aria-label="Anchor link for: an-aside-on-the-morality-of-bots">🔗</a>an aside on the morality of bots</h3> <p>I find making bots a fun activity, since once you’ve passed the step of connecting and sending messages, the rest is mostly easy (<em>cough cough regular expressions cough cough</em>) and creative work. And it’s unfortunately easy to be reckless too.</p> <p>At this time, I never considered the potentially bad effects of quoting text from a random source, viz. fetching tweets from the @horsejs account. If the source would return a message that was inconsiderate, rude, or even worse, aggressive, then the bot would replicate this behavior. It is a real issue because although the bot doesn’t think by itself and doesn’t <em>mean</em> any harm, its programmers can do better, and they should try to avoid these issues at all costs. A chat bot replicates the culture of the engineers who made it on one hand, but also contributes to propagating this culture in the chat rooms it participates in, <em>normalizing</em> it to the chat participants.</p> <p>My bot happened to be well-behaved most of the time… until one time where it was not. After noticing the incident and expressing my deepest apologies, I deactivated the module and went through the whole list of modules, to make sure none could cause any harm, in any possible way. I should have known better in the first place! I am really not trying to signal my own virtue, since I failed in a way that should have been predictable. I hope by writing this that other people may reflect about the actions of their bots as well, in case they could be misbehaving like this.</p> <h3 id="the-former-fleet-of-mozilla-bots"><a class="zola-anchor" href="#the-former-fleet-of-mozilla-bots" aria-label="Anchor link for: the-former-fleet-of-mozilla-bots">🔗</a>the former fleet of mozilla bots</h3> <p>There were a few other useful IRC bots (of which I wasn’t the author) hanging out in the Mozilla IRC rooms, notably <a rel="noopener noreferrer external" target="_blank" href="https://github.com/globau/firebot">Firebot</a> and <a rel="noopener noreferrer external" target="_blank" href="https://wiki.mozilla.org/Mrgiggles">mrggigles</a>. The latter probably started as a joke too, to enumerate puns from a list in the JavaScript channel. Then it outgrew its responsibilities by helping with a handful of requests: who can review this or this file in Mozilla’s source code? what’s the status of the continuous integration trees? can this particular C++ function used in Gecko cause a garbage collection?</p> <p>When we moved over to Matrix, the bots unfortunately became outdated, since the communication protocol (IRC) they were using was different. We could have ported them to the Matrix protocol, but the Not-Invented-Here syndrom was strong with this one: I’ve been making bots for a while, and I was personally interested in the Matrix protocol and trying out the JS facilities offered by the Matrix ecosystem.</p> <h2 id="botzilla-features"><a class="zola-anchor" href="#botzilla-features" aria-label="Anchor link for: botzilla-features">🔗</a><strong>Botzilla features</strong></h2> <p>So I’ve decided to write <a rel="noopener noreferrer external" target="_blank" href="https://github.com/bnjbvr/botzilla">Botzilla</a>, a successor in spirit to <em>meta-bot</em> and <em>mrgiggles</em>, written in TypeScript. This is a very <em>unofficial</em> bot, tailored for Mozilla’s needs but probably useful in other contexts. I’ve worked on it informally as a side-project, on my <em>copious</em> spare time. Crafting tools that show useful to other people has been sufficient a reward to motivate me to work on it, so it’s been quite fun!</p> <p><img src="https://bouvier.cc/tech/botzilla/logo.png" alt="Botzilla’s logo" /></p> <p><em>Botzilla’s logo, courtesy of <a rel="noopener noreferrer external" target="_blank" href="https://nical.github.io/index.html">Nical</a></em></p> <p>Let’s take a look at all the features that the bot offers, at this point.</p> <h3 id="uuid-generate-unique-ids"><a class="zola-anchor" href="#uuid-generate-unique-ids" aria-label="Anchor link for: uuid-generate-unique-ids">🔗</a><strong>uuid: Generate unique IDs</strong></h3> <p>This was a feature of Firebot, and easy enough to replicate, so this was the test feature for the Matrix bot. When saying <code>!uuid</code>, the bot will automatically generate a unique id (using uuid v4), guaranteed GMO-free and usable in any context that would require it. This was the first module, designed to test the framework.</p> <p><img src="https://bouvier.cc/tech/botzilla/uuid.png" alt="Demo of uuid" /></p> <h3 id="treestatus-inform-about-ci-tree-status"><a class="zola-anchor" href="#treestatus-inform-about-ci-tree-status" aria-label="Anchor link for: treestatus-inform-about-ci-tree-status">🔗</a><strong>treestatus: Inform about CI tree status</strong></h3> <p>Mozilla developers tend to interact a lot with the continuous integration trees, because code is sometimes landed, sometimes backed out (sorry/thank you sheriffs!), sometimes merged across branches. This leads to the integration trees being closed. Before we had the feature to automatically land patch stacks when the trees reopened, it was useful to be able to get the open/close status of a tree. Asking <code>!treestatus</code> will answer with a list of the status of some <em>common</em> trees. It is also possible to request the status of a particular tree, e.g. for the “mozilla-central” tree, by asking <code>!treestatus mozilla-central</code> (or just <code>central</code>, as a handy shortcut).</p> <p><img src="https://bouvier.cc/tech/botzilla/treestatus.png" alt="Demo of treestatus" /></p> <h3 id="expand-bug-status"><a class="zola-anchor" href="#expand-bug-status" aria-label="Anchor link for: expand-bug-status">🔗</a><strong>Expand bug status</strong></h3> <p>If you have ever interacted with Mozilla’s code, there’s chances that you’ve used Bugzilla, and mentioned bug numbers in conversations. The bot caches any message containing <code>bug XXX</code> and will respond with a link to this bug, the nickname of the person assigned to this bug if there’s one, and the summary of this bug, if it’s public. This is by far the most used and useful module, since it doesn’t require a special incantation, but will react automatically to a lot of messages written with no particular intent (see below where it’s explained how to not be spammy, though).</p> <p><img src="https://bouvier.cc/tech/botzilla/expand-bug.png" alt="Demo of expand-bug" /></p> <h3 id="who-can-review-x"><a class="zola-anchor" href="#who-can-review-x" aria-label="Anchor link for: who-can-review-x">🔗</a>Who Can Review X?</h3> <p>This was a very nice feature that mrgiggles had: ask for potential reviewers for a particular file in the Gecko source tree and get a list of most recent reviewers. Botzilla replicates this, when seeing the trigger: <code>who can review js/src/wasm/WasmJS.cpp?</code>. The list of potential reviewers is extracted from Mercurial logs, looking for the N last reviewers of this particular file.</p> <p>As a bonus, there’s no need to pass the full path to the file, if the file’s name is unique in the tree’s source code. Botzilla will trigger a search in Searchfox, and will use the unique name in the result list, if there’s such a unique result. The previous example thus can be shortened to <code>who can review WasmJS.cpp?</code> since the file’s name is unique in the whole code base.</p> <p><img src="https://bouvier.cc/tech/botzilla/who-can-review.png" alt="Demo of who can review" /></p> <h3 id="github-gitlab-issues-p-m-rs"><a class="zola-anchor" href="#github-gitlab-issues-p-m-rs" aria-label="Anchor link for: github-gitlab-issues-p-m-rs">🔗</a>{Github,Gitlab} {issues,{P,M}Rs}</h3> <p>It is possible for a room administrator to “connect” a given Matrix room to a Github repository. Later on, any mention of issues or pull requests by their number, e.g. <code>#1234</code>, will make Botzilla react with the summary and a link to the issue/PR at stake.</p> <p>This also works for Gitlab repositories, with slight differences: the administrator has to precise what’s the root URL of the Gitlab instance (since Gitlab can be selfhosted). Issues are caught when numbers follows a <code>#</code> sign, while merge requests are caught when the numbers follow a <code>!</code> sign.</p> <p><img src="https://bouvier.cc/tech/botzilla/gitlab.png" alt="Demo of gitlab" /></p> <h3 id="tweet-toot-post-on-twitter-mastodon"><a class="zola-anchor" href="#tweet-toot-post-on-twitter-mastodon" aria-label="Anchor link for: tweet-toot-post-on-twitter-mastodon">🔗</a>!tweet/!toot: Post on Twitter/Mastodon</h3> <p>An administrator can configure a room to tie it up to a Twitter (respectively Mastodon) user account, using API tokens. Then, any person with an administrative role can post messages with <code>!tweet something shocking for the bird site</code>(respectively <code>!toot something heartful for the mammoth site</code>). This makes it possible to allow other people to post on these social networks without the need to give them the account’s password.</p> <p>Unfortunately, the Twitter module hasn’t ever been tested, since when I’ve tried to create a developer account, Twitter accepted it after a few days but then never displayed the API tokens on the interface. The support also never answered when I asked for help. Thankfully Mastodon can be self-hosted and thus it is easier to test. I’m happy to report that it works quite well!</p> <h3 id="confession-and-histoire"><a class="zola-anchor" href="#confession-and-histoire" aria-label="Anchor link for: confession-and-histoire">🔗</a><code>confession</code> and histoire</h3> <p>It is quite common in teams to set up regular standup meetings, where everyone in the team announces what they’ve been working on in the last few days or week. It also strikes me as important for personal recognition, including towards management, to be able to <em>show off</em> (just a bit!) what you’ve accomplished recently, and to remember this when times are harder (see also <a rel="noopener noreferrer external" target="_blank" href="https://jvns.ca/blog/brag-documents/">Julia Evans’ blog post on the topic</a>).</p> <p>There’s a Botzilla module for this. Every time someone starts a message with <code>confession:</code>, then everything after the colon will be saved in a database (…wait for it!). Then, all the confessions are displayed on the <a rel="noopener noreferrer external" target="_blank" href="https://robotzilla.github.io/histoire">Histoire</a> <sup class="footnote-reference" id="fr-2-1"><a href="#fn-2">2</a></sup> website, with one message feed per user. Note it is possible to send confessions privately to Botzilla (that doesn’t affect the frontend though, which is open and public to all!), or in a public channel. Public channels somehow equate to team members, so channels also get their own pages on the frontend.</p> <p><img src="https://bouvier.cc/tech/botzilla/confession.png" alt="Demo of confession" /></p> <p><img src="https://bouvier.cc/tech/botzilla/histoire.png" alt="Screenshot of Histoire" /></p> <p>Now the fun/cursed part is how all of this <em>works</em>. This was implemented in <em>mrgiggles</em>, and I liked it a lot, since it required no kind of backend or frontend server. How so? By (ab)using Github files as the database and Github pages as the frontend. Sending a confession will trigger a request to a Github endpoint to find a database file segregated by time, then it will trigger another request to create/modify it with the content of the confession. The frontend then uses other requests to public Github APIs to read the confessions before dynamically rendering those. Astute readers will notice that under a lot of confession activity, the bot would be a bit slowed down by Github’s API use rates. In this case, there’s some exponential backoff behavior before trying to re-send unsaved confessions to Github. Overall it works great, and API limitation rates have never quite been a problem.</p> <h2 id="intrinsic-features-they-re-good-bots-bront"><a class="zola-anchor" href="#intrinsic-features-they-re-good-bots-bront" aria-label="Anchor link for: intrinsic-features-they-re-good-bots-bront">🔗</a><strong>Intrinsic features: they’re good bots, bront</strong></h2> <p>In addition to all the user-facing features, the bot has a few other interesting attributes that are more relevant to consider from a framework point of view. Hopefully some of these ideas can be useful for other bot authors!</p> <h3 id="join-all-the-rooms"><a class="zola-anchor" href="#join-all-the-rooms" aria-label="Anchor link for: join-all-the-rooms">🔗</a><strong>Join All The Rooms!</strong></h3> <p>Every time the bot is invited to a channel, be it public or private, it will join the channel, making it easy to use in general. It was implemented for free by <a rel="noopener noreferrer external" target="_blank" href="https://github.com/turt2live/matrix-js-bot-sdk">the JS framework I’ve been using</a>, and it is a definitive improvement over the IRC version of the bot.</p> <p>Sometimes Matrix rooms are upgraded to a new version of the room. The bot will try to join the upgraded room if it can, keeping all its room settings intact during the transition.</p> <h3 id="thou-shalt-not-spam"><a class="zola-anchor" href="#thou-shalt-not-spam" aria-label="Anchor link for: thou-shalt-not-spam">🔗</a><strong>Thou shalt not spam</strong></h3> <p>To avoid spamming the channel, especially for modules that are <em>reactions</em> to other messages (think: bug numbers, issues/pull requests mentions), the bot has had to learn how to keep quiet. There are two rules triggering the quieting behavior:</p> <ul> <li>if the bot has already reacted less than N minutes ago (where N is a configurable amount) in the same room,</li> <li>or if it has already reacted to some entity in a message, and there’s been fewer than M messages in between the last reaction and the last message mentioning the same entity in the same room (M is also configurable)</li> </ul> <p>If any of these two criteria is met, then the bot will keep quiet and it will not react to another similar message. The combination of these two has proven over time to be quite solid in my experience, based on observing the bot’s behavior and public reactions to its behavior.</p> <p>Some similar mechanism is used for the <em>confession</em> module: on a <em>first</em> confession, the bot will answer with a message saying it has seen the confession, including a link to where it is going to be posted, and will add an emoji “eyes” reaction to the message. Posting this long form message could be quite spammy, if there’s a lot of confessions around the same time. Under the same criteria, it will just react with an “eyes” emoji to other confessions. Later on, it’ll resend the full message, once both criterias aren’t blocking it from doing so.</p> <h3 id="decentralized-administration-self-service"><a class="zola-anchor" href="#decentralized-administration-self-service" aria-label="Anchor link for: decentralized-administration-self-service">🔗</a><strong>Decentralized administration self-service</strong></h3> <p>The bot can be administrated, by discussing with it using the <code>!admin</code> command. This can happen in both a private conversation with it, or in public channels, yet it is recommended to do so in private channels. To confirm that an admin action has succeeded, it’ll use the thumbs-up emoji on the message doing the particular action.</p> <p>To have a single administrator for the bot would be quite the burden, and it is not resilient to people switching roles, leaving the company, etc. Normally you’d solve this by implementing your own access control lists. Fortunately, Matrix already has a concept of <em>power levels</em> that assigns roles to users, among which there are the administrator and moderator roles.</p> <p>The bot will rely on this to decide to which requests it will answer. Somebody marked as an administrator or a moderator of a room can administrate Botzilla in this particular room, using <a rel="noopener noreferrer external" target="_blank" href="https://github.com/bnjbvr/botzilla#admin">the <code>!admin</code> commands</a>. There’s still a super-admin role, that must be defined in the configuration, in case things go awry. While administrators only have power over the current room, a super-admin can use its super-powers to change anything in any room. This decentralization of the administrative roles makes it easy to have different settings for different rooms, and to rely a bit less on single individuals.</p> <h3 id="key-value-store"><a class="zola-anchor" href="#key-value-store" aria-label="Anchor link for: key-value-store">🔗</a><strong>Key-value store</strong></h3> <p>In general, the bot contains a key-value store implemented in an sqlite database, making it easy to migrate and add context that’s preserved across restarts of the bot. This is used to store private information like user repository information and settings for most rooms. Conceptually, each pair of room and module has its own key-value store, so that there’s no risk of confusion between different rooms and modules. There’s also a key-value per-module store that’s applicable to all the rooms, to represent global settings. If there’s some non-global (per room) settings for a room, these are preferred over the global settings.</p> <h3 id="self-documentation"><a class="zola-anchor" href="#self-documentation" aria-label="Anchor link for: self-documentation">🔗</a>Self-documentation</h3> <p>Each chat module is implemented as a ECMAScript module and must export an help string along the main reaction function. This is then captured and aggregated as part of an <code>!help</code> command, that can be used to request help about usage of the bot. The main help message will display the list of all the enabled modules, and help about a specific module may be queried with e.g. <code>!help uuid</code>.</p> <h2 id="future-work-and-conclusion"><a class="zola-anchor" href="#future-work-and-conclusion" aria-label="Anchor link for: future-work-and-conclusion">🔗</a>Future work and conclusion</h2> <p>If I were to start again, I’d do a few things differently:</p> <ul> <li>now that the Rust ecosystem around the Matrix platform has matured a bit, I’d probably write this bot in Rust. Starting from JavaScript and moving to TypeScript has helped me catch a few static issues. I’d expect moving to Rust would help handling Matrix events faster, provide end-to-end encryption support for free, and be quite pleasant to use in general thanks to the awesome Rust tooling.</li> <li>use a real single-page app framework for the Histoire website. Maybe? I mean I’m a big fan of VanillaJS, but using it means re-creating your own Web framework like thing to make it nice and productive to use.</li> <li>despite being a fun hack, using Github as a backend has algorithmic limitations, that can make the web app sluggish. In particular, a combined feed for N users on M <em>eras</em> (think: periods) will trigger NxM Github API requests. Using a plain database with a plain API would probably be simpler at this point. This is mitigated with an in-memory cache so only the first time all the requests happen, but crafting my own requests would be more expressive and efficient, and allow for more features too (like displaying the list of rooms on the start view).</li> <li>provide a (better) commands parser. Regular expressions in this context are a bit feeble and limited. Also right now each module could in theory reuse the same command triggers as another one, etc.</li> <li>implement the chat modules in WebAssembly :-) In fact, I think there’s a whole business model which would consist in having the bot framework including a wasm VM, and interacting with different communication platforms (not restricted to Matrix). Developers in such a bot platform could choose which source language to use for developing their own modules. It ought to be possible to define a clear, restricted, WASI-like capabilities-based interface that gets passed to each chat module. In such a sandboxed environment, the responsibility for hosting the bot’s code is decoupled from the responsibility of writing modules. So a company could make the platform available, and paying users would develop the modules and host them. Imagine <code>git push</code>ing your chat modules and they get compiled to wasm and deployed on the fly. But I digress! (Please do not forget to credit me with a large $$$ envelope/a nice piece of swag if implementing this <em>at least</em> multi-billion dollars idea.)</li> </ul> <p>I’d like to finish by thanking the authors of the previous Mozilla bots, namely <strong>sfink</strong> and <strong>glob</strong>: your puppets have been incredible sources of inspiration. Also huge thanks to the people hanging in the <code>matrix-bot-sdk</code> <a rel="noopener noreferrer external" target="_blank" href="https://matrix.to/#/!matrix-bot-sdk:t2bot.io">chat room</a>, who’ve answered questions and provided help in a few occasions.</p> <p>I hope you liked this presentation of Botzilla and its features! Of course, all the code is free and open-source, including <a rel="noopener noreferrer external" target="_blank" href="https://github.com/bnjbvr/botzilla">the bot</a> as well as <a rel="noopener noreferrer external" target="_blank" href="https://github.com/robotzilla/histoire">the histoire frontend</a>. At this point it is addressing most of the needs I had, so I don’t have immediate plans to extend it further. I’d happily take contributions, though, so feel free to chime in if you’d like to implement anything! It’s also a breeze to run on any machine, thanks to Docker-based deployment. Have fun with it!</p> <section class="footnotes"> <ol class="footnotes-list"> <li id="fn-1"> <p>Karma is an IRC idiosyncrasy, in which users rate up and down other users using their nickname suffixed with ++ or –. Karma tracking consists in keeping scores and displaying those. <a href="#fr-1-1">↩</a></p> </li> <li id="fn-2"> <p>Histoire is the French for “history” and “story”. Inherited from Steve Fink’s very own mrgiggles :-) <a href="#fr-2-1">↩</a></p> </li> </ol> </section> 2019-10-10T18:00:42+00:00 2019-10-10T18:00:42+00:00 Benjamin Bouvier https://bouvier.cc/tech/github-workflow/ <p>Since I’ve been working on a <a rel="noopener noreferrer external" target="_blank" href="https://github.com/CraneStation/Cranelift">Github project</a> for a while now, I thought now would be a good time to gather ways to make it easier to work with Github pull requests (PRs). In particular, it’s easy to drown yourself in the incoming flow of Github emails.</p> <span id="continue-reading"></span> <p>This post is for you if:</p> <ul> <li>you get lost in tracking which pull requests need attention from you, be it either review requests or just mentions.</li> <li>you would like to strike a better work-life balance when it gets to Github notifications.</li> <li>you would like to filter Github email notifications in smarter ways.</li> </ul> <p>Here are a few tricks I’ve collected over the years that make it easier to deal with a few things, focusing on Github notifications and emails, since they were the largest issue for me. This is not an exhaustive list of all the nice features Github has, or all the WebExtensions that could help with Github: it is a few things that work for me and are worth sharing. Note that I go from the most mundane to the more specific advices here.</p> <h3 id="notifications-dashboard"><a class="zola-anchor" href="#notifications-dashboard" aria-label="Anchor link for: notifications-dashboard">🔗</a>Notifications dashboard</h3> <p>If you’re working on several projects, Github can end up sending you too many email notifications.</p> <p>It’s possible to disable some kinds of notifications entirely <a rel="noopener noreferrer external" target="_blank" href="https://github.com/settings/notifications">in the settings</a>, but that’s too radical for my needs.</p> <p>However, Github has a <a rel="noopener noreferrer external" target="_blank" href="https://github.com/notifications">notification dashboard</a> that displays all the activity related to repositories you’re watching or issues/pull-requests you’re involved in. It’s easy to dismiss all the notifications of all projects at once, or per project. There’s a tab on the left that allows to select more precisely your level of involvement in the issue: did you participate in it? You can also save some notifications for later, so they’re not deleted once you’ve clicked them; they’ll appear under the “Saved for later” tab — I just discovered this!</p> <p>Note that Github may also send these notifications by email, if you’ve decided to do so. In this case, I’d strongly recommend allowing the downloads of images in Github emails. Despite the bad effect on your privacy this might have by allowing user tracking, it will also synchronize the notifications’ read state, which is nice.</p> <p><img src="https://bouvier.cc/tech/github-workflow/github-notification-dashboard.png" alt="Notification dashboard count" /></p> <p>See how I am totally in control of my notifications? Truth is, I don’t need notifications in general, because I’m usually more interested in reviews I need to receive and give.</p> <h3 id="pull-requests-dashboard"><a class="zola-anchor" href="#pull-requests-dashboard" aria-label="Anchor link for: pull-requests-dashboard">🔗</a>Pull requests dashboard</h3> <p>Github allows to assign a reviewer to a pull request. At Mozilla, we require a formal review for each change in the code base, unless it’s really not meaningful (like, removing trailing whitespaces). Even documentation and tests changes may require a review, depending on the rules of the code module you’re working on.</p> <p>It is very common that a pull request is received with requests for additional changes. In this case, it is important to explicitly <strong>re-request a review</strong>, otherwise this breaks all the review tracking Github proposes.</p> <p>Now Github has two interesting pages for this:</p> <ul> <li>a list of all the <a rel="noopener noreferrer external" target="_blank" href="https://github.com/pulls">pull requests you have created</a> and that aren’t closed, so you can assign reviewers and follow PR’s progress over time;</li> <li>a list of all the <a rel="noopener noreferrer external" target="_blank" href="https://github.com/pulls/review-requested">pull requests you have been assigned to as a reviewer</a>.</li> </ul> <h3 id="navigating-files-quicker-addon"><a class="zola-anchor" href="#navigating-files-quicker-addon" aria-label="Anchor link for: navigating-files-quicker-addon">🔗</a>Navigating files quicker (addon)</h3> <p>When I know my way around a project, I’ll frequently need to see the content of a particular file or directory, that might be a few directories deep. On Github, this means going to the files view, clicking once per directory (at most), and finding the file I want.</p> <p>The pull request view doesn’t show the directory hierarchy and which files of which directory have been touched, which is a light inconvenience too.</p> <p>Good news, everyone! There is one WebExtension called <a rel="noopener noreferrer external" target="_blank" href="https://www.octotree.io/">Octotree</a> that adds a directory view within a panel to the left of Github’s UI. By default, it’s folded and doesn’t take much space; you need to hover it with the mouse to make it appear. On pull requests, it will show files that have been modified with the diff summary for each file. Note the website shows features from the PRO version, but there’s a free version that addresses the needs detailed above.</p> <p>This is an example of the Octotree panel on our project’s repository:</p> <p><img src="https://bouvier.cc/tech/github-workflow/github-octotree.png" alt="Octotree example" /></p> <p>To be honest, I haven’t investigated using the search bar, which could be quite handy for this too, especially thanks to keyboard shortcuts.</p> <h3 id="dealing-with-work-and-personal-projects"><a class="zola-anchor" href="#dealing-with-work-and-personal-projects" aria-label="Anchor link for: dealing-with-work-and-personal-projects">🔗</a>Dealing with work and personal projects</h3> <p>If you’re using Github for personal and work related projects, you might have been bothered by work emails coming into your personal mailbox. That has happened to me in the past, causing some unnecessary mental load over the weekend and unnecessarily breaking the state of relaxation.</p> <p>Fortunately, Github allows you to <a rel="noopener noreferrer external" target="_blank" href="https://github.com/settings/notifications#organization_routing">redirect emails from a particular Github Organization to a specific email address</a>. Of course, this only works when the repository is owned by an organization and you’re part of this organization.</p> <p>I’m lucky to work on such projects at the moment. It’s not a silver bullet though, because some projects are sometimes owned by personal accounts, making this trick useless. As far as I know, there are no good solutions in this case.</p> <h3 id="email-filters"><a class="zola-anchor" href="#email-filters" aria-label="Anchor link for: email-filters">🔗</a>Email filters</h3> <p>The biggest remaining offender certainly is Github emails, in general. Fortunately, Github has made it easy to filter them. I’ll mention examples in the Gmail email client, since that’s what we’re using at work, but these apply to any other modern email client too.</p> <h4 id="filter-by-project"><a class="zola-anchor" href="#filter-by-project" aria-label="Anchor link for: filter-by-project">🔗</a>Filter by project</h4> <p>Each email coming from a specific project comes with a mailing <code>list-id</code>, which is a specific header that some email clients know how to interpret. For instance, in Gmail, when you click on the small arrow next to the list of recipients, you’ll see many details about the current email, including, if there’s one, the “mailing-list” id, and a link to automatically create a filter for this mailing-list. That allows you to create a particular directory/tag in which the filter can automatically put all the emails with this id.</p> <h4 id="filter-by-reason"><a class="zola-anchor" href="#filter-by-reason" aria-label="Anchor link for: filter-by-reason">🔗</a>Filter by reason</h4> <p>In addition to filtering by project (and this is where Gmail tags / Thunderbird’s <a rel="noopener noreferrer external" target="_blank" href="https://support.mozilla.org/en-US/kb/using-saved-searches">saved searches</a> truly shine), it’s also possible to infer more information from the Github email notifications, by looking at the list of recipients or custom email headers.</p> <p>Indeed, when there’s a specific reason why an email was sent to you, Github will add a (fake) recipient in the CC field, its address username being the reason why the email was sent to you. For instance, in an email telling me that somebody requested a review from me, the email address <code>review_requested@github.com</code> will appear in the CC list. If you look at the full message, you’ll also see the custom email header <code>X-GitHub-Reason</code> set to <code>review_requested</code>.</p> <p>All the possible reasons are <a rel="noopener noreferrer external" target="_blank" href="https://help.github.com/en/articles/about-email-notifications#filtering-email-notifications">detailed in Github’s documentation</a>.</p> <p>These extra CC email addresses and email headers allow creating very powerful filters that add supplementary tags to an email. For me, they relate directly to the <em>importance</em> of the incoming email: reviews and mentions are usually something I pay very close attention to, and thus they get filtered in a special top-level tag in Gmail.</p> <p>Here’s an example of all the information you might find about a given email in Gmail: in particular, look at the CC list and mailing-list type ids.</p> <p><img src="https://bouvier.cc/tech/github-workflow/github-email-example.png" alt="Notification dashboard count" /></p> <p>Note that Gitlab also adds some <a rel="noopener noreferrer external" target="_blank" href="https://docs.gitlab.com/ee/workflow/notifications.html#email-headers">similar custom headers</a> that can be filtered by some powerful email clients. I won’t go into detail about those.</p> <h3 id="one-more-thing-mozillian-edition"><a class="zola-anchor" href="#one-more-thing-mozillian-edition" aria-label="Anchor link for: one-more-thing-mozillian-edition">🔗</a>One more thing, Mozillian edition</h3> <p>If you’re working on Mozilla code, Gecko and/or external projects, there’s this <a rel="noopener noreferrer external" target="_blank" href="https://github.com/mikeconley/myqonly/">neat addon</a> that Mike Conley made. It will add an icon to the Firefox button bar, showing you the number of requests assigned to you on Github and Phabricator, as well as the number of pending Bugzilla requests.</p> <p>It requires a minimal setup step for Github (filling your username) and Bugzilla (adding a Bugzilla API token), and then it Just Works. It smartly reuses a Phabricator token from the current Firefox Container’s session, if there’s one.</p> <p>You may think that having such a display all the time might provoke anxiety during non-working hours. And you’d be right to think so! So the author of the addon has added a feature to <strong>not</strong> display this information outside working hours, that you can define as you like. Great stuff!</p> <h3 id="that-s-it-folks"><a class="zola-anchor" href="#that-s-it-folks" aria-label="Anchor link for: that-s-it-folks">🔗</a>That’s it, folks!</h3> <p>Thanks for reading this far! I hope this helped you to some extent, allowing you to spend less time in Github and more time doing the actual work. If you have more interesting tips for using Github effectively, feel free to add a comment or ping me on <a rel="noopener noreferrer external" target="_blank" href="https://twitter.com/bnjbvr">twitter</a>!</p> 2018-07-04T18:00:42+00:00 2018-07-04T18:00:42+00:00 Benjamin Bouvier https://bouvier.cc/tech/mozilla-2018-faster-calls-and-anyref/ <p>Since this is the end of the first half-year, I think it is a good time to reflect and show some work I’ve been doing over the last few months, apart from the regular batch of random issues, security bugs, reviews and the fixing of 24 bugs found by our <a rel="noopener noreferrer external" target="_blank" href="https://en.wikipedia.org/wiki/Fuzzing">fuzzers</a>.</p> <span id="continue-reading"></span><h2 id="bug-1319203-make-js-to-webassembly-calls-blazingly-fast"><a class="zola-anchor" href="#bug-1319203-make-js-to-webassembly-calls-blazingly-fast" aria-label="Anchor link for: bug-1319203-make-js-to-webassembly-calls-blazingly-fast">🔗</a>Bug 1319203: Make JS to WebAssembly calls <em>blazingly</em> fast</h2> <p>If we want more WebAssembly (wasm) adoption, there shouldn’t be a big costly barrier between the two universes. That is, calls from one world to the other should be fast. For a very long time, calls from JS to asm.js/WebAssembly have been quite slow in Firefox. In fact, we didn’t optimize them at all. For ease and speed of implementation at the time, asm.js call activations (data structures recording information about the function being currently called in the VM) were very different from the JS ones. This difference indicated some significant structural differences, like the capability to reconstruct call stack information used by <code>Error()</code> stack frames, or just tracing the stack for garbage collection purposes. After putting a lot of hard work into refactoring and low-level changes over the last year, Spidermonkey was finally ripe for an optimization.</p> <p>When we call from JS to asm.js/wasm, the call passes through C++, does a bunch of work and then calls into a piece of glue code directly written in assembly: the <em>interpreter entry stub</em>. This stub is quite small: it just copies out the C++ arguments into the right places the wasm function being called expects, sets up some small machine state, calls into the function, then does error checking and eventually returns to the C++ caller. The critical part is JIT compilation. JIT compilation means that the code is compiled to machine code by the just-in-time compiler, IonMonkey. When a JS function has been JIT-compiled and it calls into wasm, then the caller would have to go back to C++ first, before the control flow is redirected to WebAssembly.</p> <p><img src="https://bouvier.cc/tech/mozilla-2018-faster-calls-and-anyref/2018-07-interpreter-stub.png" alt="Diagram showing interpreter entry stub" /></p> <p>Starting with Firefox 60, the JIT compiler makes no distinctions between calling a JavaScript function or a WebAssembly function, meaning it uses the same call optimizations for both kinds of function. A new piece of glue code, the <em>JIT entry stub</em>, is generated for each exported function: it converts and unboxes the arguments read from the JIT-compiled JS caller into the right primitive types as expressed in the wasm function’s signature, sets up some machine registers, calls into the wasm function being called and then converts the result into a format the JS caller will understand.</p> <p><img src="https://bouvier.cc/tech/mozilla-2018-faster-calls-and-anyref/2018-07-jit-stub.png" alt="Diagram showing JIT entry stub" /></p> <p>As you can see, the C++ step that was originally required to call wasm from JS has been completely eliminated!</p> <p>This resulted in massive speedups over a variety of different situations: when a wasm function is directly / indirectly / polymorphically called, or used as a getter/setter, or called by <code>Function.prototype.call/apply</code>, when the call is missing required arguments, etc. Here’s a brief summary of the results, but there might be a full-blown blog post about these optimizations coming on <a rel="noopener noreferrer external" target="_blank" href="https://hacks.mozilla.org/">Mozilla Hacks</a> at some point in the future. (calling 1 billion times into very simple functions, lower is better)</p> <p><img src="https://bouvier.cc/tech/mozilla-2018-faster-calls-and-anyref/2018-07-wasm-calls.png" alt="Charts showing evolution of performance" /></p> <p>This work is not entirely done yet: we can still even better optimize in the case of a function call from JS when the called wasm function is definitely known to be a unique wasm target; see the <a rel="noopener noreferrer external" target="_blank" href="https://bugzilla.mozilla.org/show_bug.cgi?id=1437065">tracking bug</a>.</p> <h2 id="bug-1422043-lazy-entry-stub-generation"><a class="zola-anchor" href="#bug-1422043-lazy-entry-stub-generation" aria-label="Anchor link for: bug-1422043-lazy-entry-stub-generation">🔗</a>Bug 1422043: Lazy entry stub generation</h2> <p>The previous bug resolution came with an important memory issue: every exported function now generates a rather big chunk of code for the JIT entry, having an impact on the memory occupied by the code itself. This would be fine in most situations where the number of exported functions is generally low. But when the wasm module exports a Table (think of the equivalent of a C++ function table with signature checks), we have to assume that every single function, including those not explicitly exported, needs entry stubs. Indeed, each function can be eventually called through the Table, after calls to <a rel="noopener noreferrer external" target="_blank" href="http://webassembly.github.io/spec/js-api/index.html#dom-table-set">WebAssembly.Table.set</a>. In fact, the existing code already suffered from this because of the interpreter entries, but it had been largely amplified by the much larger JIT entry stubs.</p> <p>To fix this, we’ve decided to lazily generate all the entry stubs for functions exported through a table. That is, if a function is <em>explicitly</em> exported, its stubs will be generated at wasm compile time, but other functions won’t have stubs yet. If a non-exported function is called through a Table, we’ll generate the entry stubs the first time it is called. This involves some fun interactions with our <a rel="noopener noreferrer external" target="_blank" href="https://hacks.mozilla.org/2018/01/making-webassembly-even-faster-firefoxs-new-streaming-and-tiering-compiler/">tiered compilation</a> mechanism, which can compile functions and create new entry stubs in the background while the running thread will generate lazy ones.</p> <p>Not only this fixed the memory regression introduced by bug 1319203, but it actually made the situation even better than the baseline, because we didn’t need to generate those interpreter entries for table-exported functions by default anymore:</p> <p><img src="https://bouvier.cc/tech/mozilla-2018-faster-calls-and-anyref/2018-07-wasm-stubs-memory.png" alt="Charts showing evolution of memory usage" /></p> <p>Since it’s not entirely readable from the chart: after the patches, the AngryBots and ZenGarden entry stubs memory usages went down to respectively 262 and 362 KB. This was also a relatively huge win in compilation times, but on such a low scale that it didn’t make a huge difference on total compile time.</p> <h2 id="bug-1447591-remove-wasm-binarytotext"><a class="zola-anchor" href="#bug-1447591-remove-wasm-binarytotext" aria-label="Anchor link for: bug-1447591-remove-wasm-binarytotext">🔗</a>Bug 1447591: Remove wasm::BinaryToText</h2> <p>WebAssembly is a binary format, and there is an equivalent human-readable and debuggable text format: the WebAssembly Text format, or <em>WAT</em> format. While SpiderMonkey once directly produced WAT for display in C++, it’s now easier for <a rel="noopener noreferrer external" target="_blank" href="https://github.com/devtools-html/debugger.html">debugger.html</a> to do so in JS. This also made the mapping between bytecode offsets and text offsets (source maps) more consistent with the display, and it could be useful in other places where this project is being used. Recently after confirming that the C++ implementation wasn’t used anymore, I was able to remove it. It’s not every day that you get a net loss of around 5,500 lines of code, which is always nice: less code means fewer bugs and less maintenance burden, especially when the code is dead.</p> <h2 id="bug-1445272-1450261-implement-basic-anyref-support"><a class="zola-anchor" href="#bug-1445272-1450261-implement-basic-anyref-support" aria-label="Anchor link for: bug-1445272-1450261-implement-basic-anyref-support">🔗</a>Bug 1445272 / 1450261: Implement basic <code>anyref</code> support</h2> <p>A new proposal has been made to the WebAssembly specification committee a few months ago: to add <a rel="noopener noreferrer external" target="_blank" href="https://github.com/WebAssembly/reference-types">reference types</a> to the type system. Reference types are a new way to represent a reference to any <em>host</em> values. In a Web environment, this means being capable of playing with JavaScript values within WebAssembly. This is a huge difference with the existing type system, which only contains primitive types: integers represented on 32 or 64 bits, IEEE754 floating-point numbers represented on 32 or 64 bits. This is also a first step for implementing <a rel="noopener noreferrer external" target="_blank" href="http://github.com/webassembly/gc">garbage collection</a> (GC) integration within WebAssembly: since these reference values have been allocated on the GC heap in JavaScript, they need to be traced during wasm execution.</p> <p>The basic implementation of this feature in the first bug allows one to use a new type, called <code>anyref</code>, as part of a function’s signature or in local variables, be it in a function definition or an imported function. This allows using JS variables within wasm and pass them around to other JS functions. The second bug implemented the capability to read and write <code>anyref</code> values in wasm Globals [1]. Since Globals can be manipulated outside of the wasm Module thanks to their JS API, and garbage collections can happen at any time in JS, we needed to implement GC barriers to make sure that the stored value would not be marked as unused during tracing. There is good literature explaining why these barriers are needed and what they do, so I will not expand too much on the topic.</p> <p>Here’s an example of usage according to latest spec drafts (and therefore subject to change for now):</p> <pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="common-lisp"><span class="giallo-l"><span>(module</span></span> <span class="giallo-l"><span> (func $alert (</span><span style="color: #0DB9D7;">import</span><span style="color: #89DDFF;"> &quot;</span><span style="color: #9ECE6A;">env</span><span style="color: #89DDFF;">&quot; &quot;</span><span style="color: #9ECE6A;">alert</span><span style="color: #89DDFF;">&quot;</span><span>) (param anyref))</span></span> <span class="giallo-l"><span> (global $global_ref (mut anyref) (ref.null anyref))</span></span> <span class="giallo-l"><span> (func (</span><span style="color: #0DB9D7;">export</span><span style="color: #89DDFF;"> &quot;</span><span style="color: #9ECE6A;">set_and_alert</span><span style="color: #89DDFF;">&quot;</span><span>) (param $param anyref) (result anyref)</span></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;"> ;; Put the previous value of $global_ref on the virtual value stack.</span></span> <span class="giallo-l"><span> get_global $global_ref</span></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;"> ;; Get the argument anyref value and store it in $global_ref.</span></span> <span class="giallo-l"><span> get_local $param</span></span> <span class="giallo-l"><span> set_global $global_ref</span></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;"> ;; Call the $alert method with the argument anyref value.</span></span> <span class="giallo-l"><span> get_local $param</span></span> <span class="giallo-l"><span> call $alert</span></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;"> ;; The previous value of $global_ref is still on the stack and will be</span></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;"> ;; returned.</span></span> <span class="giallo-l"><span> )</span></span> <span class="giallo-l"><span>)</span></span></code></pre> <p><em>Example of wasm text format using <code>anyref</code>.</em></p> <pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="javascript"><span class="giallo-l"><span style="color: #9ABDF5;">(</span><span style="color: #9D7CD8;font-style: italic;">async</span><span style="color: #BB9AF7;"> function</span><span style="color: #9ABDF5;">() {</span></span> <span class="giallo-l"><span style="color: #9D7CD8;font-style: italic;"> let</span><span style="color: #89DDFF;"> {</span><span style="color: #BB9AF7;"> instance</span><span style="color: #89DDFF;"> } =</span><span style="color: #BB9AF7;font-style: italic;"> await</span><span style="color: #C0CAF5;"> WebAssembly</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">instantiate</span><span style="color: #9ABDF5;">(</span><span style="color: #C0CAF5;">wasmBinary</span><span style="color: #89DDFF;">,</span><span style="color: #9ABDF5;"> {</span></span> <span class="giallo-l"><span style="color: #73DACA;"> env</span><span style="color: #89DDFF;">:</span><span style="color: #9ABDF5;"> {</span></span> <span class="giallo-l"><span style="color: #7AA2F7;"> alert</span><span style="color: #9ABDF5;">(</span><span style="color: #E0AF68;">obj</span><span style="color: #9ABDF5;">) {</span></span> <span class="giallo-l"><span style="color: #7AA2F7;"> alert</span><span style="color: #9ABDF5;">(</span><span style="color: #89DDFF;">`</span><span style="color: #9ECE6A;">Hello, </span><span style="color: #7DCFFF;">${</span><span style="color: #C0CAF5;">obj</span><span style="color: #89DDFF;">.</span><span style="color: #7DCFFF;">name}</span><span style="color: #9ECE6A;">!</span><span style="color: #89DDFF;">`</span><span style="color: #9ABDF5;">)</span><span style="color: #89DDFF;">;</span></span> <span class="giallo-l"><span style="color: #9ABDF5;"> }</span></span> <span class="giallo-l"><span style="color: #9ABDF5;"> }</span></span> <span class="giallo-l"><span style="color: #9ABDF5;"> })</span><span style="color: #89DDFF;">;</span></span> <span class="giallo-l"><span style="color: #C0CAF5;"> console</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">log</span><span style="color: #9ABDF5;">(</span><span style="color: #C0CAF5;">instance</span><span style="color: #89DDFF;">.</span><span style="color: #7DCFFF;">exports</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">set_and_alert</span><span style="color: #9ABDF5;">({</span></span> <span class="giallo-l"><span style="color: #73DACA;"> name</span><span style="color: #89DDFF;">: &#39;</span><span style="color: #9ECE6A;">world</span><span style="color: #89DDFF;">&#39;,</span></span> <span class="giallo-l"><span style="color: #73DACA;"> secretVal</span><span style="color: #89DDFF;">:</span><span style="color: #FF9E64;"> 42</span></span> <span class="giallo-l"><span style="color: #9ABDF5;"> }))</span><span style="color: #89DDFF;">;</span></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // alerts &quot;Hello, world!&quot;, logs null</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span style="color: #C0CAF5;"> console</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">log</span><span style="color: #9ABDF5;">(</span><span style="color: #0DB9D7;">JSON</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">stringify</span><span style="color: #9ABDF5;">(</span><span style="color: #C0CAF5;">instance</span><span style="color: #89DDFF;">.</span><span style="color: #7DCFFF;">exports</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">set_and_alert</span><span style="color: #9ABDF5;">({</span></span> <span class="giallo-l"><span style="color: #73DACA;"> name</span><span style="color: #89DDFF;">: &#39;</span><span style="color: #9ECE6A;">there</span><span style="color: #89DDFF;">&#39;</span></span> <span class="giallo-l"><span style="color: #9ABDF5;"> })))</span><span style="color: #89DDFF;">;</span></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;"> // alerts &quot;Hello, there!&quot;, logs { name: &#39;world!&#39;, secretVal: 42 }</span></span> <span class="giallo-l"><span style="color: #9ABDF5;">})()</span><span style="color: #89DDFF;">;</span></span></code></pre> <p><em>Example of JavaScript using the module defined above, passing JS values and reading them from WebAssembly.</em></p> <p>This is a very preliminary prototype and it might change in the next few months. If you feel adventurous, you can try it on Firefox Nightly by setting the <code>about:config</code> pref <code>javascript.options.wasm_gc</code> to <code>true</code>; note that we haven’t fully hooked this up to garbage collection yet, so your experimentation might occasionally throw out-of-memory exceptions. In any case, if you see something, <a rel="noopener noreferrer external" target="_blank" href="https://bugzilla.mozilla.org/enter_bug.cgi?product=Core&amp;component=Javascript%3A%20Web%20Assembly">say something</a>.</p> <p>Say you are a compiler developer, and you would like to port your language to WebAssembly, and your language uses a GC. At the moment, the only way you can do this is by compiling your garbage collector to WebAssembly, and it would be backed by the wasm Module’s memory. This works, but it won’t be very efficient. Plus, there’s already a very efficient, solidly tested, constantly improving garbage collector in your browser that uses all the possible dirty low-level tricks known to mankind, which is the GC being used for JavaScript. What if we could give you access to the garbage collector directly? Then you’d just need to give a way to define structures, and then could use a set of opcodes to allocate them, read and write fields on them, etc. At the moment, the reference types proposal only allows you to move garbage-collected values around. There’s also code in Firefox Nightly to experiment with defining your own data structures and using them, but it is very very early. If you’re interested in following us implementing more parts, this <a rel="noopener noreferrer external" target="_blank" href="https://bugzilla.mozilla.org/show_bug.cgi?id=1444925">tracking issue</a> might be of interest.</p> <p>[1] Think of a C++ “global” value, not a JavaScript “global”.</p> <h2 id="future-work"><a class="zola-anchor" href="#future-work" aria-label="Anchor link for: future-work">🔗</a>Future work</h2> <p>There is still much more work to be done on the implementation of WebAssembly in Spidermonkey, to implement other new proposals, to make it faster, or to have even better generated code.</p> <p>A big thank you for the proofreading to <a rel="noopener noreferrer external" target="_blank" href="http://whereswalden.com/">Waldo</a>, <a rel="noopener noreferrer external" target="_blank" href="https://github.com/steveklabnik">steveklabnik</a> and <a rel="noopener noreferrer external" target="_blank" href="https://twitter.com/ag_dubs">ashleygwilliams</a>. Extra thanks go to Ashley who also drew the two diagrams showing how stubs evolved.</p> 2016-04-22T15:00:42+00:00 2016-04-22T15:00:42+00:00 Benjamin Bouvier https://bouvier.cc/tech/making-asmjs-webassembly-compilation-more-parallel/ <p>In December 2015, I’ve worked on reducing startup time of asm.js programs in Firefox by making compilation more parallel. As our JavaScript engine, Spidermonkey, uses the same compilation pipeline for both asm.js and WebAssembly, this also benefitted WebAssembly compilation. Now is a good time to talk about what it meant, how it got achieved and what are the next ideas to make it even faster.</p> <span id="continue-reading"></span><h1 id="what-does-it-mean-to-make-a-program-more-parallel"><a class="zola-anchor" href="#what-does-it-mean-to-make-a-program-more-parallel" aria-label="Anchor link for: what-does-it-mean-to-make-a-program-more-parallel">🔗</a>What does it mean to make a program “more parallel”?</h1> <p>Parallelization consists of splitting a sequential program into smaller independent tasks, then having them run on different CPU. If your program is using <code>N</code> cores, it can be up to <code>N</code> times faster.</p> <p>Well, in theory. Let’s say you’re in a car, driving on a 100 Km long road. You’ve already driven the first 50 Km in one hour. Let’s say your car can have unlimited speed from now on. What is the maximal average speed you can reach, once you get to the end of the road?</p> <p>People intuitively answer “If it can go as fast as I want, so nearby lightspeed sounds plausible”. But this is not true! In fact, if you could teleport from your current position to the end of the road, you’d have traveled 100 Km in one hour, so your maximal theoritical speed is 100 Km per hour. This result is a consequence of <a rel="noopener noreferrer external" target="_blank" href="https://en.wikipedia.org/wiki/Amdahl%27s_law">Amdahl’s law</a>. When we get back to our initial problem, this means you can expect a <code>N</code> times speedup if you’re running your program with <code>N</code> cores if, and only if your program can be <strong>entirely</strong> run in parallel. This is usually not the case, and that is why most wording refers to <em>speedups <strong>up to</strong> N times faster</em>, when it comes to parallelization.</p> <p>Now, say your program is already running some portions in parallel. To make it faster, one can identify some parts of the program that are sequential, and make them independent so that you can run them in parallel. With respect to our car metaphor, this means augmenting the portion of the road on which you can run at unlimited speed.</p> <p>This is exactly what we have done with parallel compilation of asm.js programs under Firefox.</p> <h1 id="a-quick-look-at-the-asm-js-compilation-pipeline"><a class="zola-anchor" href="#a-quick-look-at-the-asm-js-compilation-pipeline" aria-label="Anchor link for: a-quick-look-at-the-asm-js-compilation-pipeline">🔗</a>A quick look at the asm.js compilation pipeline</h1> <p>I recommend to read this <a rel="noopener noreferrer external" target="_blank" href="https://blog.mozilla.org/luke/2014/01/14/asm-js-aot-compilation-and-startup-performance/">blog post</a>. It clearly explains the differences between JIT (Just In Time) and AOT (Ahead Of Time) compilation, and elaborates on the different parts of the engines involved in the compilation pipeline.</p> <p>As a TL;DR, keep in mind that <a rel="noopener noreferrer external" target="_blank" href="http://asmjs.org/">asm.js</a> is a strictly validated, highly optimizable, typed subset of JavaScript. Once validated, it guarantees high performance and stability (no garbage collector involved!). That is ensured by mapping every single JavaScript instruction of this subset to a few CPU instructions, if not only a single instruction. This means an asm.js program needs to get <em>compiled</em> to machine code, that is, translated from JavaScript to the language your CPU directly manipulates (like what GCC would do for a C++ program). If you haven’t heard, the results are impressive and you can run <a href="https://bouvier.cc/tech/making-asmjs-webassembly-compilation-more-parallel/beta.unity3d.com/jonas/DT2/">video</a> <a rel="noopener noreferrer external" target="_blank" href="https://www.unrealengine.com/html5">games</a> directly in your browser, without needing to install anything. No plugins. Nothing more than your usual, everyday browser.</p> <p>Because asm.js programs can be gigantic in size (in number of functions as well as in number of lines of code), the first compilation of the entire program is going to take some time. Afterwards, Firefox uses a caching mechanism that prevents the need for recompilation and almost instaneously loads the code, so subsequent loadings matter less*****. The end user will mostly wait for the first compilation, thus this one needs to be fast.</p> <p>Before the work explained below, the pipeline for compiling a single function (out of an asm.js module) would look like this:</p> <ul> <li>parse the function, and as we parse, emit intermediate representation (IR) nodes for the compiler infrastructure. SpiderMonkey has several IRs, including the MIR (middle-level IR, mostly loaded with semantic) and the LIR (low-level IR closer to the CPU memory representation: registers, stack, etc.). The one generated here is the MIR. All of this happens on the main thread.</li> <li>once the entire IR graph is generated for the function, optimize the MIR graph (i.e. apply a few optimization passes). Then, generate the LIR graph before carrying out register allocation (probably the most costly task of the pipeline). This can be done on supplementary helper threads, as the MIR optimization and LIR generation for a given function doesn’t depend on other ones.</li> <li>since functions can call between themselves within an asm.js module, they need references to each other. In assembly, a reference is merely an offset to somewhere else in memory. In this initial implementation, code generation is carried out on the main thread, at the cost of speed but for the sake of simplicity.</li> </ul> <p>So far, only the MIR optimization passes, register allocation and LIR generation were done in parallel. Wouldn’t it be nice to be able to do more?</p> <p><strong>*</strong> There are conditions for benefitting from the caching mechanism. In particular, the script should be loaded <a rel="noopener noreferrer external" target="_blank" href="https://developer.mozilla.org/en-US/docs/Games/Techniques/Async_scripts">asynchronously</a> and it should be of a consequent size.</p> <h1 id="doing-more-in-parallel"><a class="zola-anchor" href="#doing-more-in-parallel" aria-label="Anchor link for: doing-more-in-parallel">🔗</a>Doing more in parallel</h1> <p>Our goal is to make more work in parallel: so can we take out MIR generation from the main thread? And we can take out code generation as well?</p> <p>The answer happens to be <em>yes</em> to both questions.</p> <p>For the former, instead of emitting a MIR graph as we parse the function’s body, we emit a small, compact, pre-order representation of the function’s body. In short, a new IR. As work was starting on <a rel="noopener noreferrer external" target="_blank" href="https://github.com/webassembly/design">WebAssembly</a> (wasm) at this time, and since asm.js semantics and wasm semantics mostly match, the IR could just be the wasm <a rel="noopener noreferrer external" target="_blank" href="https://github.com/WebAssembly/design/blob/master/BinaryEncoding.md">encoding</a>, consisting of the wasm opcodes plus a few specific asm.js ones*. Then, wasm is translated to MIR in another thread.</p> <p>Now, instead of parsing and generating MIR in a single pass, we would now parse and generate wasm IR in one pass, and generate the MIR out of the wasm IR in another pass. The wasm IR is very compact and much cheaper to generate than a full MIR graph, because generating a MIR graph needs some algorithmic work, including the creation of Phi nodes (join values after any form of branching). As a result, it is expected that compilation time won’t suffer. This was a large refactoring: taking every single asm.js instructions, and encoding them in a compact way and later decode these into the equivalent MIR nodes.</p> <p>For the second part, could we generate code on other threads? One structure in the code base, the <em>MacroAssembler</em>, is used to generate all the code and it contains all necessary metadata about offsets. By adding more metadata there to abstract internal calls <strong>**</strong>, we can describe the new scheme in terms of a classic functional <code>map</code>/<code>reduce</code>:</p> <ul> <li>the wasm IR is sent to a thread, which will return a MacroAssembler. That is a <code>map</code> operation, transforming an array of wasm IR into an array of MacroAssemblers.</li> <li>When a thread is done compiling, we merge its MacroAssembler into one big MacroAssembler. Most of the merge consists in taking all the offset metadata in the thread MacroAssembler, fixing up all the offsets, and concatenate the two generated code buffers. This is equivalent to a <code>reduce</code> operation, merging each MacroAssembler within the module’s one.</li> </ul> <p>At the end of the compilation of the entire module, there is still some light work to be done: offsets of internal calls need to be translated to their actual locations. All this work has been done in <a rel="noopener noreferrer external" target="_blank" href="https://bugzilla.mozilla.org/show_bug.cgi?id=1181612">this bugzilla bug</a>.</p> <p><strong>*</strong> In fact, at the time when this was being done, we used a different superset of wasm. Since then, work has been done so that our asm.js frontend is really just another wasm emitter.</p> <p>**** ** referencing functions by their appearance order index in the module, rather than an offset to the actual start of the function. This order is indeed stable, from a function to the other.</p> <h1 id="results"><a class="zola-anchor" href="#results" aria-label="Anchor link for: results">🔗</a>Results</h1> <p>Benchmarking has been done on a Linux x64 machine with 8 cores clocked at 4.2 Ghz.</p> <p>First, compilation times of a few asm.js massive games:</p> <ul> <li><a rel="noopener noreferrer external" target="_blank" href="http://beta.unity3d.com/jonas/DT2/">DeadTrigger2</a></li> <li><a rel="noopener noreferrer external" target="_blank" href="http://beta.unity3d.com/jonas/AngryBots/">AngryBots</a></li> <li><a rel="noopener noreferrer external" target="_blank" href="https://github.com/lukewagner/PlatformerGamePacked">Platformer game</a></li> <li><a rel="noopener noreferrer external" target="_blank" href="https://www.unrealengine.com/html5">Tappy Chicken</a></li> </ul> <p>The X scale is the compilation time in seconds, so lower is better. Each value point is the best one of three runs. For the new scheme, the corresponding relative speedup (in percentage) has been added:</p> <p><img src="https://bouvier.cc/tech/making-asmjs-webassembly-compilation-more-parallel/2016-04-22_parallelization-times.png" alt="Compilation times of various benchmarks" /></p> <p>For all games, compilation is much faster with the new parallelization scheme.</p> <p>Now, let’s go a bit deeper. The Linux CLI tool <code>perf</code> has a <code>stat</code> command that gives you an average of the number of utilized CPUs during the program execution. This is a great measure of threading efficiency: the more a CPU is utilized, the more it is not idle, waiting for other results to come, and thus useful. For a constant task execution time, the more utilized CPUs, the more likely the program will execute quickly.</p> <p>The X scale is the number of utilized CPUs, according to the <code>perf stat</code> command, so higher is better. Again, each value point is the best one of three runs.</p> <p><img src="https://bouvier.cc/tech/making-asmjs-webassembly-compilation-more-parallel/2016-04-22_parallelization-cpu-utilized.png" alt="CPU utilized on DeadTrigger2" /></p> <p>With the older scheme, the number of utilized CPUs quickly rises up from 1 to 4 cores, then more slowly from 5 cores and beyond. Intuitively, this means that with 8 cores, we almost reached the theoritical limit of the portion of the program that can be made parallel (not considering the overhead introduced by parallelization or altering the scheme).</p> <p>But with the newer scheme, we get much more CPU usage even after 6 cores! Then it slows down a bit, although it is still more significant than the slow rise of the older scheme. So it is likely that with even more threads, we could have even better speedups than the one mentioned beforehand. In fact, we have moved the theoritical limit mentioned above a bit further: we have expanded the portion of the program that can be made parallel. Or to keep on using the initial car/road metaphor, we’ve shortened the constant speed portion of the road to the benefit of the unlimited speed portion of the road, resulting in a shorter trip overall.</p> <h1 id="future-steps"><a class="zola-anchor" href="#future-steps" aria-label="Anchor link for: future-steps">🔗</a>Future steps</h1> <p>Despite these improvements, compilation time can still be a pain, especially on mobile. This is mostly due to the fact that we’re running a whole multi-million line codebase through the backend of a compiler to generate optimized code. Following this work, the next bottleneck during the compilation process is parsing, which matters for asm.js in particular, which source is plain text. Decoding WebAssembly is an order of magnitude faster though, and it can be made even faster. Moreover, we have even more load-time optimizations coming down the pipeline!</p> <p>In the meanwhile, we keep on improving the WebAssembly backend. Keep track of our progress on <a rel="noopener noreferrer external" target="_blank" href="https://bugzilla.mozilla.org/show_bug.cgi?id=1188259">bug 1188259</a>!</p> 2016-03-09T18:00:42+00:00 2016-03-09T18:00:42+00:00 Benjamin Bouvier https://bouvier.cc/tech/previous-writing-about-mozilla-work/ <p>I am currently a compiler engineer at Mozilla corporation, the company making the Firefox browser among else. Our JavaScript virtual machine, Spidermonkey, is split in several tiers, including an highly optimizing Just-In-Time (JIT) compiler able to compile JavaScript to assembly at runtime. My previous work has involved efficiently compiling Float32 arithmetic to hardware instructions and implement a new SIMD API for the Web.</p> <h2 id="about-float32-optimizations"><a class="zola-anchor" href="#about-float32-optimizations" aria-label="Anchor link for: about-float32-optimizations">🔗</a>About Float32 optimizations</h2> <p>The full blog post is <a rel="noopener noreferrer external" target="_blank" href="https://blog.mozilla.org/javascript/2013/11/07/efficient-float32-arithmetic-in-javascript/">there</a>. It has been written in November 2013.</p> <p>The main idea is that if you have float32 inputs to an operation; and you cast them to doubles; and you apply an arithmetic operation to these inputs; and you cast the result back to a float32, then you’d have the same result as if you did the entire computation with float32 values and operations.</p> <p>So we’ve introduced an operation in JavaScript that converts a Number to its closest float32 IEEE754 representation: <code>Math.fround</code>. Said differently, the above equivalence says that:</p> <pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="javascript"><span class="giallo-l"><span style="color: #BB9AF7;">function</span><span style="color: #7AA2F7;"> f</span><span style="color: #9ABDF5;">(</span><span style="color: #E0AF68;">x</span><span style="color: #89DDFF;">,</span><span style="color: #E0AF68;"> y</span><span style="color: #9ABDF5;">) {</span></span> <span class="giallo-l"><span style="color: #BB9AF7;font-style: italic;"> return</span><span style="color: #C0CAF5;"> x</span><span style="color: #89DDFF;"> +</span><span style="color: #C0CAF5;"> y</span><span style="color: #89DDFF;">;</span></span> <span class="giallo-l"><span style="color: #9ABDF5;">}</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span style="color: #BB9AF7;">function</span><span style="color: #7AA2F7;"> g</span><span style="color: #9ABDF5;">(</span><span style="color: #E0AF68;">x</span><span style="color: #89DDFF;">,</span><span style="color: #E0AF68;"> y</span><span style="color: #9ABDF5;">) {</span></span> <span class="giallo-l"><span style="color: #9D7CD8;font-style: italic;"> var</span><span style="color: #BB9AF7;"> xf</span><span style="color: #89DDFF;"> =</span><span style="color: #C0CAF5;"> Math</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">fround</span><span style="color: #9ABDF5;">(</span><span style="color: #C0CAF5;">x</span><span style="color: #9ABDF5;">)</span><span style="color: #89DDFF;">;</span></span> <span class="giallo-l"><span style="color: #9D7CD8;font-style: italic;"> var</span><span style="color: #BB9AF7;"> yf</span><span style="color: #89DDFF;"> =</span><span style="color: #C0CAF5;"> Math</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">fround</span><span style="color: #9ABDF5;">(</span><span style="color: #C0CAF5;">y</span><span style="color: #9ABDF5;">)</span><span style="color: #89DDFF;">;</span></span> <span class="giallo-l"><span style="color: #BB9AF7;font-style: italic;"> return</span><span style="color: #C0CAF5;"> Math</span><span style="color: #89DDFF;">.</span><span style="color: #7AA2F7;">fround</span><span style="color: #9ABDF5;">(</span><span style="color: #C0CAF5;">xf</span><span style="color: #89DDFF;"> +</span><span style="color: #C0CAF5;"> yf</span><span style="color: #9ABDF5;">)</span><span style="color: #89DDFF;">;</span></span> <span class="giallo-l"><span style="color: #9ABDF5;">}</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span style="color: #51597D;font-style: italic;">// For all x, y that can be represented exactly as float32:</span></span> <span class="giallo-l"><span style="color: #7AA2F7;">assert</span><span style="color: #9ABDF5;">(</span><span style="color: #7AA2F7;">f</span><span style="color: #9ABDF5;">(</span><span style="color: #C0CAF5;">x</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> y</span><span style="color: #9ABDF5;">)</span><span style="color: #BB9AF7;"> ===</span><span style="color: #7AA2F7;"> g</span><span style="color: #9ABDF5;">(</span><span style="color: #C0CAF5;">x</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> y</span><span style="color: #9ABDF5;">))</span><span style="color: #89DDFF;">;</span></span></code></pre> <p>Yes, <code>===</code>. The same <code>===</code> you’ve been told <strong>not</strong> to use for floating-point Numbers. But here, we have <em>bitwise</em> equality, so we can use strict equality*.</p> <p>Processors have special instructions for carrying out float32 arithmetic, which have higher throughput than the equivalent double ones. With this result in mind, we could add a pass that would spot opportunities where the computations are equivalent (thanks to <code>Math.fround</code> hints) and emit float32 instructions instead of double instructions. This sped up a some numerical applications and games engines by a few points.</p> <p>* a careful reader would object that this is wrong for <code>x = y = NaN</code>, which I’ve put away for the sake of simplicity.</p> <h2 id="about-simd-js"><a class="zola-anchor" href="#about-simd-js" aria-label="Anchor link for: about-simd-js">🔗</a>About SIMD.js</h2> <p>The full blog post is <a rel="noopener noreferrer external" target="_blank" href="https://blog.mozilla.org/javascript/2015/03/10/state-of-simd-js-performance-in-firefox/">there</a>. It has been written in March 2015.</p> <p>Nowadays, processors have instructions sets that allow them to execute several simple arithmetic operations at once. For instance, let’s say you have two arrays of integers and you want to add each element to the corresponding one in the other array. If both arrays have size <code>N</code>, this means you’ll have to carry out <code>N</code> scalar additions. But processors can actually group these into bundles of several additions, with SIMD; for the case of 32-bits wide integers, on most modern processors, you need at most <code>Math.ceil(N / 4)</code> instructions. The blog post details what SIMD.js is and what bottlenecks we hit during implementation.</p> <h2 id="conclusion"><a class="zola-anchor" href="#conclusion" aria-label="Anchor link for: conclusion">🔗</a>Conclusion</h2> <p>This was a small reminder about previously written blog posts. If you’re into JavaScript, compilers or low-level optimization, I can only recommend you to go read the <a rel="noopener noreferrer external" target="_blank" href="https://blog.mozilla.org/javascript/">Mozilla’s JavaScript blog</a>.</p> 2015-12-31T23:59:42+00:00 2015-12-31T23:59:42+00:00 Benjamin Bouvier https://bouvier.cc/tech/hello-mozilla-world/ <h2 id="hello-mozilla"><a class="zola-anchor" href="#hello-mozilla" aria-label="Anchor link for: hello-mozilla">🔗</a>Hello Mozilla!</h2> <p>Hello world! And in advance, happy new year!</p> <p>I am going to try to blog more about my work at Mozilla, and this will happen in this new <code>mozilla</code> category. You can expect technical posts about SpiderMonkey (the JS virtual machine implementation in Mozilla Firefox), OdinMonkey / asm.js, WebAssembly (wasm), and all things Mozilla in this category.</p> <p>Feel free to reach me out via <a rel="noopener noreferrer external" target="_blank" href="https://twitter.com/bnjbvr">twitter</a>, and see you soon!</p>